Skip to content

Commit 2b6fef1

Browse files
lpcoxlpcox
authored
Expand smoke-copilot test coverage to match smoke-claude (#718)
`smoke-copilot` had 5 tests while `smoke-claude` had 10. This brings them to parity for comprehensive engine validation. ## Changes **Added shared workflows** (4 files) - `mcp-pagination.md` - MCP response size limits and pagination patterns - `github-queries-safe-input.md` - Safe-input tools for issues/PRs/discussions with jq filtering - `go-make.md` - Safe-input wrappers for Go and Make commands - `github-mcp-app.md` - GitHub App configuration template **Expanded test coverage** (5 → 10 tests) - Added: Safe Inputs GH CLI, Make Build, Tavily Web Search, Discussion Interaction, Agentic Workflows MCP - Enhanced: Serena MCP now validates `find_symbol` results (≥3 symbols) **Updated configuration** - Permissions: +`discussions: read` - Imports: 7 shared workflows - Safe-outputs: +`max: 2`, +`group: true`, +`close-older-issues: true` - Timeout: 5 → 15 minutes - Tools: +`github.toolsets: [repos, pull_requests]`, +`runtimes.go: 1.25` - Output: Issue creation + PR/discussion comments (was PR comment only) **Note:** `max-turns` excluded - Copilot engine doesn't support this feature (Claude-only). ## Test Matrix | Test | Coverage | |------|----------| | GitHub MCP | ✅ Last 2 merged PRs | | Safe Inputs GH CLI | ✅ Query via `safeinputs-gh` | | Serena MCP | ✅ `activate_project` + `find_symbol` | | Make Build | ✅ `safeinputs-make build` | | Playwright | ✅ github.com navigation | | Tavily Search | ✅ "GitHub Agentic Workflows" | | File I/O | ✅ Create + verify test file | | Bash | ✅ Command execution | | Discussions | ✅ Query + comment | | Agentic Workflows | ✅ Status introspection | <!-- START COPILOT CODING AGENT TIPS --> --- ✨ Let Copilot coding agent [set things up for you](https://github.com/github/gh-aw-mcpg/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.
2 parents fc60e38 + 892da6d commit 2b6fef1

File tree

6 files changed

+1435
-57
lines changed

6 files changed

+1435
-57
lines changed

‎.github/workflows/shared/github-mcp-app.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
---
2+
#tools:
3+
# github:
4+
# app:
5+
# app-id: ${{ vars.APP_ID }}
6+
# private-key: ${{ secrets.APP_PRIVATE_KEY }}
7+
---
8+
9+
<!--
10+
# Shared GitHub MCP App Configuration
11+
12+
This shared workflow provides repository-level GitHub App configuration for the GitHub MCP server.
13+
14+
## Configuration Variables
15+
16+
This shared workflow expects:
17+
- **Repository Variable**: `APP_ID` - The GitHub App ID for MCP server authentication
18+
- **Repository Secret**: `APP_PRIVATE_KEY` - The GitHub App private key for MCP server authentication
19+
20+
## Usage
21+
22+
Import this configuration in your workflows to enable GitHub App authentication for the GitHub MCP server:
23+
24+
```yaml
25+
imports:
26+
- shared/github-mcp-app.md
27+
```
28+
29+
The configuration will be automatically merged into your workflow's tools.github section.
30+
-->

‎.github/workflows/shared/github-queries-safe-input.md‎

Lines changed: 340 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,340 @@
1+
---
2+
safe-inputs:
3+
github-issue-query:
4+
description: "Query GitHub issues with jq filtering support. Without --jq, returns schema and data size info. Use --jq '.' to get all data, or specific jq expressions to filter."
5+
inputs:
6+
repo:
7+
type: string
8+
description: "Repository in owner/repo format (defaults to current repository)"
9+
required: false
10+
state:
11+
type: string
12+
description: "Issue state: open, closed, all (default: open)"
13+
required: false
14+
limit:
15+
type: number
16+
description: "Maximum number of issues to fetch (default: 30)"
17+
required: false
18+
jq:
19+
type: string
20+
description: "jq filter expression to apply to output. If not provided, returns schema info instead of full data."
21+
required: false
22+
env:
23+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
24+
run: |
25+
set -e
26+
27+
# Default values
28+
REPO="${INPUT_REPO:-}"
29+
STATE="${INPUT_STATE:-open}"
30+
LIMIT="${INPUT_LIMIT:-30}"
31+
JQ_FILTER="${INPUT_JQ:-}"
32+
33+
# JSON fields to fetch
34+
JSON_FIELDS="number,title,state,author,createdAt,updatedAt,closedAt,body,labels,assignees,comments,milestone,url"
35+
36+
# Build and execute gh command
37+
if [[ -n "$REPO" ]]; then
38+
OUTPUT=$(gh issue list --state "$STATE" --limit "$LIMIT" --json "$JSON_FIELDS" --repo "$REPO")
39+
else
40+
OUTPUT=$(gh issue list --state "$STATE" --limit "$LIMIT" --json "$JSON_FIELDS")
41+
fi
42+
43+
# Apply jq filter if specified
44+
if [[ -n "$JQ_FILTER" ]]; then
45+
jq "$JQ_FILTER" <<< "$OUTPUT"
46+
else
47+
# Return schema and size instead of full data
48+
ITEM_COUNT=$(jq 'length' <<< "$OUTPUT")
49+
DATA_SIZE=${#OUTPUT}
50+
51+
# Validate values are numeric
52+
if ! [[ "$ITEM_COUNT" =~ ^[0-9]+$ ]]; then
53+
ITEM_COUNT=0
54+
fi
55+
if ! [[ "$DATA_SIZE" =~ ^[0-9]+$ ]]; then
56+
DATA_SIZE=0
57+
fi
58+
59+
cat << EOF
60+
{
61+
"message": "No --jq filter provided. Use --jq to filter and retrieve data.",
62+
"item_count": $ITEM_COUNT,
63+
"data_size_bytes": $DATA_SIZE,
64+
"schema": {
65+
"type": "array",
66+
"description": "Array of issue objects",
67+
"item_fields": {
68+
"number": "integer - Issue number",
69+
"title": "string - Issue title",
70+
"state": "string - Issue state (OPEN, CLOSED)",
71+
"author": "object - Author info with login field",
72+
"createdAt": "string - ISO timestamp of creation",
73+
"updatedAt": "string - ISO timestamp of last update",
74+
"closedAt": "string|null - ISO timestamp of close",
75+
"body": "string - Issue body content",
76+
"labels": "array - Array of label objects with name field",
77+
"assignees": "array - Array of assignee objects with login field",
78+
"comments": "object - Comments info with totalCount field",
79+
"milestone": "object|null - Milestone info with title field",
80+
"url": "string - Issue URL"
81+
}
82+
},
83+
"suggested_queries": [
84+
{"description": "Get all data", "query": "."},
85+
{"description": "Get issue numbers and titles", "query": ".[] | {number, title}"},
86+
{"description": "Get open issues only", "query": ".[] | select(.state == \"OPEN\")"},
87+
{"description": "Get issues by author", "query": ".[] | select(.author.login == \"USERNAME\")"},
88+
{"description": "Get issues with label", "query": ".[] | select(.labels | map(.name) | index(\"bug\"))"},
89+
{"description": "Get issues with many comments", "query": ".[] | select(.comments.totalCount > 5) | {number, title, comments: .comments.totalCount}"},
90+
{"description": "Count by state", "query": "group_by(.state) | map({state: .[0].state, count: length})"}
91+
]
92+
}
93+
EOF
94+
fi
95+
96+
github-pr-query:
97+
description: "Query GitHub pull requests with jq filtering support. Without --jq, returns schema and data size info. Use --jq '.' to get all data, or specific jq expressions to filter."
98+
inputs:
99+
repo:
100+
type: string
101+
description: "Repository in owner/repo format (defaults to current repository)"
102+
required: false
103+
state:
104+
type: string
105+
description: "PR state: open, closed, merged, all (default: open)"
106+
required: false
107+
limit:
108+
type: number
109+
description: "Maximum number of PRs to fetch (default: 30)"
110+
required: false
111+
jq:
112+
type: string
113+
description: "jq filter expression to apply to output. If not provided, returns schema info instead of full data."
114+
required: false
115+
env:
116+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
117+
run: |
118+
set -e
119+
120+
# Default values
121+
REPO="${INPUT_REPO:-}"
122+
STATE="${INPUT_STATE:-open}"
123+
LIMIT="${INPUT_LIMIT:-30}"
124+
JQ_FILTER="${INPUT_JQ:-}"
125+
126+
# JSON fields to fetch
127+
JSON_FIELDS="number,title,state,author,createdAt,updatedAt,mergedAt,closedAt,headRefName,baseRefName,isDraft,reviewDecision,additions,deletions,changedFiles,labels,assignees,reviewRequests,url"
128+
129+
# Build and execute gh command
130+
if [[ -n "$REPO" ]]; then
131+
OUTPUT=$(gh pr list --state "$STATE" --limit "$LIMIT" --json "$JSON_FIELDS" --repo "$REPO")
132+
else
133+
OUTPUT=$(gh pr list --state "$STATE" --limit "$LIMIT" --json "$JSON_FIELDS")
134+
fi
135+
136+
# Apply jq filter if specified
137+
if [[ -n "$JQ_FILTER" ]]; then
138+
jq "$JQ_FILTER" <<< "$OUTPUT"
139+
else
140+
# Return schema and size instead of full data
141+
ITEM_COUNT=$(jq 'length' <<< "$OUTPUT")
142+
DATA_SIZE=${#OUTPUT}
143+
144+
# Validate values are numeric
145+
if ! [[ "$ITEM_COUNT" =~ ^[0-9]+$ ]]; then
146+
ITEM_COUNT=0
147+
fi
148+
if ! [[ "$DATA_SIZE" =~ ^[0-9]+$ ]]; then
149+
DATA_SIZE=0
150+
fi
151+
152+
cat << EOF
153+
{
154+
"message": "No --jq filter provided. Use --jq to filter and retrieve data.",
155+
"item_count": $ITEM_COUNT,
156+
"data_size_bytes": $DATA_SIZE,
157+
"schema": {
158+
"type": "array",
159+
"description": "Array of pull request objects",
160+
"item_fields": {
161+
"number": "integer - PR number",
162+
"title": "string - PR title",
163+
"state": "string - PR state (OPEN, CLOSED, MERGED)",
164+
"author": "object - Author info with login field",
165+
"createdAt": "string - ISO timestamp of creation",
166+
"updatedAt": "string - ISO timestamp of last update",
167+
"mergedAt": "string|null - ISO timestamp of merge",
168+
"closedAt": "string|null - ISO timestamp of close",
169+
"headRefName": "string - Source branch name",
170+
"baseRefName": "string - Target branch name",
171+
"isDraft": "boolean - Whether PR is a draft",
172+
"reviewDecision": "string|null - Review decision (APPROVED, CHANGES_REQUESTED, REVIEW_REQUIRED)",
173+
"additions": "integer - Lines added",
174+
"deletions": "integer - Lines deleted",
175+
"changedFiles": "integer - Number of files changed",
176+
"labels": "array - Array of label objects with name field",
177+
"assignees": "array - Array of assignee objects with login field",
178+
"reviewRequests": "array - Array of review request objects",
179+
"url": "string - PR URL"
180+
}
181+
},
182+
"suggested_queries": [
183+
{"description": "Get all data", "query": "."},
184+
{"description": "Get PR numbers and titles", "query": ".[] | {number, title}"},
185+
{"description": "Get open PRs only", "query": ".[] | select(.state == \"OPEN\")"},
186+
{"description": "Get merged PRs", "query": ".[] | select(.mergedAt != null)"},
187+
{"description": "Get PRs by author", "query": ".[] | select(.author.login == \"USERNAME\")"},
188+
{"description": "Get large PRs", "query": ".[] | select(.changedFiles > 10) | {number, title, changedFiles}"},
189+
{"description": "Count by state", "query": "group_by(.state) | map({state: .[0].state, count: length})"}
190+
]
191+
}
192+
EOF
193+
fi
194+
195+
github-discussion-query:
196+
description: "Query GitHub discussions with jq filtering support. Without --jq, returns schema and data size info. Use --jq '.' to get all data, or specific jq expressions to filter."
197+
inputs:
198+
repo:
199+
type: string
200+
description: "Repository in owner/repo format (defaults to current repository)"
201+
required: false
202+
limit:
203+
type: number
204+
description: "Maximum number of discussions to fetch (default: 30)"
205+
required: false
206+
jq:
207+
type: string
208+
description: "jq filter expression to apply to output. If not provided, returns schema info instead of full data."
209+
required: false
210+
env:
211+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
212+
run: |
213+
set -e
214+
215+
# Default values
216+
REPO="${INPUT_REPO:-}"
217+
LIMIT="${INPUT_LIMIT:-30}"
218+
JQ_FILTER="${INPUT_JQ:-}"
219+
220+
# Parse repository owner and name
221+
if [[ -n "$REPO" ]]; then
222+
OWNER=$(echo "$REPO" | cut -d'/' -f1)
223+
NAME=$(echo "$REPO" | cut -d'/' -f2)
224+
else
225+
# Get current repository from GitHub context
226+
OWNER="${GITHUB_REPOSITORY_OWNER:-}"
227+
NAME=$(echo "${GITHUB_REPOSITORY:-}" | cut -d'/' -f2)
228+
fi
229+
230+
# Validate owner and name
231+
if [[ -z "$OWNER" || -z "$NAME" ]]; then
232+
echo "Error: Could not determine repository owner and name" >&2
233+
exit 1
234+
fi
235+
236+
# Build GraphQL query for discussions
237+
GRAPHQL_QUERY=$(cat <<QUERY
238+
{
239+
repository(owner: "$OWNER", name: "$NAME") {
240+
discussions(first: $LIMIT, orderBy: {field: CREATED_AT, direction: DESC}) {
241+
nodes {
242+
number
243+
title
244+
author {
245+
login
246+
}
247+
createdAt
248+
updatedAt
249+
body
250+
category {
251+
name
252+
}
253+
labels(first: 10) {
254+
nodes {
255+
name
256+
}
257+
}
258+
comments {
259+
totalCount
260+
}
261+
answer {
262+
id
263+
}
264+
url
265+
}
266+
}
267+
}
268+
}
269+
QUERY
270+
)
271+
272+
# Execute GraphQL query via gh api
273+
GRAPHQL_OUTPUT=$(gh api graphql -f query="$GRAPHQL_QUERY")
274+
275+
# Transform GraphQL output to match gh discussion list format
276+
OUTPUT=$(echo "$GRAPHQL_OUTPUT" | jq '[.data.repository.discussions.nodes[] | {
277+
number: .number,
278+
title: .title,
279+
author: .author,
280+
createdAt: .createdAt,
281+
updatedAt: .updatedAt,
282+
body: .body,
283+
category: .category,
284+
labels: .labels.nodes,
285+
comments: .comments,
286+
answer: .answer,
287+
url: .url
288+
}]')
289+
290+
# Apply jq filter if specified
291+
if [[ -n "$JQ_FILTER" ]]; then
292+
jq "$JQ_FILTER" <<< "$OUTPUT"
293+
else
294+
# Return schema and size instead of full data
295+
ITEM_COUNT=$(jq 'length' <<< "$OUTPUT")
296+
DATA_SIZE=${#OUTPUT}
297+
298+
# Validate values are numeric
299+
if ! [[ "$ITEM_COUNT" =~ ^[0-9]+$ ]]; then
300+
ITEM_COUNT=0
301+
fi
302+
if ! [[ "$DATA_SIZE" =~ ^[0-9]+$ ]]; then
303+
DATA_SIZE=0
304+
fi
305+
306+
cat << EOF
307+
{
308+
"message": "No --jq filter provided. Use --jq to filter and retrieve data.",
309+
"item_count": $ITEM_COUNT,
310+
"data_size_bytes": $DATA_SIZE,
311+
"schema": {
312+
"type": "array",
313+
"description": "Array of discussion objects",
314+
"item_fields": {
315+
"number": "integer - Discussion number",
316+
"title": "string - Discussion title",
317+
"author": "object - Author info with login field",
318+
"createdAt": "string - ISO timestamp of creation",
319+
"updatedAt": "string - ISO timestamp of last update",
320+
"body": "string - Discussion body content",
321+
"category": "object - Category info with name field",
322+
"labels": "array - Array of label objects with name field",
323+
"comments": "object - Comments info with totalCount field",
324+
"answer": "object|null - Accepted answer if exists",
325+
"url": "string - Discussion URL"
326+
}
327+
},
328+
"suggested_queries": [
329+
{"description": "Get all data", "query": "."},
330+
{"description": "Get discussion numbers and titles", "query": ".[] | {number, title}"},
331+
{"description": "Get discussions by author", "query": ".[] | select(.author.login == \"USERNAME\")"},
332+
{"description": "Get discussions in category", "query": ".[] | select(.category.name == \"Ideas\")"},
333+
{"description": "Get answered discussions", "query": ".[] | select(.answer != null)"},
334+
{"description": "Get unanswered discussions", "query": ".[] | select(.answer == null) | {number, title, category: .category.name}"},
335+
{"description": "Count by category", "query": "group_by(.category.name) | map({category: .[0].category.name, count: length})"}
336+
]
337+
}
338+
EOF
339+
fi
340+
---

0 commit comments

Comments
 (0)