GitHub Copilot Metrics Plugin #8674
Open
+4,116
−11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add .specify/ directory with: - memory/constitution.md: Apache DevLake project constitution v1.0.0 - Plugin Independence principle - Three-Layer Data Model (Raw/Tool/Domain) - Test-Driven Development requirements - Migration-First Schema Changes - Apache Compliance requirements - templates/: spec, plan, tasks, checklist, agent-file templates - scripts/powershell/: automation scripts for feature workflows
Add GitHub Copilot agent mode definitions: - .github/agents/: Agent configurations (specify, plan, tasks, etc.) - .github/prompts/: Prompt files linking to agents These enable /speckit.* commands in VS Code Copilot for Spec-Driven Development workflow.
Research for building a GitHub Copilot DevLake plugin: - copilot_api_research.md: API endpoint documentation - copilot_api_actual_responses.md: Live API test results (octodemo org) - copilot_implementation_strategy.md: Option B (repo-level) approach - copilot_plugin_spec.md: Full plugin specification - copilot_research_summary.md: Research findings summary Key decisions: - Org/team-level metrics (not per-user) for privacy - Time-based before/after correlation with DORA metrics - Implementation date configuration for impact analysis - Three-phase MVP approach Ready for Spec-Driven Development implementation.
…tegy documents - Created `copilot_api_actual_responses.md` to document actual API responses and key metrics. - Developed `copilot_api_research.md` summarizing API endpoints, usage metrics, and authentication requirements. - Established `copilot_implementation_strategy.md` outlining the strategy for repository/project-level analysis. - Drafted `copilot_plugin_spec.md` detailing the plugin's specifications, data models, and API endpoints. - Compiled `copilot_research_summary.md` to summarize findings and decisions made during the research phase.
…a model and API specifications
…d error handling strategies
Implements REST endpoints for Copilot connections and scopes, plus a Test Connection helper against the GitHub billing endpoint with friendly error handling.
Adds stateful collectors for /copilot/metrics and /copilot/billing/seats, plus extractors into tool tables with Retry-After handling and unit/e2e coverage.
Marks Phase 4 (US2) tasks as completed now that collectors/extractor, fixtures, and tests are in place.
Adds Copilot Adoption dashboard panels for active/engaged users, acceptance rate, chat usage, and seat timeline based on _tool_copilot_org_metrics.
Updates quickstart with the Copilot Adoption dashboard name/variables and marks US3 tasks complete.
…and enhance Dockerfile for script execution
Support both top-level arrays and wrapped {seats:[...]} responses; mark Copilot subtasks as CROSS domain type.
…/incubator-devlake into 001-copilot-metrics-plugin
4 tasks
Co-authored-by: ewega <26189114+ewega@users.noreply.github.com>
Co-authored-by: ewega <26189114+ewega@users.noreply.github.com>
…askData Co-authored-by: ewega <26189114+ewega@users.noreply.github.com>
…-gh-copilot Rename GitHub Copilot plugin identifier and types from "copilot" to "gh-copilot"
Co-authored-by: ewega <26189114+ewega@users.noreply.github.com>
Co-authored-by: ewega <26189114+ewega@users.noreply.github.com>
Add GitHub Copilot icon to Config-UI
dosubot
bot
added
size:XXL
Contributor
|
Great job! Did you test it locally? |
Contributor
Author
|
@klesh, yes tested locally, it seems there's some pipeline failing though so I'm going to try and fix those. Any comments from yourself? |
Contributor
Author
|
Unfortunately, it seems like the metrics endpoint utilized is getting sunset: https://docs.github.com/en/enterprise-cloud@latest/rest/copilot/copilot-usage-metrics?apiVersion=2022-11-28 Poor timing. Will rework and open a new PR |
Contributor
|
I see. Take your time. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pr-type/bug-fix,pr-type/feature-development, etc.Summary
This PR adds a new copilot data-source plugin that ingests GitHub Copilot organization-level adoption metrics and ships a Grafana Adoption dashboard powered by tool-layer tables.
Key capabilities in this branch:
GitHub Copilot Plugin Implementation:
gh-copilotplugin, including API handlers for connections (connection.go), scopes (scope.go), blueprint pipeline plan generation (blueprint_v200.go), and connection testing (test_connection.go). These handlers support creating, updating, deleting, listing, and validating Copilot connections and organization scopes. [1] [2] [3] [4]init.go) to register API endpoints and helpers, and provided a standalone entrypoint for debugging collectors (copilot.go). [1] [2]Validation and Testing Enhancements:
Documentation:
README.mdfor thegh-copilotplugin, describing its purpose, setup, endpoints, data flow, limitations, and troubleshooting guidance.Build and Docker Improvements:
.dockerignoreto exclude common files, editor settings, and build outputs, improving Docker build performance and cleanliness.backend/Dockerfileto normalize shell scripts (convert line endings and set executable permissions) and ensure correct ownership for Python files, improving cross-platform compatibility and container reliability. [1] [2]Does this close any open issues?
#8664
Screenshots
Other Information
TODOs: