-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Description
Add a new endpoint to expose platform-wide latency metrics for the admin dashboard.
Motivation
Currently, latency metrics (TTFT, ITL) are only available at the organization level. Having platform-wide aggregates would help admins monitor overall system performance.
Proposed Solution
Add GET /v1/admin/platform/latency endpoint that returns:
- Average TTFT across all requests
- P95/P99 TTFT
- Average ITL (Inter-Token Latency)
- P95/P99 ITL
- Breakdown by model
Technical Details
- Query the
organization_usage_logtable for latency aggregates - Support time range filtering (start/end params)
- Consider caching for performance
Acceptance Criteria
- New endpoint returns platform-wide latency metrics
- Supports time range filtering
- Returns breakdown by model
- Documentation updated
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request