Skip to content

[Analytics] Add platform-level latency metrics endpoint #359

@PierreLeGuen

Description

@PierreLeGuen

Description

Add a new endpoint to expose platform-wide latency metrics for the admin dashboard.

Motivation

Currently, latency metrics (TTFT, ITL) are only available at the organization level. Having platform-wide aggregates would help admins monitor overall system performance.

Proposed Solution

Add GET /v1/admin/platform/latency endpoint that returns:

  • Average TTFT across all requests
  • P95/P99 TTFT
  • Average ITL (Inter-Token Latency)
  • P95/P99 ITL
  • Breakdown by model

Technical Details

  • Query the organization_usage_log table for latency aggregates
  • Support time range filtering (start/end params)
  • Consider caching for performance

Acceptance Criteria

  • New endpoint returns platform-wide latency metrics
  • Supports time range filtering
  • Returns breakdown by model
  • Documentation updated

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions