[Feature]: inference observability dashboard #78
Description
Is there an existing feature request for this?
- I have searched the existing issues
Problem or Motivation
As a user, I want a observability dashboard that correlates deployment changes, end-to-end and per-stage request latency (router/handle/replica), and autoscaler intent vs actual capacity, so that I can quickly pinpoint and resolve production performance regressions.
e.g. https://www.anyscale.com/blog/ray-serve-observability-grafana-dashboard-anyscale
Proposed Solution
Add an out-of-the-box Grafana observability dashboard (or something else) that automatically correlates ModelDeployment rollout events, end-to-end + stage-level serving latency (gateway/router/replica), and autoscaler desired vs actual replicas/GPU utilization or any other metrics that could help users diagnose issues.
Alternatives Considered
No response
Feature Area
Deployments / Model Management
How important is this feature to you?
Nice to have
Mockups or Examples
No response
Additional Context
No response