Skip to content

Comments

Emit token count metrics for vllm#62

Open
kevinle-fw wants to merge 1 commit intomainfrom
kl/emit_token_metrics_vllm
Open

Emit token count metrics for vllm#62
kevinle-fw wants to merge 1 commit intomainfrom
kl/emit_token_metrics_vllm

Conversation

@kevinle-fw
Copy link

vLLM previously didn't provide token count-based metrics including total tokens used and tokens per second. This change emits those metrics so downstream metrics collection is able to run without failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants