⚡ Bolt: optimize global pricing lookups to avoid redundant cloning

[Laisky]

⚡ Bolt: optimize global pricing lookups to avoid redundant cloning

Identify and fix a performance bottleneck where GetModelRatioWithThreeLayers
and other pricing resolution functions were calling GetGlobalModelConfig
on every request.

GetGlobalModelConfig performs a deep clone of the entire ModelConfig
struct, which includes multiple maps and slices. This is wasteful when
only a single field (like Ratio) is needed.

Introduced specialized, lightweight getters:

• GetGlobalModelRatio and GetGlobalCompletionRatio now return (float64, bool).
• Added GetGlobalVideoPricing, GetGlobalAudioPricing, and GetGlobalImagePricing.

Impact:

• Reduces latency for global pricing lookups by ~92% (678ns -> 50ns in benchmarks).
• Significantly reduces allocations and GC pressure in the request hot path.
• Avoids double cloning in video/audio/image pricing resolution.

Verified with benchmarks and unit tests.

---

PR created automatically by Jules for task 587817312917026333 started by @Laisky

Summary by CodeRabbit

• Performance Improvements

  • Optimized pricing lookups to reduce unnecessary cloning operations, delivering significant performance gains (~13.6x speedup in benchmarks).

• Bug Fixes

  • Fixed pricing fallback logic to correctly handle explicit zero pricing values instead of treating them as undefined.

• New Features

  • Introduced lightweight pricing accessors for specific model pricing types (video, audio, image).

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[github-actions]

Merging this branch will not change overall coverage

Impacted Packages	Coverage Δ	🤖
github.com/Laisky/one-api/relay/pricing	0.00% (ø)

---

Coverage by file

Changed files (no unit tests)

Changed File	Coverage Δ	Total	Covered	Missed	🤖
github.com/Laisky/one-api/relay/pricing/global.go	0.00% (ø)	0	0	0
github.com/Laisky/one-api/relay/pricing/resolver.go	0.00% (ø)	0	0	0

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

• github.com/Laisky/one-api/relay/pricing/global_test.go

[coderabbitai]

📝 Walkthrough

Walkthrough

This pull request refactors global pricing lookup to avoid expensive deep cloning of ModelConfig. It introduces lightweight getter functions returning existence flags (GetGlobalModelRatio, GetGlobalCompletionRatio) and specialized media pricing getters (GetGlobalVideoPricing, GetGlobalAudioPricing, GetGlobalImagePricing), updating callers and tests accordingly.

Changes

Cohort / File(s)	Summary
Documentation .jules/bolt.md, docs/arch/billing.md	Added performance note about ModelConfig cloning cost and updated billing fallback logic to use existence checks instead of positivity checks for global pricing ratios.
Global Pricing API relay/pricing/global.go	Updated GetGlobalModelRatio and GetGlobalCompletionRatio signatures to return (float64, bool); introduced three new specialized getters (GetGlobalVideoPricing, GetGlobalAudioPricing, GetGlobalImagePricing) that return cloned pricing configs; refactored internal logic to use boolean-returning helpers for existence checking.
Test Updates relay/pricing/global_test.go	Updated tests for GetGlobalModelRatio and GetGlobalCompletionRatio to assert on existence flag; modified TestIsGlobalPricingInitialized to trigger initialization via new signature.
Pricing Resolver relay/pricing/resolver.go	Replaced GetGlobalModelConfig calls with specialized getters (GetGlobalAudioPricing, GetGlobalImagePricing) in audio/image pricing fallback paths.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• ⚡ Bolt: optimize global pricing lookup performance #302: Both PRs modify global pricing lookup paths (GetModelRatioWithThreeLayers, GetCompletionRatioWithThreeLayers, GetVideoPricingWithThreeLayers); this PR adds boolean-returning ratio getters and specialized media pricing getters while the related PR switched to using GetGlobalModelConfig to avoid full map cloning.

Poem

🐰 No more cloning configs deep and wide,
Lightweight getters now become our guide,
Existence flags with every call,
Specialized pricing—we have them all!
Performance hops faster through the night! 🌙

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the main optimization: avoiding redundant cloning in global pricing lookups, which directly aligns with the primary change across all modified files.
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch bolt/optimize-global-pricing-lookups-587817312917026333

---

No actionable comments were generated in the recent review. 🎉

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

This PR optimizes global pricing resolution in the relay pricing layer by avoiding expensive deep-clones of adaptor.ModelConfig when callers only need a single field (e.g., Ratio) or a single nested pricing config (video/audio/image). It does this by introducing lightweight global getters and updating hot-path resolution logic and docs/tests accordingly.

Changes:

• Changed GetGlobalModelRatio / GetGlobalCompletionRatio to return (float64, bool) so callers can distinguish “missing model” from an explicit zero value without cloning full configs.
• Added targeted global getters for nested pricing configs: GetGlobalVideoPricing, GetGlobalAudioPricing, GetGlobalImagePricing (each returns a cloned sub-config).
• Updated audio/image/video resolvers and documentation + unit tests to use the new APIs and preserve “existence vs value” semantics.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
relay/pricing/global.go	Adds lightweight global pricing getters and switches three-layer ratio/video resolution to avoid full ModelConfig cloning.
relay/pricing/resolver.go	Uses new global audio/image pricing getters to avoid redundant full-config cloning in fallback resolution.
relay/pricing/global_test.go	Updates tests to validate the new (value, exists) return contract for global ratio/completion ratio getters.
docs/arch/billing.md	Updates documented three-layer pricing logic to check existence rather than > 0.
.jules/bolt.md	Records the optimization lesson learned and preferred pattern for future hot-path pricing lookups.