feat: add provider registry for flexible provider registration

[eric642]

Summary

Add ProviderRegistry to allow custom provider registration as a fallback when built-in providers are not available. This enables users to register their own provider implementations without modifying the core codebase.

Changes

• Add ProviderRegistry type with thread-safe Register/Get/IsRegistered methods
• Add ProviderRegistry() accessor with lazy initialization using sync.Once
• Add createProvider() wrapper that tries built-in first, then registry
• Update prepareProvider and hot reload to use createProvider

Type of change

Feature

Affected areas

Core (Go)

Breaking changes

No

If yes, describe impact and migration instructions.

Related issues

Closes #1400

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • Added a pluggable provider registry allowing registration of custom provider constructors with lazy, singleton initialization.
  • Provider creation now falls back to the registry when a built-in provider is not available.

• Stability

  • Registry access is thread-safe and preserves builtin-provider precedence.

• Tests

  • Comprehensive unit tests for registry behavior, concurrency, and integration with provider creation.

Walkthrough

Adds a thread-safe ProviderRegistry and lazy registry initialization to Bifrost. Provider creation now tries built-in constructors first and falls back to registry-registered constructors via a new createProvider path; provider registry is lazily initialized and exposed via Bifrost.ProviderRegistry().

Changes

Cohort / File(s)	Summary
Bifrost Core Integration core/bifrost.go	Added fields providerRegistry *ProviderRegistry and registryInitOnce sync.Once; added public ProviderRegistry() for lazy init; introduced createProvider(...) path that attempts createBaseProvider() then falls back to registry constructors; updated call sites to use the new create path.
Provider Registry Implementation core/registry.go	New file implementing ProviderRegistry with sync.RWMutex and a map of schemas.ModelProvider → ProviderConstructor; adds NewProviderRegistry(), Register(), Get(), IsRegistered(), and RegisteredProviders(); defines ProviderConstructor signature.
Registry Test Suite core/registry_test.go	New unit tests covering registry initialization, Register/Get/IsRegistered, enumeration, concurrent safety, Bifrost lazy singleton behavior, registry fallback creation, and ensuring built-ins bypass registry constructors.

Sequence Diagram

sequenceDiagram
    participant App as Application
    participant Bifrost
    participant Base as createBaseProvider
    participant Registry as ProviderRegistry
    participant Constructor as ProviderConstructor

    App->>Bifrost: createProvider(providerKey, config)
    activate Bifrost
    Bifrost->>Base: attempt createBaseProvider(providerKey, config)
    activate Base
    Base-->>Bifrost: provider or error
    deactivate Base

    alt built-in created
        Bifrost-->>App: return Provider
    else fallback to registry
        Bifrost->>Registry: ProviderRegistry().Get(providerKey)
        activate Registry
        Registry-->>Bifrost: ProviderConstructor or nil
        deactivate Registry

        alt constructor found
            Bifrost->>Constructor: constructor(config, logger)
            activate Constructor
            Constructor-->>Bifrost: Provider instance / error
            deactivate Constructor
            Bifrost-->>App: return Provider or error
        else constructor missing
            Bifrost-->>App: return error (unsupported provider)
        end
    end
    deactivate Bifrost

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇 I nibble keys and store each name,
Built-ins race first, then registry claims fame.
Thread-safe hops, constructors aligned,
New paths stitched so providers bind.
✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 10.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The pull request title accurately and concisely summarizes the main change: adding a provider registry for flexible provider registration, which is the core feature of this changeset.
Description check	✅ Passed	The pull request description covers most required template sections including Summary, Changes, Type of change, Affected areas, Breaking changes, and Related issues. However, it lacks How to test and Checklist sections.
Linked Issues check	✅ Passed	The PR successfully implements all key objectives from issue #1400: thread-safe ProviderRegistry with Register/Get/IsRegistered methods, lazy initialization via ProviderRegistry() accessor, createProvider() fallback mechanism, and comprehensive unit tests validating the implementation.
Out of Scope Changes check	✅ Passed	All changes are directly aligned with issue #1400 requirements. The PR introduces the registry system, its integration into provider creation logic, and thorough test coverage without extraneous modifications.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[eric642]

@akshaydeo Hello, can you check if this feature change is appropriate?

[akshaydeo]

@eric642 I actually like the idea, I just need to understand some implications.

One important area is how we manage requests / LLM execution across providers. We use an actor pattern: an actor is initialized at boot, then it accepts requests and responds via channels, with concurrency controlled by the configured limits.

That said, this concept could be extended to dynamically create actors in response to incoming messages. I’ll do a thorough deep dive on the implications and update here.

[eric642]

@akshaydeo，I roughly understand what you mean, but I think this advanced feature is not needed in 99% of development scenarios. The main reason is that the parameters of large model api are very complex, and there are differences in APIs among various manufacturers (whether they are compatible with the OpenAI format or not). Usually, when developing AI app, developers need to confirm the parameters and the list of supported models, and there are rarely cases where models need to be dynamically selected according to requests.

That said, this concept could be extended to dynamically create actors in response to incoming messages. I’ll do a thorough deep dive on the implications and update here.

[akshaydeo]

umm, its not about dynamically selecting models infact we never select models dynamically. This is about pre-warming the connections and workers for processing incoming LLM calls. For that we use actor pattern.

Edit:

bifrost/core/bifrost.go

Lines 3272 to 3339 in 6b0c188

	select {
	case pq.queue <- msg:
	// Message was sent successfully
	case <-pq.done:
	bifrost.releaseChannelMessage(msg)
	bifrostErr := newBifrostErrorFromMsg("provider is shutting down")
	bifrostErr.ExtraFields = schemas.BifrostErrorExtraFields{
	RequestType: req.RequestType,
	Provider: provider,
	ModelRequested: model,
	}
	return nil, bifrostErr
	case <-ctx.Done():
	bifrost.releaseChannelMessage(msg)
	bifrostErr := newBifrostErrorFromMsg("request cancelled while waiting for queue space")
	bifrostErr.ExtraFields = schemas.BifrostErrorExtraFields{
	RequestType: req.RequestType,
	Provider: provider,
	ModelRequested: model,
	}
	return nil, bifrostErr
	default:
	if bifrost.dropExcessRequests.Load() {
	bifrost.releaseChannelMessage(msg)
	bifrost.logger.Warn("request dropped: queue is full, please increase the queue size or set dropExcessRequests to false")
	bifrostErr := newBifrostErrorFromMsg("request dropped: queue is full")
	bifrostErr.ExtraFields = schemas.BifrostErrorExtraFields{
	RequestType: req.RequestType,
	Provider: provider,
	ModelRequested: model,
	}
	return nil, bifrostErr
	}
	// Re-check closing flag before blocking send (lock-free atomic check)
	if pq.isClosing() {
	bifrost.releaseChannelMessage(msg)
	bifrostErr := newBifrostErrorFromMsg("provider is shutting down")
	bifrostErr.ExtraFields = schemas.BifrostErrorExtraFields{
	RequestType: req.RequestType,
	Provider: provider,
	ModelRequested: model,
	}
	return nil, bifrostErr
	}
	select {
	case pq.queue <- msg:
	// Message was sent successfully
	case <-pq.done:
	bifrost.releaseChannelMessage(msg)
	bifrostErr := newBifrostErrorFromMsg("provider is shutting down")
	bifrostErr.ExtraFields = schemas.BifrostErrorExtraFields{
	RequestType: req.RequestType,
	Provider: provider,
	ModelRequested: model,
	}
	return nil, bifrostErr
	case <-ctx.Done():
	bifrost.releaseChannelMessage(msg)
	bifrostErr := newBifrostErrorFromMsg("request cancelled while waiting for queue space")
	bifrostErr.ExtraFields = schemas.BifrostErrorExtraFields{
	RequestType: req.RequestType,
	Provider: provider,
	ModelRequested: model,
	}
	return nil, bifrostErr
	}
	}

[eric642]

@akshaydeo, Oh oh, I get it. It would indeed be better if it can be created dynamically.

[eric642]

@akshaydeo hello，Is this submission useful? Or are there other plans?