Files
Ali Khokhar 58aef0dc8a Refactor provider runtime ownership (#925)
## Problem

Provider construction, model discovery, validation, and cleanup lived in
one registry module. API and admin routes depended on registry-shaped
app state and legacy process-level provider helpers.

## Changes

| Before | After |
| --- | --- |
| `providers.registry` mixed provider factories, config, cache,
discovery, validation, and cleanup. | `providers.runtime` splits
factories, config, cache, model cache, discovery, validation, and
runtime orchestration. |
| API and admin routes read `app.state.provider_registry` and sometimes
created registries ad hoc. | API and admin routes use app-scoped
`ProviderRuntime` through `app.state.provider_runtime`. |
| `api.dependencies` kept process-global provider cache helpers. |
`api.dependencies` resolves providers only through the app-scoped
runtime. |
| Registry-shaped tests preserved old internal boundaries. |
Runtime-shaped tests assert provider config, construction, cache,
discovery, validation, and import boundaries. |

<!-- greptile_comment -->

<details open><summary><h3>Greptile Summary</h3></summary>

This PR moves provider lifecycle ownership from the old registry module
into an app-scoped runtime package. The main changes are:

- Split provider config, factory wiring, instance cache, model cache,
discovery, validation, and cleanup into `providers.runtime` modules.
- Updated API and admin routes to resolve providers and model metadata
through `app.state.provider_runtime`.
- Removed legacy process-global provider helpers and the deleted
`providers.registry` module.
- Updated docs, smoke metadata, import-boundary checks, and tests for
the new runtime ownership model.
- Bumped the package version and lockfile metadata for the production
refactor.
</details>

<h3>Confidence Score: 5/5</h3>

The provider runtime refactor appears merge-safe with no identified
blocking issues.

The changes consistently move provider ownership to app-scoped runtime
modules and update API, admin, docs, smoke metadata, import-boundary
checks, and tests around that architecture.

<details><summary><h3><a href="https://www.greptile.com/trex"><img
alt="T-Rex"
src="https://greptile-static-assets.s3.amazonaws.com/trex/trex_green.svg"
height="20" align="absmiddle"></a> T-Rex Logs</h3></summary>

**What T-Rex did**
- Ran a baseline and head comparison of provider registry and runtime
states, verifying the after-state shows head
state\_has\_provider\_registry=False and
state\_has\_provider\_runtime=True, that GET /v1/models and admin
endpoints respond with 200, and that provider\_resolver\_called via
runtime, with assertions passing.
- Verified that the four focused provider-runtime contract tests passed
in both the before and after refactor runs, including runtime split
checks, with exit code 0.
- Identified environmental blockers that prevented the smoke-runtime
workflow from running, including uv unavailability, missing pytest for
/usr/local/bin/python, and Python 3.11 being used despite pyproject.toml
requiring \>=3.14.

<a
href="https://app.greptile.com/trex/runs/12528505/artifacts"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifactsDark.svg?v=1"><source
media="(prefers-color-scheme: light)"
srcset="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifacts.svg?v=1"><img
alt="View all artifacts"
src="https://greptile-static-assets.s3.amazonaws.com/badges/ViewAllArtifacts.svg?v=1"
height="32"></picture></a>

<sub><a href="https://www.greptile.com/trex"><img alt="T-Rex"
src="https://greptile-static-assets.s3.amazonaws.com/trex/trex_green.svg"
height="14" align="absmiddle"></a> Ran code and verified through
T-Rex</sub>
</details>

<sub>Reviews (1): Last reviewed commit: ["Refactor provider runtime
ownership"](https://github.com/alishahryar1/free-claude-code/commit/01d589488185c1f85112f1a49c47f04512846161)
| [Re-trigger
Greptile](https://app.greptile.com/api/retrigger?id=40312173)</sub>

<!-- /greptile_comment -->
2026-06-27 15:04:34 -07:00

55 lines
1.8 KiB
Python

"""Provider instance cache and cleanup."""
from __future__ import annotations
from collections.abc import Callable, MutableMapping
from config.settings import Settings
from providers.base import BaseProvider
from .factory import create_provider
ProviderCreator = Callable[[str, Settings], BaseProvider]
class ProviderCache:
"""Cache provider instances for one settings snapshot."""
def __init__(
self,
settings: Settings,
providers: MutableMapping[str, BaseProvider] | None = None,
*,
creator: ProviderCreator = create_provider,
) -> None:
self._settings = settings
self._providers = providers if providers is not None else {}
self._creator = creator
def is_cached(self, provider_id: str) -> bool:
"""Return whether a provider for this id is already cached."""
return provider_id in self._providers
def get(self, provider_id: str) -> BaseProvider:
"""Return an existing provider or create it lazily."""
if provider_id not in self._providers:
self._providers[provider_id] = self._creator(provider_id, self._settings)
return self._providers[provider_id]
async def cleanup(self) -> None:
"""Clean up every cached provider, then clear the cache."""
items = list(self._providers.items())
errors: list[Exception] = []
try:
for _provider_id, provider in items:
try:
await provider.cleanup()
except Exception as exc:
errors.append(exc)
finally:
self._providers.clear()
if len(errors) == 1:
raise errors[0]
if len(errors) > 1:
raise ExceptionGroup("One or more provider cleanups failed", errors)