Skip to content

fix(parsing): drop TextFormatT parameterization in parse_response to fix memory leak (#3084)#3088

Open
MukundaKatta wants to merge 1 commit intoopenai:mainfrom
MukundaKatta:fix/parsed-response-memory-leak
Open

fix(parsing): drop TextFormatT parameterization in parse_response to fix memory leak (#3084)#3088
MukundaKatta wants to merge 1 commit intoopenai:mainfrom
MukundaKatta:fix/parsed-response-memory-leak

Conversation

@MukundaKatta
Copy link
Copy Markdown

Why

Closes #3084.

AsyncResponses.parse() leaks pydantic schema objects without bound. The issue author captured it with a flame graph — every call allocates a fresh Rust-backed SchemaValidator/SchemaSerializer that never gets freed.

Root cause

parse_response() called construct_type_unchecked with three types parameterized by a free module-level TypeVar:

construct_type_unchecked(type_=ParsedResponseOutputText[TextFormatT], ...)
construct_type_unchecked(type_=ParsedResponseOutputMessage[TextFormatT], ...)
construct_type_unchecked(type_=ParsedResponse[TextFormatT], ...)

Pydantic can't resolve a free TypeVar, so model_rebuild(raise_errors=False) returns False. MockCoreSchema._built_memo only caches the rebuilt schema when model_rebuild succeeds — so the cache is never populated and a new schema is allocated on every call.

Fix

Drop the [TextFormatT] parameterization at the type_= argument. At runtime Python's generics are erased anyway — the constructed object's type is identical either way. Only the pydantic schema rebuild path differs:

Before After
type_ arg ParsedResponse[TextFormatT] ParsedResponse
model_rebuild always returns False succeeds
_built_memo never populated populated once, reused
Per-call allocation heavy Rust objects cache hit

This is exactly what the issue author diagnosed.

Verification

  • Ran the existing tests/lib/responses/test_responses.py suite locally: 5/5 passing — no functional regression.
  • The change is purely about schema caching behavior. Callers see no difference in returned objects or their types.

Scope

Minimal — 3 sites in one file, inline comments pointing back to #3084 for maintainability. No public API change, no test changes needed (existing tests cover the parse path).

…enai#3084)

Closes openai#3084.

parse_response() called construct_type_unchecked with three types
parameterized by a free TypeVar (ParsedResponseOutputText[TextFormatT],
ParsedResponseOutputMessage[TextFormatT], ParsedResponse[TextFormatT]).

Pydantic cannot resolve a free TypeVar, so model_rebuild(raise_errors=False)
returns False on every invocation. MockCoreSchema._built_memo only caches
the rebuilt schema when model_rebuild succeeds; when it fails, the cache
is never set and a new Rust-backed SchemaValidator/SchemaSerializer is
allocated on every parse_response() call.

Result: heavy pydantic-core objects leak without bound whenever
AsyncResponses.parse() is called in a long-lived process. This is
observable as a flame-graph spike (see openai#3084 for the screenshot).

Fix (per issue author's diagnosis): drop the [TextFormatT]
parameterization at the runtime type_= argument. At runtime Python's
generics are erased anyway — the constructed object's type is
identical either way. Only the pydantic schema rebuild path differs:
with the non-parameterized generic, model_rebuild succeeds and the
schema is cached in _built_memo.

Tests: all 5 existing tests in tests/lib/responses/test_responses.py
continue to pass (verified locally). No behavior change for callers.

Signed-off-by: Mukunda Katta <mukunda.vjcs6@gmail.com>
@MukundaKatta MukundaKatta requested a review from a team as a code owner April 15, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Async Responses Structured Outputs Memory leak

1 participant