fix(openai): add support for logprobs when using the response api#36371
fix(openai): add support for logprobs when using the response api#36371HOZHENWAI wants to merge 2 commits into
Conversation
Merging this PR will improve performance by 13%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | WallTime | test_init_time |
3.6 ms | 3.2 ms | +11.88% |
| ⚡ | WallTime | test_init_time |
3.6 ms | 3.2 ms | +13% |
| ⚡ | WallTime | test_init_time |
136 ms | 122.1 ms | +11.36% |
Comparing HOZHENWAI:HOZHENWAI/fix-missing-logprobs-openai-response (9921b2c) with master (342d8bd)
Footnotes
-
37 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
| additional_kwargs["parsed"] = content.parsed | ||
| if hasattr(content, "logprobs") and content.logprobs: | ||
| if "logprobs" in response_metadata: | ||
| # only keep the latest message logprobs |
There was a problem hiding this comment.
should this go in response_metadata, or on the associated content block?
There was a problem hiding this comment.
well, is that a recommandation or a question? The default pattern without the response api when using logprobs is to have it in the response_metadata. I just copied the pattern to make sure the outputs were compatible between use_responses_api=True and use_responses_api=False . Should this be changed?
Tbh, I don't think I tested the case with multi-turn conversation when logprobs=True and use_response_api=False, so I don't know if the pattern is different in that case.
Fixes #36211
Fix the issue where the combination of
use_responses_api=True andlogprobs=True prevented any invocation.Precommit validated and all unit-tests passing including the new ones written for this fix.
Test scope are discutable: i've decided to test that the request_payload is compatible with the response api instead of testing the output of
_construct_responses_api_payloadin the case oflogprobs=True which may have been more proper. If necessary i can do the changes. Also, i have no tested the case of refusal when logprobs =True.I also think a small update to the integration test could be useful but did not write it.
Final test made was to make sure that for the following snipped
`from langchain_openai import ChatOpenAI
client = ChatOpenAI(
model="gpt-5.2",
use_responses_api=True,
logprobs=True,
)
response = client.invoke("say hello world")`
response had the logprobs in the response_metadata which is the case with this PR.
Read the full contributing guidelines: https://docs.langchain.com/oss/python/contributing/overview
If you paste a large clearly AI generated description here your PR may be IGNORED or CLOSED!
Thank you for contributing to LangChain! Follow these steps to have your pull request considered as ready for review.
Fixes #xxline at the top is required for external contributions — update the issue number and keep the keyword. This links your PR to the approved issue and auto-closes it on merge.make format,make lintandmake testfrom the root of the package(s) you've modified.Additional guidelines:
uv.lockfiles or add dependencies topyproject.tomlfiles (even optional ones) unless you have explicit permission to do so by a maintainer.Social handles (optional)
Twitter: @
LinkedIn: https://linkedin.com/in/