Skip to content

fix(openai): add support for logprobs when using the response api#36371

Open
HOZHENWAI wants to merge 2 commits into
langchain-ai:masterfrom
HOZHENWAI:HOZHENWAI/fix-missing-logprobs-openai-response
Open

fix(openai): add support for logprobs when using the response api#36371
HOZHENWAI wants to merge 2 commits into
langchain-ai:masterfrom
HOZHENWAI:HOZHENWAI/fix-missing-logprobs-openai-response

Conversation

@HOZHENWAI
Copy link
Copy Markdown

@HOZHENWAI HOZHENWAI commented Mar 30, 2026

Fixes #36211

Fix the issue where the combination of use_responses_api=True and logprobs=True prevented any invocation.
Precommit validated and all unit-tests passing including the new ones written for this fix.
Test scope are discutable: i've decided to test that the request_payload is compatible with the response api instead of testing the output of _construct_responses_api_payload in the case of logprobs=True which may have been more proper. If necessary i can do the changes. Also, i have no tested the case of refusal when logprobs = True.
I also think a small update to the integration test could be useful but did not write it.
Final test made was to make sure that for the following snipped
`from langchain_openai import ChatOpenAI

client = ChatOpenAI(
model="gpt-5.2",
use_responses_api=True,
logprobs=True,
)

response = client.invoke("say hello world")`
response had the logprobs in the response_metadata which is the case with this PR.

Read the full contributing guidelines: https://docs.langchain.com/oss/python/contributing/overview

All contributions must be in English. See the language policy.

If you paste a large clearly AI generated description here your PR may be IGNORED or CLOSED!

Thank you for contributing to LangChain! Follow these steps to have your pull request considered as ready for review.

  1. PR title: Should follow the format: TYPE(SCOPE): DESCRIPTION
  1. PR description:
  • Write 1-2 sentences summarizing the change.
  • The Fixes #xx line at the top is required for external contributions — update the issue number and keep the keyword. This links your PR to the approved issue and auto-closes it on merge.
  • If there are any breaking changes, please clearly describe them.
  • If this PR depends on another PR being merged first, please include "Depends on #PR_NUMBER" in the description.
  1. Run make format, make lint and make test from the root of the package(s) you've modified.
  • We will not consider a PR unless these three are passing in CI.
  1. How did you verify your code works?

Additional guidelines:

  • All external PRs must link to an issue or discussion where a solution has been approved by a maintainer, and you must be assigned to that issue. PRs without prior approval will be closed.
  • PRs should not touch more than one package unless absolutely necessary.
  • Do not update the uv.lock files or add dependencies to pyproject.toml files (even optional ones) unless you have explicit permission to do so by a maintainer.

Social handles (optional)

Twitter: @
LinkedIn: https://linkedin.com/in/

@github-actions github-actions Bot added fix For PRs that implement a fix integration PR made that is related to a provider partner package integration openai `langchain-openai` package issues & PRs size: S 50-199 LOC labels Mar 30, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented Mar 30, 2026

Merging this PR will improve performance by 13%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 3 improved benchmarks
⏩ 37 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime test_init_time 3.6 ms 3.2 ms +11.88%
WallTime test_init_time 3.6 ms 3.2 ms +13%
WallTime test_init_time 136 ms 122.1 ms +11.36%

Comparing HOZHENWAI:HOZHENWAI/fix-missing-logprobs-openai-response (9921b2c) with master (342d8bd)

Open in CodSpeed

Footnotes

  1. 37 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

additional_kwargs["parsed"] = content.parsed
if hasattr(content, "logprobs") and content.logprobs:
if "logprobs" in response_metadata:
# only keep the latest message logprobs
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this go in response_metadata, or on the associated content block?

Copy link
Copy Markdown
Author

@HOZHENWAI HOZHENWAI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, is that a recommandation or a question? The default pattern without the response api when using logprobs is to have it in the response_metadata. I just copied the pattern to make sure the outputs were compatible between use_responses_api=True and use_responses_api=False . Should this be changed?
Tbh, I don't think I tested the case with multi-turn conversation when logprobs=True and use_response_api=False, so I don't know if the pattern is different in that case.

@ccurme ccurme (ccurme) self-assigned this Apr 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external fix For PRs that implement a fix integration PR made that is related to a provider partner package integration new-contributor openai `langchain-openai` package issues & PRs size: S 50-199 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

logprobs not returned when using the response api

3 participants