Azure Openai gpt-4o-2024-08-06 model is giving response as Could not parse response content as the length limit was reached - St

admin管理员组

文章数量:1431145

I am using gpt-4o-2024-08-06 model with structured output. In documentation it is mentioned that this model support Context window 128k and Max output tokens supported are 16384. I have given the code which i have followed in my project. The error i am getting is as follow.

Could not parse response content as the length limit was reached - CompletionUsage(completion_tokens=5000, prompt_tokens=21633, total_tokens=26633, completion_tokens_details=None, prompt_tokens_details=None)

How to resolve this error.
The code which I have followed is given below.

from openai import OpenAI

client = OpenAI()

class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

completion = client.beta.chatpletions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    response_format=CalendarEvent,
    max_tokens=5000
)

event = completion.choices[0].message.parsed

I am using gpt-4o-2024-08-06 model with structured output. In documentation it is mentioned that this model support Context window 128k and Max output tokens supported are 16384. I have given the code which i have followed in my project. The error i am getting is as follow.

Could not parse response content as the length limit was reached - CompletionUsage(completion_tokens=5000, prompt_tokens=21633, total_tokens=26633, completion_tokens_details=None, prompt_tokens_details=None)

How to resolve this error.
The code which I have followed is given below.

from openai import OpenAI

client = OpenAI()

class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

completion = client.beta.chatpletions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    response_format=CalendarEvent,
    max_tokens=5000
)

event = completion.choices[0].message.parsed

• azure
• azure-openai

Share Improve this question Follow edited Nov 19, 2024 at 11:56 Sanjeevani asked Nov 19, 2024 at 11:43 SanjeevaniSanjeevani 2711 silver badge77 bronze badges 5

• Why do you expect that there's a different answer than "you're exceeding the limit"? Do you think it should work with the data you're sending? Also, shouldn't you be using max_completion_tokens? max_tokens seems to have been deprecated. – Luaan Commented Nov 19, 2024 at 12:00
• The parameter you are looking for in the Playground is called Maximum length and in the API max_tokens. – Sampath Commented Nov 19, 2024 at 12:50
• @Luaan the model which i am using has 128k context window. the input i am passing is in that limit and max tokens also under the limit. but still getting the error. i want to know why i am getting this error. – Sanjeevani Commented Nov 19, 2024 at 12:56
• @Sanjeevani refer to this link for OpenAI models to large documents – Sampath Commented Nov 19, 2024 at 13:17
• max_tokens is deprecated see platform.openai/docs/api-reference/chat/… – herve Commented Nov 27, 2024 at 9:52

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 1

max_tokens is the token generated from completion, that is including output tokens and reasoning tokens.

You can check this documentation.

So, you need to give the max_tokens greater than the completion token else you will get error.

See below example.

Completion token is 17 which is difference of total and prompt tokens.

Now, if i give max_tokens as 17 i get below error.

LengthFinishReasonError: Could not parse response content as the length limit was reached - CompletionUsage(completion_tokens=17, prompt_tokens=92, total_tokens=109, completion_tokens_details=None, prompt_tokens_details=None)

When i give greater than 17 it is working fine.

So, you give the max_tokens greater than the difference of total tokens and prompt tokens, in your case it is greater than 5000.

本文标签：