Skip to main content

Set Max tokens

Set a cut-off limit for the responses of the GPT functions in the current spreadsheet. This won't shape the response, but if the response goes beyond this limit, it will be truncated.

TermDefinition
TokenTokens can be thought of as pieces of words. During processing, the language model breaks down both the input (prompt) and the result (completion) texts into smaller units called tokens. Tokens generally correspond to ~4 characters of common English text. So 100 tokens are approximately worth 75 words. See how text is split into tokens.
Token limitToken limit is the maximum total number of tokens that can be used in both the input (prompt) and the result (completion) when interacting with a language model.
Max tokensCut-off limit of the result measured in tokens. If the result is larger than this limit, it will be truncated. Helps control cost and speed. The maximum value for Max tokens is set 200 tokens below the model Token limit. For example, gpt-3.5-turbo has a capacity of 4096 tokens, so you can only set Max tokens up to 3896. This provides room for your input and prevents hitting the max token limit due to an excessively high Max tokens setting.
Prerequisites
You have opened a Google spreadsheet and selected Extensions > GPT for Sheets and Docs > Open.

  1. In the GPT for Sheets sidebar, click Settings.

  2. Set Max tokens as follows:

    • Auto: The model will automatically use the remaining available tokens (with Safe mode only).
    • Fixed: The maximum number of tokens is set manually using the slider. The value must be lower than the model limit to make room for input tokens.
      • If you expect short responses, lower the limit to get faster responses
      • If you expect long responses, increase the limit to make sure they are not truncated
      • If your response is truncated, increase the limit

        Use the slider or the Auto option so set a cut-off limit

Existing formulas will not automatically update to the new cut-off limit if the cache is enabled. To re-execute them with a new cut-off limit, you can either disable the cache, select the formulas and regenerate their results, or change a parameter in the formulas and press Enter.

What's next

Select other settings to customize how the language model operates.