Managed Inference and Agents API with Nova Lite

Table of Contents [expand]

When to Use This Model
Usage
Example curl Requests

Last updated January 30, 2026

Nova Lite is a fast and cost-effective large language model (LLM) from Amazon. It offers a multimodal solution that can process image, video, and text inputs.

Model ID: nova-lite
Region: us, eu

When to Use This Model

Nova Lite is optimized for high-throughput tasks and supports a variety of common use cases, including rapid text generation, summarization, and copywriting.

Usage

Nova Lite follows our /v1/chat/completions API schema.

To provision access to the model, attach nova-lite to your app example-app:

heroku ai:models:create -a example-app nova-lite

You can invoke nova-lite in a variety of ways:

Heroku CLI ai plugin (heroku ai:models:call)
curl
Python
Ruby
Javascript

Multimodal Support

Supported inputs: text, image, video
Supported outputs: text

Rate Limits

Maximum requests per minute: 150
Maximum tokens per minute: 800,000

Prompt Caching

Prompt caching is supported for system prompts. It isn’t supported for tools. The minimum tokens required for prompt caching is 1,000.

Example curl Requests

Text to Text

export INFERENCE_MODEL_ID=$(heroku config:get -a example-app INFERENCE_MODEL_ID)
export INFERENCE_KEY=$(heroku config:get -a example-app INFERENCE_KEY)
export INFERENCE_URL=$(heroku config:get -a example-app INFERENCE_URL)

curl $INFERENCE_URL/v1/chat/completions \
-H "Authorization: Bearer $INFERENCE_KEY" \
-d @- <<EOF
{
  "model": "$INFERENCE_MODEL_ID",
  "messages": [
    { "role": "user", "content": "Hello!" },
    { "role": "assistant", "content": "Hi there! How can I assist you today?" },
    { "role": "user", "content": "What's the weather like in Portland, Oregon right now?" }
  ],
  "temperature": 0.5,
  "max_tokens": 100,
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Fetches the current weather for a given city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string",
              "description": "The name of the city to get weather for."
            }
          },
          "required": ["city"]
        }
      }
    }
  ],
  "tool_choice": "auto",
  "top_p": 0.9
}
EOF

Image to Text

curl -X POST $INFERENCE_URL/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Forwarded-Proto: https" \
  -d @- <<EOF
{
  "model": "$INFERENCE_MODEL_ID",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "What do you see in this image?"},
      {"type": "image_url", "image_url": {"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/0/09/A_chinstrap_penguin_%28Pygoscelis_antarcticus%29_on_Deception_Island_in_Antarctica.jpg/960px-A_chinstrap_penguin_%28Pygoscelis_antarcticus%29_on_Deception_Island_in_Antarctica.jpg"}}
    ]
  }]
}
EOF

Video to Text

curl -X POST $INFERENCE_URL/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Forwarded-Proto: https" \
  -d @- <<EOF
{
  "model": "$INFERENCE_MODEL_ID",
  "stream": true,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What do you see happening in this video?"
        },
        {
          "type": "video_url",
          "video_url": {
            "url": "https://freestockfootagearchive.com/wp-content/uploads/2025/10/Green-Screen-Glitch-Grunge-Overlay-Effect-Layer.mp4"
          }
        }
      ]
    }
  ]
}
EOF

Categories