Managed Inference and Agents API with Amazon Nova Pro

Table of Contents [expand]

Last updated December 02, 2025

Amazon Nova Pro is a high-performance large language model (LLM) designed for complex tasks. It offers a multimodal solution that can process image, video, and text inputs.

  • Model ID: nova-pro
  • Region: us, eu

When to Use This Model

Amazon Nova Pro is ideal for a variety of common use cases, including advanced question-answering, content creation, and data extraction. It’s optimized to provide an ideal combination of accuracy, speed, and cost.

Usage

Amazon Nova Pro follows our /v1/chat/completions API schema.

To provision access to the model, attach nova-pro to your app example-app:

heroku ai:models:create -a example-app nova-pro

You can invoke nova-pro in a variety of ways:

Rate Limits

  • Maximum requests per minute: 150
  • Maximum tokens per minute: 800,000

Prompt Caching

Prompt caching is supported for system prompts. It isn’t supported for tools. The minimum tokens required for prompt caching is 1,000.

Example curl Request

Get started quickly with an example request:

export INFERENCE_MODEL_ID=$(heroku config:get -a example-app INFERENCE_MODEL_ID)
export INFERENCE_KEY=$(heroku config:get -a example-app INFERENCE_KEY)
export INFERENCE_URL=$(heroku config:get -a example-app INFERENCE_URL)

curl $INFERENCE_URL/v1/chat/completions \
-H "Authorization: Bearer $INFERENCE_KEY" \
-d @- <<EOF
{
  "model": "$INFERENCE_MODEL_ID",
  "messages": [
    { "role": "user", "content": "Hello!" },
    { "role": "assistant", "content": "Hi there! How can I assist you today?" },
    { "role": "user", "content": "What's the weather like in Portland, Oregon right now?" }
  ],
  "temperature": 0.5,
  "max_tokens": 100,
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Fetches the current weather for a given city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string",
              "description": "The name of the city to get weather for."
            }
          },
          "required": ["city"]
        }
      }
    }
  ],
  "tool_choice": "auto",
  "top_p": 0.9
}
EOF