Prompt caching is now automatically enabled for system prompts and tool definitions
Change effective on 02 December 2025
Heroku now automatically caches system prompts and tool definitions. Prompt caching optimizes the Heroku Managed Inference and Agents add-on to deliver faster responses for common workflows, document processing, and code-generation tools. Prompt caching support varies depending on the model, see individual model cards to learn more.
You can disable prompt caching for any request.
For more information on this new feature, see Prompt Caching.