Connecting to an Apache Kafka on Heroku Cluster

Last updated June 30, 2026

Apache Kafka on Heroku is accessible from many different programming languages and frameworks.

Apache Kafka on Heroku & SSL

Connecting to Apache Kafka on Heroku requires setting the client’s SSL configuration to not verify the server hostname. See Language Support for more information on how to set the SSL configuration depending on the language.

All connections to Kafka support SSL encryption and authentication. If you provision the cluster in a Private Space, you can connect via plaintext. Shield Spaces don’t support plaintext connections.

Connecting over SSL means all traffic is encrypted and authenticated via an SSL client certificate.

To connect over SSL, use the following environment variables:

As with all Heroku add-ons, important environment and configuration variables can change. It’s important to design your app to handle updates to these values, especially if systems outside of Heroku access these resources.

  • KAFKA_URL: A comma-separated list of SSL URLs to the Kafka brokers making up the cluster.
  • KAFKA_TRUSTED_CERT: The brokers’ SSL certificate (in PEM format), to check that you’re connecting to the right servers.
  • KAFKA_CLIENT_CERT: The required client certificate (in PEM format) to authenticate clients against the broker.
  • KAFKA_CLIENT_CERT_KEY: The required client certificate key (in PEM format) to authenticate clients against the broker.

Kafka clusters require authenticating using the provided client certificate. Kafka denies any requests not using the client certificate.

Connecting to an Apache Kafka on Heroku cluster requires disabling server hostname validation. Adjust your client SSL configuration to set ssl.endpoint.identification.algorithm to an empty string. Some Kafka clients provide their own specific option for this setting.

Only basic plans provide KAFKA_PREFIX. If you’re using a basic plan, see Multi-Tenant Apache Kafka on Heroku.

Rotating Credentials

It’s a good security practice to rotate the credentials for important services regularly. To rotate your cluster’s credentials, use the heroku kafka:credentials --reset command.

$ heroku kafka:credentials HEROKU_KAFKA_GRAY_URL --reset -a example-app
Resetting credentials for kafka-animated-39618

This command executes these steps:

  1. Generate a new credential for your cluster.
  2. Existing topics and consumer groups receive new ACLs to allow the new client certificate credential.
  3. When the new credential is ready, we update the related config vars on your Heroku app.
  4. For 5 minutes, both the old client certificate and new client certificate remain valid. This period gives time for the new client certificate to cycle into your app.
  5. After 5 minutes, the old client certificate credential expires and is no longer valid.

Language Support

There are many client libraries for Kafka across many languages. The following listed are libraries that we either helped to improve, or have up-to-date support for Kafka features, including critical capabilities like support for SSL.

Basic multi-tenant Kafka plans require a prefix on topics and consumer groups. See the differences between multi-tenant and dedicated. When integrating Kafka consumers, make sure to prefix topics and consumer groups with the value of the KAFKA_PREFIX environment variable. Otherwise, Kafka doesn’t receive the messages, and errors like Broker: Topic authorization failed or Broker: Group authorization failed can appear in Kafka debug events.

Using Kafka in Ruby Applications

We recommend using rdkafka-ruby when connecting to Kafka from Ruby.

This example shows how to write and consume messages in Ruby.

When using rdkafka-ruby, you must specify "enable.ssl.certificate.verification" => false in your client configuration to connect to Apache Kafka on Heroku.

Using Kafka in Java Applications

We recommend using kafka-clients when connecting to Kafka from Java. See the Kafka project documentation for more information.

This example shows how to write and consume messages in Java. This section of the demo app provides a good example of working with the TrustStore and KeyStore for Java Virtual Machine (JVM) apps.

On recent versions of the Java Kafka clients, you must set ssl.endpoint.identification.algorithm to an empty string to disable server hostname validation. For example, ssl.endpoint.identification.algorithm=.

Using Kafka in Go Applications

We recommend using sarama when connecting to Kafka from Go.

This example shows how to write and consume messages in Go.

Due to the strictness of the tls package in Go, when using the sarama client, you must set InsecureSkipVerify to true to disable server hostname validation.

Using Kafka in Python Applications

We recommend using confluent-kafka-python when connecting to Kafka in Python. This client library wraps the C/C++ Kafka library.

We also recommend the kafka-python library, especially for scenarios where wrapping the C/C++ libraries is less than ideal. Heroku has created the kafka-helper library to make kafka-python easier to use. To disable server hostname validation with kafka-python, set ssl_check_hostname to False.

The aiokafka client is based on the kafka-python library. To disable server hostname when using this client, set the SSL context check_hostname option to False.

Using Kafka in Node.js Applications

We recommend using kafkajs when connecting to Kafka from Node.js apps.

To disable server hostname verification with kafkajs, set ssl.rejectUnauthorized to false in your client’s SSL configuration.

Using Kafka in PHP Applications

The rdkafka extension for PHP is available on Heroku. It provides bindings for the librdkafka library and supports SSL.

Set ssl.endpoint.identification.algorithm to an empty string to disable server hostname validation, following the librdkafka configuration properties.

Using Kafka in Other Languages or Frameworks

The Confluence Client wiki lists clients and code examples for other languages and frameworks.