Monitoring JVM Metrics with the Heroku Java Agent
Last updated 24 October 2016
Table of Contents
The Heroku Java memory logging agent is an extremely lightweight Java agent that can be used to send memory usage information to your logs. This article describes how to configure the Heroku Java agent and connect its log output to the Librato add-on to enable monitoring and alerting.
Adding the agent to a Maven app
Use the Maven dependency plugin to download the Heroku agent and copy it into your
target/dependency directory by adding this configuration to the plugins section of your
<build> ... <plugins> ... <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <version>2.3</version> <executions> <execution> <phase>package</phase> <goals><goal>copy</goal></goals> <configuration> <artifactItems> <artifactItem> <groupId>com.heroku.agent</groupId> <artifactId>heroku-javaagent</artifactId> <version>1.5</version> <destFileName>heroku-javaagent.jar</destFileName> </artifactItem> </artifactItems> </configuration> </execution> </executions> </plugin> </plugins> </build>
Then open your
Procfile and attach the agent to your app by adding the
-javaagent option to the
java command. A typical entry might look like this:
web: java -javaagent:target/dependency/heroku-javaagent.jar=stdout=true,lxmem=true -cp target/app.jar com.example.Main
Commit your changes to Git and redeploy. Then run
heroku logs -t and you’ll see something like this periodically in your logs:
source=web.1 measure.mem.jvm.heap.used=33M measure.mem.jvm.heap.committed=376M measure.mem.jvm.heap.max=376M source=web.1 measure.mem.jvm.nonheap.used=19M measure.mem.jvm.nonheap.committed=23M measure.mem.jvm.nonheap.max=219M source=web.1 measure.threads.jvm.total=21 measure.threads.jvm.daemon=11 measure.threads.jvm.nondaemon=1 measure.threads.jvm.internal=9
These logs can be funneled into an add-on provider to enable alerting and monitoring.
Monitoring with Librato
Librato’s log integration allows you to publish your own custom metrics directly to Librato through your Heroku log stream. This includes the logs from the Heroku Java agent.
Create a new Librato add-on by running this command:
$ heroku addons:create librato
Then open the add-on interface in the Heroku Dashboard or by running:
$ heroku addons:open librato
The default “Heroku Overview” space includes charts for response time, error codes, and more. But it does not include JVM metrics by default.
To view JVM metrics, create a new space. Then add a new chart and choose “Stacked”. Librato will prompt you to choose the metrics to include in the chart. Search for “mem.jvm.heap.used” and “mem.jvm.nonheap.used”. Save your changes, and you’ll see something like this:
The stacked chart displays the combined measurements of heap and non-heap memory. For more information on what these categories mean, see our Dev Center article on Troubleshooting Memory Issues in Java Applications.
Alerting with Librato
To enable alerting, you must upgrade your Librato add-on to a paid plan. For example:
$ heroku addons:upgrade librato:nickel
Then open the Librato dashboard again, and select the option to create a new alert. The interface will prompt you to choose a metric to base the alerting on. Select “mem.jvm.nonheap.used”, which is a common value to be concerned about. Then select a maximum threshold such as 100, which refers to megabytes because the Java agent reports in megabytes. The alert will look like this:
You can configure notification preferences through the Librato interface.
For more information on diagnosing Java memory problems, see the Dev Center article on Troubleshooting Memory Issues in Java Applications.
The Heroku Java agent is an open source project. You can find the source code for the agent on Github.