Run Non-web Java Dynos on Heroku

Last updated December 16, 2019

Some applications can benefit from splitting logic into multiple components:

  1. A web process that is consumed by the end-user
  2. One or more non-web processes to perform background and admin tasks.

A non-web process can be either:

  1. A long running process in a Worker dyno, that is waiting on events (either on a database or from a message queue)
  2. A command executed in a one-off dyno, which can be invoked manually from the command line or from a service like the Heroku Scheduler

Prerequisites

  • Basic Java knowledge, including an installed version of the JVM and Maven.
  • Basic Git knowledge, including an installed version of Git.

Sample application

A simple app that demonstrates the one-off and worker process types can be created as two simple Java classes and a build file:

sampleapp/
  pom.xml
  src/
    main/
      java/
        OneOffProcess.java
        WorkerProcess.java

src/main/java/OneOffProcess.java

public class OneOffProcess
{
    public static void main(String[] args)
    {
        System.out.println("OneOffProcess executed.");
    }
}

src/main/java/WorkerProcess.java

public class WorkerProcess
{
    public static void main(String[] args)
    {
        while(true) {
            try {
                Thread.sleep(1000);
            } catch(InterruptedException e) {}

            System.out.println("Worker process woke up");
        }
    }
}

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.example</groupId>
  <artifactId>herokujavasample</artifactId>
  <version>1.0-SNAPSHOT</version>

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
          <artifactId>appassembler-maven-plugin</artifactId>
          <version>1.1.1</version>
          <configuration>
            <assembleDirectory>target</assembleDirectory>
            <programs>
                <program>
                    <mainClass>WorkerProcess</mainClass>
                    <name>worker</name>
                </program>
                <program>
                    <mainClass>OneOffProcess</mainClass>
                    <name>oneoff</name>
                </program>
            </programs>
          </configuration>
          <executions>
              <execution>
                  <phase>package</phase><goals><goal>assemble</goal></goals>
              </execution>
          </executions>
      </plugin>
    </plugins>
  </build>

</project>

The app assembler plugin generates a convenient launch script for starting your application. A single pom.xml can define multiple web, worker or admin processes.

You can clone this project from GitHub.

Run locally

To build your application simply run:

$ mvn package

Run the worker with:

$ sh target/bin/worker
Worker process woke up
Worker process woke up
Worker process woke up
...

(use target\bin\worker.bat on Windows). Run the one-off process with:

$ sh target/bin/oneoff
OneOffProcess executed.

That’s it. You are now ready to deploy to Heroku.

Deploy to Heroku

Create a Procfile

You declare how you want your application executed in a Procfile in the project root. Create this file as below:

worker: sh target/bin/worker

There is no need to add commands that you want executed in a one-off dyno to a Procfile - the one-off dyno mechanism lets you specify the command when you launch the one-off dyno.

Deploy to Heroku

Commit your changes to Git:

$ git init
$ git add .
$ git commit -m "Ready to deploy"

Create the app:

$ heroku create
Creating empty-fire-9222... done, stack is heroku-18
http://empty-fire-9222.herokuapp.com/ | git@heroku.com:empty-fire-9222.git
Git remote heroku added

Deploy your code:

$ git push heroku master
Counting objects: 66, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (31/31), done.
Writing objects: 100% (66/66), 15.74 KiB, done.
Total 66 (delta 10), reused 30 (delta 9)

-----> Heroku receiving push
-----> Java app detected
-----> Installing Maven 3.0.3..... done
-----> executing /app/tmp/repo.git/.cache/.Maven/bin/mvn -B -Duser.home=/tmp/build_14lc6nws0m7oc -Dmaven.repo.local=/app/tmp/repo.git/.cache/.m2/repository -DskipTests=true clean install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building herokujavaworker 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.513s
[INFO] Finished at: Mon Nov 28 15:44:32 UTC 2011
[INFO] Final Memory: 12M/490M
[INFO] ------------------------------------------------------------------------
-----> Discovering process types
       Procfile declares types -> worker
-----> Compiled slug size is 12K
-----> Launching... done, v5
   http://empty-fire-9222.herokuapp.com deployed to Heroku

Scaling worker processes

You can now start and scale your worker dynos using a command like this:

$ heroku ps:scale worker=1
Scaling worker processes... done, now running 1

By scaling your workers to more than one dyno you can have more listeners and thereby consume and process more messages simultaneously. To look at the logs for your worker dyno, you can use the command:

$ heroku logs --tail
2011-12-14T00:52:26+00:00 heroku[slugc]: Slug compilation started
2011-12-14T00:52:54+00:00 heroku[web.1]: State changed from created to down
2011-12-14T00:52:55+00:00 heroku[slugc]: Slug compilation finished
2011-12-14T00:53:17+00:00 heroku[worker.1]: State changed from created to starting
2011-12-14T00:53:17+00:00 heroku[api]: Scale to worker=1 by jesper@heroku.com
2011-12-14T00:53:17+00:00 heroku[worker.1]: Starting process with command `sh target/bin/worker`
2011-12-14T00:53:18+00:00 heroku[worker.1]: State changed from starting to up
2011-12-14T00:53:19+00:00 app[worker.1]: Worker process woke up
2011-12-14T00:53:20+00:00 app[worker.1]: Worker process woke up
2011-12-14T00:53:21+00:00 app[worker.1]: Worker process woke up

One-off dynos

If your process is a command you wish to run manually on an as needed basis, you can do so with a one-off dyno. Use the heroku run command to start a one-off dyno and execute the command:

$ heroku run "sh target/bin/oneoff"
Running sh target/bin/oneoff attached to terminal... up, run.1
OneOffProcess executed.

Scheduling jobs

Applications that need to execute jobs based on some interval of time or frequency can use the Scheduler add-on or can define a custom clock process with a library like Quartz.

Scheduled jobs versus workers

Scheduling one-off processes is a good way to perform admin tasks such as clearing a cache or triggering the creation of a report that is sent over email. These types of events happen infrequently and don’t need to scale up or down.

Worker processes are good for processing work that is being queued up by a front-end web process or by other worker processes. The workload may vary depending on the traffic to your app and you can scale up the number of workers so you can perform more work in parallel. You can also use a worker process if you simply need to process events more frequently than every 10 minutes.

Feedback

Log in to submit feedback.