Skip Navigation
Show nav
Heroku Dev Center
  • Get Started
  • Documentation
  • Changelog
  • Search
  • Get Started
    • Node.js
    • Ruby on Rails
    • Ruby
    • Python
    • Java
    • PHP
    • Go
    • Scala
    • Clojure
  • Documentation
  • Changelog
  • More
    Additional Resources
    • Home
    • Elements
    • Products
    • Pricing
    • Careers
    • Help
    • Status
    • Events
    • Podcasts
    • Compliance Center
    Heroku Blog

    Heroku Blog

    Find out what's new with Heroku on our blog.

    Visit Blog
  • Log inorSign up
View categories

Categories

  • Heroku Architecture
    • Dynos (app containers)
    • Stacks (operating system images)
    • Networking & DNS
    • Platform Policies
    • Platform Principles
  • Command Line
  • Deployment
    • Deploying with Git
    • Deploying with Docker
    • Deployment Integrations
  • Continuous Delivery
    • Continuous Integration
  • Language Support
    • Node.js
    • Ruby
      • Rails Support
      • Working with Bundler
    • Python
      • Background Jobs in Python
      • Working with Django
    • Java
      • Working with Maven
      • Java Database Operations
      • Working with the Play Framework
      • Working with Spring Boot
      • Java Advanced Topics
    • PHP
    • Go
      • Go Dependency Management
    • Scala
    • Clojure
  • Databases & Data Management
    • Heroku Postgres
      • Postgres Basics
      • Postgres Getting Started
      • Postgres Performance
      • Postgres Data Transfer & Preservation
      • Postgres Availability
      • Postgres Special Topics
    • Heroku Data For Redis
    • Apache Kafka on Heroku
    • Other Data Stores
  • Monitoring & Metrics
    • Logging
  • App Performance
  • Add-ons
    • All Add-ons
  • Collaboration
  • Security
    • App Security
    • Identities & Authentication
    • Compliance
  • Heroku Enterprise
    • Private Spaces
      • Infrastructure Networking
    • Enterprise Accounts
    • Enterprise Teams
    • Heroku Connect (Salesforce sync)
      • Heroku Connect Administration
      • Heroku Connect Reference
      • Heroku Connect Troubleshooting
    • Single Sign-on (SSO)
  • Patterns & Best Practices
  • Extending Heroku
    • Platform API
    • App Webhooks
    • Heroku Labs
    • Building Add-ons
      • Add-on Development Tasks
      • Add-on APIs
      • Add-on Guidelines & Requirements
    • Building CLI Plugins
    • Developing Buildpacks
    • Dev Center
  • Accounts & Billing
  • Troubleshooting & Support
  • Integrating with Salesforce
  • Language Support
  • Go
  • Background Jobs with Que-Go

Background Jobs with Que-Go

English — 日本語に切り替える

Last updated January 26, 2022

Table of Contents

  • Getting started
  • Application overview
  • Testing the application

Web servers should focus on serving users as quickly as possible. Any non-trivial work that could slow down your user’s experience should be done asynchronously outside of the web process.

Worker queues are commonly used to accomplish this goal. For a more in-depth description of the worker queue architectural pattern, read the Worker Dynos, Background Jobs and Queueing article. Here we’ll demonstrate this pattern using a sample Go application using Que-Go and Heroku Postgres.

This article assumes that you have the Heroku CLI and the Go toolchain installed.

 

Que-Go is compatible with the Que Ruby library, enabling sharing of queues and jobs between the two languages.

Getting started

Follow the steps below to create a copy of the example application in your Heroku account:

Via dashboard

  1. Click here to Deploy

Via CLI

Install the Heroku CLI and then follow these instructions inside of a terminal.

$ go get -u github.com/heroku-examples/go-queue-example/...
$ cd $GOPATH/src/github.com/heroku-examples/go-queue-example
$ heroku create
$ heroku addons:add heroku-postgresql
$ git push heroku master
$ heroku ps:scale worker=1

Application overview

The application has two processes:

  • queue-example-web : A Negroni-based application that accepts URLs in the form of a POST’d JSON document containing a url for indexing by a worker.

  • queue-example-worker: The worker processes jobs containing a url that should be indexed.

Because these are separate processes, they can be scaled independently based on specific application needs. Read the Process Model article for a more in-depth understanding of Heroku’s process model.

There is some shared code in shared.go that focuses on setting up the database, the database connections and the que client.

Web process

cmd/queue-example-web/main.go queues up the requested URLs into a queue for processing by the worker process.

Worker process

cmd/queue-example-worker/main.go runs 2 worker Go routines that process messages from the queue as they come in. The workers don’t do anything interesting with the data and just output the received data into the log stream. This is where long running processing logic would be put.

Shared logic

shared.go contains some shared initialization and setup logic that is used by both.

Testing the application

You can watch the interaction between the two processes by observing the log stream:

$ heroku logs --tail -a <app name>

In a different terminal we’ll use cURL to submit a URL for “indexing”.

$ curl -XPOST "https://<app name>.herokuapp.com/index" -d '{"url": "http://google.com"}'

Back in the first terminal you should see output like the following:

2015-06-23T18:29:35.663096+00:00 heroku[router]: at=info method=POST path="/index" host=<app name>.herokuapp.com request_id=84f9d369-7d6e-4313-8f16-9db9bb7ed251 fwd="76.115.27.201" dyno=web.1 connect=19ms service=31ms status=202 bytes=141
2015-06-23T18:29:35.623878+00:00 app[web.1]: [negroni] Started POST /index
2015-06-23T18:29:35.644483+00:00 app[web.1]: [negroni] Completed 202 Accepted in 20.586125ms
2015-06-23T18:29:37.750543+00:00 app[worker.1]: time="2015-06-23T18:29:37Z" level=info msg="Processing IndexRequest! (not really)" IndexRequest={http://google.com}
2015-06-23T18:29:37.753021+00:00 app[worker.1]: 2015/06/23 18:29:37 event=job_worked job_id=1 job_type=IndexRequests

Above, we can see the web process servicing the request and returned a 202 and a few milliseconds later the worker picks up the queued request and “processes” it.

Keep reading

  • Go

Feedback

Log in to submit feedback.

Using WebSockets on Heroku with Go Getting Started on Heroku with Go

Information & Support

  • Getting Started
  • Documentation
  • Changelog
  • Compliance Center
  • Training & Education
  • Blog
  • Podcasts
  • Support Channels
  • Status

Language Reference

  • Node.js
  • Ruby
  • Java
  • PHP
  • Python
  • Go
  • Scala
  • Clojure

Other Resources

  • Careers
  • Elements
  • Products
  • Pricing

Subscribe to our monthly newsletter

Your email address:

  • RSS
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku Blog
    • Heroku News Blog
    • Heroku Engineering Blog
  • Heroku Podcasts
  • Twitter
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku
    • Heroku Status
  • Facebook
  • Instagram
  • Github
  • LinkedIn
  • YouTube
Heroku is acompany

 © Salesforce.com

  • heroku.com
  • Terms of Service
  • Privacy
  • Cookies
  • Cookie Preferences