Heroku CI: Browser and User Acceptance Testing (UAT)

Last updated 26 May 2017

On browser testing / UAT

Heroku CI provides support for browser testing, or “user acceptance testing” (UAT) by providing a options for installing browsers in your test run dyno. Users can choose between Google Chrome stable, beta, and unstable release channels supported by either the new --headless flag or Xvfb.

It’s important to note that while we provide options for installing browsers, it is the developers responsibility to drive the browser. Each language, web framework, and/or test framework tends to have it’s own prescription on how to drive and run tests in a browser (most commonly, this is done with Selenium). Most of these solutions can be supported with Heroku CI. We will be adding UAT tech stacks and set-ups below as they are tested and proven.

Browsers available

While we offer a few browsers to accommodate various needs, we strongly urge you to consider Google Chrome with --headless support first. It’s stable, has strong parity with user browsers, and is designed to enable this kind of testing.

Google Chrome (via --headless)

Google Chrome recently added support for a --headless command line option, which means you no longer need a window server to run browser tests. This configuration is optimal for UAT as it uses your browser, but doesn’t waste cycles rendering content to the screen. As such, this is our prescribed browser solution.

However, please note that if you driving the browser with Selenium, filling in fields still requires a window to map keypresses to the appropriate keycode. If you are filling in fields in your tests via Selenium, you’ll need the Xvfb powered Chrome instead (see below).

To use headless Google Chrome for Heroku CI runs, add the buildpack to the environments.test.buildpacks section of your app.json. For example, if you were using Chrome for NodeJs tests:

{
  "environments": {
    "test": {
      "buildpacks": [
        { "url": "heroku/nodejs" },
        { "url": "https://github.com/heroku/heroku-buildpack-google-chrome" }
      ]
    }
  }
}

Now google-chrome (from the stable release channel by default) will be available in your test run, and it will automatically run in headless mode (the --headless flag is always used). See the buildpack documentation for additional details.

Google Chrome (via Xvfb)

Before Google Chrome added --headless support, the only way to run Google Chrome without a GUI was via Xvfb (X virtual framebuffer). Xvfb essentially creates a virtual display that Chrome can render to. While this is less optimum for performance, it provides better interoperability with some testing tools, most notably, Selenium.

To use Google Chrome with Xvfb for Heroku CI runs, add the buildpack to the environments.test.buildpacks section of your app.json. For example, if you were using Chrome for Ruby tests:

{
  "stack": "cedar-14",
  "environments": {
    "test": {
      "buildpacks": [
        { "url": "heroku/ruby" },
        { "url": "https://github.com/heroku/heroku-buildpack-xvfb-google-chrome" }
      ]
    }
  }
}

Now, you’ll have an Xvfb display running in your test run dyno, and google-chrome (from the stable release channel by default) will be available. Running google-chrome will automatically use the running Xvfb display. See the buildpack documentation for additional details.

PhantomJS

PhantomJS is a headless browser built on NodeJS. It’s smaller and easier to install than Chrome, so it’s been a common choice for UAT for quite some time. Please note that Heroku CI runs can be configured to use PhantomJS, but we do not presently offer direct support for it, for two reasons: 1) It’s installed via a 3rd party buildpack, and 2) PhantomJS is no longer maintained.

To use PhantomJS for Heroku CI runs, add the buildpack to the environments.test.buildpacks section of your app.json. For example, if you were using PhantomJS in Ruby tests:

{
  "environments": {
    "test": {
      "buildpacks": [
        { "url": "heroku/ruby" },
        { "url": "https://github.com/stomita/heroku-buildpack-phantomjs" }
      ]
    }
  }
}

If you are using NodeJS, you may want to consider using the phantomjs-prebuilt instead.

Testing with Selenium

Heroku CI supports testing with Selenium via the Chrome buildpacks mentioned above. Additionally, to run Selenium in Heroku CI, you’ll need:

1. Chromedriver - You’re language or framework may already install this, if not, please see our Chromedriver buildpack
2. Language bindings to Selenium - these will vary according to the language you selected.
3. Special configuration to point chromedriver to the correct Chrome binary location (learn more)