Shared Workforce

This add-on is operated by Pigment Ltd

Human power for your app

Shared Workforce

Last Updated: 03 April 2014

Table of Contents

Shared Workforce is an add-on for providing human powered content analysis.

Shared Workforce also gives you on-demand access to human-powered text editing, photo cropping and photo rotating.

By adding a few lines of code to your app, you can send text or images to be checked by our human workers. Their responses will be returned to your app where you can define actions to perform. The Ruby Gem fully integrates with Rails providing a simple syntax for defining tasks. If you’re not using Rails, our REST api is also very simple to get started with.

You can view a live demo sandbox app at http://catsify.herokuapp.com or watch a video tutorial.

Provisioning the add-on

Shared Workforce can be attached to a Heroku application via the CLI:

A list of all plans available can be found here.

$ heroku addons:add sharedworkforce
-----> Adding sharedworkforce to sharp-mountain-4005... done, v18 (free)

Using with Rails 3.x

Ruby on Rails applications will need to add the following entry into their Gemfile specifying the Shared Workforce client library.

gem 'shared_workforce'

Update application dependencies with bundler.

$ bundle install
Your bundle is complete!

Defining tasks

For a full list of task types and their parameters, please see the docs.

For a screencast showing a typical Rails integration, watch our video tutorial.

A class defines the content (for example, an image url) and instructions that the human worker will see when they work on your task, and the methods to be run once the task has been completed. It can include the following attributes:

Attribute Example Purpose
title "Tag our photo" A short but descriptive title to be shown to the workers to help them easily recognise and understand your task.
instruction "Look at the photo and choose all that apply" 1 or 2 sentences instructing the worker on how to perform your task.
answer_type "choice"

Sets the type of answer you would like for your content.

Possible options are:

  • "choice": choose one option from a list

  • "tags": choose any number of options from a list

  • "edit": edit the text in the 'text' attribute

  • "crop": crop the photo (see image_crop_ratio)

  • "rotate": rotate the photo

answer_options 'Offensive', 'Contains Nudity' Sets the options the workers can choose from in a 'choice' or 'tags' task.
responses_required 1 Sets the number of workers you would like responses from.
text "Time is an illusion. Lunchtime doubly so." Text you would like to be moderated (optional).
image_crop_ratio 1 For a crop task, the ratio you would like the image cropping to. (for example, 1 for square or omit for unconstrained).

Example of a task definition

To configure these attributes in your task, create a task class in an app/tasks directory (or anywhere in the load path).

app/tasks/tag_photo_task.rb:

class TagPhotoTask
  include SharedWorkforce::Task

  title 'Please tag our photo'

  instruction 'Please look at the photo and tick all that apply.'

  answer_type :tags
  answer_options ['Offensive', 'Contains Nudity', 'Blurry or low quality', 'Upside down or sideways']

  responses_required 1

  on_success :moderate_photo

  def setup(photo)
    self.image_url = photo.url
  end

  def moderate_photo(photo, responses)
    # responses => [{:answer=>['Offensive', 'Contains Nudity']}]
    if responses.map {|r| r[:answer]}.flatten.include?('Offensive')
      photo.delete
      photo.user.quarantine("Uploded offensive photo")
    end
  end
end

Note that the task definition includes a setup method which is called automatically whenever the task is initialized. In the example, the task's image_url (the image shown to the worker) is set from the photo model's url attribute. Any of the task's attributes can be set this way.

Class level attributes are a handy way of defining data that doesn’t change between each task. Attributes set on the instance will always override attributes set at the class level.

In most cases, you’ll want to explicitly set default task values at the class level (like title and instruction). Setting text and image_url values (i.e. the content in question) will usually be done in the setup method.

Once you have created a task definition, you can request real human responses for a model instance by calling its create method. This can be done in an after_create callback in one of your Active Record models. This will be covered in more detail in the next section, ”Publishing tasks”.

Publishing tasks

When you publish a task it will be queued for a human worker to complete. You can publish a task in an after_create callback on a typical Active Record model:

class Photo < ActiveRecord::Base

  after_create :request_tags

  def request_tags
    TagPhotoTask.create(self)
  end
end

Note that in the example, the photo model instance (self) is used an argument to the TagPhotoTask.create method. This argument will be available in the setup method and the callback method as shown in the example of a Task Definition above.

When the response(s) from the human workers are collected, the method specified in the on_success attribute in your task definition will be called. Typically this will take about 15 minutes. You can check the Shared Workforce web site for an up to date status on the current response time.

Unit testing

You can test your task definition by calling its methods directly.

photo = Factory(:photo)
task = ClassifyPhotoTask.new(photo)
task.moderate_photo(photo, [{:answer=>['Offensive']}])
photo.user.should be_quarantined

Developing and testing locally with real responses

Testing locally enables you to gain an understanding of how the human workers will respond to your tasks. This provides an opportunity to improve your copy and structure to get the response you are looking for.

For testing your code, it's highly recommended that you develop offline and write suitable unit tests.

To test locally, retrieve your API key from your app configuration:

$ heroku config | grep SHAREDWORKFORCE_API_KEY
SHAREDWORKFORCE_API_KEY => acdc30b2-14c5-46ee-ba35-11d50edc65ec

And set SHAREDWORKFORCE_API_KEY before running your server or console:

$ SHAREDWORKFORCE_API_KEY=acdc30b2-14c5-46ee-ba35-11d50edc65ec rails c

Alternatively you can use dotenv to read environment files from a .env file during development. Use the following command to add the SHAREDWORKFORCE_API_KEY values retrieved from heroku config to .env.

$ heroku config -s | grep SHAREDWORKFORCE_API_KEY >> .env
$ more .env

Add add .dotenv to your Gemfile

gem 'dotenv', :groups => [:development]

Credentials and other sensitive configuration values should not be committed to source-control. In Git exclude the .env file with: echo .env &gt;&gt; .gitignore.

When your task has been completed by a human, you can use a rake task to collect and process the responses. (The rake task is only for your convenience during development. For apps running in production, a webhook is used to process responses.)

$ rake sw:collect

Developing offline

When developing locally you can optionally disable calls to Shared Workforce to minimize dependencies on remote services. This can be done in an initializer:

SharedWorkforce.configure do |config|
  if Rails.env.test? || Rails.env.development?
    config.request_class = SharedWorkforce::TaskRequest::BlackHole
  end
end

Demo app

A demo Rails app demonstrating a Shared Workforce integration is available at https://github.com/sharedworkforce/sharedworkforce-demo-rails. The app’s README contains details on how to install and configure the app.

Dashboard

The Shared Workforce dashboard allows you to monitor your completed tasks, work on tasks yourself and report problems.

You can also see a log of the 100 most recent API events to aid in debugging.

Shared Workforce API logs

The dashboard can be accessed via the CLI:

$ heroku addons:open sharedworkforce
Opening sharedworkforce for sharp-mountain-4005…

or by visiting the Heroku apps web interface and selecting the application in question. Select Shared Workforce from the Add-ons menu.

Shared Workforce Add-ons Dropdown

Migrating between plans

Application owners should carefully manage the migration timing to ensure proper application function during the migration process.

Use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade sharedworkforce:newplan
-----> Upgrading sharedworkforce:newplan to sharp-mountain-4005... done, v18 ($49/mo)
       Your plan has been updated to: sharedworkforce:newplan

Removing the add-on

Shared Workforce can be removed via the CLI.

$ heroku addons:remove sharedworkforce
-----> Removing sharedworkforce from sharp-mountain-4005... done, v20 (free)

Support

All Shared Workforce support and runtime issues should be logged with Heroku Support at https://support.heroku.com. Any non-support related issues or product feedback is welcome by emailing erica@sharedworkforce.com

Additional resources

Additional resources are available at: