Rcmmndr

This add-on is operated by Rcmmndr

Recommendation as a Service

Rcmmndr

Last Updated: 19 March 2014

The Rcmmndr add-on is currently in beta.

Table of Contents

Rcmmndr (http://addons.heroku.com/rcmmndr) is a recommendation engine add-on based on Apache Mahout.

Adding a recommendation engine to an application will enable the application’s users to discover items or other people they might not have found by themselves. Providing similar functionality with hard-coded search algorithms may be difficult once the application gets bigger and complex.

Rcmmndr is a collaborative filtering solution, which essentially means that, it will take all the current preferences that have been made in your application, and try to guess new preferences for a specific user. It is not specific to item types, but it only depends on the present preferences data. You can recommend books, movies, cars to your users, or even other users to follow.

Rcmmndr is easily accessible via a REST API and has supported thin client libraries for Java, Ruby and Python.

You can refer to the Official Mahout documentation or “Mahout In Action” book if you would like to get extensive information on how the recommendations are done. But to summarize; Rcmmndr will take all the preferences that you have created and calculate a user similarity based on these preferences, which then will be used to make recommendations to the users. Sometimes there might not be enough information to make recommendations for a specific user, at this case the server will return an empty json body.

Rcmmndr is using a sensible-default recommendation algorithm,so it is ready to make recommendations as soon as you put your data in. But you can also customize the recommender and tune it better for your domain.

Provisioning the add-on

Rcmmndr can be attached to a Heroku application via the CLI:

A list of all plans available can be found here

$ heroku addons:add rcmmndr
-----> Adding rcmmndr to sharp-mountain-4005... done, v18 (free)

Once RCMMNDR has been added a RCMMNDR_ACCESS_URL setting will be available in the app configuration and will contain the canonical URL used to access the newly provisioned RCMMNDR service instance. This can be confirmed using the heroku config:get command.

$ heroku config:get RCMMNDR_ACCESS_URL
http://api.rcmmndr.com/api_key/{your_api_key}

After installing Rcmmndr the application should be configured to fully integrate with the add-on.

API Reference

This is the general api reference that you can refer to see how the web service is implemented.

Preference API

You can do the crud operations about preferences using this api. Please refer to bulk api for batch operations.

Rcmmndr is a collaborative filtering service. The flow starts with creating preferences. A single preference contains:

  • user_id : the actual user on your app that made the preference
  • item_id : the item that the user expressed the preference for, such as a book or a movie
  • rating: the level of the preference if applicable.

So for instance, if you own an ecommerce website and you want to create a recommendation widget based on your sales, you might want to set a default value to the rating field which is the same for all users. But if you have a rating value as well you can use it.

Creating a preference

api_key/{apiKey}/preference/{uid}/{iid}/{rating}

curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/preference/1/101/5"

This command will add a preference for user with id 1 to the item 101 with a weight of 5.

Deleting preferences with specific user id

To delete all the preferences that belong to a specific user id:

curl -X DELETE "http://api.rcmmndr.com/api_key/{API_KEY}/preference/{uid}"

Deleting all preferences

This will delete all the preferences that you have created on that account

curl -X DELETE "http://api.rcmmndr.com/api_key/{API_KEY}/preference/_all"

Exporting your preference data

You can get all your data in csv format like the following:

curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/preference/_all"

Recommendation API

Get a recommendation for a user

api_key/{apiKey}/recommend/{uid}

curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/recommend/1"

This will return the top 10 recommendations for user 1

Bulk API

Please note that currently,the process of loading a dump file will override your existing data.

To load a preferences file dump:

curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/preference/_bulk" --data-binary @preferences.tsv -H "Content-Type: text/plain"

To load a gzip compressed preferences file dump:

curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/preference/_bulk" --data-binary @preferences.tsv.gz -H "Content-Type: text/plain" -H "Content-Encoding: gzip"

Stats API

You can use this api to check for your account statistics. Stats API will return the results in json format.

curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/_stats"

Settings API

You can use the settings api to finetune your recommender and check how different algorithms behave on your dataset.

Get Current Settings

curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/_settings"
{"response":"tenant has no customizations"}

If you have defined a custom recommender, you will see the definition here. Rcmmndr uses a JSON based DSL to express custom recommenders. If you do not see any customizations such as the case above, it means that you are using the default recommender. (GenericUserBasedRecommender)

Update Settings

curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/_settings" -H "Content-Type:application/json" -d '
        {
             "recommender": {
                  "impl":"GenericBooleanPrefUserBasedRecommender",
                  "params":{
                       "UserSimilarity" : {
                            "impl":"LogLikelihoodSimilarity"
                       },
                       "UserNeighborhood":{
                            "impl":"NearestNUserNeighborhood",
                            "params": {
                                 "n":20
                            }
                       }
                  }
             }
        }
'

This will use a boolean recommender (discarding the preference value) instead of the default GenericUserBasedRecommender.

Now if you do a get on the same Rest url, you will see that your settings has been updated:

{
         "recommender": {
              "impl":"GenericBooleanPrefUserBasedRecommender",
              "params":{
                   "UserSimilarity" : {
                        "impl":"LogLikelihoodSimilarity"
                   },
                   "UserNeighborhood":{
                        "impl":"NearestNUserNeighborhood",
                        "params": {
                             "n":20
                        }
                   }
              }
         }
    }

Available Recommenders

Different recommender algorithms or same algorithms with different parameters will lead to different recommendations on the same dataset for the same users. You can finetune the system as you wish using the settings DSL. Please remember check the Apache Mahout Documentation for choosing the suitable Recommender.

Using GenericUserBasedRecommender
{
"recommender": {
    "impl": "GenericUserBasedRecommender",
    "params": {
        "UserSimilarity": {
            "impl": "PearsonCorrelationSimilarity"
        },
        "UserNeighborhood": {
            "impl": "NearestNUserNeighborhood",
            "params": {
                "n": 2
            }
        }
    }
}
}

You can post the descriptor above to switch to GenericUserBasedRecommender. A UserBasedRecommender requires a UserSimilarity and a User Neighborhood. Please check the Apache Mahout Documentation for details.

Using GenericBooleanPrefUserBasedRecommender

Sometimes you may not have meaningful preference values to feed into the recommender, in this case you can give a default value (ex: 1.0) to all preferences and use GenericBooleanPrefUserBasedRecommender, which will ignore the preference value. Using GenericBooleanPrefUserBasedRecommender is the same as using other user based recommenders

{
"recommender": {
    "impl": "GenericBooleanPrefUserBasedRecommender",
    "params": {
        "UserSimilarity": {
            "impl": "PearsonCorrelationSimilarity"
        },
        "UserNeighborhood": {
            "impl": "NearestNUserNeighborhood",
            "params": {
                "n": 2
            }
        }
    }
}
}
Supported User Similarity Algorithms
  • PearsonCorrelationSimilarity
  • LogLikelihoodSimilarity
  • TanimotoCoefficientSimilarity
  • EuclideanDistanceSimilarity

are currently supported.Please refer to the Apache Mahout Documentation for details.

Custom User Similarity Algorithms

You can also provide your own similarity logic that compares the two user ids and returns a similarity value based on those ids. Rcmmndr uses Mvel expressions to support this.

{
"recommender": {
    "impl": "GenericUserBasedRecommender",
    "params": {
        "UserSimilarity": {
            "custom": "if (Math.abs(id2-id1) > 1000000000) { return -1.0; } else { return 1.0; }"
        },
        "UserNeighborhood": {
            "impl": "NearestNUserNeighborhood",
            "params": {
                "n": 3
            }
        }
    }
}
}

In the example above, id2 and id1 is compared, if the difference is greater than one billion, it is decided that these users are not alike at all.

Rcmmndr does not keep any additional information about the items or users in the system. In order to benefit from this feature, you need to provide your ids in a sensible way, from which you can get some decisional information based on mathematical expressions.

Supported User Neighborhood Algorithms

NearestNUserNeighborhood

is currently supported. You can change the size of the User Neighborhood with the “n” parameter. N should be between the range 2-50 at the moment.

GenericItemBasedRecommender
{
"recommender": {
    "impl": "GenericItemBasedRecommender",
    "params": {
        "ItemSimilarity": {
            "impl": "PearsonCorrelationSimilarity"
        }
    }
}
}

You can post the descriptor above to switch to GenericItemBasedRecommender. An ItemBasedRecommender requires an ItemSimilarity. Please check the Apache Mahout Documentation for details.

Supported Item Similarity Algorithms
  • PearsonCorrelationSimilarity
  • LogLikelihoodSimilarity
  • TanimotoCoefficientSimilarity
  • EuclideanDistanceSimilarity

are currently supported.Please refer to the Apache Mahout Documentation for details.

Custom Item Similarity Algorithms

You can also provide your own similarity logic that compares the two item ids and returns a similarity value based on those ids. Rcmmndr uses Mvel expressions to support this.

{
"recommender": {
    "impl": "GenericItemBasedRecommender",
    "params": {
        "ItemSimilarity": {
            "custom": "if (Math.abs(id2-id1) > 1000000000) { return -1.0; } else { return 1.0; }"
        }
    }
}
}

In the example above, id2 and id1 is compared, if the difference is greater than one billion, it is decided that these items are not alike at all.

Rcmmndr does not keep any additional information about the items or users in the system. In order to benefit from this feature, you need to provide your ids in a sensible way, from which you can get some decisional information based on mathematical expressions.

Estimation API

You can get an estimation to a user’s specific preference for an item as follows:

curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/estimate/{user_id}/{item_id}"

Evaluation API

While experimenting with these different recommenders, you might want to check which recommender performs better with your domain by evaluating how closely the estimated preferences match the actual preferences.

For this purpose Mahout provides some built in evaluators, and some of them are currently available in the Evaluation Api.

Supported Evaluators
  • AverageAbsoluteDifferenceRecommenderEvaluator
  • RMSRecommenderEvaluator

You can check the details of these evaluators form the Mahout documentation but in short, Root-mean-square (RMSRecommenderEvaluator) more heavily penalizes the estimates that are much different from the expected ones.

Evaluation is done by spitting the dataset into two, one is used for running the original recommender and the other is used for comparing the results.

Evaluator Parameters

trainingPercentage: For instance 0.7 means that the evaluator will train with the %70 of the data and test with %30 of the data.

evaluationPercentage: 0.1 means that %10 of the data will be used in evaluation process. Note that during the beta, this propery might be limitied to a lower default until the development is finalized.

AverageAbsoluteDifferenceRecommenderEvaluator
{
"evaluator": {
    "impl": "AverageAbsoluteDifferenceRecommenderEvaluator",
    "params": {
        "trainingPercentage": 0.1,
        "evaluationPercentage": 0.1
    }
}
}
RMSRecommenderEvaluator
{
"evaluator": {
    "impl": "RMSRecommenderEvaluator",
    "params": {
        "trainingPercentage": 0.7,
        "evaluationPercentage": 0.2
    }
}
}
Sample Usage
curl --include -X GET http://api.rcmmndr.com/api_key/{API_KEY}/_evaluate -d  '{
"evaluator": {
"impl": "AverageAbsoluteDifferenceRecommenderEvaluator",
"params": {
"trainingPercentage": 0.1,
"evaluationPercentage": 0.1
}
}
}' -H "Content-Type:application/json"
Assessing the Result

The evaluation result will be a double value similar to the following:

{
"score": 0.9635416666666665
}

This is the average difference between the estimated preferences and the actual preferences. The result above was extracted from a dataset whose preferences were in the range 1 to 5. So the result means that on average our recommendation is of by 0.9 ,Depending on your requirements you can interpret this in different ways,for our test scenario its does not seem great but not too bad as well. You can try this evaluation with different evaluator and recommender settings.

A perfect recommendation’s score would be 0. Meaning that there are no differences at all. But this is probably impossible without some domain specific hints to the recommender.

Creating Multiple Datasets for a Single Account

It is possible to create multiple datasets by specifying a data type. The examples above does not mention a type, so all the preferences and settings are saved to a default internal type. To specify a custom type, you can simply add the type path variable to the url, following the api key. For instance, the http url to create a preference will be something like this:

curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/type/{TYPE_NAME}/preference/1/101/5"

Using with Ruby or Rails 3.x

You can use the RubyRcmmndr gem to start making recommendations.

gem 'RubyRcmmndr', '0.0.1', :git => 'https://github.com/Rcmmndr/RubyRcmmndr.git'

Update application dependencies with bundler.

$ bundle install

Creating a Client Instance

require 'RubyRcmmndr'
client = RubyRcmmndr::Client.new("API_KEY")

Adding Preferences

client.create_preference(1,102,2.5)

this will add user ‘1’ a preference of ‘2.5’ for the item ‘102’

Making Recommendations

recs=client.get_recommendation(1)

the Ruby Data Structure ‘recs’ contains the recommendation results as itemid,value pairs

Usage Statistics

You can check the usage statistics for your account as follows:

client.get_usage_stats()["total_preferences"]
client.get_usage_stats()["distinct_keys"]

Deleting Preferences

Deleting Preferences for specific user

client.delete_preferences_of_user(1)

this will delete all the preferences of user ‘1’

Deleting All Preferences

client.delete_all_preferences

This will delete all the data in your account.

Bulk API

Bulk API is a better way of inserting many number of preference values. It is done by posting a text file (comma or tab separated) to the server.

client.bulk_update_preferences("preferences.tsv")

where the contents of preferences.tsv are similar to the following: (userID,itemID,preference)

196,242,3
186,302,3
22,377,1
244,51,2
166,346,1

Using with Python/Django

You can use the PyRcmmndr module to start making recommendations.

Installation

pip install git+https://github.com/Rcmmndr/pyrcmmndr.git

Creating a Client Instance

import pyrcmmndr
pyrcmmndrClient=pyrcmmndr.RcmmndrClient("API_KEY")

Adding Preferences

pyrcmmndrClient.create_preference(1,101,1.0)

this will add user ‘1’ a preference of ‘1.0’ for the item ‘100’

Making Recommendations

recs=pyrcmmndrClient.get_recommendation(1)

the dictionary ‘recs’ will contain the recommendation results as itemid,value pairs

Usage Statistics

You can check the usage statistics for your account as follows:

stats=pyrcmmndrClient.get_usage_stats()
assert stats['total_preferences'] == 1
assert stats['distinct_keys'] == 1

Deleting Preferences

Deleting Preferences for specific user

pyrcmmndrClient.delete_preferences_of_user(1)

this will delete all the preferences of user with id ‘1’

Deleting All Preferences

pyrcmmndrClient.delete_all_preferences()

This will delete all the data in your account.

Bulk API

Bulk API is a better way of inserting many number of preference values. It is done by posting a text file (comma or tab separated) to the server.

pyrcmmndrClient.bulk_update_preferences('preferences.tsv')

where the contents of preferences.tsv are similar to the following: (userID,itemID,preference)

196,242,3
186,302,3
22,377,1
244,51,2
166,346,1

Using with Java

You can use the rest client wrapper for Java and other JVM based languages.

https://github.com/Rcmmndr/java-client

Creating a Client Instance

rcmmndrClient = new DefaultRcmmndrClient("API_KEY");

Creating a Client Instance with custom Http Client (Apache Commons)

rcmmndrClient = new DefaultRcmmndrClient("API_KEY",httpClientInstance);

Adding Preferences

rcmmndrClient.createPreference(1l, 100l, 1.0f);

this will add user ‘1’ a preference of ‘1.0’ for the item ‘100’

Making Recommendations

rcmmndrClient.getRecommendation(10l);

this will return the recommendations for user 10 as a map

Usage Statistics

defaultRcmmndrClient.getUsageStats().getDistictKeys()

this will return how many different users are in the system who have at least one preference

defaultRcmmndrClient.getUsageStats().getTotalNumberOfPreferences()

this will return the total number of preference objects

Deleting Preferences

Deleting Preferences for specific user

defaultRcmmndrClient.deletePreferencesOfUser(5l)

this will delete all the preferences of user 5

Deleting All Preferences

defaultRcmmndrClient.deleteAllPreferences()

This will delete all the data in your account.

Bulk API

Bulk API is a better way of inserting many number of preference values. It is done by posting a text file (comma or tab separated) to the server.

defaultRcmmndrClient.bulkUpdatePreferences(new FileInputStream(new File("…/preferences.tsv")))

where the contents of preferences.tsv are similar to the following: (userID,itemID,preference)

196,242,3
186,302,3
22,377,1
244,51,2
166,346,1

Using with Clojure

There is a ready to run clojure webapp sample located here:

https://github.com/Rcmmndr/rcmmndr-clojure-sample

This sample is currently using the Java client

Monitoring

You can access the Rcmmndr dashboard from the CLI.

$ heroku addons:open rcmmndr

Migrating between plans

Application owners should carefully manage the migration timing to ensure proper application function during the migration process.

Use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade rcmmndr:basic
-----> Upgrading rcmmndr:basic to sharp-mountain-4005... done, v18 ($49/mo)
Your plan has been updated to: rcmmndr:basic

Removing the add-on

Rcmmndr can be removed via the CLI.

This will destroy all associated data and cannot be undone!

$ heroku addons:remove rcmmndr
-----> Removing rcmmndr from sharp-mountain-4005... done, v20 (free)

Support

All Rcmmndr support and runtime issues should be submitted via on of the Heroku Support channels. Any non-support related issues or product feedback is welcome at support@rcmmndr.com