This add-on is operated by Rcmmndr
Recommendation as a Service
Rcmmndr
Last updated January 30, 2017
This article is a work in progress, or documents a feature that is not yet released to all users. This article is unlisted. Only those with the link can access it.
The Rcmmndr add-on is currently in beta.
Table of Contents
Rcmmndr (https://elements.heroku.com/addons/rcmmndr) is a recommendation engine add-on based on Apache Mahout.
Adding a recommendation engine to an application will enable the application’s users to discover items or other people they might not have found by themselves. Providing similar functionality with hard-coded search algorithms may be difficult once the application gets bigger and complex.
Rcmmndr is a collaborative filtering solution, which essentially means that, it will take all the current preferences that have been made in your application, and try to guess new preferences for a specific user. It is not specific to item types, but it only depends on the present preferences data. You can recommend books, movies, cars to your users, or even other users to follow.
Rcmmndr is easily accessible via a REST API and has supported thin client libraries for Java, Ruby and Python.
You can refer to the Official Mahout documentation or “Mahout In Action” book if you would like to get extensive information on how the recommendations are done. But to summarize; Rcmmndr will take all the preferences that you have created and calculate a user similarity based on these preferences, which then will be used to make recommendations to the users. Sometimes there might not be enough information to make recommendations for a specific user, at this case the server will return an empty json body.
Rcmmndr is using a sensible-default recommendation algorithm,so it is ready to make recommendations as soon as you put your data in. But you can also customize the recommender and tune it better for your domain.
Provisioning the add-on
Rcmmndr can be attached to a Heroku application via the CLI:
A list of all plans available can be found here
$ heroku addons:create rcmmndr
-----> Adding rcmmndr to sharp-mountain-4005... done, v18 (free)
Once RCMMNDR has been added a RCMMNDR_ACCESS_URL
setting will be available in the app configuration and will contain the canonical URL used to access the newly provisioned RCMMNDR service instance. This can be confirmed using the heroku config:get
command.
$ heroku config:get RCMMNDR_ACCESS_URL
http://api.rcmmndr.com/api_key/{your_api_key}
After installing Rcmmndr the application should be configured to fully integrate with the add-on.
API Reference
This is the general api reference that you can refer to see how the web service is implemented.
Preference API
You can do the crud operations about preferences using this api. Please refer to bulk api for batch operations.
Rcmmndr is a collaborative filtering service. The flow starts with creating preferences. A single preference contains:
- user_id : the actual user on your app that made the preference
- item_id : the item that the user expressed the preference for, such as a book or a movie
- rating: the level of the preference if applicable.
So for instance, if you own an e-commerce website and you want to create a recommendation widget based on your sales, you might want to set a default value to the rating field which is the same for all users. But if you have a rating value as well you can use it.
Creating a preference
api_key/{apiKey}/preference/{uid}/{iid}/{rating}
curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/preference/1/101/5"
This command will add a preference for user with id 1 to the item 101 with a weight of 5.
Deleting preferences with specific user id
To delete all the preferences that belong to a specific user id:
curl -X DELETE "http://api.rcmmndr.com/api_key/{API_KEY}/preference/{uid}"
Deleting all preferences
This will delete all the preferences that you have created on that account
curl -X DELETE "http://api.rcmmndr.com/api_key/{API_KEY}/preference/_all"
Exporting your preference data
You can get all your data in csv format like the following:
curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/preference/_all"
Recommendation API
Get a recommendation for a user
api_key/{apiKey}/recommend/{uid}
curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/recommend/1"
This will return the top 10 recommendations for user 1
Bulk API
Please note that currently,the process of loading a dump file will override your existing data.
To load a preferences file dump:
curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/preference/_bulk" --data-binary @preferences.tsv -H "Content-Type: text/plain"
To load a gzip compressed preferences file dump:
curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/preference/_bulk" --data-binary @preferences.tsv.gz -H "Content-Type: text/plain" -H "Content-Encoding: gzip"
Stats API
You can use this api to check for your account statistics. Stats API will return the results in json format.
curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/_stats"
Settings API
You can use the settings api to finetune your recommender and check how different algorithms behave on your dataset.
Get Current Settings
curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/_settings"
{"response":"tenant has no customizations"}
If you have defined a custom recommender, you will see the definition here. Rcmmndr uses a JSON based DSL to express custom recommenders. If you do not see any customizations such as the case above, it means that you are using the default recommender. (GenericUserBasedRecommender)
Update Settings
curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/_settings" -H "Content-Type:application/json" -d '
{
"recommender": {
"impl":"GenericBooleanPrefUserBasedRecommender",
"params":{
"UserSimilarity" : {
"impl":"LogLikelihoodSimilarity"
},
"UserNeighborhood":{
"impl":"NearestNUserNeighborhood",
"params": {
"n":20
}
}
}
}
}
'
This will use a boolean recommender (discarding the preference value) instead of the default GenericUserBasedRecommender.
Now if you do a get on the same Rest url, you will see that your settings has been updated:
{
"recommender": {
"impl":"GenericBooleanPrefUserBasedRecommender",
"params":{
"UserSimilarity" : {
"impl":"LogLikelihoodSimilarity"
},
"UserNeighborhood":{
"impl":"NearestNUserNeighborhood",
"params": {
"n":20
}
}
}
}
}
Available Recommenders
Different recommender algorithms or same algorithms with different parameters will lead to different recommendations on the same dataset for the same users. You can finetune the system as you wish using the settings DSL. Please remember check the Apache Mahout Documentation for choosing the suitable Recommender.
Using GenericUserBasedRecommender
{
"recommender": {
"impl": "GenericUserBasedRecommender",
"params": {
"UserSimilarity": {
"impl": "PearsonCorrelationSimilarity"
},
"UserNeighborhood": {
"impl": "NearestNUserNeighborhood",
"params": {
"n": 2
}
}
}
}
}
You can post the descriptor above to switch to GenericUserBasedRecommender. A UserBasedRecommender requires a UserSimilarity and a User Neighborhood. Please check the Apache Mahout Documentation for details.
Using GenericBooleanPrefUserBasedRecommender
Sometimes you may not have meaningful preference values to feed into the recommender, in this case you can give a default value (ex: 1.0) to all preferences and use GenericBooleanPrefUserBasedRecommender, which will ignore the preference value. Using GenericBooleanPrefUserBasedRecommender is the same as using other user based recommenders
{
"recommender": {
"impl": "GenericBooleanPrefUserBasedRecommender",
"params": {
"UserSimilarity": {
"impl": "PearsonCorrelationSimilarity"
},
"UserNeighborhood": {
"impl": "NearestNUserNeighborhood",
"params": {
"n": 2
}
}
}
}
}
Supported User Similarity Algorithms
- PearsonCorrelationSimilarity
- LogLikelihoodSimilarity
- TanimotoCoefficientSimilarity
- EuclideanDistanceSimilarity
are currently supported.Please refer to the Apache Mahout Documentation for details.
Custom User Similarity Algorithms
You can also provide your own similarity logic that compares the two user ids and returns a similarity value based on those ids. Rcmmndr uses Mvel expressions to support this.
{
"recommender": {
"impl": "GenericUserBasedRecommender",
"params": {
"UserSimilarity": {
"custom": "if (Math.abs(id2-id1) > 1000000000) { return -1.0; } else { return 1.0; }"
},
"UserNeighborhood": {
"impl": "NearestNUserNeighborhood",
"params": {
"n": 3
}
}
}
}
}
In the example above, id2 and id1 is compared, if the difference is greater than one billion, it is decided that these users are not alike at all.
Rcmmndr does not keep any additional information about the items or users in the system. In order to benefit from this feature, you need to provide your ids in a sensible way, from which you can get some decisional information based on mathematical expressions.
Supported User Neighborhood Algorithms
NearestNUserNeighborhood
is currently supported. You can change the size of the User Neighborhood with the “n” parameter. N should be between the range 2-50 at the moment.
GenericItemBasedRecommender
{
"recommender": {
"impl": "GenericItemBasedRecommender",
"params": {
"ItemSimilarity": {
"impl": "PearsonCorrelationSimilarity"
}
}
}
}
You can post the descriptor above to switch to GenericItemBasedRecommender. An ItemBasedRecommender requires an ItemSimilarity. Please check the Apache Mahout Documentation for details.
Supported Item Similarity Algorithms
- PearsonCorrelationSimilarity
- LogLikelihoodSimilarity
- TanimotoCoefficientSimilarity
- EuclideanDistanceSimilarity
are currently supported.Please refer to the Apache Mahout Documentation for details.
Custom Item Similarity Algorithms
You can also provide your own similarity logic that compares the two item ids and returns a similarity value based on those ids. Rcmmndr uses Mvel expressions to support this.
{
"recommender": {
"impl": "GenericItemBasedRecommender",
"params": {
"ItemSimilarity": {
"custom": "if (Math.abs(id2-id1) > 1000000000) { return -1.0; } else { return 1.0; }"
}
}
}
}
In the example above, id2 and id1 is compared, if the difference is greater than one billion, it is decided that these items are not alike at all.
Rcmmndr does not keep any additional information about the items or users in the system. In order to benefit from this feature, you need to provide your ids in a sensible way, from which you can get some decisional information based on mathematical expressions.
Estimation API
You can get an estimation to a user’s specific preference for an item as follows:
curl -X GET "http://api.rcmmndr.com/api_key/{API_KEY}/estimate/{user_id}/{item_id}"
Evaluation API
While experimenting with these different recommenders, you might want to check which recommender performs better with your domain by evaluating how closely the estimated preferences match the actual preferences.
For this purpose Mahout provides some built in evaluators, and some of them are currently available in the Evaluation Api.
Supported Evaluators
- AverageAbsoluteDifferenceRecommenderEvaluator
- RMSRecommenderEvaluator
You can check the details of these evaluators form the Mahout documentation but in short, Root-mean-square (RMSRecommenderEvaluator) more heavily penalizes the estimates that are much different from the expected ones.
Evaluation is done by spitting the dataset into two, one is used for running the original recommender and the other is used for comparing the results.
Evaluator Parameters
trainingPercentage: For instance 0.7 means that the evaluator will train with the %70 of the data and test with %30 of the data.
evaluationPercentage: 0.1 means that %10 of the data will be used in evaluation process. Note that during the beta, this propery might be limitied to a lower default until the development is finalized.
AverageAbsoluteDifferenceRecommenderEvaluator
{
"evaluator": {
"impl": "AverageAbsoluteDifferenceRecommenderEvaluator",
"params": {
"trainingPercentage": 0.1,
"evaluationPercentage": 0.1
}
}
}
RMSRecommenderEvaluator
{
"evaluator": {
"impl": "RMSRecommenderEvaluator",
"params": {
"trainingPercentage": 0.7,
"evaluationPercentage": 0.2
}
}
}
Sample Usage
curl --include -X GET http://api.rcmmndr.com/api_key/{API_KEY}/_evaluate -d '{
"evaluator": {
"impl": "AverageAbsoluteDifferenceRecommenderEvaluator",
"params": {
"trainingPercentage": 0.1,
"evaluationPercentage": 0.1
}
}
}' -H "Content-Type:application/json"
Assessing the Result
The evaluation result will be a double value similar to the following:
{
"score": 0.9635416666666665
}
This is the average difference between the estimated preferences and the actual preferences. The result above was extracted from a dataset whose preferences were in the range 1 to 5. So the result means that on average our recommendation is of by 0.9 ,Depending on your requirements you can interpret this in different ways,for our test scenario its does not seem great but not too bad as well. You can try this evaluation with different evaluator and recommender settings.
A perfect recommendation’s score would be 0. Meaning that there are no differences at all. But this is probably impossible without some domain specific hints to the recommender.
Creating Multiple Datasets for a Single Account
It is possible to create multiple datasets by specifying a data type. The examples above does not mention a type, so all the preferences and settings are saved to a default internal type. To specify a custom type, you can simply add the type path variable to the url, following the api key. For instance, the http url to create a preference will be something like this:
curl -X POST "http://api.rcmmndr.com/api_key/{API_KEY}/type/{TYPE_NAME}/preference/1/101/5"
Using with Ruby or Rails 3.x
You can use the RubyRcmmndr gem to start making recommendations.
gem 'RubyRcmmndr', '0.0.1', :git => 'https://github.com/Rcmmndr/RubyRcmmndr.git'
Update application dependencies with bundler.
$ bundle install
Creating a Client Instance
require 'RubyRcmmndr'
client = RubyRcmmndr::Client.new("API_KEY")
Adding Preferences
client.create_preference(1,102,2.5)
this will add user ‘1’ a preference of ‘2.5’ for the item ‘102’
Making Recommendations
recs=client.get_recommendation(1)
the Ruby Data Structure ‘recs’ contains the recommendation results as itemid,value pairs
Usage Statistics
You can check the usage statistics for your account as follows:
client.get_usage_stats()["total_preferences"]
client.get_usage_stats()["distinct_keys"]
Deleting Preferences
Deleting Preferences for specific user
client.delete_preferences_of_user(1)
this will delete all the preferences of user ‘1’
Deleting All Preferences
client.delete_all_preferences
This will delete all the data in your account.
Bulk API
Bulk API is a better way of inserting many number of preference values. It is done by posting a text file (comma or tab separated) to the server.
client.bulk_update_preferences("preferences.tsv")
where the contents of preferences.tsv are similar to the following: (userID,itemID,preference)
196,242,3
186,302,3
22,377,1
244,51,2
166,346,1
Using with Python/Django
You can use the PyRcmmndr module to start making recommendations.
Installation
pip install git+https://github.com/Rcmmndr/pyrcmmndr.git
Creating a Client Instance
import pyrcmmndr
pyrcmmndrClient=pyrcmmndr.RcmmndrClient("API_KEY")
Adding Preferences
pyrcmmndrClient.create_preference(1,101,1.0)
this will add user ‘1’ a preference of ‘1.0’ for the item ‘100’
Making Recommendations
recs=pyrcmmndrClient.get_recommendation(1)
the dictionary ‘recs’ will contain the recommendation results as itemid,value pairs
Usage Statistics
You can check the usage statistics for your account as follows:
stats=pyrcmmndrClient.get_usage_stats()
assert stats['total_preferences'] == 1
assert stats['distinct_keys'] == 1
Deleting Preferences
Deleting Preferences for specific user
pyrcmmndrClient.delete_preferences_of_user(1)
this will delete all the preferences of user with id ‘1’
Deleting All Preferences
pyrcmmndrClient.delete_all_preferences()
This will delete all the data in your account.
Bulk API
Bulk API is a better way of inserting many number of preference values. It is done by posting a text file (comma or tab separated) to the server.
pyrcmmndrClient.bulk_update_preferences('preferences.tsv')
where the contents of preferences.tsv are similar to the following: (userID,itemID,preference)
196,242,3
186,302,3
22,377,1
244,51,2
166,346,1
Using with Java
You can use the rest client wrapper for Java and other JVM based languages.
https://github.com/Rcmmndr/java-client
Creating a Client Instance
rcmmndrClient = new DefaultRcmmndrClient("API_KEY");
Creating a Client Instance with custom Http Client (Apache Commons)
rcmmndrClient = new DefaultRcmmndrClient("API_KEY",httpClientInstance);
Adding Preferences
rcmmndrClient.createPreference(1l, 100l, 1.0f);
this will add user ‘1’ a preference of ‘1.0’ for the item ‘100’
Making Recommendations
rcmmndrClient.getRecommendation(10l);
this will return the recommendations for user 10 as a map
Usage Statistics
defaultRcmmndrClient.getUsageStats().getDistictKeys()
this will return how many different users are in the system who have at least one preference
defaultRcmmndrClient.getUsageStats().getTotalNumberOfPreferences()
this will return the total number of preference objects
Deleting Preferences
Deleting Preferences for specific user
defaultRcmmndrClient.deletePreferencesOfUser(5l)
this will delete all the preferences of user 5
Deleting All Preferences
defaultRcmmndrClient.deleteAllPreferences()
This will delete all the data in your account.
Bulk API
Bulk API is a better way of inserting many number of preference values. It is done by posting a text file (comma or tab separated) to the server.
defaultRcmmndrClient.bulkUpdatePreferences(new FileInputStream(new File("…/preferences.tsv")))
where the contents of preferences.tsv are similar to the following: (userID,itemID,preference)
196,242,3
186,302,3
22,377,1
244,51,2
166,346,1
Using with Clojure
There is a ready to run clojure webapp sample located here:
https://github.com/Rcmmndr/rcmmndr-clojure-sample
This sample is currently using the Java client
Monitoring
You can access the Rcmmndr dashboard from the CLI.
$ heroku addons:open rcmmndr
Migrating between plans
Application owners should carefully manage the migration timing to ensure proper application function during the migration process.
Use the heroku addons:upgrade
command to migrate to a new plan.
$ heroku addons:upgrade rcmmndr:basic
-----> Upgrading rcmmndr:basic to sharp-mountain-4005... done, v18 ($49/mo)
Your plan has been updated to: rcmmndr:basic
Removing the add-on
Rcmmndr can be removed via the CLI.
This will destroy all associated data and cannot be undone!
$ heroku addons:destroy rcmmndr
-----> Removing rcmmndr from sharp-mountain-4005... done, v20 (free)
Support
All Rcmmndr support and runtime issues should be submitted via on of the Heroku Support channels. Any non-support related issues or product feedback is welcome at support@rcmmndr.com