Treasure Data
Last updated June 20, 2023
Table of Contents
Treasure Data Add-On lets Heroku users collect, store and analyze large amounts of data without sinking a lot of time learning about Hadoop, MPP databases and other powerful but time-consuming technologies. With Treasure Data, you can start analyzing your data today, not weeks and months later.
We are proven to scale. We store more than 2 trillion data records for our customers, adding hundreds of millions of data records every hour. Our customers run thousands of queries against their data on our system every day.
The Treasure Data Add-on offers instant setup of ‘Log Everything’ infrastructure for tracking, and understanding the user activities of your apps.
Our customers can collect user activities from their applications instantly and analyze the data through a SQL-like query language (Apache Hive) for better understanding of your users behavior. Typical use cases include:
- Building Reporting Feature to Your Customers
- Daily / Hourly Reports of Your Business Metrics
- Ranking Calculation
- Conversion Path Analytics
Provisioning the add-on
A list of all plans available can be found here.
One configuration parameter that Treasure Data introduces into your Heroku app is TREASURE_DATA_API_KEY.
Treasure Data can be attached to a Heroku application via the CLI:
$ heroku addons:create treasure-data
-----> Adding treasure-data to sharp-mountain-4005... done, v18 (free)
heroku addons:open treasure-data will lead you to our web console if you want.
CLI Setup
1) TD Toolbelt setup
At first, please download and install the Treasure Data Toolbelt for your development environment.
- Download MacOS Installer
- Download Windows Installer
- Linux (Redhat/CentOS, Debian/Ubuntu)
- or ‘gem install td’ for those who are familiar with Ruby.
2) Heroku CLI setup
heroku-td CLI plugin is also required to bridge between heroku CLI and td CLI. Once you install the CLI plugin, you will be able to execute the heroku td family of commands.
$ heroku plugins:install https://github.com/treasure-data/heroku-td.git
$ heroku td
usage: heroku td [options] COMMAND [args]
Analyze Your Dataset
Please access to your site to call TD.event.post(). After several minutes, the data upload is done by td gem. heroku td tables shows your uploaded dataset.
$ heroku td tables
+------------+--------+------+-----------+
| Database | Table | Type | Count |
+------------+--------+------+-----------+
| production | login | log | 31232 |
| production | follow | log | 3132 |
| production | pay | pay | 132 |
+------------+--------+------+-----------+
Now you can issue the query into cloud by heroku td query command. The example query below counts login per day.
$ td query -w -d testdb \
"SELECT \
TD_TIME_FORMAT(time, "yyyy-MM-dd", "PDT") AS day, \
COUNT(1) AS cnt \
FROM login \
GROUP BY TD_TIME_FORMAT(time, "yyyy-MM-dd", "PDT") \
ORDER BY cnt"
+------------+------+
| day | cnt |
+------------+------+
| 2012-05-26 | 4981 |
| 2012-05-27 | 4481 |
| 2012-05-28 | 481 |
+------------+------+
Migrating between plans
Application owners should carefully manage the migration timing to ensure proper application function during the migration process.
Use the heroku addons:upgrade command to migrate to a new plan.
$ heroku addons:upgrade treasure-data:developer
-----> Upgrading treasure-data:developer to sharp-mountain-4005... done
Your plan has been updated to: treasure-data:developer
developer is one of our plans. You can find out more about our price plans here.
Removing the add-on
Treasure Data can be removed via the CLI.
Note that your data stays on Treasure Data even if you remove the addon. If you want to purge your data, here is how you do it.
$ heroku addons:destroy treasure-data
-----> Removing treasure-data from sharp-mountain-4005... done, v20 (free)
Support
All Treasure Data support and runtime issues should be submitted via on of the Heroku Support channels. Any non-support related issues or product feedback is welcome at support@treasure-data.com.