Stackhero S3 (Object Storage)
Last updated September 12, 2023
Table of Contents
Stackhero Object Storage provides an object storage, based on MinIO, compatible with the S3 protocol and running on a fully dedicated instance.
The Stackhero Object Storage add-on provides the following features:
- Unlimited transfers
- A private instance (dedicated VM)
- A dedicated public IP (v4)
- TLS encryption (aka SSL)
- An automatic backup every 24 hours
- A graphical web UI (MinIO console)
- One click to update to new MinIO versions
- Super fast servers with SSD disks
Provisioning the Add-on
Stackhero Object Storage can be attached to a Heroku application via the CLI:
A list of all plans available can be found here.
$ heroku addons:create ah-s3-object-storage-stackhero --app <your app name>
-----> Adding ah-s3-object-storage-stackhero to sharp-mountain-4005... done
After provisioning Stackhero Object Storage, the STACKHERO_MINIO_ROOT_ACCESS_KEY
and STACKHERO_MINIO_ROOT_SECRET_KEY
config variables are available in your app’s configuration. They contain the “root” credentials.
Review config var values with the heroku config:get
CLI command:
$ heroku config:get STACKHERO_MINIO_ROOT_ACCESS_KEY
2765UKKnWOjjFtcXYSzW
Don’t directly use the “root” credentials in your apps. Create a dedicated user and use it in your apps.
Creating a User
A user is identified by an access key and a secret key. You can create a user using the MinIO web console or the MinIO client.
Creating a user with the MinIO Web Console
To create a user on MinIO, you first have to connect to your Stackhero dashboard:
$ heroku addons:open ah-s3-object-storage-stackhero
Opening ah-s3-object-storage-stackhero for sharp-mountain-4005
Select your MinIO service and choose the Console
link. Provide your credentials. They can be found in the STACKHERO_MINIO_ROOT_ACCESS_KEY
and STACKHERO_MINIO_ROOT_SECRET_KEY
config variables. You can retrieve them with the heroku config
command or directly in Stackhero dashboard by choosing Configure
.
In the MinIO console, select Users, and then select Create a user.
Define an access key
, a secret key
and select the policies you want (for example, readwrite
).
Creating a User with the MinIO Client
Use the Heroku Local command-line tool to configure, run, and manage process types specified in your app’s Procfile. Heroku Local reads configuration variables from an .env
file. Use heroku config
to view your app’s config variables. Use the following command for each value that you want to add to your .env
file:
heroku config:get <STACKHERO_MINIO_VARIABLE> -s >> .env
For example:
$ echo "STACKHERO_MINIO_ACCESS_KEY=<ACCESS_KEY>" >> .env
$ echo "STACKHERO_MINIO_SECRET_KEY=<SECRET_KEY>" >> .env
Replace <ACCESS_KEY>
and <SECRET_KEY>
with appropriate values.
Don’t commit credentials and other sensitive configuration values to source-control. In Git, exclude the .env
file with: echo .env >> .gitignore
.
For more information, see Heroku Local.
Connecting to MinIO
There are S3 clients for most popular programming language. We recommend you chose a library that supports TLS encryption. We recommend you use the MinIO SDKs to connect to your object storage. For more information about available SDKs, see the MinIO official documentation.
Another way to connect is to use the MinIO CLI interface. The CLI interface can send large files directly from your terminal. For more information about the MinIO CLI interface, see the MinIO CLI official documentation.
Connecting with Java
The following example shows how to connect using the MinIO Java client:
MinioClient minioClient =
MinioClient.builder()
.endpoint("https://" + System.getenv("STACKHERO_MINIO_HOST"))
.credentials(System.getenv("STACKHERO_MINIO_ACCESS_KEY"), System.getenv("STACKHERO_MINIO_SECRET_KEY"))
.build();
For more information about connecting with Java, see MinIO Java SDK documentation.
Connecting with Ruby
To connect with Ruby, use the AWS SDK gem. Use the following to install the gem:
$ gem 'aws-sdk'
$ bundle install
The following example shows how to connect to MinIO using the AWS SDK Ruby gem:
require 'aws-sdk'
Aws.config.update(
endpoint: 'https://' + ENV["STACKHERO_MINIO_HOST"],
access_key_id: ENV["STACKHERO_MINIO_ACCESS_KEY"],
secret_access_key: ENV["STACKHERO_MINIO_SECRET_KEY"],
force_path_style: true,
region: 'us-east-1'
)
rubys3_client = Aws::S3::Client.new
# put_object operation
rubys3_client.put_object(
key: 'testobject',
body: 'Hello from MinIO!!',
bucket: 'testbucket',
content_type: 'text/plain'
)
# get_object operation
rubys3_client.get_object(
bucket: 'testbucket',
key: 'testobject',
response_target: 'download_testobject'
)
print "Downloaded 'testobject' as 'download_testobject'. "
For more information about how to use the AWS SDK for Ruby with MinIO, see the MinIO Ruby documentation.
Connecting with Python
Connecting with AWS SDK (boto)
Install the AWS SDK (boto) package:
$ pip install boto3
$ pip freeze > requirements.txt
The following example shows how to connect to MinIO with Python and AWS SDK (boto) package:
#!/usr/bin/env/python
import os
import boto3
from botocore.client import Config
s3 = boto3.resource('s3',
endpoint_url='https://' + os.environ.get('STACKHERO_MINIO_HOST'),
aws_access_key_id=os.environ.get('STACKHERO_MINIO_ACCESS_KEY'),
aws_secret_access_key=os.environ.get('STACKHERO_MINIO_SECRET_KEY'),
config=Config(signature_version='s3v4'),
region_name='us-east-1')
# upload a file from local file system '/home/john/piano.mp3' to bucket 'songs' with 'piano.mp3' as the object name.
s3.Bucket('songs').upload_file('/home/john/piano.mp3','piano.mp3')
# download the object 'piano.mp3' from the bucket 'songs' and save it to local FS as /tmp/classical.mp3
s3.Bucket('songs').download_file('piano.mp3', '/tmp/classical.mp3')
print "Downloaded 'piano.mp3' as 'classical.mp3'. "
For more information about how to use the AWS SDK for Python with MinIO, see the MinIO Python documentation.
Connecting with the MinIO SDK
Install the MinIO package:
$ pip install minio
$ pip freeze > requirements.txt
The following example shows how to connect using the MinIO Python SDK:
#!/usr/bin/env/python
from minio import Minio
from minio.error import S3Error
def main():
# Create a client with the MinIO server playground, its access key
# and secret key.
client = Minio(
endpoint=os.environ.get('STACKHERO_MINIO_HOST') + ':443,
secure=True,
access_key=os.environ.get('STACKHERO_MINIO_ACCESS_KEY'),
secret_key=os.environ.get('STACKHERO_MINIO_SECRET_KEY'),
)
# Make 'asiatrip' bucket if not exist.
found = client.bucket_exists("asiatrip")
if not found:
client.make_bucket("asiatrip")
else:
print("Bucket 'asiatrip' already exists")
# Upload '/home/user/Photos/asiaphotos.zip' as object name
# 'asiaphotos-2015.zip' to bucket 'asiatrip'.
client.fput_object(
"asiatrip", "asiaphotos-2015.zip", "/home/user/Photos/asiaphotos.zip",
)
print(
"'/home/user/Photos/asiaphotos.zip' is successfully uploaded as "
"object 'asiaphotos-2015.zip' to bucket 'asiatrip'."
)
if __name__ == "__main__":
try:
main()
except S3Error as exc:
print("error occurred.", exc)
For more information about how to use the MinIO Python client, see the MinIO Python SDK documentation.
Connecting with PHP
Install the AWS SDK for PHP package:
$ composer require aws/aws-sdk-php
The following example shows how to connect to MinIO using the AWS SDK for PHP package:
<?php
// Include the SDK using the Composer autoloader
date_default_timezone_set('America/Los_Angeles');
require 'vendor/autoload.php';
$s3 = new Aws\S3\S3Client([
'version' => 'latest',
'region' => 'us-east-1',
'endpoint' => 'https://' . getenv('STACKHERO_MINIO_HOST'),
'use_path_style_endpoint' => true,
'credentials' => [
'key' => getenv('STACKHERO_MINIO_ACCESS_KEY'),
'secret' => getenv('STACKHERO_MINIO_SECRET_KEY')
],
]);
// Send a PutObject request and get the result object.
$insert = $s3->putObject([
'Bucket' => 'testbucket',
'Key' => 'testkey',
'Body' => 'Hello from MinIO!!'
]);
// Download the contents of the object.
$retrive = $s3->getObject([
'Bucket' => 'testbucket',
'Key' => 'testkey',
'SaveAs' => 'testkey_local'
]);
// Print the body of the result by indexing into the result object.
echo $retrive['Body'];
For more information about how to use the AWS SDK for PHP with MinIO, see the MinIO PHP documentation.
Connecting with Go
Connecting with the AWS SDK for Go
Install the AWS SDK package for GO:
$ go get github.com/aws/aws-sdk-go-v2
$ go get github.com/aws/aws-sdk-go-v2/config
The following example shows how to connect to MinIO using the AWS SDK for GO package:
package main
import (
"fmt"
"os"
"strings"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/service/s3/s3manager"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
func main() {
bucket := aws.String("newbucket")
key := aws.String("testobject")
// Configure to use MinIO Server
s3Config := &aws.Config{
Credentials: credentials.NewStaticCredentials(os.Getenv("STACKHERO_MINIO_ACCESS_KEY"), os.Getenv("STACKHERO_MINIO_SECRET_KEY"), ""),
Endpoint: aws.String("https://" + os.Getenv("STACKHERO_MINIO_HOST")),
Region: aws.String("us-east-1"),
DisableSSL: aws.Bool(false),
S3ForcePathStyle: aws.Bool(true),
}
newSession := session.New(s3Config)
s3Client := s3.New(newSession)
cparams := &s3.CreateBucketInput{
Bucket: bucket, // Required
}
// Create a new bucket using the CreateBucket call.
_, err := s3Client.CreateBucket(cparams)
if err != nil {
// Message from an error.
fmt.Println(err.Error())
return
}
// Upload a new object "testobject" with the string "Hello World!" to our "newbucket".
_, err = s3Client.PutObject(&s3.PutObjectInput{
Body: strings.NewReader("Hello from MinIO!!"),
Bucket: bucket,
Key: key,
})
if err != nil {
fmt.Printf("Failed to upload data to %s/%s, %s\n", *bucket, *key, err.Error())
return
}
fmt.Printf("Successfully created bucket %s and uploaded data with key %s\n", *bucket, *key)
// Retrieve our "testobject" from our "newbucket" and store it locally in "testobject_local".
file, err := os.Create("testobject_local")
if err != nil {
fmt.Println("Failed to create file", err)
return
}
defer file.Close()
downloader := s3manager.NewDownloader(newSession)
numBytes, err := downloader.Download(file,
&s3.GetObjectInput{
Bucket: bucket,
Key: key,
})
if err != nil {
fmt.Println("Failed to download file", err)
return
}
fmt.Println("Downloaded file", file.Name(), numBytes, "bytes")
}
For more information about how to use the AWS SDK for Go with MinIO, see the MinIO GO documentation.
Connecting with the MinIO SDK
Install the MinIO SDK package for GO:
$ GO111MODULE=on go get github.com/minio/minio-go/v7
The following example shows how to connect to MinIO using the MinIO SDK:
package main
import (
"log"
"github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
)
func main() {
endpoint := os.Getenv("STACKHERO_MINIO_HOST")
accessKeyID := os.Getenv("STACKHERO_MINIO_ACCESS_KEY")
secretAccessKey := os.Getenv("STACKHERO_MINIO_SECRET_KEY")
useSSL := true
// Initialize minio client object.
minioClient, err := minio.New(endpoint, &minio.Options{
Creds: credentials.NewStaticV4(accessKeyID, secretAccessKey, ""),
Secure: useSSL,
})
if err != nil {
log.Fatalln(err)
}
log.Printf("%#v\n", minioClient) // minioClient is now set up
}
For more information about using the MinIO Go SDK, see the MinIO GO SDK documentation.
Connecting with Node.js
Connecting with the AWS SDK for Node.js
Install the AWS SDK package for Node.js:
$ npm install --save aws-sdk
The following example shows how to connect to MinIO using the AWS SDK package for Node.js:
const process = require('process');
const AWS = require('aws-sdk');
const s3 = new AWS.S3({
accessKeyId: process.env.STACKHERO_MINIO_ACCESS_KEY,
secretAccessKey: process.env.STACKHERO_MINIO_SECRET_KEY,
endpoint: 'https://' + process.env.STACKHERO_MINIO_HOST,
s3ForcePathStyle: true, // needed with minio?
signatureVersion: 'v4'
});
// putObject operation.
s3.putObject(
{ Bucket: 'testbucket', Key: 'testobject', Body: 'Hello from MinIO!!' },
(err, data) => {
if (err)
console.log(err)
else
console.log('Successfully uploaded data to testbucket/testobject');
}
);
// getObject operation.
const file = require('fs').createWriteStream('/tmp/mykey');
s3.getObject({ Bucket: 'testbucket', Key: 'testobject' })
.on('httpData', chunk => file.write(chunk))
.on('httpDone', () => file.end())
.send();
For more information about how to use the AWS SDK for Node.js with MinIO, see the MinIO Node.js documentation.
Connecting with the MinIO SDK
Install the MinIO SDK package for Node.js:
$ npm install --save minio
The following example shows how to connect to MinIO with the MinIO SDK package for Node.js:
const process = require('process');
const Minio = require('minio');
// Instantiate the minio client with the endpoint
// and access keys as shown below.
const minioClient = new Minio.Client({
endpoint: process.env.STACKHERO_MINIO_HOST,
port: 443,
useSSL: true,
accessKeyId: process.env.STACKHERO_MINIO_ACCESS_KEY,
secretAccessKey: process.env.STACKHERO_MINIO_SECRET_KEY,
});
// File that needs to be uploaded.
const file = '/tmp/photos-europe.tar'
// Make a bucket called europetrip.
minioClient.makeBucket(
'europetrip',
'us-east-1',
err => {
if (err) return console.log(err)
console.log('Bucket created successfully in "us-east-1".')
const metaData = {
'Content-Type': 'application/octet-stream',
'X-Amz-Meta-Testing': 1234,
'example': 5678
}
// Using fPutObject API upload your file to the bucket europetrip.
minioClient.fPutObject(
'europetrip',
'photos-europe.tar',
file,
metaData,
(err, etag) => {
if (err) return console.log(err)
console.log('File uploaded successfully.')
}
);
}
);
For more information about the MinIO Node.JS SDK, see the MinIO Node.js SDK documentation.
Accessing the Dashboard and MinIO Console Web UI
You can see your instance usage, restart your instance, and apply updates from the Stackhero dashboard. You can also access the MinIO console web UI.
Access the dashboard via the CLI:
$ heroku addons:open ah-s3-object-storage-stackhero
Opening ah-s3-object-storage-stackhero for sharp-mountain-4005
You can also visit the Heroku Dashboard, select your application, and then select Stackhero Object Storage from the Add-ons menu.
Upgrading Your Plan
You can’t downgrade an existing add-on.
Use the heroku addons:upgrade
command to migrate to a new plan.
$ heroku addons:upgrade ah-s3-object-storage-stackhero:newplan
-----> Upgrading ah-s3-object-storage-stackhero:newplan to sharp-mountain-4005... done
Your plan has been updated to: ah-s3-object-storage-stackhero:newplan
Manage migration timing to ensure your application is available during the migration process.
Removing the Add-on
You can remove Stackhero Object Storage via the CLI:
This destroys all associated data and can’t be undone!
$ heroku addons:destroy ah-s3-object-storage-stackhero
-----> Removing ah-s3-object-storage-stackhero from sharp-mountain-4005... done
Support
Stackhero Object Storage support and runtime issues should be submitted via one of the Heroku Support channels. We recommend adding support@stackhero.io in copy for urgent issues.
Additional Resources
- Object Storage (MinIO) documentation by Stackhero
- MinIO managed cloud