Stackhero Object Storage (S3 compatible)

This add-on is operated by Stackhero

Object storage on dedicated instances with unlimited transfers

Stackhero Object Storage (S3 compatible)

Last updated January 24, 2022

Stackhero Object Storage provides an object storage, based on MinIO, compatible with the S3 protocol and running on a fully dedicated instance.

The Stackhero Object Storage add-on provides the following features:

  • Unlimited transfers
  • A private instance (dedicated VM)
  • A dedicated public IP (v4)
  • TLS encryption (aka SSL)
  • An automatic backup every 24 hours
  • A graphical web UI (MinIO console)
  • One click to update to new MinIO versions
  • Super fast servers with SSD disks

Provisioning the Add-on

Stackhero Object Storage can be attached to a Heroku application via the CLI:

A list of all plans available can be found here.

$ heroku addons:create ah-s3-object-storage-stackhero --app <your app name>
-----> Adding ah-s3-object-storage-stackhero to sharp-mountain-4005... done

After provisioning Stackhero Object Storage, the STACKHERO_MINIO_ROOT_ACCESS_KEY and STACKHERO_MINIO_ROOT_SECRET_KEY config variables are available in your app’s configuration. They contain the “root” credentials.

Review config var values with the heroku config:get CLI command:

$ heroku config:get STACKHERO_MINIO_ROOT_ACCESS_KEY
2765UKKnWOjjFtcXYSzW

Don’t directly use the “root” credentials in your apps. Create a dedicated user and use it in your apps.

Creating a User

A user is identified by an access key and a secret key. You can create a user using the MinIO web console or the MinIO client.

Creating a user with the MinIO Web Console

To create a user on MinIO, you first have to connect to your Stackhero dashboard:

$ heroku addons:open ah-s3-object-storage-stackhero
Opening ah-s3-object-storage-stackhero for sharp-mountain-4005

Select your MinIO service and choose the Console link. Provide your credentials. They can be found in the STACKHERO_MINIO_ROOT_ACCESS_KEY and STACKHERO_MINIO_ROOT_SECRET_KEY config variables. You can retrieve them with the heroku config command or directly in Stackhero dashboard by choosing Configure.

In the MinIO console, select Users, and then select Create a user. Define an access key, a secret key and select the policies you want (for example, readwrite).

Creating a MinIO user

Creating a User with the MinIO Client

Use the Heroku Local command-line tool to configure, run, and manage process types specified in your app’s Procfile. Heroku Local reads configuration variables from an .env file. Use heroku config to view your app’s config variables. Use the following command for each value that you want to add to your .env file:

heroku config:get <STACKHERO_MINIO_VARIABLE> -s  >> .env

For example:

$ echo "STACKHERO_MINIO_ACCESS_KEY=<ACCESS_KEY>" >> .env
$ echo "STACKHERO_MINIO_SECRET_KEY=<SECRET_KEY>" >> .env

Replace <ACCESS_KEY> and <SECRET_KEY> with appropriate values.

Don’t commit credentials and other sensitive configuration values to source-control. In Git, exclude the .env file with: echo .env >> .gitignore.

For more information, see Heroku Local.

Connecting to MinIO

There are S3 clients for most popular programming language. We recommend you chose a library that supports TLS encryption. We recommend you use the MinIO SDKs to connect to your object storage. For more information about available SDKs, see the MinIO official documentation.

Another way to connect is to use the MinIO CLI interface. The CLI interface can send large files directly from your terminal. For more information about the MinIO CLI interface, see the MinIO CLI official documentation.

Connecting with Java

The following example shows how to connect using the MinIO Java client:

MinioClient minioClient =
  MinioClient.builder()
    .endpoint("https://" + System.getenv("STACKHERO_MINIO_HOST"))
    .credentials(System.getenv("STACKHERO_MINIO_ACCESS_KEY"), System.getenv("STACKHERO_MINIO_SECRET_KEY"))
    .build();

For more information about connecting with Java, see MinIO Java SDK documentation.

Connecting with Ruby

To connect with Ruby, use the AWS SDK gem. Use the following to install the gem:

$ gem 'aws-sdk'
$ bundle install

The following example shows how to connect to MinIO using the AWS SDK Ruby gem:

require 'aws-sdk'

Aws.config.update(
  endpoint: 'https://' + ENV["STACKHERO_MINIO_HOST"],
  access_key_id: ENV["STACKHERO_MINIO_ACCESS_KEY"],
  secret_access_key: ENV["STACKHERO_MINIO_SECRET_KEY"],
  force_path_style: true,
  region: 'us-east-1'
)

rubys3_client = Aws::S3::Client.new

# put_object operation

rubys3_client.put_object(
  key: 'testobject',
  body: 'Hello from MinIO!!',
  bucket: 'testbucket',
  content_type: 'text/plain'
)

# get_object operation

rubys3_client.get_object(
  bucket: 'testbucket',
  key: 'testobject',
  response_target: 'download_testobject'
)

print "Downloaded 'testobject' as  'download_testobject'. "

For more information about how to use the AWS SDK for Ruby with MinIO, see the MinIO Ruby documentation.

Connecting with Python

Connecting with AWS SDK (boto)

Install the AWS SDK (boto) package:

$ pip install boto3
$ pip freeze > requirements.txt

The following example shows how to connect to MinIO with Python and AWS SDK (boto) package:

#!/usr/bin/env/python
import os
import boto3
from botocore.client import Config

s3 = boto3.resource('s3',
                    endpoint_url='https://' + os.environ.get('STACKHERO_MINIO_HOST'),
                    aws_access_key_id=os.environ.get('STACKHERO_MINIO_ACCESS_KEY'),
                    aws_secret_access_key=os.environ.get('STACKHERO_MINIO_SECRET_KEY'),
                    config=Config(signature_version='s3v4'),
                    region_name='us-east-1')

# upload a file from local file system '/home/john/piano.mp3' to bucket 'songs' with 'piano.mp3' as the object name.
s3.Bucket('songs').upload_file('/home/john/piano.mp3','piano.mp3')

# download the object 'piano.mp3' from the bucket 'songs' and save it to local FS as /tmp/classical.mp3
s3.Bucket('songs').download_file('piano.mp3', '/tmp/classical.mp3')

print "Downloaded 'piano.mp3' as  'classical.mp3'. "

For more information about how to use the AWS SDK for Python with MinIO, see the MinIO Python documentation.

Connecting with the MinIO SDK

Install the MinIO package:

$ pip install minio
$ pip freeze > requirements.txt

The following example shows how to connect using the MinIO Python SDK:

#!/usr/bin/env/python
from minio import Minio
from minio.error import S3Error

def main():
  # Create a client with the MinIO server playground, its access key
  # and secret key.
  client = Minio(
    endpoint=os.environ.get('STACKHERO_MINIO_HOST') + ':443,
    secure=True,
    access_key=os.environ.get('STACKHERO_MINIO_ACCESS_KEY'),
    secret_key=os.environ.get('STACKHERO_MINIO_SECRET_KEY'),
  )

  # Make 'asiatrip' bucket if not exist.
  found = client.bucket_exists("asiatrip")
  if not found:
    client.make_bucket("asiatrip")
  else:
    print("Bucket 'asiatrip' already exists")

  # Upload '/home/user/Photos/asiaphotos.zip' as object name
  # 'asiaphotos-2015.zip' to bucket 'asiatrip'.
  client.fput_object(
    "asiatrip", "asiaphotos-2015.zip", "/home/user/Photos/asiaphotos.zip",
  )
  print(
    "'/home/user/Photos/asiaphotos.zip' is successfully uploaded as "
    "object 'asiaphotos-2015.zip' to bucket 'asiatrip'."
  )


if __name__ == "__main__":
  try:
    main()
  except S3Error as exc:
    print("error occurred.", exc)

For more information about how to use the MinIO Python client, see the MinIO Python SDK documentation.

Connecting with PHP

Install the AWS SDK for PHP package:

$ composer require aws/aws-sdk-php

The following example shows how to connect to MinIO using the AWS SDK for PHP package:

<?php

// Include the SDK using the Composer autoloader
date_default_timezone_set('America/Los_Angeles');
require 'vendor/autoload.php';

$s3 = new Aws\S3\S3Client([
  'version' => 'latest',
  'region'  => 'us-east-1',
  'endpoint' => 'https://' . getenv('STACKHERO_MINIO_HOST'),
  'use_path_style_endpoint' => true,
  'credentials' => [
    'key'    => getenv('STACKHERO_MINIO_ACCESS_KEY'),
    'secret' => getenv('STACKHERO_MINIO_SECRET_KEY')
  ],
]);


// Send a PutObject request and get the result object.
$insert = $s3->putObject([
  'Bucket' => 'testbucket',
  'Key'    => 'testkey',
  'Body'   => 'Hello from MinIO!!'
]);

// Download the contents of the object.
$retrive = $s3->getObject([
  'Bucket' => 'testbucket',
  'Key'    => 'testkey',
  'SaveAs' => 'testkey_local'
]);

// Print the body of the result by indexing into the result object.
echo $retrive['Body'];

For more information about how to use the AWS SDK for PHP with MinIO, see the MinIO PHP documentation.

Connecting with Go

Connecting with the AWS SDK for Go

Install the AWS SDK package for GO:

$ go get github.com/aws/aws-sdk-go-v2
$ go get github.com/aws/aws-sdk-go-v2/config

The following example shows how to connect to MinIO using the AWS SDK for GO package:

package main

import (
  "fmt"
  "os"
  "strings"

  "github.com/aws/aws-sdk-go/aws"
  "github.com/aws/aws-sdk-go/aws/credentials"
  "github.com/aws/aws-sdk-go/service/s3/s3manager"
  "github.com/aws/aws-sdk-go/aws/session"
  "github.com/aws/aws-sdk-go/service/s3"
)

func main() {
  bucket := aws.String("newbucket")
  key := aws.String("testobject")

  // Configure to use MinIO Server
  s3Config := &aws.Config{
    Credentials:      credentials.NewStaticCredentials(os.Getenv("STACKHERO_MINIO_ACCESS_KEY"), os.Getenv("STACKHERO_MINIO_SECRET_KEY"), ""),
    Endpoint:         aws.String("https://" + os.Getenv("STACKHERO_MINIO_HOST")),
    Region:           aws.String("us-east-1"),
    DisableSSL:       aws.Bool(false),
    S3ForcePathStyle: aws.Bool(true),
  }
  newSession := session.New(s3Config)

  s3Client := s3.New(newSession)

  cparams := &s3.CreateBucketInput{
    Bucket: bucket, // Required
  }

  // Create a new bucket using the CreateBucket call.
  _, err := s3Client.CreateBucket(cparams)
  if err != nil {
    // Message from an error.
    fmt.Println(err.Error())
    return
  }

  // Upload a new object "testobject" with the string "Hello World!" to our "newbucket".
  _, err = s3Client.PutObject(&s3.PutObjectInput{
    Body:   strings.NewReader("Hello from MinIO!!"),
    Bucket: bucket,
    Key:    key,
  })
  if err != nil {
    fmt.Printf("Failed to upload data to %s/%s, %s\n", *bucket, *key, err.Error())
    return
  }
  fmt.Printf("Successfully created bucket %s and uploaded data with key %s\n", *bucket, *key)

  // Retrieve our "testobject" from our "newbucket" and store it locally in "testobject_local".
  file, err := os.Create("testobject_local")
  if err != nil {
    fmt.Println("Failed to create file", err)
    return
  }
  defer file.Close()

  downloader := s3manager.NewDownloader(newSession)
  numBytes, err := downloader.Download(file,
  &s3.GetObjectInput{
    Bucket: bucket,
    Key:    key,
  })
  if err != nil {
    fmt.Println("Failed to download file", err)
    return
  }
  fmt.Println("Downloaded file", file.Name(), numBytes, "bytes")
}

For more information about how to use the AWS SDK for Go with MinIO, see the MinIO GO documentation.

Connecting with the MinIO SDK

Install the MinIO SDK package for GO:

$ GO111MODULE=on go get github.com/minio/minio-go/v7

The following example shows how to connect to MinIO using the MinIO SDK:

package main

import (
  "log"

  "github.com/minio/minio-go/v7"
  "github.com/minio/minio-go/v7/pkg/credentials"
)

func main() {
  endpoint := os.Getenv("STACKHERO_MINIO_HOST")
  accessKeyID := os.Getenv("STACKHERO_MINIO_ACCESS_KEY")
  secretAccessKey := os.Getenv("STACKHERO_MINIO_SECRET_KEY")
  useSSL := true


  // Initialize minio client object.
  minioClient, err := minio.New(endpoint, &minio.Options{
    Creds:  credentials.NewStaticV4(accessKeyID, secretAccessKey, ""),
    Secure: useSSL,
  })
  if err != nil {
    log.Fatalln(err)
  }

  log.Printf("%#v\n", minioClient) // minioClient is now set up
}

For more information about using the MinIO Go SDK, see the MinIO GO SDK documentation.

Connecting with Node.js

Connecting with the AWS SDK for Node.js

Install the AWS SDK package for Node.js:

$ npm install --save aws-sdk

The following example shows how to connect to MinIO using the AWS SDK package for Node.js:

const process = require('process');
const AWS = require('aws-sdk');

const s3  = new AWS.S3({
  accessKeyId: process.env.STACKHERO_MINIO_ACCESS_KEY,
  secretAccessKey: process.env.STACKHERO_MINIO_SECRET_KEY,
  endpoint: 'https://' + process.env.STACKHERO_MINIO_HOST,
  s3ForcePathStyle: true, // needed with minio?
  signatureVersion: 'v4'
});

// putObject operation.
s3.putObject(
  { Bucket: 'testbucket', Key: 'testobject', Body: 'Hello from MinIO!!' },
  (err, data) => {
    if (err)
      console.log(err)
    else
      console.log('Successfully uploaded data to testbucket/testobject');
  }
);

// getObject operation.
const file = require('fs').createWriteStream('/tmp/mykey');
s3.getObject({ Bucket: 'testbucket', Key: 'testobject' })
  .on('httpData', chunk => file.write(chunk))
  .on('httpDone', () => file.end())
  .send();

For more information about how to use the AWS SDK for Node.js with MinIO, see the MinIO Node.js documentation.

Connecting with the MinIO SDK

Install the MinIO SDK package for Node.js:

$ npm install --save minio

The following example shows how to connect to MinIO with the MinIO SDK package for Node.js:

const process = require('process');
const Minio = require('minio');

// Instantiate the minio client with the endpoint
// and access keys as shown below.
const minioClient = new Minio.Client({
  endpoint: process.env.STACKHERO_MINIO_HOST,
  port: 443,
  useSSL: true,
  accessKeyId: process.env.STACKHERO_MINIO_ACCESS_KEY,
  secretAccessKey: process.env.STACKHERO_MINIO_SECRET_KEY,
});

// File that needs to be uploaded.
const file = '/tmp/photos-europe.tar'

// Make a bucket called europetrip.
minioClient.makeBucket(
  'europetrip',
  'us-east-1',
  err => {
    if (err) return console.log(err)

    console.log('Bucket created successfully in "us-east-1".')

    const metaData = {
      'Content-Type': 'application/octet-stream',
      'X-Amz-Meta-Testing': 1234,
      'example': 5678
    }
    // Using fPutObject API upload your file to the bucket europetrip.
    minioClient.fPutObject(
      'europetrip',
      'photos-europe.tar',
      file,
      metaData,
      (err, etag) => {
        if (err) return console.log(err)
        console.log('File uploaded successfully.')
      }
    );
  }
);

For more information about the MinIO Node.JS SDK, see the MinIO Node.js SDK documentation.

Accessing the Dashboard and MinIO Console Web UI

You can see your instance usage, restart your instance, and apply updates from the Stackhero dashboard. You can also access the MinIO console web UI.

Access the dashboard via the CLI:

$ heroku addons:open ah-s3-object-storage-stackhero
Opening ah-s3-object-storage-stackhero for sharp-mountain-4005

You can also visit the Heroku Dashboard, select your application, and then select Stackhero Object Storage from the Add-ons menu.

Upgrading Your Plan

You can’t downgrade an existing add-on.

Use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade ah-s3-object-storage-stackhero:newplan
-----> Upgrading ah-s3-object-storage-stackhero:newplan to sharp-mountain-4005... done
       Your plan has been updated to: ah-s3-object-storage-stackhero:newplan

Manage migration timing to ensure your application is available during the migration process.

Removing the Add-on

You can remove Stackhero Object Storage via the CLI:

This destroys all associated data and can’t be undone!

$ heroku addons:destroy ah-s3-object-storage-stackhero
-----> Removing ah-s3-object-storage-stackhero from sharp-mountain-4005... done

Support

Stackhero Object Storage support and runtime issues should be submitted via one of the Heroku Support channels. We recommend adding support@stackhero.io in copy for urgent issues.

Additional Resources

Feedback

Log in to submit feedback.