This add-on is operated by Crazy Ant Labs

Managed SFTP/FTPS cloud storage as a service with Amazon S3 and HTTP access.

Last updated March 05, 2023

SFTP To Go allows you to add fully-managed cloud SFTP storage to your Heroku applications. SFTP is a standard network protocol used for secure file access, transfer, and management.

Having a managed SFTP service enables you to participate in secure data exchange with partners and customers in a standard protocol, without purchasing and running your own SFTP servers and storage. SFTP To Go is based on AWS offerings, making it highly scalable and highly available with multi-AZ architecture.

With SFTP To Go, you can also use FTPS and S3 to access your files and share them with others.

Provisioning the add-on

SFTP To Go can be attached to a Heroku application via the CLI:

$ heroku addons:create sftptogo
-----> Adding sftptogo to sharp-mountain-4005... done, v18 (free)

After you provision SFTP To Go, the SFTPTOGO_URL config var is available in your app’s configuration. It contains the URL to your provisioned SFTP service, including the root username, password, and host. You can confirm this via the heroku config:get command:

$ heroku config:get SFTPTOGO_URL
sftp://user:pass@server.sftptogo.com

The root user has read/write access to the entirety of your SFTP service.

Once you provision the SFTP To Go add-on, there is nothing to set up. You and your application can immediately access your SFTP service.

Local setup

Environment setup

After you provision the add-on, you must locally replicate its configuration variables to ensure that your development environment can operate against the service.

Use the Heroku Local command-line tool to configure, run and manage process types specified in your app’s Procfile. Heroku Local reads configuration variables from a .env file. To view all of your app’s config vars, type heroku config. Use the following command for each value that you want to add to your .env file:

$ heroku config:get SFTPTOGO_URL -s  >> .env

For more information, see the Heroku Local article.

Configuring DNS for custom subdomain

After provisioning the add-on, you can add a DNS record to use your subdomain instead of SFTP To Go’s host name. To do so, copy your add-on host name from the dashboard and add a CNAME record to point to the host name. For example:

Consult your DNS provider’s documentation for specific instructions on creating CNAME records.

Using with Linux/Mac command line interface

Start an interactive SFTP session using the following command:

$ sftp user@host

You will then be prompted to enter your password.

Some of the simple commands you can use with SFTP:

Using with data integration platforms

Modern data integration platforms allow you to easily read data files from SFTP or write files into SFTP, ultimately supporting easy integration of data from a variety of sources for further analysis and BI processes.

For example, to connect from Integrate.io, a data integration add-on found in Heroku, to SFTP To Go, create an SFTP connection through Integrate.io. Go to SFTP To Go’s dashboard, copy the host, user and password, and paste them in Integrate.io’s new SFTP Connection page, according to the steps described here.

Using with graphical client tools

There are many available SFTP and FTPS GUI clients, such as Cyberduck, WinSCP, FileZilla, Transmit and CloudBerry. For example, to use Cyberduck with SFTP To Go, copy the value of your SFTPTOGO_URL config var to CyberDuck and click Quick Connect.

Using with Ruby

Ruby applications need to add the following entry into their Gemfile specifying the Net-SFTP client library:

gem 'net-sftp'

Update application dependencies with bundler:

$ bundle install

To download a file:

require 'net/sftp'
require 'uri'

sftptogo_url = ENV['SFTPTOGO_URL']

uri = URI.parse(sftptogo_url)

Net::SFTP.start(uri.host, uri.user, :password => uri.password) do |sftp|
  # download a file or directory from the remote host
  sftp.download!("/path/to/remote", "/path/to/local")
end

Using with Node

Node applications need to add the following entry into their package.json, specifying the dependency on the ssh2-sftp-client library:

{
  "name": "your-app-name",
  "dependencies": {
    "ssh2-sftp-client": "^5.2.2" ...
    }
}

You can also install it locally:

$ npm install ssh2-sftp-client

To list files:

let Client = require('ssh2-sftp-client');
let sftp = new Client();

sftp.connect({
  host: 'myserver.sftptogo.com',
  port: '22',
  username: 'username',
  password: '******'
}).then(() => {
  return sftp.list('/home');
}).then(data => {
  console.log(data, 'the data info');
}).catch(err => {
  console.log(err, 'catch error');
});

Using with Go

In your app’s go.mod file, add dependencies to the following:

require (
    golang.org/x/crypto/ssh
    golang.org/x/crypto/ssh/agent
    github.com/pkg/sftp
)

The following code shows how to connect to SFTP To Go (using the environment variable), list files, upload and download files.

package main

import (
    "fmt"
    "io"
    "os"
    "net"
    "net/url"
    "bufio"
    "strings"
    "path/filepath"

    "golang.org/x/crypto/ssh"
    "golang.org/x/crypto/ssh/agent"
    "github.com/pkg/sftp"
)

func main() {
    // Get SFTP To Go URL from environment
    rawurl := os.Getenv("SFTPTOGO_URL")

    parsedUrl, err := url.Parse(rawurl)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to parse SFTP To Go URL: %s\n", err)
        os.Exit(1)
    }

    // Get user name and pass
    user := parsedUrl.User.Username()
    pass, _ := parsedUrl.User.Password()

    // Parse Host and Port
    host := parsedUrl.Host
    // Default SFTP port
    port := 22

    hostKey := getHostKey(host)

    fmt.Fprintf(os.Stdout, "Connecting to %s ...\n", host)

    var auths []ssh.AuthMethod

    // Try to use $SSH_AUTH_SOCK which contains the path of the unix file socket that the sshd agent uses
    // for communication with other processes.
    if aconn, err := net.Dial("unix", os.Getenv("SSH_AUTH_SOCK")); err == nil {
        auths = append(auths, ssh.PublicKeysCallback(agent.NewClient(aconn).Signers))
    }

    // Use password authentication if provided
    if pass != "" {
        auths = append(auths, ssh.Password(pass))
    }

    // Initialize client configuration
    config := ssh.ClientConfig{
        User: user,
        Auth: auths,
        // Uncomment to ignore host key check
        //HostKeyCallback: ssh.InsecureIgnoreHostKey(),
        HostKeyCallback: ssh.FixedHostKey(hostKey),
    }

    addr := fmt.Sprintf("%s:%d", host, port)

    // Connect to server
    conn, err := ssh.Dial("tcp", addr, &config)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to connecto to [%s]: %v\n", addr, err)
        os.Exit(1)
    }

    defer conn.Close()

    // Create new SFTP client
    sc, err := sftp.NewClient(conn)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to start SFTP subsystem: %v\n", err)
        os.Exit(1)
    }
    defer sc.Close()

    //*
    //* List working directory files
    //*
    listFiles(*sc, ".")
    fmt.Fprintf(os.Stdout, "\n")

    //*
    //* Upload local file to remote file
    //*
    uploadFile(*sc, "./local.txt", "./remote.txt")
    fmt.Fprintf(os.Stdout, "\n")

    //*
    //* Download remote file to local file
    //*
    downloadFile(*sc, "./remote.txt", "./download.txt")
    fmt.Fprintf(os.Stdout, "\n")
}

func listFiles(sc sftp.Client, remoteDir string) (err error) {
    fmt.Fprintf(os.Stdout, "Listing [%s] ...\n\n", remoteDir)

    files, err := sc.ReadDir(remoteDir)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to list remote dir: %v\n", err)
        return
    }

    for _, f := range files {
        var name, modTime, size string

        name = f.Name()
        modTime = f.ModTime().Format("2006-01-02 15:04:05")
        size = fmt.Sprintf("%12d", f.Size())

        if f.IsDir() {
            name = name + "/"
            modTime = ""
            size = "PRE"
        }
        // Output each file name and size in bytes
        fmt.Fprintf(os.Stdout, "%19s %12s %s\n", modTime, size, name)
    }

    return
}


// Upload file to sftp server
func uploadFile(sc sftp.Client, localFile, remoteFile string) (err error) {
    fmt.Fprintf(os.Stdout, "Uploading [%s] to [%s] ...\n", localFile, remoteFile)

    srcFile, err := os.Open(localFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to open local file: %v\n", err)
        return
    }
    defer srcFile.Close()

    // Make remote directories recursion
    parent := filepath.Dir(remoteFile)
    path := string(filepath.Separator)
    dirs := strings.Split(parent, path)
    for _, dir := range dirs {
        path = filepath.Join(path, dir)
        sc.Mkdir(path)
    }

    // Note: SFTP To Go doesn't support O_RDWR mode
    dstFile, err := sc.OpenFile(remoteFile, (os.O_WRONLY|os.O_CREATE|os.O_TRUNC))
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to open remote file: %v\n", err)
        return
    }
    defer dstFile.Close()

    bytes, err := io.Copy(dstFile, srcFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to upload local file: %v\n", err)
        os.Exit(1)
    }
    fmt.Fprintf(os.Stdout, "%d bytes copied\n", bytes)

    return
}

// Download file from sftp server
func downloadFile(sc sftp.Client, remoteFile, localFile string) (err error) {

    fmt.Fprintf(os.Stdout, "Downloading [%s] to [%s] ...\n", remoteFile, localFile)
    // Note: SFTP To Go doesn't support O_RDWR mode
    srcFile, err := sc.OpenFile(remoteFile, (os.O_RDONLY))
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to open remote file: %v\n", err)
        return
    }
    defer srcFile.Close()

    dstFile, err := os.Create(localFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to open local file: %v\n", err)
        return
    }
    defer dstFile.Close()

    bytes, err := io.Copy(dstFile, srcFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to download remote file: %v\n", err)
        os.Exit(1)
    }
    fmt.Fprintf(os.Stdout, "%d bytes copied\n", bytes)

    return
}

// Get host key from local known hosts
func getHostKey(host string) ssh.PublicKey {
    // parse OpenSSH known_hosts file
    // ssh or use ssh-keyscan to get initial key
    file, err := os.Open(filepath.Join(os.Getenv("HOME"), ".ssh", "known_hosts"))
    if err != nil {
        fmt.Fprintf(os.Stderr, "Unable to read known_hosts file: %v\n", err)
        os.Exit(1)
    }
    defer file.Close()

    scanner := bufio.NewScanner(file)
    var hostKey ssh.PublicKey
    for scanner.Scan() {
        fields := strings.Split(scanner.Text(), " ")
        if len(fields) != 3 {
            continue
        }
        if strings.Contains(fields[0], host) {
            var err error
            hostKey, _, _, _, err = ssh.ParseAuthorizedKey(scanner.Bytes())
            if err != nil {
                fmt.Fprintf(os.Stderr, "Error parsing %q: %v\n", fields[2], err)
                os.Exit(1)
            }
            break
        }
    }

    if hostKey == nil {
        fmt.Fprintf(os.Stderr, "No hostkey found for %s", host)
        os.Exit(1)
    }

    return hostKey
}

Using with other languages

SFTP To Go can generate code snippets for you in various other languages as well. To get a code snippet, click the ... menu button for one of your credentials and then click <> code snippets.

Amazon S3 access

In addition to the SFTP protocol, SFTP To Go allows you to use S3 APIs to access the same data. This allows you to share data with some users or applications using SFTP and with others, using S3, out of the box. When the add-on is provisioned, the following environment variables are added to your app to allow programmatic access using S3 APIs:

• SFTPTOGO_AWS_ACCESS_KEY_ID - IAM access key
• SFTPTOGO_AWS_SECRET_ACCESS_KEY - IAM secret
• SFTPTOGO_AWS_BUCKET - bucket name
• SFTPTOGO_AWS_REGION - AWS region

You can also use the values of these environment variables to access your data using Amazon S3 CLI and GUI clients.

Note that the values (especially access key and secret) may change, so please refer to the environment variables only

Using with the AWS CLI

If you have Amazon S3 access, you can also use the AWS CLI to interact with your files.

After you install the AWS CLI, you can configure it with the SFTPTOGO_AWS_* settings in your app or through your dashboard:

$ aws configure
AWS Access Key ID [None]: <$SFTPTOGO_AWS_ACCESS_KEY_ID>
AWS Secret Access Key [None]: <$SFTPTOGO_AWS_SECRET_ACCESS_KEY>
Default region name [None]: <$SFTPTOGO_AWS_REGION>

The following table lists the most popular commands used when activating the AWS CLI

Dashboard

Use The SFTP To Go dashboard to:

1. Access your SFTP server information (host, user name, and password),
2. Rotate your password. This generates a new password and changes the URI in the SFTPTOGO_URL config var.

1. Get storage and bandwidth metrics to monitor your add-on usage.
2. Add/remove/modify/activate/deactivate credentials.

You can access the dashboard via the CLI:

$ heroku addons:open sftptogo
Opening sftptogo for sharp-mountain-4005

or by visiting the Heroku Dashboard and selecting the application in question. Select SFTP To Go from the Add-ons menu.

REST API and API keys

When SFTP To Go is provisioned, the SFTPTOGO_API_KEY variable is added to your app’s environment variables.

SFTP To Go’s APIs allow you to create, modify and delete objects such as credentials, webhooks and inbound network rules automatically. For more details, see our API reference.

Migrating between plans

You can either use the UI to upgrade or downgrade SFTP To Go, or use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade sftptogo:newplan
-----> Upgrading sftptogo:[[newplan]] to sharp-mountain-4005... done, v18 ($50/mo)
       Your plan has been updated to: sftptogo:newplan

Removing the add-on

You can remove SFTP To Go via the CLI:

$ heroku addons:destroy sftptogo
-----> Removing sftptogo from sharp-mountain-4005... done, v20 (free)

Before removing SFTP To Go, you can download your data using your favorite sftp client.

Security

• The data you store on SFTP To Go is encrypted in transit and at rest on the server side.
• Our physical infrastructure is hosted on Amazon Web Services’ (AWS) data centers.
• AWS Security Groups (virtual firewalls) are utilized to restrict access to our network from external networks and between systems internally.
• Backend access to the OS level and AWS console is limited to Crazy Ant Labs staff and requires key authentication and multi factor authentication.

To learn more about SFTP To Go’s security you can visit our full documentation.

Support

All SFTP To Go support and runtime issues should be submitted via one of the Heroku Support channels. Any non-support related issues or product feedback is welcome at hello@sftptogo.com.

Additional Resources

Check out SFTP To Go’s full documentation to learn about many additional features, such as:

• Credentials and permissions
• Management APIs
• Authentication Methods
• Inbound network rules
• Webhook events
• SFTP Connection via Static IP addresses

Known issues and remedies

• Troubleshooting slow connection time
• How to handle FileZilla connection timeout to SFTP To Go
• Uploaded files are corrupted or empty
• Supported and unsupported file operations in SFTP
• Known limitations with FTPS