Skip Navigation
Show nav
Heroku Dev Center
  • Get Started
  • Documentation
  • Changelog
  • Search
  • Get Started
    • Node.js
    • Ruby on Rails
    • Ruby
    • Python
    • Java
    • PHP
    • Go
    • Scala
    • Clojure
  • Documentation
  • Changelog
  • More
    Additional Resources
    • Home
    • Elements
    • Products
    • Pricing
    • Careers
    • Help
    • Status
    • Events
    • Podcasts
    • Compliance Center
    Heroku Blog

    Heroku Blog

    Find out what's new with Heroku on our blog.

    Visit Blog
  • Log inorSign up
View categories

Categories

  • Heroku Architecture
    • Dynos (app containers)
    • Stacks (operating system images)
    • Networking & DNS
    • Platform Policies
    • Platform Principles
  • Command Line
  • Deployment
    • Deploying with Git
    • Deploying with Docker
    • Deployment Integrations
  • Continuous Delivery
    • Continuous Integration
  • Language Support
    • Node.js
    • Ruby
      • Working with Bundler
      • Rails Support
    • Python
      • Background Jobs in Python
      • Working with Django
    • Java
      • Working with Maven
      • Java Database Operations
      • Working with the Play Framework
      • Working with Spring Boot
      • Java Advanced Topics
    • PHP
    • Go
      • Go Dependency Management
    • Scala
    • Clojure
  • Databases & Data Management
    • Heroku Postgres
      • Postgres Basics
      • Postgres Getting Started
      • Postgres Performance
      • Postgres Data Transfer & Preservation
      • Postgres Availability
      • Postgres Special Topics
    • Heroku Redis
    • Apache Kafka on Heroku
    • Other Data Stores
  • Monitoring & Metrics
    • Logging
  • App Performance
  • Add-ons
    • All Add-ons
  • Collaboration
  • Security
    • App Security
    • Identities & Authentication
    • Compliance
  • Heroku Enterprise
    • Private Spaces
      • Infrastructure Networking
    • Enterprise Accounts
    • Enterprise Teams
    • Heroku Connect (Salesforce sync)
      • Heroku Connect Administration
      • Heroku Connect Reference
      • Heroku Connect Troubleshooting
    • Single Sign-on (SSO)
  • Patterns & Best Practices
  • Extending Heroku
    • Platform API
    • App Webhooks
    • Heroku Labs
    • Building Add-ons
      • Add-on Development Tasks
      • Add-on APIs
      • Add-on Guidelines & Requirements
    • Building CLI Plugins
    • Developing Buildpacks
    • Dev Center
  • Accounts & Billing
  • Troubleshooting & Support
  • Integrating with Salesforce
  • Heroku Enterprise
  • Heroku Connect (Salesforce sync)
  • Reading Data from Salesforce with Heroku Connect

Reading Data from Salesforce with Heroku Connect

English — 日本語に切り替える

Last updated May 16, 2022

Table of Contents

  • SOAP API
  • Bulk API
  • Streaming API
  • API Call Usage
  • Resolve Read Errors
  • Resolve Synchronization Issues
  • Diagnose Perceived Performance Problems

Heroku Connect automatically chooses the most efficient method to transfer data between your database and your Salesforce organization by employing a set of best practices that take into account data change volume and the many subtle details of Salesforce API operation. This is true for both reading from and writing to Salesforce.

Salesforce offers a variety of APIs that are best suited to specific integration requirements and Heroku Connect utilizes two of them for data transfer: SOAP and Bulk. In addition the Streaming API is used to consume Push Topic notifications for mappings that use accelerated polling.

In this article we’ll discuss how Heroku Connect uses these APIs to sync (read) data from Salesforce, as well as how to address common read errors.

Heroku Connect can only be used with Salesforce editions that have API access. Some plan types, including trial versions, do not have API access by default and cannot be used with Heroku Connect.

SOAP API

Heroku Connect primarily uses the SOAP API for interactions with your Salesforce organization. The API has been optimized for operations involving a small number of records and is therefore used by Heroku Connect for the following tasks when there are up to 10,000 records to be processed:

  • initially loading data from your Salesforce organization for a new mapping
  • reloading all data into an existing mapping
  • reading changes from your Salesforce organization

The SOAP API is also used for administrative tasks such as counting records and querying for mapping fields.

Lastly, the SOAP API is also used when writing data to Salesforce, though the threshold of records to be processed is lower. You can read more about this here.

SOAP API calls made by Heroku Connect do not count towards your API request limits. They are counted in your Salesforce org, but they do not count towards API license utilization.

Bulk API

The Bulk API is optimized for loading large sets of data making use of asynchronous processing to retrieve batches of records. Heroku Connect makes use of the Bulk API for the following tasks when there are more than 10,000 records to be read from Salesforce:

  • initially loading data from your Salesforce organization for a new mapping
  • reloading all data into an existing mapping
  • reading changes from your Salesforce organization

The Bulk API can also be used when writing changes to Salesforce on read-write mappings, although the threshold of records to be processed is lower. You can read more about how the Bulk API is used when writing data to Salesforce here.

Bulk API calls do not count towards your Bulk API limits. Bulk data load jobs created by Heroku Connect will appear in your list of jobs in Salesforce, but will not increment your count of batches processed in the last 24 hours.

Streaming API

When you enable accelerated polling for a mapping, Heroku Connect uses the Streaming API to essentially listen for changes in Salesforce, rather than having to wait for changes to be seen in the next polling period. It does so by creating a Push Topic that will send notifications to trigger polling when data is changed in Salesforce.

Push Topics created by Heroku Connect have a hc_ naming prefix, for example hc_123, and will automatically be deleted either when you switch to polling mode or delete the mapping.

 

Streaming API calls do not count towards the daily Streaming API event limit that applies to your Salesforce organization, provided that your connection has been authorized on April 5, 2017 or later.

The following API limit needs to be considered when configuring a mapping to use accelerated polling:

  • Push Topics created by Heroku Connect count towards the ‘Maximum number of topics (PushTopic records) per organization’.

Heroku Connect’s Streaming API event consumption will not be included when you check your current Streaming API event usage in Salesforce:

  • Log in to your Salesforce organization.
  • Go to Setup and type Company Information in the Quick Find box.
  • Click on Company Information.
  • Streaming API Events, Last 24 Hours shows the number of notifications received and the maximum allowed. It does not include notifications received by Heroku Connect.

API Call Usage

The number of API calls used by Heroku Connect will depend on a number of factors:

  • the total number of mappings
  • the directionality of each mapping (read only or read/write)
  • the synchronization mode of each mapping (polling or streaming) (see synchronization mode)

Depending on the API used and the directionality of the mapping, Heroku Connect will batch changes differently.

For changes to records being read from Salesforce, Heroku Connect will use:

  • the SOAP API in batches of up to 2000 records OR
  • the Bulk API in batches of up to 150,000 records

For changes to records being written to Salesforce, Heroku Connect will use:

  • the SOAP API in batches of up to 200 records OR
  • the Bulk API in batches of 2,000 to 10,000 records

None of these calls will count towards your daily SOAP or Bulk API limits.

Resolve Read Errors

Read errors can occur from either insufficient view permissions on the objects being mapped, unsupported object types, or problems with an object’s attributes.

View Permissions

Heroku Connect requires “View All” permissions, which is scoped at the individual object level, for optimal operation. In certain cases where an object doesn’t have the option to set “View All” permissions, “View All Data” can be used instead. For Salesforce organizations with large record sets, some operations may not be possible without those permissions. The system will notify members or collaborators on your application if an operation requiring those permissions fails without them.

Unsupported Objects

Heroku Connect supports read of all standard and custom objects. You can find a full list of supported objects here Knowledge Base objects, and standard or custom objects requiring special API handling or without required attributes such as SystemModStamp are not mappable. Heroku Connect attempts to detect and suppress these unmappable objects from presentation in the UI.

Common Read Errors

  • INVALID_TYPE_FOR_OPERATION... : This or a similar error will be displayed in the logs when Heroku Connect cannot query for a mapped object.

  • MALFORMED_QUERY... : This or a similar error will be displayed in the logs when Heroku Connect cannot map an object because it requires object specific filters.

  • Cannot poll changes for table '>fieldname<' because it contains neither... : This or a similar error will be displayed in the logs when Heroku Connect cannot map an object because it lacks a required attribute.

  • Skipped fields on >mapping name< not present in Salesforce: >fieldname<, >fieldname<... : This error will display in the logs when Heroku Connect cannot import a field in an exported configuration.

Remove the ineligible object or field from the mapping to resolve these errors.

Resolve Synchronization Issues

There are a number of issues that mappings on a connection can encounter that require user intervention. Check the state of your mapping in the Heroku Connect dashboard, CLI, or API and see Mapping States Reference for more info.

Diagnose Perceived Performance Problems

A number of factors can affect sync performance with Heroku Connect. Follow the steps in Diagnosing Heroku Connect Performance Issues to diagnose perceived read performance issues.

Keep reading

  • Heroku Connect (Salesforce sync)

Feedback

Log in to submit feedback.

Writing Data to Salesforce with Heroku Connect Writing Data to Salesforce with Heroku Connect

Information & Support

  • Getting Started
  • Documentation
  • Changelog
  • Compliance Center
  • Training & Education
  • Blog
  • Podcasts
  • Support Channels
  • Status

Language Reference

  • Node.js
  • Ruby
  • Java
  • PHP
  • Python
  • Go
  • Scala
  • Clojure

Other Resources

  • Careers
  • Elements
  • Products
  • Pricing

Subscribe to our monthly newsletter

Your email address:

  • RSS
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku Blog
    • Heroku News Blog
    • Heroku Engineering Blog
  • Heroku Podcasts
  • Twitter
    • Dev Center Articles
    • Dev Center Changelog
    • Heroku
    • Heroku Status
  • Facebook
  • Instagram
  • Github
  • LinkedIn
  • YouTube
Heroku is acompany

 © Salesforce.com

  • heroku.com
  • Terms of Service
  • Privacy
  • Cookies
  • Cookie Preferences