In May we released the first version of Heroku Connect, a service that makes it easy to build Heroku apps that share data with your Salesforce deployment.
Today we released our first major update to the service, bringing new speed and scale enhancements to all Heroku Connect users. Together, these enhancements lower latency on Heroku Connect synchronization, provide developers with more granular controls and improve insight into their Force.com API utilization.
Event Driven Synchronization from Force.com to Heroku Postgres
One of the top requests from the first Heroku Connect customers was to reduce the latency of synchronization between Force.com and Heroku Postgres. With this release, developers can take advantage of event driven synchronization, which is a new configuration option available on each object. Using the Force.com Streaming API, this mode proactively notifies Heroku Connect of any pending changes, rather than waiting for the service to poll Salesforce for updates. So instead of having to wait for a periodic poll to get new updates for Salesforce, the changes are sent in real time.
Access Mode by Object
With this release of Heroku Connect, developers can set the read-only / read-write status of their system on a per object basis. Previously this configuration setting was applied to all objects in a deployment. With the new fine grained access controls, companies can enjoy greater flexibility in how Postgres write operations are propagated back to Salesforce.
Fast writes from Heroku Postgres to Force.com
Heroku Connect now asynchronously batches updates from Heroku Postgres to Force.com every 10 seconds. This change greatly reduces latency of changes being sent from Heroku Postgres to Force.com, and in conjunction with event driven updates, makes bi-directional synchronization near real time.
New Polling Configuration Controls
The Salesforce polling interval is now configurable on a per object basis, from 2 to 60 minutes. Since developers can set the interval as low as every 2 minutes per mapping, "Fast Sync" mode is now removed. The main benefit of less frequent polling is lower API usage. Setting the sync frequency to 60 minutes is useful for controlling API utilization where latency is less important than management of API calls, particularly Bulk API calls, when there are different systems and processes updating large data sets within a given Salesforce deployment.
Estimated Force.com API Utilization and Capping
Heroku Connect now provides an estimated Force.com API utilization range per object based on specified poll frequency and estimated volume. This estimate is provided for planning purposes during configuration. In practice, Heroku Connect will now manage the total API utilization for each mapping to a maximum of 3000 calls per rolling 24 hour period.
Some customers with large data sets, for instance developers performing an initial sync of Salesforce tables containing more than 100 million records, or sustained update rates of >100K records per minute inside Heroku Postgres, may wish to have this constraint removed to further reduce latency. Please contact Support for assistance.
Additional performance and scale enhancements
With the release, additional performance and architecture improvements have enabled Heroku Connect to scale to virtually any sized Salesforce org or object; synchronizing hundreds of millions of rows on a single object is now easily available.
More Information
For an engineering perspective on event driven data sync, check out the post on the Heroku Engineering Blog. Heroku Connect documentation can be found in the Heroku Dev Center. Additional detail regarding the Streaming API/ PushTopics is available in the Force.com documentation found here.