Text-based communication has a long history weaved into the evolution of the Internet, from IRC and XMPP to Slack and Discord. And where there have been humans, there have also been chatbots: scriptable programs that respond to a user’s commands, like messages in a chat room.
Chatbots don't require much in terms of computational power or disk storage, as they rely heavily on APIs to send actions and receive responses. But as with any kind of software, scaling them to support millions of user’s requests across the world requires a fail-safe operational strategy. Salesforce offers a Live Agent support product with a chatbot integration that reacts to customer inquiries.
In this post, we'll take a look at how the team uses Heroku for their chatbot's multi-regional requirements.
How users interact with the chatbot
Live Agent is an embeddable chatbot that can be added to any website or mobile app. Users can engage in a conversation with the chatbot, asking questions and performing actions along the way. For example, if a bank customer wants to learn how to set up two-factor authentication, they could ask the chatbot for guidance, rather than call the bank directly.
The aim of Live Agent is to augment a human support agent's capabilities for responding to events that happen at a high scale. Because everybody learns and interacts a little bit differently, it's advantageous to provide help through various mediums, like videos and documentation. Chatbots offer another channel, with guided feedback that offers more interactive information. Rather than providing a series of webpages with static images, a chatbot can make processes friendlier by confirming to users their progress as they go through a sequence of steps.
Live Agent hooks into Apex, a Java-like programming language that is tied directly into Salesforce's object models, allowing it to modify and call up CRM records directly. You can also have a Live Agent chatbot call out to any API and pretty much do anything on the web.
With their open-ended nature, chatbots can perform endless operations across a variety of communication platforms. Facebook Messenger, for example, is the third most popular app in the world, and you could have a Live Agent backend running on the Messenger platform to respond to user queries.
Running Live Agent on Heroku
With such a large scope across disparate mediums, there's a significant number of requests coming into Live Agent chatbots and vast amounts of data they can access. It may surprise you to learn that there are only eight engineers responsible for running Live Agent! In addition to coding the features, they own the entire product. This means that they are also responsible for being on-call for pager rotations and ensuring that the chatbots can keep up with incoming traffic.
The small team didn't want to waste time configuring their platform to run on bare metal or on a cloud VM, and they didn't want the administrative overhead of managing databases or other third-party services. Since Salesforce customers reside all over the world, the Live Agent chatbots must also be highly available across multiple regions.
The Live Agent team put its trust into Heroku to take care of all of those operational burdens. Heroku already manages millions of Postgres databases for our customers, and we have a dedicated staff to manage backups, perform updates, and respond to potential outages. The Live Agent chatbot runs on Java, and Heroku's platform supports the entire Java ecosystem, with dedicated Java experts to handle language and framework updates, providing new features and responding to security issues.
In order to serve their customers worldwide, the core Live Agent infrastructure matches Heroku's availability in every region around the world. All of their services are managed by Heroku, ensuring that their Heroku Postgres, Redis, and Apache Kafka dependencies are blazing fast no matter where a request comes from.
The beauty of it all is how simple it is to scale, without any of Live Agent's team needing to be responsible for any of the maintenance and upkeep.
Leveraging Terraform for replication and Private Spaces for security
The Live Agent platform is comprised of ten separate apps, each with their own managed add-ons and services. To fully isolate the boundaries of communication, the collection of apps are deployed into a Heroku Private Space. Private Spaces establish an isolated runtime for the apps to ensure that the data contained within the network is inaccessible from any outside service.
Private Spaces are available in a variety of regions; if a new region becomes available, the Live Agent team wanted to be able to automatically redeploy the same apps and add-ons there. And if they ever need to create a new app, they also wanted to add it to all of the Private Spaces in those geographic locations.
To easily replicate their architecture, the Live Agent team uses Terraform to automate deployment and configuration of the Live Agent platform. Terraform is the driver behind everything they do on Heroku. With it, they can explicitly and programmatically define their infrastructure--the apps and add-ons, custom domains, and logging and profiling setup--and have it securely available in any region, instantly. Whenever a new configuration is necessary, they can implement that update with just a few lines of code and make it live everywhere with the merge of a pull request.
For example, to automatically set up a Node.js Heroku app that requires a Postgres database and logging through Papertrail, a Terraform config file might just look something like this:
resource "heroku_app" "server" {
name = "my-app"
region = "us"
provisioner "local-exec" {
command = "heroku buildpacks:set heroku/nodejs --app ${heroku_app.server.name}"
}
}
resource "heroku_addon" "database" {
app = "${heroku_app.server.name}"
plan = "heroku-postgresql:hobby-dev"
}
# Papertrail addon (for logging)
resource "heroku_addon" "logging" {
app = "${heroku_app.server.name}"
plan = "papertrail:choklad"
}
Here are some details on how to use Terraform with Heroku.
Learning more
If you'd like to learn more about how Live Agent uses Heroku to scale their platform, our podcast Code[ish], has an interview with their team, where they dive into more of the technical specifics.
We also have not one but two posts on dev.to listing all the DevOps chores which Heroku automatically takes care of for you.