For developers and businesses offering a web-based product, automated browser testing is a critical tool to ensure continuous delivery of a reliable service. Developers write browser tests by scripting actions against a real browser, simulating real usage by navigating, selecting, and making assertions about web pages and their document elements.
In this post, we introduce a new community buildpack that helps with automated browser testing. The new buildpack resolves installation reliability problems in the existing Chrome browser buildpacks for Heroku apps.
Browser Testing on Heroku
Developers can manually run browser tests on their machines to support writing and debugging tests. They can automate browser tests with continuous integration tools like Heroku CI to run in response to code updates and catch new problems on feature branches before they’re merged and released. They can also automate browser tests with a continuous end-to-end testing service. For example, running the test suite every hour to catch new problems with a customer-facing app.
At Heroku, we use automated browser testing to ensure the reliability of the Heroku Dashboard, our primary web interface. Continuous testing of the dashboard and related interfaces throughout their lifecycle, from feature development to monitoring the production system, is essential for early bug detection, quality assurance, and adaptability.
Heroku engineers found one long-standing issue regularly disrupts browser testing. Occasionally, automated Chrome browser tests all fail due to a version mismatch of the installed Chrome and Chromedriver components, like this example error message:
This version of ChromeDriver only supports Chrome version N
Current browser version is M
While it seems like the answer is to set a specific version number, Chrome is an evergreen browser. The browser continuously refreshes itself with security updates and features. Setting a specific version is discouraged because the browser quickly falls out of date.
Introducing A New Community Buildpack
To solve this cycle of version mismatches as Chrome updates itself, we created the Chrome for Testing Heroku Buildpack. We were able to release this buildpack because the Chrome development team addressed the long-standing problem of keeping Chrome and Chromedriver versions updated and aligned with each other for automated testing environments.
To use this new Chrome for Testing buildpack in Heroku apps, head over to the Heroku Elements Marketplace and install the Chrome for Testing Heroku Buildpack.
If the app is already using Chrome, make sure to remove existing Chrome and Chromedriver buildpacks before installing the new buildpack. To install Chrome for Testing on an app, add heroku-community/chrome-for-testing as the first buildpack:
heroku buildpacks:add -i 1 heroku-community/chrome-for-testing
By default, this buildpack downloads the latest Stable release, which Google provides. You can control the channel of the release by setting the app’s GOOGLE_CHROME_CHANNEL config variable to Stable, Beta, Dev, or Canary, and then deploy and build the app.
After the app deploys with the Chrome for Testing buildpack, chrome and chromedriver executables are installed on the PATH in dynos, available for browser automation tools like Selenium WebDriver and Puppeteer. We welcome feedback about this buildpack on its GitHub repository. Happy testing!