When building web applications, unit testing your individual components is certainly important. However, end-to-end testing provides assurance that the final user experience of your components chained together matches the expected behavior. Testing web application behavior locally in your browser can be helpful, but this approach isn’t efficient or reliable, especially as your application grows more complex.
Ideally, end-to-end tests in your browser are automated and integrated into your CI pipeline. Every time you commit a code change, your tests will run. Passing tests gives you the confidence that the application — as your end users experience it — behaves as expected.
With Heroku CI, you can run end-to-end tests with headless Chrome. The Chrome for Testing Heroku Buildpack installs Google Chrome Browser (chrome) and chromedriver in a Heroku app. You can learn more about this Heroku Buildpack in a recent post.
In this article, we’ll walk through the simple steps for using this Heroku Buildpack to perform basic end-to-end testing for a React application in Heroku CI.
Brief Introduction to our React App
Since this is a simple walkthrough, we’ve built a very simple React application, consisting of a single page with a link and a form. The form has a text input and a submit button. When the user enters their name in the text input and submits the form, the page displays a simple greeting with the name included.
It looks like this:
Super simple, right? What we want to focus on, however, are end-to-end tests that validate the end-user experience for the application. To test our application, we use Jest (a popular JavaScript testing framework) and Puppeteer (a library for running headless browser testing in either Chrome or Firefox).
If you want to download the simple source code and tests for this application, you can check out this GitHub repository.
The code for this simple page is in src/App.js:
import React, { useState } from 'react';
import { Container, Box, TextField, Button, Typography, Link } from '@mui/material';
function App() {
const [name, setName] = useState('');
const [greeting, setGreeting] = useState('');
const handleSubmit = (e) => {
e.preventDefault();
setGreeting(`Nice to meet you, ${name}!`);
};
return (
<Container maxWidth="sm" style={{ marginTop: '50px' }}>
<Box textAlign="center">
<Typography variant="h4" gutterBottom>
Welcome to the Greeting App
</Typography>
<Link href="https://pptr.dev/" rel="noopener">
Puppeteer Documentation
</Link>
<Box component="form" onSubmit={handleSubmit} mt={3}>
<TextField
name="name"
label="What is your name?"
variant="outlined"
fullWidth
value={name}
onChange={(e) => setName(e.target.value)}
margin="normal"
/>
<Button variant="contained" color="primary" type="submit" fullWidth>
Say hello to me
</Button>
</Box>
{greeting && (
<Typography id="greeting" variant="h5" mt={3}>
{greeting}
</Typography>
)}
</Box>
</Container>
);
}
export default App;
Running In-Browser End-to-End Tests Locally
Our simple set of tests is in a file called src/tests/puppeteer.test.js. The file contents look like this:
const ROOT_URL = 'http://localhost:8080';
describe('Page tests', () => {
const inputSelector = 'input[name="name"]';
const submitButtonSelector = 'button[type="submit"]';
const greetingSelector = 'h5#greeting';
const name = 'John Doe';
beforeEach(async () => {
await page.goto(ROOT_URL);
});
describe('Puppeteer link', () => {
it('should navigate to Puppeteer documentation page', async () => {
await page.click('a[href="https://pptr.dev/"]');
await expect(page.title()).resolves.toMatch('Puppeteer | Puppeteer');
});
});
describe('Text input', () => {
it('should display the entered text in the text input', async () => {
await page.type(inputSelector, name);
// Verify the input value
const inputValue = await page.$eval(inputSelector, el => el.value);
expect(inputValue).toBe(name);
});
});
describe('Form submission', () => {
it('should display the "Hello, X" message after form submission', async () => {
const expectedGreeting = `Hello, ${name}.`;
await page.type(inputSelector, name);
await page.click(submitButtonSelector);
await page.waitForSelector(greetingSelector);
const greetingText = await page.$eval(greetingSelector, el => el.textContent);
expect(greetingText).toBe(expectedGreeting);
});
});
});
Let’s highlight a few things from our testing code above:
- We’ve told Puppeteer to expect an instance of the React application to be up and running at
http://localhost:8080. For each test in our suite, we direct the Puppeteerpageto visit that URL. - We test the link at the top of our page, ensuring that a link click redirects the browser to the correct external page (in this case, the Puppeteer Documentation page).
- We test the text input, verifying that a value entered into the field is retained as the input value.
- We test the form submission, verifying that the correct greeting is displayed after the user submits the form with a value in the text input.
The tests are simple, but they are enough to demonstrate how headless in-browser testing ought to work.
Minor modifications to package.json
We bootstrapped this app by using Create React App. However, we made some modifications to our package.json file just to make our development and testing process smoother. First, we modified the start script to look like this:
"start": "PORT=8080 BROWSER=none react-scripts start"
Notice that we specified the port that we want our React application to run on (8080) We also set BROWSER=none, to prevent the opening of a browser with our application every time we run this script. We won’t need this, especially as we move to headless testing in a CI pipeline.
We also have our test script, which simply runs jest:
"test": "jest"
Start up the server and run tests
Let’s spin up our server and run our tests. In one terminal, we start the server:
~/project$ npm run start
Compiled successfully!
You can now view project in the browser.
Local: http://localhost:8080
On Your Network: http://192.168.86.203:8080
Note that the development build is not optimized.
To create a production build, use npm run build.
webpack compiled successfully
With our React application running and available at http://localhost:8080, we run our end-to-end tests in a separate terminal:
~/project$ npm run test
FAIL src/tests/puppeteer.test.js
Page tests
Puppeteer link
✓ should navigate to Puppeteer documentation page (473 ms)
Text input
✓ should display the entered text in the text input (268 ms)
Form submission
✕ should display the "Hello, X" message after form submission (139 ms)
● Page tests › Form submission › should display the "Hello, X" message after form submission
expect(received).toBe(expected) // Object.is equality
Expected: "Hello, John Doe."
Received: "Nice to meet you, John Doe!"
36 | await page.waitForSelector(greetingSelector);
37 | const greetingText = await page.$eval(greetingSelector, el => el.textContent);
> 38 | expect(greetingText).toBe(expectedGreeting);
| ^
39 | });
40 | });
41 | });
at Object.toBe (src/tests/puppeteer.test.js:38:28)
Test Suites: 1 failed, 1 total
Tests: 1 failed, 2 passed, 3 total
Snapshots: 0 total
Time: 1.385 s, estimated 2 s
Ran all test suites.
And… we have a failing test. It looks like our greeting message is wrong. We fix our code in App.js and then run our tests again.
~/project$ npm run test
> project@0.1.0 test
> jest
PASS src/tests/puppeteer.test.js
Page tests
Puppeteer link
✓ should navigate to Puppeteer documentation page (567 ms)
Text input
✓ should display the entered text in the text input (260 ms)
Form submission
✓ should display the "Hello, X" message after form submission (153 ms)
Test Suites: 1 passed, 1 total
Tests: 3 passed, 3 total
Snapshots: 0 total
Time: 1.425 s, estimated 2 s
Ran all test suites.
Combine server startup and test execution
We’ve fixed our code, and our tests are passing. However, starting up the server and running tests should be a single process, especially as we intend to run this in a CI pipeline. To serialize these two steps, we’ll use the start-server-and-test package. With this package, we can use a single script command to start our server, wait for the URL to be ready, and then run our tests. Then, when the test run finishes, it stops the server.
We install the package and then add a new line to the scripts in our package.json file:
"test:ci": "start-server-and-test start http://localhost:8080 test"
Now, running npm run test:ci invokes the start-server-and-test package to first start up the server by running the start script, waiting for http://localhost:8080 to be available, and then running the test script.
Here is what it looks like to run this command in a single terminal window:
~/project$ npm run test:ci
> project@0.1.0 test:ci
> start-server-and-test start http://localhost:8080 test
1: starting server using command "npm run start"
and when url "[ 'http://localhost:8080' ]" is responding with HTTP status code 200 running tests using command "npm run test"
> project@0.1.0 start
> PORT=8080 BROWSER=none react-scripts start
Starting the development server...
Compiled successfully!
You can now view project in the browser.
Local: http://localhost:8080
On Your Network: http://172.16.35.18:8080
Note that the development build is not optimized.
To create a production build, use npm run build.
webpack compiled successfully
> project@0.1.0 test
> jest
PASS src/tests/puppeteer.test.js
Page tests
Puppeteer link
✓ should navigate to Puppeteer documentation page (1461 ms)
Text input
✓ should display the entered text in the text input (725 ms)
Form submission
✓ should display the "Hello, X" message after form submission (441 ms)
Test Suites: 1 passed, 1 total
Tests: 3 passed, 3 total
Snapshots: 0 total
Time: 4.66 s
Ran all test suites.
Now, our streamlined testing process runs with a single command. We’re ready to try our headless browser testing with Heroku CI.
Running Our Tests in Heroku CI
Getting our testing process up and running in Heroku CI requires only a few simple steps.
Add app.json file
We need to add a file to our code repository. The file, app.json, is in our project root folder. It looks like this:
{
"environments": {
"test": {
"buildpacks": [
{ "url": "heroku-community/chrome-for-testing" },
{ "url": "heroku/nodejs" }
],
"scripts": {
"test": "npm run test:ci"
}
}
}
}
In this file, we specify the buildpacks that we will need for our project. We make sure to add the Chrome for Testing buildpack and the Node.js buildpack. Then, we specify what we want Heroku’s execution of a test script command to do. In our case, we want Heroku to run the test:ci script we’ve defined in our package.json file.
Create a Heroku pipeline
In the Heroku dashboard, we click New ⇾ Create new pipeline.
We give our pipeline a name, and then we search for and select the GitHub repository that will be associated with our pipeline. You can fork our demo repo, and then use your fork for your pipeline.
After finding our GitHub repo, we click Connect and then Create pipeline.
Add an app to the pipeline
Next, we need to add an app to our pipeline. We’ll add it to the Staging phase of our pipeline.
We click Create new app…
This app will use the GitHub repo that we’ve already connected to our pipeline. We choose a name and region for our app and then click Create app.
With our Heroku app added to our pipeline, we’re ready to work with Heroku CI.
Enable Heroku CI
In our pipeline page navigation, we click Tests.
Then, we click Enable Heroku CI.
Just like that, Heroku CI is up and running.
- We’ve created our Heroku pipeline.
- We’ve connected our GitHub repo.
- We’ve created our Heroku app.
- We’ve enabled Heroku CI.
- We have an `app.json` file that specifies our need for the Chrome for Testing and Node.js buildpacks, and tells Heroku what to do when executing the `test` script.
That’s everything. It’s time to run some tests!
Run tests (manual trigger)
On the Tests page for our Heroku pipeline, we click the New Test ⇾ Start Test Run to manually trigger a run of our test suite.
As Heroku displays the output for this test run, we see immediately that it has detected our need for the Chrome for Testing buildpack and begins installing Chrome and all its dependencies.
After Heroku installs our application dependencies and builds the project, it executes npm run test:ci. This runs start-server-and-test to spin up our React application and then run our Jest/Puppeteer tests.
Success! Our end-to-end tests run, using headless Chrome via the Chrome for Testing Heroku Buildpack.
By integrating end-to-end tests in our Heroku CI pipeline, any push to our GitHub repo will trigger a run of our test suite. We have immediate feedback in case any end-to-end tests fail, and we can configure our pipeline further to use review apps or promote staging apps to production.
Conclusion
As the end-to-end testing in your web applications grows more complex, you’ll increasingly rely on headless browser testing that runs automatically as a part of your CI pipeline. Manually running tests is neither reliable nor scalable. Every developer on the team needs a singular, central place to run the suite of end-to-end tests. Automating these tests in Heroku CI is the way to go, and your testing capabilities just got a boost with the Chrome for Testing Buildpack.
When you’re ready to start running your apps on Heroku and taking advantage of Heroku CI, sign up today.