End-to-end testing - Automating user actions in the browser

Posted Jan 21, 2021



One of the simplest ways to test a web app is to open it in a browser, try all the features and see if anything doesn't work.

Wouldn't it be great if you could do this automatically?

Well, that's the idea behind end-to-end testing, commonly abbreviated as "E2E".

How it works #

When you run an end-to-end test, the test runner will take over your browser. It will then programmatically navigate to your app, click buttons, fill out form fields, and so on.

It will then check the page state to decide if the test passed or failed.

Since this simulates the way a real user would experience the app in production it provides highly relevant feedback on the app integrity.

E2E testing vs unit testing #

The concept of unit testing is that we test atomic parts of the app in isolation, which could mean testing individual functions, classes, or components.

While this sort of testing is definitely useful, there are certain kinds of bugs that it may miss.

The concept of end-to-end testing is to test your running app to ensure the individual pieces work together.

This means that end-to-end testing may uncover bugs relating to your network connectivity, database setup, and third party integrations - things that unit tests may miss.

How to write an end-to-end test #

End-to-end tests are generally designed around the main use cases of your app. For example, let's say you had an eCommerce site. What would be some useful tests?

In this case, we'd probably want to know:

  • Can a user search for an item to buy?
  • Can they add that item to the shopping cart?
  • Can they checkout and pay?

These are highly important use cases for an eCommerce app, so we'd probably want end-to-end tests for each one.

Let's now look at another example, and this time we'll go into detail and actually see some code.

Example E2E test #

We're now going to think about another very common web app use case that would deserve an end-to-end test: does the site's sign-up form work?

Once you've identified the use case you want to test, you should now break it down into its constituent steps.

So, how does a user fill out a form on a typical site?

  • Firstly, they'd open the website in their browser and navigate to the form page
  • Then they'd select and fill out the name field
  • Then the email fields
  • They'd then click the submit button
  • And finally, they'd wait to be redirected to the "success" page.

To write this end-to-end test, all we need to do is tell the browser to programmatically follow those steps and confirm they work.

Declaring an E2E test #

Now we have to choose an end-to-end test framework to write our test. There's a variety available, including Protractor, CodeceptJS, Puppeteer, TestCafe, and more.

For this example, we're going to use the JavaScript-based Cypress framework.

Firstly, let's create a describe block in order to declare a new test suite.

describe('Website tests', () => {

})

We'll then declare a new test called fills out form successfully.

describe('Website tests', () => {
it('fills out form successfully', () => {

})
})

The browser object #

The Cypress framework provides an API which represents the browser we'll be automating. This is accessible as the global cy object.

Since the first step of our test is to navigate to the form page, we'll use the visit method of the cy object and pass to it the path of the page.

describe('Website tests', () => {
it('fills out form successfully', () => {
cy.visit('/subscribe')
})
})

Test commands #

The next step is for the browser to fill out the name field of the form.

To command the browser to do this, we use the get API method and pass it a CSS selector targeting the name input. We then use the type method to provide a value to the field.

describe('Website tests', () => {
it('fills out form successfully', () => {
cy.visit('/subscribe')
cy.get('input[name=name]').type('Sam')
})
})

Next, we fill out the email field in much the same way. Hopefully, you're getting the idea now - end-to-end testing is mostly just telling the browser how to programmatically do what a user would normally do.

describe('Website tests', () => {
it('fills out form successfully', () => {
cy.visit('/subscribe')
cy.get('input[name=name]').type('Sam')
cy.get('input[name=email]').type('sam@test.com')
})
})

Now we need to click the submit button. The browser can once again locate this using the get method, and then we call click.

describe('Website tests', () => {
it('fills out form successfully', () => {
cy.visit('/subscribe')
cy.get('input[name=name]').type('Sam')
cy.get('input[name=email]').type('sam@test.com')
cy.get('input[type=submit]').click()
})
})

E2E test assertions #

Finally, all tests need an assertion, in other words, a way to determine if the test passed or failed.

In this case, we're going to assert that after the form submits that the url will be that of the success page.

You might also check the database state to be really sure, but I won't show that code here to keep things simple.

describe('Website tests', () => {
it('fills out form successfully', () => {
cy.visit('/subscribe')
cy.get('input[name=name]').type('Sam')
cy.get('input[name=email]').type('sam@test.com')
cy.get('input[type=submit]').click()
cy.url().should('include', '/success')
})
})

Running a test #

Now our test is written, so let's go ahead and run it.

To do that, we can go to the terminal and type cypress run.

$ npx cypress run

Once the test starts running, the test runner will open the chosen browser and automate the steps we declared in our test.

Unless you're in headless mode, you'll actually see it zooming through the various steps, almost like a ghost has taken over your browser!

Image source: https://docs.cypress.io/guides/core-concepts/test-runner.html

Once the test completes, the browser automation will end. Back in the terminal, you'll see a report telling you whether or not the test passed.

CI/CD #

In this example, we assumed that the test suite would be run locally on your computer system.

But actually, end-to-end testing is most powerfully used as part of a continuous intergration or continuous deployment pipeline (CI/CD) where your app is built, tested, and deployed by a virtual server in the cloud.

Build -> Test (unit and/or E2E) -> Deploy

By having end-to-end tests in your deployment pipeline, you can be more confident that bugs won't sneak into your production app.

Choosing an E2E framework #

The end-to-end framework space is changing rapidly. There are several frameworks that are all attempting to solve the problem in different ways.

So, how do you choose the right E2E test framework?

Cypress #

We saw an example before of the Cypress framework. Cypress is great because it allows you to write both unit tests as well as end-to-end tests.

It also has excellent user-experience thanks to easy installation and its amazing debugging capabilities.

However, Cypress only supports Chrome-based browsers and Firefox. If you need to test in Safari, you might need a different solution.

CodeceptJS #

Of course, all the various frameworks will have their pros and cons. We don't have time to cover them all now, but I will mention one more called CodeceptJS.

Their novel solution is to be an abstraction on top of other E2E frameworks ensuring that you can get any feature you need while keeping a consistent API.

Be sure to let us know your favorite framework in the comments.

Drawbacks of E2E #

While end-to-end testing can be a powerful tool, like everything else, it has drawbacks that you should consider before deciding to use it.

Firstly, compared to unit testing, end-to-end tests run very slowly, taking much longer to give you feedback.

This is because end-to-end tests run a real app where there's a delay between page loads, database loads, server responses, etc.

Indeed, a typical end-to-end test suite will take several minutes to run. In large apps, though, it's not uncommon for it to take up to an hour!

Another drawback is that end-to-end tests aren't so great at pinpointing bugs.

For example, let's say our test for the subscribe form actually failed. Since the whole app is being tested, it may not be clear whether the failure occurred in the frontend or the backend, let alone in which part of the code.

For these reasons, the best practice for testing a web app is to have a suite of unit tests AND a suite of E2E tests.

With these two together you are testing your app both in detail and the level of the bigger picture, with each testing method covering the shortcomings of the other.

Wrap up #

So in summary, end-to-end testing is a great way of identifying bugs before they end up in the hands of users.

As the price of cloud services drops and browser sophistication increases, it's becoming increasingly easier - and more important - for small teams to implement.

In smaller companies without dedicated QA staff, the web developer may be responsible for implementing end-to-end testing, which is why now, more than ever, it's an important skill for developers to learn.

You might also like...


Did you enjoy this review? To help us bring you more just like it, consider sponsoring Dev Reviews on Patreon!

Become a Patron