Cypress.io
When testing is easy, developers build better things faster and with confidence.
&
Amir Rustamzadeh
Director of DX
@amirrustam
WEBCAST
Building an E2E Strategy that Scales
Ask Questions at Sli.do
event code #cy-faire
David Mamujee
Software Engineer
github.com/dmamujee
About me
- Front-end Engineer @ Faire
- Located in Waterloo, Ontario, Canada
github.com/dmamujee
Agenda
- Implementation and structure of our E2E suite
- Flaky test management
- Ways we leverage E2E suite
Ask Questions at Sli.do
event code #cy-faire
- Founded in 2017, Faire is a wholesale marketplace connecting independent brands and retailers. Currently serving over 250,000 retailers and 30,000 brands.
- Our mission is to empower entrepreneurs to chase their dreams
- Our aim is to grow and support the wholesale industry and make it easier for local retailer and wholesale brands to find one another and grow their businesses
- Raised $260M in Series F round at $7B valuation in June 2021
Why invest in E2E testing?
- Had existing front-end component and unit tests, but no tests interacting with the backend
- When releasing to production, we had unscalable manual testing script
- As engineering team scales and product complexity increases, need tool to ensure quality
Why use Cypress?
- Easy to get started
- Good documentation and examples, easy for new engineers to ramp-up
- Easy to debug and manage tests with Cypress Dashboard
- Easy to scale with parallelization
E2E repo organization
- Started with mono-repo for FE
- Migrated to multi-repo for various FE areas
- Used git submodule to have common implementation of test utilities
- Planning to migrate to use internal npm package with shared e2e utilities, have test implementation per repo
Test parallelization
Flaky Test Management ❄️
Pros ✅
- Simple to setup and configure
- All latest and greatest changes on staging
- Closely resembles production
- Flags breakages to staging
Cons 🚩
- Prone to transient breakages
- Configuration values can be modified, breaking tests
- Staging data can be unstable
Testing Against Staging
Cypress Cucumber Preprocessor 🥒
Feature: Orders
Background:
Given a brand exists
And the brand has 5 products
And a retailer exists
And I am logged in as a retailer
Scenario: Return Order
Given I order and return items
Then Test order page reflects returned items
Scenario: Report Issue
Given I create a new order
And I report an issue on the order
Then I view the order and see the reported issue- Creates human-readable definition of the test scenario
Cypress Cucumber Preprocessor 🥒
Feature: Orders
Background:
Given a brand exists
And the brand has 5 products
And a retailer exists
And I am logged in as a retailer
@Ignore
Scenario: Return Order
Given I order and return items
Then Test order page reflects returned items
@Flaky
Scenario: Report Issue
Given I create a new order
And I report an issue on the order
Then I view the order and see the reported issue- Creates human-readable definition of the test scenario
- Creates infrastructure to tag tests
Using the Cucumber Tags
- Want to block PRs on E2E test failure
- Don't want to block PRs on flaky failures
- Create "@Flaky" tag for unstable tests. Don't block PR merging when flaky tests fails
- Flaky test demotion/graduation
Current initiatives to reduce flake ❄️
- Run E2E test suite against a local backend instance
-
Only run relevant E2E tests in a PR
- Create relevancy map using istanbul.js to determine which files an E2E tests uses
- On PRs, only run E2E tests that use the files being changed
Ways we leverage our E2E test suite
Testing configuration changes ⚙️
- Use setting values for experimentation, feature flag, and configuration values
- Modifying these values can break E2E test suite
- Modifying setting values can cause regressions
- Before changing setting value, can run E2E suite with updated value
Cypress Cucumber Preprocessor 🥒
Scenario Outline: Sign up with specific countries
Given I am located in "<country>"
When I visit the brand page
Then the correct products should display with the right price: "<price>"
When I click on the product tile button
And I sign up with
| country | storeZipCode | language |
| <country> | <postal_code> | <language> |
Then I should be on the product details page
Examples:
| country | postal_code | language | price |
| USA | 90210 | ENG | $100 |
| DEU | 51149 | DEU | 100 € |
| FRA | 75001 | FRA | 100 € |
| DEU | 51149 | ENG | €100 |
| FRA | 75001 | ENG | €100 |
| GBR | SW1A 2AA | ENG | £100 |- Create data table for easy definition of multiple test scenarios
- Easily test many different combinations of mobile/desktop with different languages
Testing Emails 📫
- Using E2E tests, test triggering and content of emails
- Tests functionality of email infrastructure
Continuous Deployment 🔁
- Goal is to create stable infrastructure to deploy frontend and backend releases automatically with confidence
Q & A time
Amir Rustamzadeh
@amirrustam
Building an E2E Strategy that Scales
David Mamujee
Software Engineer
Thank you 👏
github.com/dmamujee
Director of DX
Cypress and Faire
By Cypress.io
