Gleb Bahmutov

VP of Engineering

Cypress.io

@bahmutov

Wait, before you write

your next test...

our planet is in imminent danger

https://lizkeogh.com/2019/07/02/off-the-charts/

+3 degrees Celsius will be the end.

we have to act today

ME

you

you

you

you

vote & lobby & rebel

Airplane

flying fast and high

steaming trail

fast propeller

blue like a policeman

2 seats

Number 576

☠ Pirate Ship ☠

Very tall

Lots of flags 🚩🚩🚩🚩🚩

Strong cannons πŸ’£

Climbing ropes

Attacks other ships

Tries to find treasure πŸ†πŸ’°Β 

if I can help I will

@bahmutov

gleb.bahmutov@gmail.com

If there is a company that fights global climate catastrophe and needs JavaScript and testing skills - I will do for free.

Example: https://fab.earth

Take a deep breath

Testing is a drag

Testing does not pay

Testing is boring

This presentation

How I avoid testing

How I pick tests to write

How I make testing fun

Dr. Gleb Bahmutov, PhD

these slides

πŸ¦‰ @bahmutov

I test because

I doubt myself

The simplest way to gain confidence: @ts-check

Lint Pyramid

Prettier

ESLint

@ts-check

Code is easier to read and understand

Actual linter: catches JS things that can go wrong

Strict(er) linter: catches JS and your type errors

You should try

https://slides.com/bahmutov/ts-without-ts

Step 1: code in JS

const add = (a, b) => a + b
add(2, 'foo')

Step 2: add comment

VSCode extension "Document This"

https://marketplace.visualstudio.com/items?itemName=joelday.docthis

Nice: IntelliSense

Step 3: @ts-check

CLI tsc check

$ npx tsc --noEmit --allowJs app.js 
app.js:9:8 - error TS2345: Argument of type '"foo"' is not assignable 
                           to parameter of type 'number'.

9 add(2, 'foo')
         ~~~~~


Found 1 error.

You are not documenting your code enough

IntelliSense in JS test files when you are using Cypress. Hovering over ".type" command

Documentation page for "cy.type" at https://on.cypress.io/type

Do we make more mistakes writing code or using it?

I think using it

Write code with confusing API once

1x developer

10x developers trip over it

-10x developer πŸŽ‰ 🎊

Make your code simpler to understand and to use. No one should be Einstein to fix a small bug

Static types, linting, JSDoc comments, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples, examples

E2E

integration

unit

Testing Pyramid β–³

(click down arrow to see types of tests)

Code

Test

API

Test

Web app

Test

There is only a test appropriate for the thing you doubt works

Thing is a function

const add = (a, b) => a + b
it('adds numbers', () => {
  expect(add(2, 3)).to.equal(5)
})

you write a test

Thing is a component

import { HelloState } from '../src/hello-x.jsx'
import { HelloState } from '../src/hello-x.jsx'
import React from 'react'
it('changes state', () => {
  cy.mount(<HelloState />)
  cy.contains('Hello Spider-man!')
  const stateToSet = { name: 'React' }
  cy.get(HelloState).invoke('setState', stateToSet)
  cy.get(HelloState)
    .its('state')
    .should('deep.equal', stateToSet)
  cy.contains('Hello React!')
})

you write a test

Thing is an API

it('yields result that has log messages', () => {
  cy.api({ url: '/' }, 'hello world')
  .then(({ messages }) => {
    const logs = Cypress._.filter(messages, {
      type: 'console',
      namespace: 'log'
    })
    expect(logs, '1 console.log message').to.have.length(1)
    expect(logs[0]).to.deep.include({
      type: 'console',
      namespace: 'log',
      message: 'processing GET /'
    })
  })
})

you write a test

https://github.com/bahmutov/cy-api

Thing is an API

you write a test

https://github.com/bahmutov/cy-api

Thing is a webapp

it('adds todos', () => {
  cy.visit('/')
  cy.get('.new-todo')
    .type('write code{enter}')
    .type('write tests{enter}')
    .type('deploy{enter}')
  cy.get('.todo').should('have.length', 3)
})

you write a test

Thing is a webapp

you write a test

Thing is a webapp's style

it('draws pizza correctly', function () {
  cy.percySnapshot('Empty Pizza')

  cy.enterDeliveryInformation()
  const toppings = ['Pepperoni', 'Chili', 'Onion']
  cy.pickToppings(...toppings)

  // make sure the web app has updated
  cy.contains('.pizza-summary__total-price', 'Total: $12.06')
  cy.percySnapshot(toppings.join(' - '))
})

you write a test

Thing is a webapp's style

you write a test

<style type="text/css">
  .st0{fill:#FFD8A1;}
  .st1{fill:#E8C08A;}
- .st2{fill:#FFDC71;}
+ .st2{fill:#71FF71;}
  .st3{fill:#DFBA86;}
</style>

changes crust color SVG

Thing is a webapp's style

you write a test

Thing is accessability

it('has good contrast', () => {
  cy.visit('/')
  cy.injectAxe()
  cy.checkA11y({
    runOnly: ['cat.color'],
  })
})

you write a test

https://glebbahmutov.com/blog/fix-low-contrast-with-cypress/

You can always write a test using the right tool

https://on.cypress.io/plugins

What should I test?

How long should my test be?

1. Carefully collect user stories and feature requirments

2. Write matching tests

Keep updating user stories and tests to keep them in sync

What should I test?

Are we testing all features?

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

typical Todo application http://todomvc.com/

Are we testing all features?

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

it('adds todos', () => { ... })
it('completes todos', () => { 
  ... })

Ohhh, we don't have a test for Feature C

Keeping track manually

Are we testing all features?

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

it('adds todos', () => { ... })
it('completes todos', () => { 
  ... })
it('deletes todos', () => { 
  ... })

Keeping track manually

1. Carefully collect user stories and feature requirments

2. Write matching tests

Keep updating user stories and tests to keep them in sync

HARD

  • first test
  • two more tests
  • hundreds of tests!!!

Keep updating user stories and tests to keep them in sync

Are we testing all features?

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

source code

Keeping track via code coverage

Are we testing all features?

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

source code

it('adds todos', () => { ... })
it('completes todos', () => { 
  ... })

Keeping track via code coverage

Are we testing all features?

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

source code

it('adds todos', () => { ... })
it('completes todos', () => { 
  ... })

green: lines executed during the tests

red: lines NOT executed during the tests

Keeping track via code coverage

Are we testing all features?

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

source code

it('adds todos', () => { ... })
it('completes todos', () => { 
  ... })
it('deletes todos', () => { 
  ... })

Keeping track via code coverage

Are we testing all features?

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

source code

it('adds todos', () => { ... })
it('completes todos', () => { 
  ... })
it('deletes todos', () => { 
  ... })

Code coverage from tests indirectly

measures implemented features tested

Keeping track via code coverage

⚠️ 100% Code Coverage β‰  0🐞

Feature A

User can add todo items

Feature B

User can complete todo items

Feature C

User can delete todo items

❚❚❚❚❚
  ❚❚❚❚❚ ❚❚ ❚❚❚❚
  ❚❚❚
❚❚❚❚

❚❚❚ ❚❚❚❚❚❚❚
❚❚❚❚
❚❚
  ❚❚❚❚❚ ❚❚
❚❚❚❚❚❚
❚❚❚
❚❚❚❚❚
❚❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚
  ❚❚❚❚❚❚ ❚❚❚ ❚❚
❚
❚❚

source code

it('adds todos', () => { ... })
it('completes todos', () => { 
  ... })
it('deletes todos', () => { 
  ... })

Unrealistic tests; subset of inputs

code does not implement the feature correctly

Code Coverage with @cypress/code-coverage

  1. Instrument code (YOU)

  2. Run tests

  3. Report results

https://on.cypress.io/code-coverage

demo https://github.com/cypress-io/cypress-example-todomvc-redux

it('adds todos', () => {
  cy.visit('/')
  cy.get('.new-todo')
    .type('write code{enter}')
    .type('write tests{enter}')
    .type('deploy{enter}')
  cy.get('.todo').should('have.length', 3)
})

E2E tests are extremely effective at covering a lot of app code

Coverage as a guide to writing E2E tests

it('adds todos', () => {
  cy.visit('/')
  cy.get('.new-todo')
    .type('write code{enter}')
    .type('write tests{enter}')
    .type('deploy{enter}')
  cy.get('.todo').should('have.length', 3)
})

Coverage as a guide to writing E2E tests

it('adds todos', () => {
  cy.visit('/')
  cy.get('.new-todo')
    .type('write code{enter}')
    .type('write tests{enter}')
    .type('deploy{enter}')
  cy.get('.todo').should('have.length', 3)
})

We have tested "add todo"

Need tests

https://github.com/cypress-io/cypress-example-todomvc-redux

Write more end-to-end tests

https://github.com/cypress-io/cypress-example-todomvc-redux

Ughh, missed it

Can we always write an E2E test?

https://github.com/cypress-io/cypress-example-todomvc-redux

What is this code doing?

Can I write an E2E test to hit this code?

Sometimes you cannot

This code should be unreachable from the user interface

https://github.com/cypress-io/cypress-example-todomvc-redux

Write a Unit Test!

import {getVisibleTodos} from '../../src/selectors'

describe('getVisibleTodos', () => {
  it('throws an error for unknown visibility filter', () => {
    expect(() => {
      getVisibleTodos({
        todos: [],
        visibilityFilter: 'unknown-filter'
      })
    }).to.throw()
  })
})

@cypress/code-coverage plugin

We got this line from the unit test

@cypress/code-coverage plugin

Nice job, @cypress/code-coverage plugin

combines e2e and unit test coverage automatically

Use Code Coverage

  1. Guide test writing
  2. Stop build if coverage drops below X%
  3. Send to 3rd party service

Who long should my test be?

Most unit tests are very short

  1. Arrange

  2. Act

  3. Assert

Tests are kept short because they run in the terminal. Their shortness helps to debug a failed test.

Write longer tests

Go ahead

Meaningful Long Test

https://github.com/cypress-io/cypress-example-realworld

When a test runs for too long...

Multi-page form example

https://github.com/bahmutov/cypress-example-forms

The test is too long

  1. Split into 3 tests

  2. End each test with a "checkpoint"

  3. Starts next test from a "checkpoint"

class MasterForm extends React.Component {
  constructor (props) {
    super(props)
    if (window.Cypress) {
      window.app = this
    }
  }
  ...
}

Inside your app code

Now tests can control the application directly

https://www.cypress.io/blog/2019/01/03/stop-using-page-objects-and-start-using-app-actions/

expose app reference

cy.contains('Next').click()

cy.log('Second page')
cy.contains('h1', 'Book Hotel 2')
cy.window()
  .its('app.state')
  .should('deep.equal', startOfSecondPageState)

End of the first test

cy.window()
  .its('app')
  .invoke('setState', startOfSecondPageState)

cy.log('Second page')
cy.contains('h1', 'Book Hotel 2')
cy.get('#username').type('JoeSmith', typeOptions)

Start of the second test

Start a test right from the state at the end of the previous test

Talk summary

Document, lint and test your code

Code coverage is a tool, not a goal

Do not accept slow tests

πŸ¦‰ @bahmutov

Wait, before you write

your next test...

Angry or happy? Let me know @bahmutovΒ  Β  Β @cypress_io

https://cypress.slides.com/cypress-io/next-test

MulΘ›umesc frumos πŸ‘

Wait, before you write your next test...

By Cypress.io

Wait, before you write your next test...

This presentation looks at the variety of ways we can test a typical web application. From unit tests to end-to-end, from functional to visual testing, from happy paths to the edge cases - there are a lot of solutions available, and each new technique gives us additional confidence in our application. This presentation will teach you how to guide test writing using code coverage, how to reach into the application code to test edge cases, and how to decide when to stub a network API request and when to let it go through.

  • 2,867

More from Cypress.io