&

Gleb Bahmutov

VP of Engineering

@bahmutov

WEBCAST

Questions via Sli.do #cygofundme

Recording at https://on.cypress.io/webinars

Todd Williams

QA Engineer

/in/twilliams15

GoFundMe Charity

GoFundMe Charity is a subscription-free fundraising platform with powerful tools to make fundraising easy for charities

Stats

5-hour overnight test runs (~1,000 tests)
Random 10% (~100) tests fail from flake
- Brittle code
- Poor coding/QA practices
Difficult to debug & update

Focus

Getting developers to write tests
Gaining confidence w/ the test suite

Questions via Sli.do #cygofundme

Getting developers

to write tests

QA as support

https://www.atlassian.com/inside-atlassian/qa

Why don't developers write tests?

Takes too much time

While devs prepare for a new feature,

QA helps set up the tests.

Then we just fill in the blanks.

Not good at testing

No problem, QA has you covered

Not used to writing tests

QA helps create a habit by writing unfinished tests.

Red, Green, Refactor

No fun

Selenium isn't very fun

What tools should we use?

Questions via Sli.do #cygofundme

Previous Solution

Selenium, PHP Webdriver, PHPUnit
Paratest, SauceLabs, TestLodge
Utils package, Swagger CodeGen

That's a lot of work

Current Solution

npm install cypress

describe('Sign up', () => {
  it('Users can create a new account')
})

describe('Sign in', () => {
  it('Users can log in as a charity')
  it('Users can log in as a human')
})

Example

beforeEach(() => {
  cy.visit('/signin/form')
})

describe('Sign up', () => {
  it('Users can create a new account')
})

describe('Sign in', () => {
  it('Users can log in as a charity')
  it('Users can log in as a human')
})

beforeEach(() => {
  cy.visit('/signin/form')
})

describe('Sign up', () => {
  beforeEach(() => {
    cy.contains('Sign up')
      .click()
  })

  it('Users can create a new account')
})

describe('Sign in', () => {
  it('Users can log in as a charity')
  it('Users can log in as a human')
})

beforeEach(() => {
  cy.visit('/signin/form')
})

describe('Sign up', () => {
  beforeEach(() => {
    cy.contains('Sign up')
      .click()
  })

  it('Users can create a new account')
})

describe('Sign in', () => {
  describe('Charity', () => {
    let charityUser
    before(() => {
      cy.charitySetup()
        .then(response => {
          charityUser = response.body
        })
    })

    it('Users can log in as a charity')
  })

  describe('Human', () => {
    let humanUser
    before(() => {
      cy.humanSetup()
        .then(response => {
          humanUser = response.body
        })
    })

    it('Users can log in as a human')
  })
})

beforeEach(() => {
  cy.visit('/signin/form')
})

describe('Sign up', () => {
  beforeEach(() => {
    cy.contains('Sign up')
      .click()
  })

  it('Users can create a new account')
    // enter email & password
    // click submit
    // assert 201 response from api
    // assert success message in ui
})

describe('Sign in', () => {
  describe('Charity', () => {
    let charityUser
    before(() => {
      cy.charitySetup()
        .then(response => {
          charityUser = response.body
        })
    })

    it('Users can log in as a charity')
      // enter email & password
      // click submit
      // assert 200 response from api
      // assert url is charity home screen
  })

  describe('Human', () => {
    let humanUser
    before(() => {
      cy.humanSetup()
        .then(response => {
          humanUser = response.body
        })
    })

    it('Users can log in as a human')
      // enter email & password
      // click submit
      // assert 200 response from api
      // assert url is human home screen
  })
})

Check-ins

Watch the code reviews to help inform best practices
Add/update tests as features change

Review:

Getting developers to write tests

Provided better tools
Help set up the tests
Provide continued support

Gaining confidence

A new set of best practices

Moved away from page object model
Made tests deterministic
Created better setups
Added data-qa selectors
Wised up to waiting
No focus on the browser

Questions via Sli.do #cygofundme

No more POM

Too much time to write
Too difficult to debug
Leads to repeated steps

w/ Cypress

abstraction isn't necessary

it('uses page object model', () => {
  login
    .visit()
    .enterEmail('tester@example.com')
    .enterPassword('password')
    .submit();
});

it('avoids page object model', () => {
  cy.visit('/login');
  cy.get('#email')
    .type('tester@example.com')
  cy.get('#password')
    .type('password')
  cy.get('#submit')
    .click();
});

using POM

just use cy commands

$this->testUtility->enterText(
  $this->donateLocators->first_name,
  $this->donateEntity->getBillingFirstName()
);
$this->testUtility->enterText(
  $this->donateLocators->last_name,
  $this->donateEntity->getBillingLastName()
);
$this->testUtility->enterText(
  $this->donateLocators->email,
  $this->donateEntity->getUserEmail()
);

legacy codebase (php)

public function enterText($element, $text, $verifyText = true, $timeout = null)
{
  $timeout = $this->setTimeout($timeout);

  if (Driver::isMobileBrowser()) {
    $foundElement = $this->waitForElementToAppear($element, $timeout);
  } else {
    $foundElement = $this->scrollToElement($element, $timeout);
  }

  $foundElement->clear();
  $foundElement->sendKeys($text);

  if ($verifyText && !(CONFIG['browser'] == 'FIREFOX' && CONFIG['useSaucelabs'])) {
    $this->assertEquals($text, $foundElement->getAttribute("value"));
  }
}

legacy codebase (php)

Deterministic Tests

Random tests suck
Difficult to debug

w/ Cypress

you can mock and stub!

$username = $entity_data['username'];
if ($entity_type == EntityTypes::PROJECT) {
  $username .= '/' . $entity_data['project_username'];
} elseif ($entity_type == EntityTypes::FUNDRAISER) {
  $entity_type = EntityTypes::PROJECT; // due to $entity_type being used in url below
}

$this->donateEntity->setAmount(100);
$this->donateEntity->setCharityId($entity_data['charity_id']);
$this->donateEntity->setFeeStructureType($entity_data['fee_structure_type']);
if ($this->donateEntity->getPaymentProcessor() == PaymentProcessors::PPGF) {
  $this->donateEntity->setCardCvv('324');
} else {
  $this->donateEntity->setCardNumber('4111111111111111');
}
if ($widget) {
  $fees_data = $this->donateSupport->widgetFeeCalculations($entity_data['charity_id'], $entity_data['fee_id'], $this->donateEntity->getAmount());
} else {
  $fees_data = $this->donateSupport->feeCalculations($entity_data['charity_id'], $this->donateEntity->getAmount());
}

it(`
Users can donate to a team:
* Logged out
* PPGF
* USD
* Includes Tip
* $25, one-time
`, () => {
  cy.visit(`/donate/project/${fundraiser.team_slug}`);

  // step 1
  cy.contains('$25')
    .click();
  cy.contains('Continue')
    .click();

  // step 2: billing info
  cy.get('#first_name')
    .type('First');
  cy.get('#last_name')
    .type('Last');
  cy.get('#email')
    .type('testdonation@example.com');
  cy.get('#address')
    .type('123 Main St');
  cy.get('#country')
    .select('United States');
  cy.get('#city')
    .type('LA');
  cy.get('#state')
    .select('CA');
  cy.get('#zip')
    .type('12345');

  // step 2: payment info
  cy.get('#shown_card_number')
    .type(getRandomPPGFTestCard());
  cy.get('#exp_month')
    .type('01');
  cy.get('#exp_year')
    .type('2030');
  cy.get('#cvc')
    .type('123');

  cy.contains('DONATE')
    .click();

  // success
  cy.url()
    .should('include', '/share');
});

Smarter setup

Don't use existing data that can change
Don't repeat the setup for every test

w/ Cypress

use before and beforeEach

and nest 'em!

https://on.cypress.io/writing-and-organizing-tests

Add selectors just for testing

Classes and IDs can be hard to read
Prone to flake

w/ Cypress

build your own cy.getTestID

Questions via Sli.do #cygofundme

Cypress Best Practices

Cypress.Commands.add('getDataQA', (value) => {
  return cy.get(`[data-qa="${value}"]`)
})

support/commands.js

it('works', () => {
  cy.getDataQA('like-magic')
    .click()
})

some.spec.js

Smarter waiting

Too much waiting

w/ Cypress

use route aliases or assertions

cy.server()
  .route({
    method: 'GET',
    url: '**/users/*',
  })
  .as('users')

// ...

cy.wait('@users')

// ...

https://on.cypress.io/network-requests

cy.visit('/react-app')
  .get('.loading')
  .should('not.be.visible')

// ...

No focus on the browser

Cross-browser testing is overrated
Adds time without adding value
Difficult to write specific tests

Default: Firefox
Cypress local: Edge
Cypress CI: Chrome

Stats

~10 minute runtime (600+ tests)
Very little test flake (1 or 2 tests/run)
Easier to read, update, & debug

Questions via Sli.do #cygofundme

https://on.cypress.io/test-retries

In conclusion

Getting developers to write tests
- Switch tools
- QA as support
Gaining confidence w/ the test suite
- No POM
- Deterministic tests
- Smart setup and waiting
- Custom selectors
- No cross-browser

Todd Williams

QA Engineer

/in/twilliams15