Visual Regression Service

Visual regression testing with WebdriverIO.

Installation

wdio-visual-regression-service uses wdio-screenshot for capturing screenshots.

You can install wdio-visual-regression-service via NPM as usual:

1

$ npm install wdio-visual-regression-service --save-dev

Instructions on how to install WebdriverIO can be found here.

An example repository using the wdio-visual-regression service can be found here.

Configuration

Setup wdio-visual-regression-service by adding visual-regression to the service section of your WebdriverIO config and define your desired comparison strategy in visualRegression.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

// wdio.conf.js

var path = require('path');
var VisualRegressionCompare = require('wdio-visual-regression-service/compare');

function getScreenshotName(basePath) {
  return function(context) {
    var type = context.type;
    var testName = context.test.title;
    var browserVersion = parseInt(context.browser.version, 10);
    var browserName = context.browser.name;
    var browserViewport = context.meta.viewport;
    var browserWidth = browserViewport.width;
    var browserHeight = browserViewport.height;

    return path.join(basePath, `${testName}_${type}_${browserName}_v${browserVersion}_${browserWidth}x${browserHeight}.png`);
  };
}

exports.config = {
  // ...
  services: [
    'visual-regression',
  ],
  visualRegression: {
    compare: new VisualRegressionCompare.LocalCompare({
      referenceName: getScreenshotName(path.join(process.cwd(), 'screenshots/reference')),
      screenshotName: getScreenshotName(path.join(process.cwd(), 'screenshots/screen')),
      diffName: getScreenshotName(path.join(process.cwd(), 'screenshots/diff')),
      misMatchTolerance: 0.01,
    }),
    viewportChangePause: 300,
    viewports: [{ width: 320, height: 480 }, { width: 480, height: 320 }, { width: 1024, height: 768 }],
    orientations: ['landscape', 'portrait'],
  },
  // ...
};

Options

Under the key visualRegression in your wdio.config.js you can pass a configuration object with the following structure:

• compare Object
  screenshot compare method, see Compare Methods

• viewportChangePause Number ( default: 100 )
  wait x milliseconds after viewport change. It can take a while for the browser to re-paint. This could lead to rendering issues and produces inconsistent results between runs.

• viewports Object[{ width: Number, height: Number }] ( default: [current-viewport] ) (desktop only)
  all screenshots will be taken in different viewport dimensions (e.g. for responsive design tests)

• orientations String[] {landscape, portrait} ( default: [current-orientation] ) (mobile only)
  all screenshots will be taken in different screen orientations (e.g. for responsive design tests)

Compare Methods

wdio-visual-regression-service allows the usage of different screenshot comparison methods.

VisualRegressionCompare.LocalCompare

As it’s name suggests LocalCompare captures screenshots locally on your computer and compares them against previous runs.

You can pass the following options to it’s constructor as object:

• referenceName Function
  pass in a function that returns the filename for the reference screenshot. Function receives a context object as first parameter with all relevant information about the command.

• screenshotName Function
  pass in a function that returns the filename for the current screenshot. Function receives a context object as first parameter with all relevant information about the command.

• diffName Function
  pass in a function that returns the filename for the diff screenshot. Function receives a context object as first parameter with all relevant information about the command.

• misMatchTolerance Number ( default: 0.01 )
  number between 0 and 100 that defines the degree of mismatch to consider two images as identical, increasing this value will decrease test coverage.

For an example of generating screenshot filenames dependent on the current test name, have a look at the sample code of Configuration.

VisualRegressionCompare.SaveScreenshot

This method is a stripped variant of VisualRegressionCompare.LocalCompare to capture only screenshots. This is quite useful when you just want to create reference screenshots and overwrite the previous one without diffing.

You can pass the following options to it’s constructor as object:

• screenshotName Function
  pass in a function that returns the filename for the current screenshot. Function receives a context object as first parameter with all relevant information about the command.

Usage

wdio-visual-regression-service enhances an WebdriverIO instance with the following commands:

• browser.checkViewport([{options}]);
• browser.checkDocument([{options}]);
• browser.checkElement(elementSelector, [{options}]);

All of these provide options that will help you to capture screenshots in different dimensions or to exclude unrelevant parts (e.g. content). The following options are available:

• exclude String[]|Object[] (not yet implemented)
  exclude frequently changing parts of your screenshot, you can either pass all kinds of different WebdriverIO selector strategies that queries one or multiple elements or you can define x and y values which stretch a rectangle or polygon

• hide String[]
  hides all elements queried by all kinds of different WebdriverIO selector strategies (via visibility: hidden)

• remove String[]
  removes all elements queried by all kinds of different WebdriverIO selector strategies (via display: none)

• viewports Object[{ width: Number, height: Number }] (desktop only)
  Overrides the global viewports value for this command. All screenshots will be taken in different viewport dimensions (e.g. for responsive design tests)

• orientations String[] {landscape, portrait} (mobile only)
  Overrides the global orientations value for this command. All screenshots will be taken in different screen orientations (e.g. for responsive design tests)

• misMatchTolerance Number
  Overrides the global misMatchTolerance value for this command. Pass in a number between 0 and 100 that defines the degree of mismatch to consider two images as identical,

• viewportChangePause Number
  Overrides the global viewportChangePause value for this command. Wait x milliseconds after viewport change.

License

MIT