WebdriverRTC

WebdriverRTC Build Status

This project is an extension to WebdriverIO and enables your client instance to grep statistical data from a running WebRTC peer connection. According to the w3 WebRTC draft all RTCPeerConnection objects provide a method called getStats which returns a RTCStats object with useful information about things like packet losts or the audio input level which can be helpful in order to test your network connection or environment (e.g. did my “Mute” button really work?).

This means that you can access all statistical data from chrome://webrtc-internals using Selenium as part of your integration tests.

chrome-internals

Prerequisites

To use WebdriverRTC you need at least WebdriverIO >=v2.5.0 (not available yet but already merged into master).

How does it work

WebdriverRTC masquerades the url command and injects a script after the page has been loaded to overwrite the standard RTCPeerConnection interface and get access to all created RTCPeerConnection objects. After you start analyzing it repeats calling the getStats method with a specific interval and saves all results to an internal object lying in the window scope. Then you can use WebdriverRTC commands to access these information. Currently only the Chrome browser is supported. But there’s more to come.

Example

First install WebdriverRTC via NPM:

1
$ npm install webdriverrtc

Then enhance your client instance using the init method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// init WebdriverIO
var matrix = require('webdriverio').multiremote({
    'browserA': {
        desiredCapabilities: {
            browserName: 'chrome'
        }
    },
    'browserB': {
        desiredCapabilities: {
            browserName: 'chrome'
        }
    }
});

var WebdriverRTC = require('webdriverrtc');
WebdriverRTC.init(matrix);

Now start your selenium session and do everything required to establish a WebRTC connection. Note that you need to run WebdriverIO in multiremote mode if you don’t have something fancy that autoconnects your browser. Multiremote can be really helpful in these situations where you need to control more then one browser. After the connection was established run startAnalyzing, make a pause for a specific amount of time and then grab the stats for that time period.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
matrix
    .init()
    .url('https://apprtc.appspot.com/r/' + channel)
    .click('#confirm-join-button')
    .pause(5000)
    .startAnalyzing()
    .getConnectionInformation(function(err, connectionType) {
        console.log(connectionType);
    })
    .pause(10000)
    .getStats(10000, function(err, mean, median, max, min, rawdata) {
        console.log('mean:', mean);
        console.log('median:', median);
        console.log('max:', max);
        console.log('min:', min);
        console.log('rawdata', rawdata); // contains the complete RTCStatsReport with even more information (mostly browser specific)
    })
    .end();

Commands

WebdriverRTC enhances your client with a small amount of new commands in order to use this plugin properly:

startAnalyzing(options)

Start with WebRTC analyzing.

options

instance (String):
instance name of a specific multibrowser instance you want to use for your analyzing

interval (Number):
defines the interval on which getStats will get called

selectorMethod (Function):
if you want to take stats of a specific RTCPeerConnection object you can use this function to return that object. Also necessary if your app creates an object immediatelly after the page got loaded.

Example:

1
2
3
4
5
6
7
matrix.startAnalyzing({
    instance: 'browserA',
    interval: 500,
    selectorMethod: function() {
        return appController.call_.pcClient_.pc_;
    }
});

getConnectionInformation(callback)

Get basic information about the connection. Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
matrix.getConnectionInformation(function(connection) {
    console.log(connection);
    /**
     * returns:
     * {
     *     "transport": "udp",
     *     "remote": {
     *         "candidateType": "local",
     *         "ipAddress": "192.168.1.7",
     *         "port": "52193"
     *     },
     *     "local": {
     *         "candidateType": "local",
     *         "ipAddress": "192.168.1.7",
     *         "port": 55375
     *     }
     * }
     */
});

getStats(duration, callback)

Returns all stats within given duration in different formats.

duration

You can specify a specific time frame in which you want to receive the stats. If you pass in a number you will receive stats within the last x (your number) ms. You can also be more specific and pass in an object with from and to attribues and desired timestamps as value respectively. If you pass in null, you will receive the last taken stat trace.

Type: Number|Object

callback

Provides stat results in different formats: mean, median, max, min. You can also get access to the raw stat report with even more browser specific information. Example: get stats of the last 10 seconds:

1
2
3
4
5
6
7
8
9
matrix
    .pause(10000)
    .getStats(10000, function(err, mean) {
        /**
         * this test would fail if you are too loud during the test ;-)
         */
        assert.ok(max.audio.outbound.inputLevel < 1000, 'You are too loud!');
        expect(video.rtt).to.be.within(0, 15);
    });

This is how an example result object does look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
{
    "audio": {
        "inbound": {
            "bytesReceived": 31431,
            "jitter": 0.5,
            "packetsReceived": 295.83,
            "packetsLost": 0,
            "outputLevel": 8112.5
        },
        "outbound": {
            "jitter": 0.83,
            "rtt": 1.5,
            "packetsLost": 0,
            "packetsSent": 297,
            "bytesSent": 30884.33,
            "inputLevel": 465.33
        }
    },
    "video": {
        "captureJitterMs": 25,
        "encodeUsagePercent": 75,
        "frameWidthInput": 640,
        "captureQueueDelayMsPerS": 0.83,
        "bandwidth": {
            "actualEncBitrate": 160375,
            "availableReceiveBandwidth": 325032.67,
            "targetEncBitrate": 154050.5,
            "transmitBitrate": 160351.5,
            "retransmitBitrate": 0,
            "bucketDelay": 6.67,
            "availableSendBandwidth": 154050.5
        },
        "frameRateSent": 16,
        "avgEncodeMs": 8.5,
        "bytesSent": 71676.5,
        "frameWidthSent": 640,
        "frameHeightInput": 480,
        "rtt": 3.17,
        "frameHeightSent": 480,
        "packetsLost": 0,
        "packetsSent": 100,
        "frameRateInput": 14.5
    }
}

Examples

There are two examples prepared which show how easy it is to trace WebRTC statistics. Before running them make sure you have the current beta version of WebdriverIO installed:

1
$ npm install git://github.com/webdriverio/webdriverio#master

Then start the getstats.demo.js by running:

1
$ node ./examples/getstats.demo.js

It should start two Selenium sessions and should trace the WebRTC connection, created on https://apprtc.appspot.com. You will get the result formatted as mean, median, max, min. If you want to use a fake video instead of using the camera on your computer just download the following two y4m files by calling:

1
2
$ curl -O https://media.xiph.org/video/derf/y4m/sign_irene_qcif.y4m
$ curl -O https://media.xiph.org/video/derf/y4m/silent_qcif.y4m

The other examples will prove that you can let your tests fail according on the results of the recorded stat. Run the script and start to scream or clap in your hands and it will return with an error message.

1
$ node ./examples/scream.demo.js