Documentation / Video

Video

The stack (easy with Docker)

We use FFmpeg to record a video at 30 fps of the screen (you can configure the number of frames per second). The easiest way is to use our Docker container with pre-installed FFmpeg, but if you use the npm version, you can record a video too. Video works on Linux and OS X at the moment.

Once we have the video we use Visual Metrics (built by Pat Meenan) to analyse it and get SpeedIndex and other visual metrics. If you use our Docker container you get that for free, otherwise you need to install FFmpeg, ImageMagick and a few Python libraries (pyssim, opencv-python, numpy, scipy) — see the installation instructions for your OS.

We record the video in two steps. First we turn the background orange (this is used by VisualMetrics to know when the navigation starts), set the background to white, and let the browser go to the URL. The video is recorded lossless. Then, when the video has been analysed, we remove the orange frames and convert the video to a compressed mp4.

The video will look something like this: watch on YouTube.

There are a couple of things that you can do to configure the video and the metrics.

SpeedIndex and other Visual Metrics

To collect Visual Metrics like firstVisualChange, SpeedIndex, visualComplete85%, visualComplete95% visualComplete99% and lastVisualChange you add the parameter --visualMetrics. The video will then be recorded, analysed and then removed.

Keep or remove the video

If you want to keep the video when you collect metrics or only want the video, just add --video to the list of parameters.

Firefox window recorder

If you use Firefox you can use the built-in window recorder (instead of using FFmpeg) to record the video. The Mozilla team uses it to make sure recording the video doesn't add any overhead. Turn it on with --firefox.windowRecorder.

Video quality

You can change the number of frames per second (default is 30) by using --browsertime.videoParams.framerate. If you have a large server with a lot of extra CPU you can increase the rate. You should probably not decrease it below 30, since that will affect the precision of Visual Metrics.

You can also change the constant rate factor (see https://trac.ffmpeg.org/wiki/Encode/H.264#crf) to change the quality of the video. CRF is applied in the second step (we first record the video as lossless as possible).

Skip converting the video

When you record a video, the video is first recorded with settings to make the recording as fast as possible and with low overhead. Then the video is converted to a format that works better in most video players. If you want to speed up your tests, you may want to remove the video conversion, you can do that with --browsertime.videoParams.convert false.

Remove timer and metrics from the video

The video will by default include a timer and show when visual metrics happen. If you want the video without any text/timer, just add --browsertime.videoParams.addTimer false.

Filmstrip parameters

When the video is analysed with VisualMetrics screenshots for a filmstrip are also created, and shown on the per-iteration page in the HTML report.

Page to page

By default we only show screenshots that differ or have a metric collected at the same time. If you want to see all the screenshots, you can do that with --filmstrip.showAll.

If you don't use those images you can turn them off with --videoParams.createFilmstrip false.

If you use them and want them the same size as the video (they are 200x200 by default), you can turn on full size with --videoParams.filmstripFullSize.

If you want to change the quality (compression level 0-100) of the images, you do that with --videoParams.filmstripQuality.

XVFB

If you run the Docker container we will automatically set up XVFB as a virtual frame buffer. If you run without Docker but still want to use XVFB, add --xvfb and sitespeed.io will start XVFB automatically. You only need to make sure it is installed.

Collect visual elements metrics

You can choose to collect when visual elements are visible (and at their final position) on the screen. Turn it on with --visualElements to collect Visual Metrics from elements. Works only with --visualMetrics turned on (default in Docker). By default you will get visual metrics from the largest image within the viewport and the largest H1.

If you want to add your own element, the easiest way is to follow the Element Timing API: annotate your element with the elementtiming annotation. This will work even if your browser does not support the Element Timing API!

Say that we want to know when the sitespeed.io logo appears on the screen. Then we add the annotation and give that image a unique name (so we know what we are measuring). It will look like this:

<img src="/img/team.png" alt="sitespeed.io Logo with My Little Pony style cats" elementtiming="logo">

If you cannot edit the HTML of the site, you can still measure when elements are painted on the screen, but it's a little more work. You can configure picking up your own defined elements with --scriptInput.visualElements. Give the element a name and select it with document.body.querySelector. Use it like this: --scriptInput.visualElements name:domSelector. Add multiple instances to measure multiple elements.

Visual Metrics will use these elements and calculate when they are visible and fully rendered. These metrics will also be included in the HAR file, so you can look at the waterfall and see when elements are visible within the viewport.

Let's say that we want to measure when the logo of sitespeed.io is painted on screen and we cannot add the annotation.

Logo sitespeed.io

Open up DevTools/web inspector and select the image:

Web Inspector selecting the logo

The next step is to find that element using document.body.querySelector. Do that in your web console. The logo has a unique class, so let's use that: document.body.querySelector(".navbar-brand").

Then try it out in sitespeed.io. Let's name the element ... logo :)

docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:41.1.0 -b chrome https://www.sitespeed.io/ --scriptInput.visualElements "logo:.navbar-brand"  --visualElements

And you can see the result in the Visual Metrics section:

The logo in the result

Compare two video runs (combine two videos)

One of the things we love about WebPageTest is the video where you can compare two different runs. Since sitespeed.io is serverless, it isn't something you can do on the fly. Instead we created a simple tool you can use. The only thing you need is Docker!

  1. curl -O https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/tools/combineVideos.sh
  2. chmod +x combineVideos.sh
  3. Download the videos you want to compare, let's call them file1.mp4 and file2.mp4.
  4. ./combineVideos.sh file1.mp4 file2.mp4
  5. You are done!

You will then have the two videos side by side, slightly slowed down to make it easier to compare them: watch on YouTube.