Web Performance:
Perception & Metrics

http://instartlogic.github.io/p/SFHTML5/psi.html

Visual Metrics

  1. SpeedIndex (SI)
  2. Perceptual Speed Index (PSI)

Speed Index

Metric on above-fold visual Quality of Experience

  • Created by Patrick Meenan (Google)
  • Used on WebPage Test

Speed Index

Aggregate function on quickness of above-the-fold visual completion:

  • speed index graph for both sites
  • speed index graph for staples4,462
  • speed index graph for wolfermans5,902

equation for speed index

Measurement of visual progress in Speed Index

  • Frame-by-frame VC progress is computed from pixel-histogram comparisons

    equation for speed index

  • Pixel-wise similarity (mean histogram difference a.k.a. MHD) doesn’t capture visual perception!
    • Perception of Shape / Color / Object similarity

Pixel-wise similarity doesn’t capture shape similarity

Black/White = 50/50             MHD (Mean Histogram Difference) = 0

  • box that is 50% white and 50% black
  • box that is 50% white and 50% black
  • box that is 50% white and 50% black
  • box that is 50% white and 50% black
  • box that is 50% white and 50% black
  • box that is 50% white and 50% black

Pixel-wise similarity doesn’t capture color similarity

Speed Index

Aggregate function on quickness of above-the-fold visual completion:

  • speed index graph for both sites
  • speed index graph for staples4,462
  • speed index graph for wolfermans5,902

equation for speed index

Proposal for a perceptually oriented visual QoE metric

  • Update: Frame-by-frame VC progress computation using SSIM

Perceptual Speed Index

Frame-by-frame VC progress computation using SSIM

equation for perceptual speed index

Without Jitter

With Jitter

PSI v. SI

  • SI and PSI: linearly correlated
  • Visual jitter / layout thrashing? PSI > SI
    • PSI appears higher when visual jitter exists (Pop-up ads / large lay-out changes / etc.)
  • SSIM based visual progress measurements match human perception more closely than MHD
  • SSIM / MHD swap doesn’t affect websites without visual jitter

Staples

Wolfermans

PSI v. SI

Speed Index

  • Primarily focused on progress of above-fold loading
  • Does not account for layout stability

Perceptual Speed Index

  • A perceptually oriented metric to measure above-fold visual QoE
  • Designed to account for visual jitter (layout stability)
  • Complementary to SI

Resources

Performance Projects launch

  1. Progressive Web Applications
  2. Lighthouse
Go launch

Thank you

Speed Perception: Understanding and Measuring Perceived Performance

http://instartlogic.github.io/p/SFHTML5