Notes on: Raiders of the Fast Start at Perfmatters

#

At #PerfMatters, Katie Sylor-Miller gave a presentation entitled “Raiders of the Fast Start” (best title ever). Here are my notes.

  • Performance archaeology uncovers insights into your development and performance culture.
  • Start with a hypothesis and do a comprehensive survey and additional resource. When you understand the context, start excavating and finally interpret your discoveries.

Hypothesis

  • Focused on improving the mobiler performance of the Etsy listing page.
  • Code for mobile listing page started in 2013 with multiple teams, so lots of legacy code baggage.
  • RUM DOM Content Loaded times tied most closely to the user experience of Etsy users, so focused on that. 90% of visits were under 5 seconds. 52% of visitors were 2 seconds or less.
  • Looking at conversion rate versus DOMContentLoaded they discovered that conversions steadily declined for every second added to DOMContentLoaded time.
  • Their ultimate hypothesis was that improving the performance of the listing page would increase conversions.

Survey

  • Etsy had the basics in place already, so they focused on improving the critical rendering path. Focusing on how quickly a user receives confirmation the page is loading and how quickly they can interact with a page.
  • They ran their listing page through WebPageTest on an iPhone 6 over a slow 3g network to level set. Start Render (their primary focus) was 8.5 seconds and DOM Content Loaded was at 12.1 seconds.
  • Their listing page had five CSS files loaded up front that blocked CSSOM construction and Start Render despite most of that CSS not being used.
  • Their main listing image, the thing they wanted to load first, was the 36th resource actually requested.
  • Identified four primary areas of improvement: lazyload images, reduce CSS file size, switch to SVGs for icons and background images, reduce JavaScript file size.

Excavate

  • 35 of their 37 images could be moved out of the critical rendering path. Synthetic testing showed a significant performance improvement on slower networks, while faster networks remained similar.
  • Started with 5 CSS files totaling 98kb of CSS over the network.
  • To identify unused CSS, they used Selenium scripts to open pages in the browser and ran unCSS on them to identify unused CSS and create a file with only the CSS in use.
  • The problem with the automation was that they didn’t capture all the various states of a page and they had to keep adding more every time they found a bug. So there was manual work involved.
  • They put the lazyloading and unused CSS optimizations out as an experiment to a subset of their users and saw a 6% reduction in DOM Content Loaded and a 13.2% reduction in page load time. Conversion also increased.
  • Originally, Etsy had five background images in CSS and one icon font. While the over the wire cost wasn’t very high, the extra HTTP requests weren’t necessary so they switched to SVG. The impact on metrics was negligible and didn’t impact conversion.
  • These optimizations combined to create a 44% decrease in start render time, 10% decrease in Time to First Interactive, 60% reduction in CSS file size and 32% reduction in image weight.
  • Etsy had two primary JS files on the listing page. A page specific file that was 56kb with 121 dependencies and a global base file that was 142kb with 124 dependencies.
  • Manually reducing JS dependencies saw an improvement in performance metrics but a reduction in conversions.
  • Chrome’s Code Coverage tab reports unused JS and CSS code. There was no API, so Etsy built a tool called Vimes to make code coverage scale.
  • Vimes re-writes JS so that each function logs itself. Those logs are sent to the server to be aggregated and mapped back to source files to get an idea of what functions were called and when.
  • Running Vimes lead to a 28% reduction in page specific file and a 37% reduction in the global base file and an improvement in conversion.

Interpret

  • Etsy’s experiments clearly showed that performance directly impacts conversion. Nothing compares to having your own numbers to show when you need to convince folks that performance matters.
  • Frontend performance is just as important, if not more, than server-side performance. It’s important to prioritize it so that legacy baggage doesn’t accumulate and negatively impact the user experience.
  • Our experiences aren’t our user’s experiences. The experiments Etsy ran made little impact on high-end machines and networks, but significantly improved the experience for less than ideal scenarios.
  • Your front-end architecture should match your culture. For example, if you have a culture of experimentation, your architecture needs to be set up in a way that encourages and supports that experimentation.
  • Architect for deletion. It needs to be easy to identify what gets used and what doesn’t so that you can minimize you tech debt. Every line of code we write today will end up as someone’s legacy code.