How I use Chrome’s Lighthouse to improve site performance

Over my years of personal experience, I’ve found to be true, that maintenance has its way of killing perfectionism. We try so hard to solve all the things, but all the things are never solved.

Don’t just take my word for it. Listen to xkcd:

I’m just gonna leave this here… (tldr: it’s mathematically impossible to solve all the things)

Though it may seem that the natural state of things is to defy one’s every attempt. The more effort we put toward fixing an issue, three other undesirable ones appear. An endless string of code snippets yell out “help me!” as one human struggles in vain to afford time for all of them.

Sorry, little binary buddies, I simply don’t have enough hours in my sprint!

We’ve gotta triage.

It’s technological triage, and it sucks. It can be a depressing fact but it’s important reminder of what we, as developers, are really trying to do each day:

  1. Add judiciously
  2. Maintain enough to quell the behemoth
  3. Sleep
  4. Shower
  5. Repeat.

(Feel free to add another shower or three in that mix. It does wonders to your concentration.)

How do we allocate the resources in a balanced way? How do we know when to fix, when to ignore, when to push, when to take a break? How do we balance the equation?

Money and time balanced on a scale

Time is money, bro.

As a developer always striving for the perfect mix of obsession and expediency, I like metrics that tell me where I should focus more of my time in my bug hunting efforts. Metrics are great shortcuts for my time and sanity.

Where do I need to focus? Ah, right, in Area X, because that’s what this intelligently-informed metric says, and I’m in no position or mood to question it.

This week, as I was dedicating my life to resolving all my site’s bugs, Google’s Lighthouse became my new best friend.

Lighthouse is a potent potpourri of performance metrics.

Some of you may have noticed that I’ve made several releases to my site over this past month. In fact, I’m already building a recipe for the batch of hotfixes hot on the tail of my site’s version 1.5.0 release, which I pushed a few days ago.

I plowed through a ton of the top-grass.
  • little UI tweaks
  • minor script bugs
  • ya know, the small things.

Plus, I added some scoops of whipped-cream topping.

  • lazyloadable images
  • search-engine chummy JSON-LD
  • and some cool Google Analytics indicators. Yummy.

Now, I’m seeing mostly shrub and hidden weeds.

This is where the real maintenance work begins.

So, to help me sort though the brush and weeds, I fired up Lighthouse and ran it on my site’s heaviest page, the home page.

… and got this result:

My score wasn’t impressive (unless you’re impressed by bad scores).

Yeah, you could say my tweet was understating the issue.

According to Google, my site evidently needs a lot of TLC.

I expected excellent scores in the SEO and best practices realms, but the 29 score on performance was a bitter pill. In developing my own site over a 100-megabit connection, I had neglected the constraints put upon my visitors from 3G Land, while Lighthouse caught them with ease.

Face, meet egg.

Though my initial performance disappointed me a bit, this report has an upside for me: A failure is a superb opportunity to succeed!

In its adorable language of red, orange, and green, Lighthouse is telling me that I:

  1. am not perfect (but no surprise there, really).
  2. should have used it earlier.
  3. now have a laundry list of todos it’s provided me and knows I can accomplish. (Otherwise, why would it tell me these things?)

So, I set out on a quest to improve upon my first-round score.

In A World, where bugs rule the galaxy, a HeroHuman awaits, armed with devtools, data, and a killer demeanor, enough to slay even the toughest critters.

Don LaFontaine, beyond the grave, saying all the things he should have said but was never paid to say. R.I.P.

Results after first round of improvements

Over a handful of hours, and also probably from as many cups of coffee, I was able to improve my site’s Lighthouse score dramatically!

Though the Performance metric doesn’t yet report me a score in the green range, it’s a 20+ point increase from my previously terrible score, a boost into the orange range, and orange is better than red here.

TL;DR: Lighthouse clearly thinks there’s a significant improvement to be noted — Woohoo me!

But, we’re not yet in green territory.

Why? Because Lighthouse is especially unhappy about some things I don’t have direct control over.

That, of course, is not an excuse for me. A responsible supervisor wouldn’t look at that and think “no problem here.” Lighthouse is Google’s product. If I want to please the gorilla, then I should definitely listen to its demands.

Now, I have a solid case for spending time and resources toward developing a more efficient solution. The next step: Pinpoint exactly what are the problems whereas Lighthouse has given a high-level analysis.

Major takeaways

This meticulous process to discover, classify and prioritize those problems led me to some pretty neat conclusions.

Twitter’s platform.js is a bloated #whale.
Illustration of a dead whale beached
A literal example of deadweight. #grimshot

Twitter’s impact on a page’s performance is gross. I’m not surprised that my use of external libraries would aggravate Google’s effort to minimize my home page’s weight. But, I was shocked to see just how much unnecessary code there is in Twitter’s payload. Evidently, ~95% of the their lib code is deadweight to my site.

For just displaying a simple timeline, I definitely don’t need everything.

Why can’t Twitter just be nice and offer minimal subsets for major cases, like when someone just wants to embed a simple timeline? Feels to me like an oversight on Twitter’s end, or it’s some heavy desire of theirs to accomplish a load-once scenario, but at the end user’s expense. Either way, not cool, and I’m not happy about it.

Clearly if I want to make good on my commitment to ace the Lighthouse exam, I need to make a better solution for my site.

I need to eliminate or sharply reduce site’s reliance on Twitter’s platform.js script.

Instead of loading that gorilla’s weight, I’m going to roll my own. I’ll poll my timeline via their API, then I’ll cache and render the pretty things from their JSON response data. Perhaps I’ll even package my solution as an open-source WordPress plugin. #efficiency #ownership #community

TL;DR for this post, in one phrase: Big Bugs First.

While it can be appealing to us to always kill the small stuff first, the “small stuff first” strategy doesn’t always lead us to the best outcome. If we always focus on the small stuff, we may never touch the big stuff.

Size matters.

Magnified view of roach on human skin
My skin crawls just by looking at this bug. Imagine how your repository feels.

Until I kill my site’s dependency on a severely bloated external library, I’d be ignoring the biggest way to lower my site’s technical debt.

If I ignore this Big Bug and resolve all the small bugs, I could still end up with my site having a terrible Lighthouse score, because the Big Bug is so damn big.

Focus on the Big Bugs. Don’t ignore the small ones, but especially don’t ignore the big ones. The goal is as much about the user’s perceived experience (the big changes) as the overall picture (a string of small changes).

Conclusions

  • Just running my personal site through Lighthouse told me a lot in terms of major areas for improvement. Without that free rubric, my site may have gone on to annoy countless number of poor souls bound to a depressing 3G (or worse!) connection. What a horrible outcome that would have made.
  • Though my site is aimed at audiences that have near-constant access to high-speed internet, I don’t want to ignore a large segment of the world community by prejudging them on the basis of their connection speed. And I certainly want to be super-conscious of Lighthouse’s opinion of any site I aim at a broader audience.

Where to find Lighthouse

Screenshot of the Audits tab in Chrome's Developer Tools pane

I highly recommend adding Lighthouse to your regular workflow. You can find it in Chrome and Chromium, in the Developer Tools pane, in the Audits tab.

Why you need React for your site’s UI: contextual awareness

When heavyweights ring in, I take notice.

The devs at TechCrunch recently undertook a total overhaul of their website, and they chose React to power it. They focused their efforts around contextual experience. In other words: they focused especially on a need to give their site more context awareness and greater capabilities for case-by-case intelligent prediction about each visitor’s behavior.

Whereas other teams may still focus on page views, they’re focusing on page engagement.

Our premise was that reading an article should feel enjoyable, and that it should be frictionless and fun to get more context surrounding whatever you’re consuming, whenever you want it. For a while now, behavior on the web has been shaped by the networks. People find links, they click on links and they’re dropped into a story without context. Links aren’t enough, they need to be able to catch up or read ahead and, in general, be better informed by the product. The burden shouldn’t be on them to put the story together.

To provide a truly context-driven experience, we need a context-driven interface. We need to be able to make things happen as quickly as our visitor’s mind is running. So, we need our service to become more contextually aware, so our our communication between our services, and our interface’s conversation with the user, will carry less overhead. Minimal overhead via optimal compression — that’s what we’re aiming for.

Graph showing impact of message vs. time taken to communicate
Hit that sweet spot.

This is why I’ve turned my attention into React. As I look for ripe areas for growth in my skill set, I see a field of delicious fruit here. React was a newbie just a few years ago, and Angular was gaining popularity, but neither were deployed on a mass scale. Nowadays, half of the unsolicited messages I receive from recruiters specifically mention React in their desirable skills list. It’s harvest season, yo.

So, what’s so great about React?

More than a fad, React delivers more front-end potential.

The street cred of making a React site is obvious. Every developer loves the opportunity to show-off their front-end skills. (Just take my site as an example: I’ve refreshed it three times already over the last year. Flexing muscles is super fun work!)

But for a tech to gain a critical mass of support, it needs to deliver real, measurable advantages. They need to be seen and felt by those who get to make the calls. You need to draw up bullet points and put them in front of the people who decide when and how the web’s major properties undergo an overhaul. Then, the magic can happen.

How to make the case?

So, how do we determine whether React is an appropriate tech for our web properties? React is a very magical thing indeed. But the benefit isn’t as tangible as some other front-end features.

Easy sells: low friction + high benefit

Here’s an example of a tangible feature: grid frameworks. (btw: you’re currently reading a page gridded up with Bootstrap 4.1. #tmyk)

A grid with ultraviolet colored lines and a bright horizon
Ahhh, this grid. So beauty simplict. More lovely yas.
  • Grids immediately make our life less painful. No more carrying around ad hoc sets of CSS rules. We won’t have to manually cobble them together for each project we undertake.
  • Grids impart a foundation for a common style guide. Those guides help us to deliver us a consistently familiar, relatively easy means for prototyping, theming, and delivering on a schedule. Wins all around.

It’s a no-brainer decision: Learn a few more rules and workflow modifications in return for a huge boost in productivity.

Tron grid, light cycle, and competitor in a glowing suit
… and win enough of the dollars to afford your very own Tron suit and grid land. Neon EVERYTHING.

Less-easy sells: paradigm shifts

Sometimes, the new awesome tool doesn’t make itself easy.

React is one of those. It reveals its secrets only after you’ve taken some time to learn it. It doesn’t sacrifice its complexity for the sake of easy comprehension. It’s complicated — and it’s lightning.

That’s a good thing. We don’t want React to sacrifice its awesome parts for the sake of any laziness.

We see most of the web’s visitor traffic happens now over-the-air. It’s all about the wireless devices: smartphones, tablets, hotspot-ed laptops, the IoT ecosystem, and whatever else humanity cooks up tomorrow.

To really understand the case for React, have a better understanding for the full potential of React as a force of technology in the future world.

Possibly even harder sells: the subconscious effect

Cross-sectional depiction of a typical iceberg in a body of water
To find this prize, were going to have to go deeper.

So, we want to use React for our next project. Let’s consider how to achieve our goal.

We won’t be selling eye candy. Though we can manipulate our new React-driven UI’s components however we see fit, React still remains transparent to a website’s visitor.

… Which causes us another dilemma: Not only is React going to be potentially difficult for us and our team to wrap our heads around, it will be even more difficult for a non-dev to understand why we want to overhaul these things on the company’s time and dime.

We’re selling benefits that accrue at the subconscious level

With React, we actually sell the intangible qualities of experience. How do we communicate those things effectively?

In doubt about the benefits of investing heavily in your users’ subconscious experience? Look to Google. The brains of Mountain View spend a ton of time and money working to shave milliseconds off the load time for their results pages. A millisecond is a small fraction of the amount of time it takes for us to blink, but it’s an unprofitable delay to Google’s ad-selling rate. Data compiled from billions of searches shows that human behavior does change in terms of milliseconds of wait time. Perhaps the user never consciously has such reaction. Yet, it’s a very significant one that some people at Google regularly lose sleep over.

Google's survey on a 5-point scale asking users "How fast or slow is Google Search?"
If your users feel your site is slow, where else does that affect their opinion? How else may they think you’re slow? :thinking_face:

We should pay attention to our users subconscious as well. The user’s experience is where magic truly happens.

How is React different from anything that’s come before it?

React is a paradigm shift for front-end developers.

Remember 1998 brought us XMLHttpRequest and gave us our first real taste of dynamic pages? We had JavaScript already, but now we could load assets long after the page had completed initial script execution. It was a game changer. Dude. It was POWER.

React is another leap of that magnitude.

Remember that time when you jumped into Node and nothing made sense anymore, because all the things became asynchronous?

Asian boy coping with a headache
Yeah, it can be like that.

Yeah, it can be like that.

Now, everything is objects.

React is a framework that formally brings object-oriented (OO) process down to the HTML level of front-end web page construction. If you’ve ever gone from a language like C (which is non-OO) to a language like Java, where nearly everything is strictly an object, you remember the headaches delivered to you ad nauseum when you learned that the object-relationship model was now your pesky BFF. (And if transitioning to OO didn’t make you pain, I dare you to tackle a purely functional language, like Lisp, and avoid headaches then. It eventually happens to all of us, buddy.)

Pages are obsolete.

In The Land Of React, what you’re so used to calling a “page” is no longer a page; it’s now just an initial state of the DOM. The “page” your server sends a client is now just a bare shell that contains a div sporting a unique id attribute — nothing you’d call “content.” The shell is filled dynamically, according to your design, which you define meticulously using React’s object-oriented infrastructure. The page you intend to display to your visitors is now just a constellation of structured React components, with each having the capability to govern its behavior and its children’s behavior, and to interact with components up and down its family chain via life-cycle methods, props, and properly-crafted callbacks.

It’s practically like having a program, rather than a page, running in your browser. When you hear the terms single-page app (SPA) or progressive web app (PWA), it’s likely driven (or should be driven) by a framework like React.

All the pain, but with sweet benefits!

Initially, there’s likely going to be pain for your brain.

The flip side is nice, though. Once you get over the hump of conceptualization, you get to enjoy:

  • The time to first byte (TTFB) is now near-zero. Happiness will increase in all things around you.
  • Also, you have greater control over the time to first meaningful paint (TTFMP). This number is a hugely important metric for search. Google specifically mentions TTFMP in dev docs here, and in discussion here about SERP signals for mobile visitors. Through its AMP initiative, Google continues to force the entire web to move forward on the leading edge, as they must think about TTFMP and TTFB as major considerations of SEO.
  • Now, thanks to the natural separation of your site’s presentation and controller layers from the API and other back-end services, your front-end’s production environment is handled on the client’s hardware. It doesn’t run any expensive IT infrastructure you may or may not have bought for this exact moment in time. Be sure to make more advantage of client’s languishing CPU cycles, by the way. (Want to someday emulate a successful native app? Go for it, and leverage HTML5! Now you have the power, literally.)
  • As a bonus from your hard work separating the front-end and back-end constituents of your app, you’re now free to develop any front-end experience. With a considerably lower cost of startup overhead, you can formalize your data layer’s exposures as a versioned API spec, and develop the front-end with confidence in your team’s capability to experience no unintended breaking changes. (wp-json, anyone? data as a service, amirite?)
So yummy
Soooooo yummy.

That’s just a highlight of the benefits I’ve uncovered. I’m finding more of them while I continue to learn and play with React, and I expect I’ll write in more detail about some of them as I make more progress.

Yeah, it’s also a super-fun power tool!

Let’s be real here: Power tech is adrenaline. If I had unlimited time in the day, I’d go beast for stuff like React, consumer APIs, variable-centric style sheeting (incidentally, I’m currently working on one as a stealth hobby), SVG visuals… and the list goes on.

So, yeah: consider the immense upside! Take up React, get through the growing pains, and enjoy being in the ranks of devs in high demand and great prospects for future employment. 🙂

Google study shows cancerous ad injection malware has infected 5.5% of the Internet world

Have you heard the news? Google’s advertising platform has exploitable flaws! Oh no! But we already knew this, didn’t we? I mean, no system is perfect, right?

Truth is, yes, this is not a new thing: Google’s system has long been gamed by many nefarious players. The activity that Google has decided to highlight this week is a sneaky practice called ad injection. The gist of it is, malicious software devs cook up a compromised browser plugin that makes a broad request for permissions and convinces you to install it. Then, as you surf the web and visit pages that serve up Google ads, the plugin inserts its own advertising in place of the ads that Google is trying to show you, in a way that is so well disguised as to keep you from ever noticing there’s something fishy going on. You view – and possibly click – the interposing ads, instead of the ads that Google’s advertising clients want you to see. That creates potential for the malware to trick Google into charging AdWords advertisers for your click action even though you were actually clicking the ad that the malicious ad injecting plugin served you. Very evil, and really not in the best interests of anyone but the handful of people who are distributing the malware. So, not something we, or Google, wants to encourage by any means.

Just like we don’t want to encourage the spread of cancer. And seeing as ad injection is actively corroding the life source of the Internet, I think it deserves a proper medical name befitting a cancer. Let’s call it adinjectoritis.

Fortunately it is in Google’s better interests to eradicate adinjectoritis, and so eradicating it they have begun to do. Last year over a period of a few months, Google joined forces with awesome folks from University of California’s Berkeley and Santa Barbara campuses, in order to conduct a study that scientifically investigated the nature and extent of adinjectoritis within the body silico of the Internet. What they found is pretty shocking: more than 1 in every 20 unique IPs that hit Google’s ad properties are definitely or potentially compromised by ad injection software. All that lost traffic adds up to many millions of unserved ad impressions, lots of forfeited revenue for Google and lots of potentially wasted ad dollars spent by marketing departments on AdWords campaigns. Google estimates that more than 3,000 AdWords advertisers are being routinely victimized by ad injection, and they cite many major retailers as examples in their report.

If we want the Internet to be accessible and affordable to as much of the world as possible, then it must be a healthy and hospitable place for profitable advertising. Allowing or even encouraging developers of ad injection software is wholly counter-productive to this end – not even to mention how destructive the long-term effects of such activity could be to the financial incentives that motivate so many site owners who feed our digital consumption addiction.

We love our cat pictures, don’t we? Advertising pays for those. Here’s to hoping that Google can wield its influence over the web enough to vaccinate us from this digitally contagious cancer.

Adinjectoritis, you’re on the verge of extinction.