Book Notes - The Checklist Manifesto

Books have always been important to me. My grandmother was a librarian and as she babysat me, I was able to spend hundreds, if not thousands, of hours searching, consuming, and falling in love with books. That passion continues today, but I find that I have less time to devote to extended, deep periods of reading.

I’m going to follow CGP Grey’s lead and start publicly sharing notes on books I’ve read. Here’s his explanation of why he started:

I’m going to attempt, for a little while anyway, to make public some of my notes from some of the books that I’ve read. This is partly because people are forever asking what I’m reading, but it’s mostly as a way to try and encourage myself to read both more deeply and more frequently – a target I have been trying, and failing, to hit for all my adult life.

All projects and hobbies of mine eventually die from lack of attention if they cannot serve multiple purposes. So it is my hope that these notes will add even more reason to engage more frequently with long-form writing.

There isn’t anything close to a public clamor for my reading list, but I do hope that this practice helps me with his second goal - increasing my incentives to read books with more depth and more frequency. Additionally, I hope that formalizing this process will help me draw better connections between the books I read and what it could mean for education and/or individual productivity.

With that said, I’m going to apply Grey’s template to the last book I finished, The Checklist Manifesto:

The Big Idea

Professionals of all varieties - programmers, doctors, teachers, pilots, etc. - perform cognatively demanding tasks on regular basis in high-stakes settings. They apply the skills they’ve honed through training, their understanding of their field, and their general experience to execute complex tasks. But no matter how competent or prepared these professionals are, they still make mistakes.

“In a complex environment, experts are up against two main difficulties. The first is the fallibility of human memory and attention, especially when it comes to mundane, routine matters that are easily overlooked under the strain of more pressing events.”

“A further difficulty, just as insidious, is that people can lull themselves into skipping steps even when they remember them. In complex processes, after all, certain steps don’t always matter. Perhaps the elevator controls on airplanes are usually unlocked and a check is pointless most of the time. Perhaps measuring all four vital signs uncovers a worrisome issue in only one out of fifty patients. “This has never been a problem before,” people say. Until one day it is.”

There’s nothing wrong with confidence in your ability to practice your profession, but it does breed a degree hubris that even when seemingly benign, can lead to significant errors. Gwande provides several examples of this in the medical field, where skipping seemingly obvious steps can lead to someone having their right knee opened up even if they’re in for an ACL repair on their left knee.

It’s not hard to see how this could translate to the classroom. A teacher might typically use exit tickets to gauge how much students understood the concepts taught in each lesson. It’s easy to imagine a scenario where the teacher assumes a lesson went well and forgoes this step. This may only happen once in a while, but it could be the case that while the lesson seemed to go well, students actually misunderstood the concept that was taught. It’s a much lower-stakes situation than a botched surgery, but even this small disruption could alter the success and pacing of future lessons.

The concept of minimizing mistakes in routine processes is also applicable to data analysis. Each data analysis project involves follows the broad format of importing, cleaning, transforming, and reporting data. If small steps along the way aren’t double-checked, a tiny error can lead to significant problems with the results.

Gwande stresses that using checklists to minimize error isn’t meant to dumb down the work of professionals. Instead, he argues that:

“It is common to misconceive how checklists function in complex lines of work. They are not comprehensive how-to guides, whether for building a skyscraper or getting a plane out of trouble. They are quick and simple tools aimed to buttress the skills of expert professionals. And by remaining swift and usable and resolutely modest, they are saving thousands upon thousands of lives.”

Key Passages

When the B-17 bomber was developed in the mid-1930’s, it was the most complex aircraft ever built. It had four engines when nearly every plane had one or maybe two engines. This, along with the plane’s other features, made the task of piloting it incredibly complex. The first time the plane was tested, it crashed shortly after takeoff not due to any mechanical issue, but to pilot error. The new design was simply too complex for one person to manage on their own.

Development on the plane continued and another test flight was scheduled. This time, the pilots developed what is now a staple of aviation: a checklist. It was concise, but it ultimately helped flight crews man the B-17 without incident from that point forward.

I found this to be a terrific example of intellectual humility by the test pilots. They were extremely well-trained, but they were willing to admit to themselves that while they were experts, their brains weren’t reliable enough to ensure perfect execution every single time.

“Substantial parts of what software designers, financial managers, firefighters, police officers, lawyers, and most certainly clinicians do are now too complex for them to carry out reliably from memory alone. Multiple fields, in other words, have become too much airplane for one person to fly.”

Educators and data analysts alike find themselves with “too much airplane” right now. We need to have the humility to:

  1. Admit that these are jobs we can’t do alone.
  2. Develop tools to help overcome the problem of “too much airplane.”

Actionable Items

“You want people to make sure to get the stupid stuff right. Yet you also want to leave room for craft and judgment and the ability to respond to unexpected difficulties that arise along the way”

Checklists must strike an important balance. They need to cover enough critical points in a process to minimize the “stupid stuff” that could happen, but they also need the flexibility and brevity to be useful in a real-world setting. Gwande explains the difference between a good checklist and a bad one:

“Bad checklists are vague and imprecise. They are too long; they are hard to use; they are impractical. They are made by desk jockeys with no awareness of the situations in which they are to be deployed. They treat the people using the tools as dumb and try to spell out every single step. They turn people’s brains off rather than turn them on.

Good checklists, on the other hand, are precise. They are efficient, to the point, and easy to use even in the most difficult situations. They do not try to spell out everything—a checklist cannot fly a plane. Instead, they provide reminders of only the most critical and important steps—the ones that even the highly skilled professionals using them could miss. Good checklists are, above all, practical.”

After developing a good, concise checklist comes the most critical step: implementation. As Gwande first developed his surgical checklist, he practiced using it with his surgical team to figure out what worked, what didn’t, and how they could improve it before using it in an actual surgical setting.

Once the checklist was tested, it needed to gain adoption with other surgons and their teams. Gwande started introducing it to chief surgons - the idea was that if a hospital’s leadership was able to embrace the change, others would follow. This is excatly what happened, and the incidence of surgical errors began to drop. But then something else began to happen:

“Spot surveys of random staff members coming out of surgery after the checklist was in effect did indeed report a significant increase in the level of communication. There was also a notable correlation between teamwork scores and results for patients—the greater the improvement in teamwork, the greater the drop in complications.”

The checklist process actually helped keep surgical teams more aware of what each person was doing by establishing a template for regular communication between everyone in the operating room. It’s an example of the hidden benefit of checklists:

“Just ticking boxes is not the ultimate goal here. Embracing a culture of teamwork and discipline is.”

Setting up checklists can not only improve processes, it can help build stronger teams. For educators and analysts alike, that’s a goal worthy of pursuit.


Tinkering

The most recent episode of Mad Dogs & Englishmen1 starts off with a discussion of buying and assembling and AR-15 from scratch. It’s an interesting process, but the conversation took an even more interesting turn as they focused on the decline of a tinkering ethos in our society.

The machines and devices we use are often too complex to repair, disassemble, or customize. If you open the hood of your car, you won’t be able to see or even access many of the most critical parts - you’ll most likely see a series of injection-molded plastic meant to not only protect but to discourage amateur tinkering. It is more likely that you’ll be able to tinker with your car by software (via OBD-II compatible devices like Automatic) than by mechanical manipulation.

Understanding how to tinker/hack software is an increasingly valuable skill, but we should also recognize and cultivate the ability to take apart, assemble, fix, and customize physical objects. The Maker Movement is a great example of keeping mechanical tinkering alive by encouraging kids and adults alike to get their hands dirty by blending both digital and physical craftsmanship.

As more of our lives become digitized, I hope that we continue to find ways to maintain some form of physical tinkering.

  1. Mad Dogs and Englishmen is a podcast featuring National Review writers Kevin Williamson and Charles C.W. Cooke 


Intellectuals are Freaks

Michael Lind on the weakness of intellectuals’ approach to inequality:

The views of intellectuals about social reform tend to be warped by professional and personal biases, as well. In the U.S. the default prescription for inequality and other social problems among professors, pundits, and policy wonks alike tends to be:  More education! Successful intellectuals get where they are by being good at taking tests and by going to good schools. It is only natural for them to generalize from their own highly atypical life experiences and propose that society would be better off if everyone went to college — natural, but still stupid and lazy. Most of the jobs in advanced economies — a majority of them in the service sector — do not require higher education beyond a little vocational training. Notwithstanding automation, for the foreseeable future janitors will vastly outnumber professors, and if the wages of janitors are too low then other methods — unionization, the restriction of low-wage immigration, a higher minimum wage — make much more sense than enabling janitors to acquire BAs, much less MAs and Ph.Ds.

I’m a member of the very intellectual class Lind describes. I share his concern that our “intellectual class” is too isolated from the experiences of the majority of our fellow citizens and that this can lead to ideas/policies that would work great for Belmont but fail to address the needs of Fishtown.

But does that mean the “intellectual class” solution of “more education” is the wrong answer?

Every year, fewer jobs are available to those with only a high school degree. If we want to expand opportunity for working class Americans, it means ensuring a greater portion earn a BA - it also means that we need more people completing career/technical education and earning 2-year degrees.

Lind argues for short-term protectionist solutions to address low wages: more unionization, less immigration, and higher minimum wages. These may sound attractive, but they are ultimately stopgap measures that don’t address the systemic challenge in our economy, illustrated in the graphic below:

Why did wages for the Western working class stagnate while the Asian middle class grew just as much as the income of the most affluent? As Yuval Levin explains in The Fractured Republic:

“In the 1950s, the nation produced more than half of the world’s total manufactured goods—a mind-boggling, if obviously transitory, advantage.

The equality of the postwar era was a unique moment in American history, subsidized in large part by the devastating impact of WWII on competing economies. As other nations rebuilt and technology facilitated the rise of globalization, the wage premium for American labor withered away. Protectionist policies can only attempt to insulate against this trend - they don’t address the fundamental challenge that work historically performed by high school graduates in the US can now increasingly be performed more cheaply abroad or by robots.

The long-term solution is obvious: more education (both career and college oriented) to help more people access highly-skilled jobs with high wages, but we also need to consider shorter-term solutions to help make this transition less disruptive without resorting to protectionism. This will certainly involve a re-thinking of our social welfare system, either by re-imagining A Better Way to run our current system, or replacing it with something more radical, like a Universal Basic Income.

While I disagree with Lind’s proposed solution to inequality, I agree that more members of the “intellectual class” need to get outside of our bubbles to better understand the lives of working class Americans. Our nation is currently a fractured republic and coming apart - those of us in the “intellectual class” need to actively work to reverse the isolation we experience and cultivate more relationships and empathy for people of all backgrounds in our society.


Time Tracking

I tweeted this out earlier today:

Why do I have time tracking charts that break down when I do certain kinds of work?

A book (Deep Work) and a podcast (Cortex).

I know that the ability to focus is important to the work I do, but Deep Work by Cal Newport helped articulate precisely why it’s extremely important to cultivate the ability to perform “deep work.” I recommend reading the book, but the tl;dr is that in order to succeed in today’s economy, one needs to be able to perform extremely cognitively demanding work with speed and quality. In order to accomplish that kind of work, one needs to be able to “focus intently without distraction.”

The constant stream of email, texts, meetings, and phone calls that are commonplace in modern offices help keep us informed and connected to our coworkers, but it often prevents us from being able to maintain sustained periods of focus on cognitively demanding tasks. Newport makes several recommendations to combat distraction, but I found the most helpful advice was to schedule every minute of my day.

The purpose of scheduling every minute of your day isn’t to hold yourself to an itinerary without any allowance for deviations. Instead, it’s a way to be more intentional about how you’re spending your time. More importantly, it can help you better track the amount of deep work you’re actually accomplishing.

I currently use my calendar as a guide for how I’m going to spend my week. There are always meetings I have to work around, but every other minute at work is allocated to a specific type of work. Despite my best intentions, reality doesn’t always align with the calendar I set, so I needed another system to help me better track the work I actually accomplish.

Enter Cortex.

Cortex is a podcast featuring Myke Hurley and C.G.P. Grey talking about how they get their work done. In their third episode, Grey reveals that he uses Launch Center Pro and Due as a time tracking system. The workflow goes something like this:

  • Start a 40 minute timer in Due & begin working
  • When the timer goes off, open Launch Center Pro, which has several actions associated with a specific type of work you do
  • Trigger the action associated with the work you just performed, which adds a row to a spreadsheet detailing the time and type of work you’re doing.
  • Reset timer and continue working.

Like any good nerd, I’ve tweaked this system a little. I’ve integrated Drafts actions with the Launch Center Pro actions so I can capture a little more context about the work I’m doing. I also use R Markdown to convert the spreadsheet of my time tracking data into charts in a PDF that help me understand how I’m spending my time.

This took a little bit of work to set up, but it’s been extremely helpful. Every Friday afternoon, I look at the PDF produced by the R Markdown script I wrote a few months ago. It’s helped me realize how productive I am before 9am. I now make sure that before I leave the office, I’m clear on the work I want to tackle the first thing the following morning. Conversely, looking at the data also shows me I still have to work on making the afternoons on Thursdays and Fridays more productive.

If you’re interested in improving your capacity for deep work, start with tracking your time.


Ezra Klein interviews Yuval Levin

This is a thoughtful, in-depth conversation that covers from the thesis of Levin’s latest book, The Fractured Republic, his experience working in the Bush (43) Administration, the most interesting policy ideas on each side of the political spectrum, and more.

This is the first Ezra Klein Show episode I’ve listened to - I’m looking forward to adding others to my Overcast queue.