The Scientific Method

Back in the 1990s I was fortunate enough to work for a smart, energetic man. In a way, working for him — or at least in the position he gave me — helped change the trajectory of my career into what I wanted it to be.

Skipping 99% of that story ... one thing he did exceptionally well was troubleshoot problems, and troubleshoot them very fast. I didn’t know it at the time, but he was using something called The Scientific Method. After observing him for a while, I saw him repeat these steps so precisely that I thought he must have them on a tattoo on the inside of his eyelids:

  1. Observe some feature, in our case, a bug
  2. Hypothesize a model consistent with the observations
  3. Predict future events the hypotheses should yield
  4. Verify the predictions by making further observations
  5. Validate by repeating

For our purposes the last step wasn’t usually too terribly necessary; by the time we got to Step 4 we knew if we were right or wrong.

The other amazing thing he did was to do this faster than anyone I have ever worked with. He would observe, formulate a hypothesis, make a prediction, then test it as fast as anyone I have ever seen. Besides doing this fast, he did it verbally; he allowed me to participate in the process, if often only as a sounding board. He would say, “We’re seeing X, so logically the cause must be Y, so if we tweak this, we should see Z”, or something to that effect.

How he did this was important also. He would state it with such confidence that you just knew that Y and Z would be as expected. But to my surprise, he was often wrong. But that didn’t matter to him, for him it was a game, and all of those small failures were irrelevant as long as he had the success in the end. (Because he stated his case so strongly, when the hypothesis failed to be true, he seemed very happy to know that that specific theory wasn’t the cause.)

Could he have taken a few more moments to consider each hypothesis? Sure, but he was a man of action, and while someone else might still be thinking through all of the possibilities, he had iterated through several possibilities, and almost had the problem solved in a matter of minutes.

The Scientific Method

I was reminded of this person today when I saw this “Scientific method” section in a book named Algorithms:

Whenever you’re trying to resolve bugs, I encourage you to follow the steps in the Scientific Method until you know them like the back of your hand. As with the person I worked with, don’t over-think things too much; come up with a hypothesis that fits the known data, test it, repeat.)

(Of course it also helps to know your tools, like how to read a stack trace, to look at log files, etc.)