Software metrics: A Tufte-inspired graph for software development projects

By Alvin Alexander. Last updated: June 4, 2016

I couldn't sleep last night so I started reading Edward Tufte's "The Visual Display of Quantitative Information", and I started thinking about software project metrics that might be useful to the team members on a software development project (including developers, business analysts, domain experts, and stakeholders), and how I would show these metrics in simple but data-filled Tufte charts and graphs.

As a little background, my recent experience is being the project lead for a multi-million dollar software project consisting of several implementation phases over the last 4.5 years, and I'm also a Certified Function Point Specialist (CFPS), and very familiar Function Point Analysis (FPA) and software cost estimating. Given this background, I feel somewhat qualified to write about software metrics. I also own all of Edward Tufte's books, and saw him speak on one of his tours, so I'm at least familiar with the concepts of Tufte charts/graphs.

Possible Tufte inspired software metrics graphs

Some of the software metrics I considered are shown below. When thinking about how to graph any of these software project variables, I'd have these variables posted on a Y1 axis on the left of the graph, and the X axis would be time. I think it's also important to show "number of developers" or "number of team members" with any of these variables, so I'd also want to plot those variables on the same graph, presumably on a Y2 axis on the right side of the graph.

Source lines of code (SLOC) per day
Total SLOC
SLOC churn/day
Bugs opened/day
Bugs closed/day
Total bugs
Project cost versus time
(Many more deleted here; you'll see why)

(If you're read some of my writings before you know I don't believe in SLOC as a project metric, and I do believe in Function Points combined with complexity factors. I just used SLOC above to simplify this initial discussion.)

I considered creating all of these "software project metrics" graphs, and many more, but -- I don't think any of them are important. I believe Mr. Tufte would call them "chartjunk".

Tufte charts and software metrics: What stakeholders need

Having worked with project stakeholders very closely over the years I think the only important questions from their perspective are:

Are you done yet?
If not, when do you really think you'll be done?
How much is this going to cost?

If you agree with these questions you can skip these two paragraphs ... but if you disagree with me, think about it this way: You walk into the car repair place, and they've had your car for two days already. What questions are you going to ask them? Now look at those questions again.

One thing I've learned about software development is that project stakeholders are incredibly vulnerable when it comes to software development projects, much more vulnerable than you are walking into the auto repair place (at least in terms of cost). It's all about trust. They're basically saying "Here's a big pile of money, please don't screw me." When they keep hearing things like "Just two more weeks" over and over again, pretty soon they know they're getting screwed. And once they lose confidence in you, you'll soon be gone, and they're going to find someone who really knows what they're doing.

Software project metrics - One good Tufte graph

Now, given that those are the only important questions to project stakeholders, what metrics are really relevant?

IMHO the only thing you can do -- get ready for it -- is use Function Point Analysis (FPA) to make an initial estimate of the project size, add in the complexity factors, and then prepare a graph at the end of each development iteration showing the functionality delivered versus the overall estimate, like this:

  ^
  |
e |
s |
t |
i |............................................. 
m |                 (original baseline estimate)
a |
t |
e |         .. (current delivered functionality)
  |      ...
  |  ....
  |.. 
  +-------------------------------------------->
                   (time)

Of course with 99% of all software projects what really happens is that you end up chasing a moving target. It's like if you were to start running a 5K race, but then as you're running along the finish line is moved to 6K, 7K, etc. So that simple graph turns into something a little more complicated, like this:

  ^
  |            ............................  (2)
e |            . 
s |       ......  (1)
t |       .
i |........         (original baseline estimate)
m |                 
a |
t |
e |         .. (current delivered functionality)
  |      ...
  |  ....
  |.. 
  +-------------------------------------------->
                   (time)

(I hope Mr. Tufte won't condemn me for these fancy ASCII graphics. :)

In this chart "(1)" refers to the new overall project estimate following a change in application functionality, and "(2)" represents the second change. You continue to show the baseline, but in this case, line (2) is the new current project estimate.

I think this graph is a lot more like a real project, but, it's what a stakeholder rarely sees. He hears about such things from his goal donors and project managers, but he never sees this in a graph.

And I think this is the only software metrics chart that's really needed. It's like Martin Fowler has said about UML: As a trained engineer he thought UML was important, but at the end of the day, did it really contribute to more accurate code getting produced faster? I have the same feeling here. You can have all the fancy graphs you want showing SLOC, bugs/defects, developers -- and all those things don't matter very much. The only thing that matters is "Are you done?", followed by "If not, how much longer?", followed by "How much is this going to cost?"

Tufte charts and software metrics - Summary

Wow, this could easily turn into a Tufte charts and graphs Master's thesis, but to keep it moderately short I'll bring this to a quick close, and be a little anti-Tufte and summarize my thoughts about software development metrics in the following bullet points:

Using FPA you can take very educated guesses at overall application size very early in the project lifecycle. Please, please, please, take those "off the cuff" estimates and put them in your back pocket, and use a technology like FPA.
- Note #1: Using FPA can be very fast, and usually accounts for less than 1% of a project's overall budget.
- Note #2: If you use FPA over time at your company you'll also know the average scope creep of previous projects at your company. (Hmm, that could come in handy ...)
Now, regarding that second chart, yes, this trend happens on 99% of all software development projects.
No, there isn't much the industry has to offer to keep this from happening. Users touch the software and realize what they thought they wanted isn't what they really want. (And, as the Scrum guys will tell you, software development is an empirical process -- it has to be this way.)
While this makes it very hard to accurately predict total cost and project duration -- especially on very large projects -- people that run projects have to try, it's the only way you'll get better and give the sponsors what they need.
For project sponsors, the people that can be your best friends at times like these are experienced goal donors, project managers, and developers. And when I say "experienced", I mean experienced on a project of this size and complexity level. They should already know that you don't really want what you said you wanted, and they should be able to use their best practices to try to get through that problem.
You need to mix in less-experienced people on these projects so they can gain experience for the future.

Update

I originally wrote this article in July of 2007, and just updated it today in October of 2009. From what I've seen the Scrum methodology guys have things about right. They have stand-up meetings every day where they ask developers to answer three questions that are remarkably close to what I just wrote: