I took some heat for my article Time to Vanquish the Mythical Man Month, where I brought up was the idea that the analysis in the Mythical Man Month might be wrong, and consequently, I might recommend different ways to manage a big project. After hearing my comments at the Agile Boston presentation, Dan Mezick sent me an interesting followup question about that. I'll review the issue here, and then the question.
The Mythical Man Month is a classic book from Frederick Brooks, who was responsible the IBM OS-360 project. He considers the problem of large software projects, which tend to move slowly and have low productivity. There are two frustrating aspects to this. The first frustrating aspect is that everyone gets slowed down. A programmer who can produce 100 debugged lines per day on a small project might only manage 10 on a big project. The second frustrating aspect is that it seems impossible to speed up by adding labor. Brooks famously says something like "Nine women can't produce a baby in one month".
Brooks hypothesizes that this problem stems from a sort of communication overload. If N people are working on a project, they have N^2 ways to talk to each other, and that gets to be a lot of work as N increases. He then suggests some ways to reduce communication load through specialization (for example, a tool master) and modularization, breaking the project into functional groups that only communicate inside a small part of the hierarchy.
It's clear that big projects are slow, hard to manage, and hard to accelerate. However, I am not persuaded by Brooks' analysis of the problem. There are 6 billion people in the world, but I don't have to send them all a Christmas card, or write on their facebook walls. Nobody forces you to communicate with them.
I propose an alternate hypothesis based on dependencies. In a big project, a lot people are waiting for components from other people. If 100 people are working, and 50 people are waiting for something, then you are already down to 50% of potential productivity. The problem could actually become a lot worse if everyone is waiting for a few critical components.
If you believe the dependency hypothesis, then you believe that more communication is better, because it helps people work around the obstacles. You also believe in sharing code, because if someone is waiting for something, and they get desperate enough, they can fix it themselves. If dependency is the problem, then big open source projects should scale better than projects with more rigid task assignments, and there is some evidence that this is the case.
Dan Mezick wrote:
You state it is not the N^2 connecting points but rather the dependencies generated between code and coders...basically queueing issues. That being the case, does that tend to steer your design decisions towards [dependency injection, inversion of control, loose coupling architectures?? Do you make design decisions with intention to diminish or eliminate code dependencies?
My response illustrated how you would work differently if you were concerned about communication, or about dependency.
I agree that reducing dependency by using modular code is really important. However, it's so important, that it's important in any methodology. I'm not sure that we place a bigger emphasis on it than anybody else. And, it's more important when you are doing maintenance, and you want to make sure that changes and upgrades in one place aren't causing a requirement for changes and testing somewhere else. For new development, you are often in the situation where you just need a capability. It's not implemented, or it's implemented wrong, whether it's coupled or not. In this case, the organizational structure becomes important. Can you fix it yourself? Do you have the ability to communicate with the person who you are waiting for? In this case, you might want increased communication and visibility, which is the opposite of the Mythical Man Month recommendation, and maybe even goes against the idea of modularity. For example, you might want people to commit incomplete or even broken code to the daily build, so that you can see what is working and what isn't, rather than doing the modular thing and keeping it to themselves until it is ready for someone else to use.