Current Articles | RSS Feed RSS Feed

Time to Vanquish the Mythical Man Month

Posted by Andy Singleton on Sun, Aug 10, 2008
  
  

Developing big and complicated systems quickly remains a difficult problem.  Big, complicated systems cost a lot more per "function point" than smaller systems, and they famously seem to resist attempts to accelerate them.  It often can't be done, even when a big organization is willing to spend unlimited funds to get results.  However, the availability of unlimited funds should provide motivation for the entrepreneurial among us.  I think these big projects CAN be accelerated, using ideas pioneered in open source projects.

The Mythical Man Month is an analysis by Fred Brooks of the problems involved in truly large software projects, using the example of the effort that IBM started in 1967 to build OS/360, the operating system that would go on to run most of the world's mainframe computers.  Brooks noted that "Adding manpower to a late software project makes it later" (Brook's law).  He also pointed out that "Nine women can't make a baby in one month" - a reference to the problem with the idea that you can add more people and get productive "man months" out of them.

You have probably read this book.  At a recent gathering of 25 CTO's, I asked who had read The Mythical Man Month, and 23 hands went up.  It's a great book, but it's dogma now, the conventional wisdom.

The statistics in Software Assessments, Benchmarks and Best Practices put numbers to the scaling effect.  Productivity is measured at 15 function points per developer month for software having 100 function points, declining to 5 per developer month for projects having 10,000 function points, at which point 67 people were involved.  The author notes that "projects having more than 100,000 function points are more likely to fail than to succeed."  Typical staff size on these projects was 690.  There lie dragons.

Software development productivity has improved enormously in the past 30 years.  Today's developer has fast hardware, dynamic scripting languages, instant access to tools, documentation, and advice over the Internet, and decades of experience in development process to draw on.  As a result a contemporary developer able to produce a lot more "function points" per day than a developer could produce 30 years ago, or even 10 years ago at the height of the Internet bubble. 

If one person today can do the work of 5 people from days gone by, that in itself ameliorates the problem. For any given project, our staff only needs to be one fifth as big.  We see small entrepreneurial teams doing work that big companies did 15 years ago, and bootstrap entrepreneurs producing products that would have required millions in venture funding ten years ago.   If you don't want to fail, don't take on big projects.  You can wait for them to become small.  However, this is an unsatisfying solution, because our software becomes continually more complicated, and over time, the functionality expected by the user inccreases just as fast as our developer productivity.  To produce great systems, we must slay the dragons.

I should also note the the scaling problems are actually worse for small projects.  For example, when you go from 1 person to 2 people, you get a big decline in developer productivity.  I usually estimate that you need two developers working 40 hours per week to replace one developer working 60 hours per week.  That is exactly why good developers work such long hours.  Improved scaling ratios will have a disproportionate benefit for small teams that need vacations.

I am one of the few development managers who feels free to ignore Brook's law.  I will add a lot of developers to any project, if I can afford it.  My limits are imposed not by productivity, but by the fact that I have limited funds for my own product development, and that my clients, like me, are cheap entrepreneurial bastards.

Linus Torvalds provides a more famous example of a developer ignoring Brook's law.  Although much of Linux is built by a small core of very dedicated developers, there are currently about 1000 kernel contributors.  This has produced a steady, rather than a declining, scaling effect. The kernel has consistently grown by about 10% per year.

I think that Brooks is fundamentally wrong about the cause of the scaling problem.  He points the finger at communication problems.  If you have N people working on a project, you have N squared communication channels, and each person needs to spend more time communicating.  Furthermore, if a new person tries to come into a big project, they have to get a lot of information from a lot of different people, so the ramp up time is increased. 

Brooks proposes various tactics to reduce the complexity of communication, including having a master architect produce a well-documented architecture and manual, and having designated "tool makers" to supply each team and subteam.  Essentially, he's recommending that we translate a network of technical communications into a hierarchy with a lot fewer connections.

If we compare Brooks' recommendations to typical practices for Linux development, there isn't much overlap.  Code is accepted hierarchically, moving from "contributors" to "maintainers", but this is the output, not the raw material.  It's done to control quality, not to provide information.  Tools and architecture ideas are expected to come from a variety of sources.

I think the scaling problem is not a communication problem; it's a dependency problem.  It's not necessarily true that when you work on a bigger system, you need to do more communication.  However, it is always true that you depend on more things, so you are more frequently waiting for something else to get finished.  Take a close look at a development team of 100 people.  At any given time, 50 of them are waiting for something from someone else.  That effect alone can account for the observed 50% degradation in productivity compared to a six person team where a blocked team member can demand, or build, an immediate fix.

From this point of view, you can see that open source projects have a huge scaling advantage because all code is shared.  If someone is waiting for a component, and they are frustrated enough, and talented enough, they can just fix the problem.  The code is all shared, anybody can build and fix any component, and the responsibility for critical components can move around.  The answer to the dependency problem is less hierarchy, and fewer official tool builders, not more.

If we do grant some truth to the idea that bigger systems require more time from each developer for communication, we can see that are ways that modern developers reduce this time.  Most notably, they share code.  Code is a relatively crude form of communication compared with deliberate documentation, but it's accurate, and it's immediately available.

Furthermore, Internet projects tend to communicate in writing, on tickets, blogs, mailing lists, and wikis that are accessible to all team members.  This changes the communication from a network of conversations that take time from a lot of people, to a simple text search that a team member can do individually.  Perhaps this is why the time spent on conference calls is, in my observation, a good indicator of management problems.

So, what are the takeways for me?

  • It is possible to be productive in big groups, and I think it could be a lucrative area of study.
  • It is important to share code, and very important to accept new components and fixes not only from designated developers, but from the users of that code.
  • It is important to share ideas in writing.
  • Control quality by being hierarchical and rigorous about how you test and accept changes, not how you generate them.

 

Tags: ,

COMMENTS

I think you're right to some degree - there is definitely more opportunity to work as a cohesive unit, and with the advent of Wikis and source control a lot of that manual sharing of information can now be handled, especially if design decisions etc are documented well. 
 
I think there is still some truth to the idea though that you cannot just throw people at a project and expect it to get done faster.

posted @ Sunday, August 10, 2008 7:42 PM by Sid Savara


Hi Andy, 
 
You propose an interesting counterpoint to the arguments raised in The Mythical Man Month, but I believe the projects that are discussed in the novel are different enough from Linux projects that they cannot be compared against each other well. 
 
For instance, the degree of specificity required by a typical large software project versus that of the Linux project is an issue that arises. My relatively ignorant view of Linux is that a problem presents itself, and an individual (or team) solves the problem to the best of it's ability. The same approach is used by small firms producing products. They do what they think is best. 
 
Larger firms, on the other hand, are much more specific in their requirements. They don't turn to a developer and say "Hack until this works." The approach draws more on "You need to do this in this specific manner, no questions." The problem of communication, and lost efficiency, arises when those who do the legwork are separate from those who do the overall design. 
 
Perhaps if a large company is better able to trust its developers to do the project well, then we wouldn't see the sharp decline in productivity. Of course, not all developers are created equally, but if you eliminate the need to communicate by sticking with broad outlines and general compatibility, then the issues do not arise. 
 
Thanks for reading!

posted @ Sunday, August 10, 2008 8:00 PM by Shamiq Islam


@Shamiq 
 
Not sure I agree with your assessment of large firms and their requirements. Every job I've worked at has a mixture of solid, well planned requirements, and then bugs that pop up and need to be fixed in enterprise applications ASAP - just make it work, pronto! The same thing happens close to crunch time as requirements change. 
 
I think you and I agree on this point though, that it is easier to do work separately with minimal communication (as much open source software has) if the software is already fairly stable and people are just working off on branches/components that are sectioned off from each other.

posted @ Sunday, August 10, 2008 8:23 PM by Sid Savara


@Sid 
 
I am quite critical of large firms, so my bias does appear in the writing (I didn't realize it was that obvious when i first wrote it...). 
 
You are right in terms of our shared philosophy. Focused, distinct units work very well. If the need for communication appears, then direct questions to the head of an individual branch should be able to sort out the difficulties. 
 
By the way, thanks to Andy for letting us use his comments as a basis for our discussion.

posted @ Sunday, August 10, 2008 8:31 PM by Shamiq Islam


Your opinions are nice and everything, but whereas Mythical Man Month was a scientific study, you fail to provide any statistics that backup your loosly connected theories. 
 
Let's take, for example, the following claim you made: 
 
"Software development productivity has improved enormously in the past 30 years. Today's developer has fast hardware, dynamic scripting languages, instant access to tools, documentation, and advice over the Internet, and decades of experience in development process to draw on. As a result a contemporary developer able to produce a lot more "function points" per day than a developer could produce 30 years ago, or even 10 years ago at the height of the Internet bubble." 
 
Let's dissect this paragraph: 
 
"Software development productivity has improved enormously in the past 30 years." 
 
Define "enormously." And how do you know this to be true? 
 
"Today's developer has fast hardware, dynamic scripting languages, instant access to tools, documentation, and advice over the Internet, and decades of experience in development process to draw on." 
 
Again, where this list comes from seems magical, with the implication being that all of these things have had a direct effect on a still-yet-to-be proven dramatic improvement in software success. For example, dynamic scripting languages are more prevalent on the web than they are in primary building blocks of corporate applications (this is slowly changing, which is good news). One could even argue that dynamic languages are specifically suited for rapid development/prototype on the web. Perl isn't so much different from PHP, and is in fact probably more relevant to corporate applications (and it's still being developed with Perl 5 being a big rewrite with YAML, etc.), yet Perl has been around for a while and hasn't necessarily fixed productivity problems. 
 
"As a result a contemporary developer able to produce a lot more "function points" per day than a developer could produce 30 years ago, or even 10 years ago at the height of the Internet bubble."" 
 
Languages have evolved, yes, and thank goodness! But has success evolved? You don't make a very compelling argument. 
 
Thanks, 
Bob

posted @ Sunday, August 10, 2008 9:05 PM by Bob Yakovic


You do make some good points, however you are talking about team sizes and equating team sizes to adding people to a late project. 
 
Brooks' Law says that adding people to a LATE project makes it LATER.  
 
It doesn't say big teams are bad. You are talking about team sizes.

posted @ Sunday, August 10, 2008 9:33 PM by kevin


Hi Andi, 
 
Your point of view makes a lot of sense. 
 
I've always been surprised by the way a company works (in terms of software projects). I've seen thousand of guys in open source organizing themselves and achieving great results whereas I've seen less complex projects fail in the enterprise world. 
 
I come to the conclusion that a company could work with the methodologies used in the open source world — as you mentionned — if it provides such great results. 
 
But finally I've come to a different conclusion. The enterprise world and the open source world are fundamentally different, and major difference is: the willingness of developers to participate in the common effort. 
 
In the most company, developers consider only their job as a job, they don't share a specific enthusiasm with the final outcome — an the open source developer choose to participate to a specific project. 
 
This is why the open source methodology won't work for most companies and they'll have to use the classic methodology of software developement (i.e. requirements, specs) in order to control the outcome quality. No matter the productivity.

posted @ Sunday, August 10, 2008 9:50 PM by Fred Brunel


I think that you are missing what Brooks's law really says. It basically says that serial tasks are can't be performed in parallel. That's why nine women can't have one baby in one month. But three women can have three babies in nine months, whereas one woman would take 27 months to perform the same task. 
The more you can isolate and parallelize, the more people you can add to the project.

posted @ Monday, August 11, 2008 9:41 AM by BA


Be careful when you try to slain the werewolf with your new shinny silver bullet! Countless made that claim and failed. 
I would say: feel free to ignore "The Mythical Man-Month" at your own peril! Although, you'll not be alone in the tar pit... 

posted @ Monday, August 11, 2008 4:57 PM by Gabriel C.


@Andy: 
You propose that projects are limited by dependencies more than communication. I seriously doubt this is the case really. 
 
Project planning is a pretty well known field, and ensuring that tasks are done in the correct order wrt dependencies is not something you have to invent. Eg take a look at "critical chain project management", which at least the last 3 SW companies I've been at have used. 
 
What I see is limiting is not so much dependencies but complexity of products and finding out whom to talk to. If you are in an organization with 10 people this is not a problem. Currently I have 500+ co-developers, and a software product that matches that in scale. Needless to say this puts a very different requirement on communications than a smaller project.

posted @ Tuesday, August 12, 2008 8:31 AM by Hast


http://www.it-cortex.com/Stat_Failure_Rate.htm

posted @ Tuesday, August 12, 2008 2:46 PM by Hello


You're kidding right?  
 
Brooks' law: adding manpower to a *late* software project makes it later. 
 
Linux has the enviable luxury of never being late. There are no schedules, no market deadlines to meet. It's done when its... actually its never done.  
 
On projects for which there is a paying client or a competitor waiting to eat your lunch, ignore the core of Brooks' advice at your peril. 
 
Adding manpower early to an understaffed project will improve its chances of on time completion but as sure as someone on the internet is being called Hitler, there will reach a point where you will get negative returns for every engineer you add to that project.

posted @ Saturday, August 16, 2008 3:52 PM by Ifeanyi Echeruo


Hello,  
 
Your point is good that Open Source software project doesn't suffered from Brook's Law. But, as other may have mentioned, I want to add one more point to contradict your stand. 
 
Those Open Source contributor are already familiar to the project by the time they've been promoted. Not only they have already become power user for the project before starting to give some code. But also, the selection process is quite complex. Those who make enough good patch submission (via e-mail, forum, IM, whatever it is.) are more likely to be promoted compared to those who are newcomer to the project. Who are be able to submit numerous patch should have pretty good understanding in the project. Time that those new "contributor" to spend time learning the project would be reduced. 
 
But, on the other hand, those commercial software project doesn't have this kind of advantage. Their selection process may involved with some background interview, but this doesn't make candidates have some insight knowledge about the project they're going to join. So they have little knowledge by the time those candidates become full-time developer for the project. They have to familiar themselves to the project, from their colleagues, a lot. So the whole team has to spend some amount of time teaching new developers. 
 
Just to point out some other characteristics which makes open-source projects differed from commercial projects, and made Brook's Law less relevant from open-source projects. 
 
PS. Sorry for my weak English.

posted @ Thursday, August 21, 2008 1:20 AM by ekkmanz


Hi Andy, 
 
I Agree with you in almost 100%, but I think there is an concept used in all these analysis that is: "At a given time, any developer in a project can build n function points". 
 
The skills and the personality of a developer are very important. 
 
One good can do the job of 5 developers.

posted @ Friday, August 22, 2008 4:25 AM by Jorgebg


Add

posted @ Wednesday, September 10, 2008 7:33 PM by Gisele


I agree that tools have made development easier, however no one is mentioning the increase in software complexity and customer's expectations over the years. Lets take a website development project as an example.  
 
Back in 1997 it was just HTML, some javascript if you wanted to be really cool and use a simple WYSIWYG to do most of the work and nobody really bothered much if an image was 2 px wider. Today its HTML, CSS, AJAX, XHTML, dynamic content, not to mention website security like sql injection attacks, cross site scripting and Browser compatibility. 
 
Fortunately tools did improve the process coupled with better methodologies like DSDM and SCRUM to mention a few, however we can't really say life is easier now since work load has increased 
 
I think one should uses laws like Brook's law as guidelines and not treat them as absolutes that cannot change. One should try to use different techniques and measure outcomes. Software engineering is not an exact science yet, mostly because it deals with people not machines.

posted @ Thursday, October 16, 2008 5:34 AM by Adrian Hedley


A very thought provoking essay. 
 
 
 
An analogue of the problem can be posed this way: Can three authors write a trilogy in one third of the time? Certainly with careful techniques, through careful story-planning, and minimizing the number of characters that pass from one volume to another, a trilogy can appear to be consistent, even though each author has only a passing knowledge of what the other authors had written. 
 
 
 
Similarly, improvement in software techniques, better understanding of componentization, and a better software stack (three tier is now de-facto) leads to more understandable processes and better separation of concerns. 
 
 
 
Software mashups is a refutation of the mythical man-month, in the sense that interesting programs can be stitched together with minimal communication overhead between development teams. 
 

posted @ Thursday, February 12, 2009 9:38 AM by Chui Tey


I solved the MMM NSB but I can't get funding. It took me 5 years to solve it. I had to rethink everything about programming to figure it out. 
I figured out how to develop (build) software 1000x faster and reuse all code and reduce bugs by more than 90% and build code based on human thinking instead of machine thinking (meaning). If you wanna fund me contact me. Yes, I'm in Silicon Valley.

posted @ Friday, August 14, 2009 1:39 AM by Blake Southwood


I think there is another key aspect to Linux (and most open-source software) that you are forgetting: in Linux, the user and the client are the same person. 
 
When writing an enterprise application, they frequently are not. 
 
This (obviously) has implications towards productivity and communication. While it is not a refutation, it still damages your argument to run an enterprise application like an open-source application.

posted @ Thursday, October 08, 2009 7:50 PM by Keith


The communication problem comes from the dependencies. If I have no dependencies, I don't need to talk to anybody. If I have to interface with 100 poorly documented libraries, I have to talk to the 100 authors. Or I have to reverse engineer the libraries, which is just a different way to be unproductive. 
 
b.t.w. The Linux kernel is a poor model for this discussion. It shows that dependencies are relevant, but we have no idea how much developer time it takes to create linux. Open source developer time is "free, as in beer", so nobody needs to track it or find ways to improve productivity. 

posted @ Monday, October 19, 2009 9:34 AM by Mark


Happy New Year -- My youngest son, a computer-programming-major, brought home a copy of the 1995 anniversary edition of MMM and I found this post while looking for info on the revision (now 15 years old). I enjoyed Andy's article and the discussion, especially the insights related to the difference between open source projects and "bespoken" development products. However, I am disheartened that the software community is still debating points in a book that was already 10 years old when I was using it before said son was born!

posted @ Friday, January 01, 2010 1:16 PM by Kate


Comments have been closed for this article.

Follow Assembla

twitter facebook youtube linkedin googleplus

Get Started

blog CTA button

Subscribe by Email

Your email: