March 29, 2012 by huionn
Over the years, based on observation on the open source projects I use, the software companies I was working in, I think there is a certain pattern for productivity of software productivity over time.
At the beginning (Phase I), new technologies are explored, new frameworks are applied and the business requirement is not clear. Therefore this is the learning and exploring phase for the development team.
When the team becomes productive with the platform, the productivity start accelerating (Phase II). More people will be put into the team at this phase. As the infrastructure is basically complete, every team member is familiar with the APIs and modules.
Phase III is the maturing period when the product is launched to market. Based on new requirement, more and more enhancement is added into the product. As the application is still relatively new, customization is still relatively cheap.
In Phase IV, the product is basically mature and feature-rich. Before this stage, software quality is often ignored for faster time to market. On the other hands, for the team that invest more in architecture and enforce software quality, this is the pay-off. (Nevertheless, I agree that shipped product is better than perfect product.)
I have observed that a few open source projects achieve Curve A through evolving architecture. New technologies are applied to their adaptive core. The architecture is modular that extension module can be plugged into the core. The examples I am aware are Spring Framework, Nuxeo (CMS), NetBeans, Eclipse, JBoss and Joomla!. I don’t mean these projects are perfect, but at least they confine the sloppiness in limited scope.
Projects achieve Curve B through refactoring and maybe partial rewrite. Unused and obsolete feature are abandoned, new features are applied through extensions. Design patterns are applied to the hotspots. The software quality is maintained defensively. Nevertheless the slight decrease in productivity is inevitable as the codebase is increasing.
Most companies I was working in follows Curve C. As quality is less tangible than revenue and cost, the quality deteriorates until it becomes too late to notice and too costly to recover. As the development cost rise, it eats into profit margin and therefore there is urgent need to rush for delivery. Then “quick fix” starts appear here and there. Duplication increases as developers are afraid to modify existing codes. The software is being bogged down by its own weight.
At the end, there is no silver bullet to overcome this – dilemma between short term growth and long term sustainability. The success of those open source projects are because of their top talents in teams and unlikely to be imitated elsewhere. But the lesson I learn is that software debt in architecture has highest interest rate. Software quality issue can be fixed but architecture defect is risky and costly to “patch”.