The recent 10 year anniversary of the Agile Manifesto is a good reason to be reflective on where software practices have come from and where they might be going. Plus the Oscar’s are on TV so I have taken refuge in my office and I’ve run out of new books to read.

Parallel Worlds

I do not think Agile software development methods have crossed over to the mainstream developer. That may seem a confusing and contentious statement, given on how much noise and words are generated in the community, but I suspect it to be true. We’re in a curious time where the two worlds of people-over-process and process-over-people seldom now need to coincide. These people do not meet each other at conferences.

I keep coming across entire offices where unit testing is considered some new ‘pointless what if’ dark art, where refactoring code is seen as a weight dragging schedules down to the abyss and business’s treat software as some hell-forsaken never-ending cost-center that needs to be continuously kept in check through arbitrary metrics. This ‘Dark Matter’ to balance the equation of industry numbers is out there. I know it’s wrong to wave assumptions around like a baby with a loaded gun, but I’d take a bet that less than 10% of software teams working today actually follow the principles behind agile.

Additionally, I think these two worlds are actually diverging rather converging, in that those who agree with the cut and thrust of using an agile process are now on a path of enthusiastically pursuing it, while those who have chose not to have more or less now decided to ignore it. Agility in the processes, the teams and the ethos you regard quality with, is heading in a direction that’s diametrically opposed to what’s considered ‘traditional’ software development. Both camps are pretty well entrenched on their respective paths, ready to disown each other’s foolish and ultimately doomed ways.

Elephants

The elephant in the room for adopting Agile processes is what to do when you reach the impasse with those running the business: Of what to do when you come down to the ideological gap between putting trust in people and delayed gratification in front of the cold and hard reality of the immediate bottom-line results and dollar revenue streams. One solution has been the ‘productization’ of the agile concept (and which excellently discussed by William Pietri here) but that just seems to leave both ends of the spectrum disappointed – we now have Scrumbuts fighting the long death-march dual-front of both Agilists and Non-Agilists with their disdain for lack of purity in a weak middle-ground of nothingness.

The underlying problem is that it seems to come down to having to completely change the culture of an existing business. This can be done internally, and often is done by heroic souls today, but like the advice of how to eat an elephant (‘one bite at a time’) after a while anyone’s going to get pretty sick of tasting just bad elephant every day. I see a lot of people who would previously tried to change an organization within now feel it’s easier to just go compete with their former organization’s business – ‘If you can’t beat em, take their lunch’. The lack of repeatability of costs and features makes it a frustrating job to try to change the mindset of a business through the necessary terms and language they need to use; that of short-term-centric revenue and expense.

It seems the future of Agile is not to change existing processes and thought, but for people to now separate software away from intransigent businesses and see it as either something they either buy or hire 3rd party specialists to go and build for them. If building software is not your core competency then perhaps embrace it entirely or treat it with the contractual distrust you probably would be more comfortable with.

It might sound a little nihilist for a ‘You can’t get there from here’ gloomy prognostication, but what I’m trying to say is that the biggest lesson of Agile software might not be to change how everyone can do better software but to make the gap between good and bad so obvious as to split into a specialization of core and non-core competencies.

Small is Beautiful

The most major difference to software development over the last 10 years for me has been the amount of successfully working software that can be produced by an increasingly smaller and smaller sized team. Frameworks have specialized in lots of different areas and languages have gradually improved. You can get a lot done by clambering over the back of a giant’s head and it seems like the days of massive teams are increasingly edge-cases for the insane few.

Objects have won as both an accepted mainstream design and language construct, even surviving the barrage of whatever set-theory could be misused from relational databases. We even seem temporarily safe from top-down code generation tools (which I swear are correlated in their arrival by solar sunspot cycles). Sharing code is now more social and learning has been commoditized to the extent that a diaspora of interests seems normal. Not so many people are proud of just knowing a single language inside-out, but of the breadth of techniques they like to know and use.

The Future

The most simplistic thing I could do about predicting what would happen next in software development (the Oscar’s have now finished) would just be to extrapolate what we have seen over last 10 years and just project a straight line forward. This precludes new invention, which is never a good idea to bet against, but it’s still an interesting exercise.

On that basis, over the next 10 years perhaps we’ll see more temporary teams and self-employment. As teams get smaller, as frameworks get more specialized and productive then do we head for a future where we form temporary collectives of interest? A team can’t get much smaller than one, and while teams will still get together to collaborate, the future of software developers may well be more like Doctors, or Lawyers or Prostitutes (yes, let’s not talk ourselves up just yet)– specialized but self-supporting, collaborating but independent.

The combination of businesses not wanting to write their own bespoke software, of teams getting smaller and more specialized and of software production become more efficient (or at least repeatable). The story of Agile may not have been how to solve the world’s software problems but to highlight what is hopeless and what is achievable.