Italian Agile Day

Date for the Italian Agile Day 2008 has finally been defined! Needless to say, I am happy this will be again in Bologna. :-)
See you there!

Me and MSProject

Yesterday evening I attended Craig Larman's brilliant talk -  in Skills Matter, in which he deeply criticized some commons dysfunctions of traditional project management, and doomed MsProjects and Gantt charts as useless and dangerous in software development.

I couldn't agree more with him, so imagine my surprise when today I discovered that on my naymz page I was actually sponsored by MsProject!

Needless to say, I am quite disappointed. :-(

I know it's AdWords driven, and that some keywords in my profile triggered the ad. But...

Why should I eat dog food?

I was reading this interesting article that my friend Mirco Casoni wrote on Mokabyte. He basically reviews and criticize the idea of framework and the unfortunate implementations of the framework concept he happened to see as a consultant (implementations which I probably know very well too, since we’ve been colleagues for years).

One of the main points touched by the article was the long-term ineffectiveness of in-house frameworks. A solution that was developed internally, to bridge a gap in standard implementations, turns later into a drag factor; the short term advantage becomes burden of legacy code and so on.

The old java legacy
One of the reason of “the rise of java frameworks” at the very beginning of this millennium (sounds epical, doesn’t it?) was the low level of abstraction of the java specification itself. JDBC, EJB, J2EE Web were coming out-of-the box without a high level implementation. SUN accompanied the technology with publications such as “Core J2EE Patterns” that used to be a bible in these years. It really sounded like “here is a bunch of small lego pieces hat you can put together to make ... everything! And here (but not in the same place) are the guidelines to make something useful and working properly”.

Unfortunately this is not what many people wanted: people wanted an efficient and solid architecture, easy to set up and reuse. But J2EE offered too many choice-points (how to implement a MVC Model 2? how to deal with JDBC exception? How to make EJB perform? and having the right answer to all he questions was a tough job. So, frameworks were seen as a way to formalize and distill the body of knowledge captured in various patterns book, and to prevent the rest of the team from making awkward choices.

Later on, the implementation of the java platform got better: old EJBs were dropped and many related EJB patterns sunk with them and JSF finally looked like an official standard implementation of a Model 2 MVC framework. It has to be noted that this happened after Spring got momentum, and after the MVC landscape got overcrowded...

Put in this way, there were tons of reasons to build frameworks in those years, because there were holes (or craters) to be filled ...anyhow. Unfortunately some solutions were better than nothing, providing a short term advantage, but rapidly transforming into a legacy nightmare.

If you love somebody, set them free
Some in-house frameworks look definitely naive now, but they looked like (and probably were) a reasonable solution, at the time they were designed (well ...not all of them). Specific design decisions apart, what really makes the difference between a friendly framework and a “traitor” is the cost of getting rid of them. Vendor design frameworks with a very high vendor lock-in potential (it’s part of the game, on the business side), but in-house framework designers often do the same mistake, with no real good reason. Depending on third-party is generally seen as a risk. Depending on yourself is often a higher risk, but seldom considered.

Documentation quality and availability (as Mirco points out) are other key factors, that should make you favor a standard solutions instead of your in-house framework. We used to have a good MVC solution, but switched to Struts as soon as it became a viable option, and so on. Few companies are large enough to afford production of documentation for an in-house framework (one of the reasons why they prefer to sponsor open source projects), so you have to be ready to leap from your solution to the promising open source one, sooner or later.

A framework fills a gap. This means that there is no standard solution for the given problem (otherwise, a framework would probably already be available) and means also that the in-house solution might be non-optimal. This is fine, we’ll probably get more informations about the optimal solution by implementing ours. What we should know in advance, is that we should “design for replacement”, meaning that our framework design solutions should have the primary requirement to be easily discardable. Thus limiting pervasiveness of our solutions.

Often, this is not a technology issue. In-house frameworks have “fathers” that make it difficult to replace them (more or less consciously or willingly) or simply slowing down necessary decisions. Can you be in love with a piece of code? Some do.
   
Which framework now?
Does it still make sense to think frameworks? Well, it largely depends: applications frameworks such as Spring, Hibernate or also Grails offer an off-the-shelf solution to many common architectural problems, leaving architects free to concentrate on application-specific domain decisions. On the other hand they’re all domain-neutral, and a domain specific framework could be of great value for a product company, especially if it is designed to be technology-neutral, and could provide great value in the mid-term.

Farewell Vista

I am finally getting rid of Windows Vista. My transition to Mac OS X is almost complete (some folks said that Vista is Steve Jobs' ultimate marketing tool).

There are still a couple of open issues (I have tons of valuable material in Word 2007 and PowerPoint 2007) and I am tied up with the new fonts that Microsoft added in Vista and Office 2007. So, looks like I am moving and keeping all the Office 2007 stuff (which by the way is an excellent tool) and dropping Vista (which by the way is crap) and all the related performance and productivity drag. Just to summarize, here is one of the reasons why I quit.

Agile SOA Presentation slides posted on Slideshare

I just posted the slides from my in-the-brain session @SkillsMatter on slideshare. If you want to have a look, just follow this link.

Sustainable Software Architecture

A software development project is a bounded activity. One of the key goals of Software Architecture is to find the best trade-off (or the sweet spot) for a given project ecosystem.
Such ecosystem is the result of many combined factors such as:

• project size
• team size
• team location (es. co-located vs distributed or offshore team)
• team skills (experienced developers, contractors, folks just hired, and so on)
• team members availability (not all teams are well-formed at the time the project starts)
• architect avalability
• turnover
• logistics
• marketing constraints
• deadline constraints
• etc. etc.

I learned to know that keeping software architecture clean and preserved from all this is a dead end road. Those ecosystem constraints affect the way a project is carried on, and will also affect the optimal architecture for a given project. Put in another way, there's no optimal architecture for a given problem, unless you include all the variables in.

As Kevin Seal pointed out at the last week SM event: "Architecture is about things that are expensive to change", and in the open source era, the most expensive things to change are time (which can't be reverted) and people, which have normally a long and inefficient learning cycle.

This makes choices like "choosing the implementation language" an expensive one, because despite the free availability of development environments for different platforms, team skills might have to be formed, and properly skilled.

Make the architecture fit the ecosystem

Answering to questions like "What's the right tool for a given load requirement?" is a non trivial job, but it's the one we're (hopefully) prepared to do. It's right in the software architect mindset.

What I've found more tricky is the need to define just the minimum affordable level of architecture, in a given application. In an ideal world we would like to have the most reliable architecture, making coding easy and meeting all of the nonfunctional requirements. We would also like to have all the team members getting familiar with that, and understanding roles of every architectural component and the reason they've been implemented the way they are.

Too much of a dream. Architecture definition is often a time bounded activity also. For a consultant,  the Architecture Specification Document might be a deliverable mentioned in the contract, so even if there are more efficient ways to deliver architecture information (podcasts, meetings, comedies, tattoos, ...just to name a few) a document must be prepared. But the contained information must be delivered some other way...

Pairing with programmers, or simply coding are great way to get a grasp of coding reality (meaning that architects might learn something really useful from the way architecture and other coding tools are used), but also a bit naive when the matter is "How to deliver architecture information". Architecture have (I mean they must have) a broader scope than developers, taking into account long term factors, while a developer is generally focusing on a problem that has to be solved now. And developers are not all the same. Seasoned veterans and college guys have peculiar skills ad interests, and a completely different background.

Moreover, properly training the team might be a goal for a company investing it its own development team while it might be not a viable option for an external or a contractors-based team. This might sound a bit cynical (it is), but even if I prefer teaching (after all, it's part of my job) as much as I can about the architecture in place, I have to face the fact that information will flow on a limited bandwidth channel: there will not be as many chances as I would like to discuss the architecture, there would not be as many questions, and developers might not be that curiuos, after all.

Finding the sweet spot

Generally, I think that the ideal sweet spot is doing as much architecture as it can be managed, by the team (buy I usually learn about the point location by surpassing it). This is not a fixed line. A good team will probably ask for a more elegant architecture, al long as the project advances, pushing (or pulling) architecture evolution. In some critical situations, like long projects with a lot of turnover, the architecture should be robust enough to prevent developers to harm themselves, keeping the learning curve as small as possible. Some more work for the architecture team, but usually a bit sadder.

How much Architecture is Enough?

During my London staying, I had also the chance to attend Simon Brown's in-the-brain event, which turned out to be more two-way than mine, with Simon softly facilitating a really open discussion. As it is often the case, interesting ideas emerged. Here are some of my side notes, mixed with some of my following ruminations.

Communicating the architecture

One of the biggest challenges seems to be finding the right way to effectively communicate the architecture. It turned out that architecture documents are not the best way to convey architecture information alternative and possibly more creative ways are needed, spanning from Wikis, to podcasts, to live sessions. Honestly, the Google Chrome Comics is a great way to deliver an architectural view. But maybe it's not a viable option for common teams.

Also, a prototype could bring so much information and is particularly efficient in edge situations, where the team is not properly trained or skilled and needs to produce some code anyway. Bad as it may sound, also cut&paste information propagation (the worst case scenario) might be preferable to paralysis or anarchy.

Championing the architecture

Without an active championing role, the architecture is at risk of being twisted or thwarted. Without an active role, an architectural specification is just a piece of paper.

Non Functional requirements

Managing non functional requirements seems to be a hot spot for many agile teams: requirements do not naturally show up as user stories, so their emergency has to be somehow facilitated. Also, automatic tests on nonfunctional requirements are necessary.

Strictly tying delivered software to business value might eventually put non functional requirements aside, but having a fully operating passed load test delivers both business value and confidence.

Resisting temptations

A pragmatic strategy to define if an occasional need needs to be transformed into an Architecture requirement is to count the number of times you've been hit by the need to refactor a given thing, with 'three' as a threshold for a rule of thumb. Before getting to the threshold one should "resist to the temptation" of refactoring the code to a more elegant solution.

Personal note: it's interesting to notice that this works only in teams where you have an efficient information flow within the team. If development process does not facilitate this sort of information exchange (es. no daily scrum meetings), your counting might reach '20' before the need for a given refactoring is communicated to the architect-in-charge (whatever that means).

Sustainable architectures

I also advocated the cause of "sustainable architectures": putting too much stuff on an architecture definition might result valueless if the team isn't able to grasp it. Sometimes it's better keep everything as simple as possible, or to step back and start with a less sophisticated solution, and invest in training or improving the team technical skills (or the architect communication ones) before building and obscure cathedral.

Deferrable architectures

A key point is that architecture is about things that are expensive to change in software applications. Interestingly, a too sophisticated architecture (architects love to create frameworks) is more frequent than the need to heavily refactor an existing one. Honestly, I think that in many cases, simply defining a clean layering architecture might be enough, to allow a clean solution of emerging architectural requirements.

Personal note: also, the cost of change has largely dropped: if we compare heavyweight frameworks that were popular some years ago, with what can be achieved with Spring and AOP in terms of pluggable, orthogonal requirements, it's clear that the scenario has changed radically, at least in the JEE landscape.

Should architects be part of the team?

It turned out that architects are sitting on the borderline of the team. There are architects that are definitely not in the team (and absolutely do not code), and some which are more willing to pair with developers, improving team skills and gaining feedback on the correctness of the architectural choices. At the end of the day, developers are architecture primary users, and, as an architect, I really would have my choices tested and challenged by a real feedback. On the other hand, an architect should preserve its ability to think long term, so can't be fully involved in coding, especially tasks which are on the critical path. Moreover, they might be external roles, hired on a short term basis, or with reduced availability, so being part of a fully involved Scrum team might be problematic. Doing Scrum by the book, the Team might ask for an external architecture support as long as the team thinks they need it.

Project start effect (personal note)

As a consultant, I might have a slightly biased perspective, but in many times, the architecture definition has just been the result of a "resource allocation incident". Sometimes, the team was just set and ready-to-code, so there wasn't really much time to have them wait for a good architecture specification to start coding. Sometimes, I joined a project when development had already started, so I had to impose an architecture over a (barely) existing another. Finally, some other projects would not even start before an "architectural view" documentation is developed, as this is part of the project budget negotiation by the management (SOA is a typical example of this case); so when the project starts you have more or less a "specification" already in place. In other words, the early days really influenced the whole project direction, and this rarely has to do with architecture, per se.

Tales from my first in-the-brain session

Just right after my long awaited holidays, I rolled to London as part of my recently started collaboration with Skills Matter as a trainer. As a side activity, I had the chance to held a so-called "in the brain" session, giving a presentation about how to make agility work in a typical SOA scenario.

I must admit that I am quite happy with the result: despite a couple of night spent fine-tuning the slides, with my laptop in miserable shape (I had no battery left, and the 'o' key was activated by pressing 'p' and 'i' meaning that when I typed "pippo" the result on the screen was "poiopopoo"), everything went quite smoothly and I had definitely a good time. Also, the pub follow-up has been largely appreciated.

By the way, if you're interested in the content, a podcast is available on Skills Matter website.

Tags: agile, SOA, Scrum, podcast