Not Dead Yet

Like it or not, David Heinemeier Hanson did it. By starting the debate around the “Is TDD Dead” topic, he forced the whole agile community to re-think about many issues that were taken for granted. You may like his opinion or not, you may prefer to choose his side or Kent Beck side, or the more radical one from Robert C. Martin. Or you may also appreciate the way Martin Fowler proposed to sort out the issue, in a way that is a great example of “this is how we discuss things in the agile community”.

I personally liked the fact that many collateral discussion spread out on the topic, the best ones I had with Francesco Fullone and Matteo Vaccari. Here are some of the thoughts they helped shaping.

Who’s paying for what

I remember the old argument that 90% time on code is spent on maintenance, so it does make a lot of sense to write software that’s easier to read an maintain. This is what I normally do, or, better, this is what I relentlessly keep trying to do.

But the more I look around the more I realise that this statement leads to a quite naive approach. This is not how most developers are working right now.

Let me explain that.

It’s not about the code

Given 90% of time spent on code is in fact maintenance, it makes a lot of sense to try to improve this 90% instead of the 10% spent writing new stuff. Unless…

…unless we stop focusing about code and start looking at human beings surrounding it. Some call them developers, but I am really more concerned about the human being, not the role.

Humans move, evolve, grow, move to another team and another company. Humans will be far away the moment evolution will be needed on a given piece of code. They won’t be there to take the praise for well-written maintainable code as well as they won’t be there for talking the blame for horrible legacy code.

Even worse: once they’ve left, they might be blamed also for supposedly well-written code, if the new developer who’s now taking care of their beloved source code has a different programming style. Or simply doesn’t like their indentation style, or what was intended to be good coding practice at the time the code was written.

If I don’t plan to stay around a piece of software for a long time, writing maintainable code is dangerously close to an exercise of style. Many practices are said to repay themselves in the long term but no one stays around long enough to catch the benefit

Being short-term sustainable

Yep. In the long term.

That’s the thing I really don’t like. It sounded like reasonable first time I heard it, but now it has the sinister sound of I have no evidence to support it, just faith.

And in a world where software development workforce is made not only by internal developers, but also consultants, freelancers, contractors, offshore developers and part-time hackers (just to name a few) the implicit assumption that

if you don’t code right, technical debt will bite your back 

is flawed.

Well, technical debt is a bad thing. It’s probably worse than we think. Companies deliver horrible services due to technical debt, they struggle, collapse and ultimately sink for technical debt.

But well… who cares! Big companies will survive anyway. For small companies… it’s just darwinian selection. As a developer I’ll move somewhere else soon (disclaimer: I have a bias for not hiring developers who previously worked in companies that sunk).

So, I guess the real story with technical debt should be rephrased like

if you don’t code right, technical debt will bite somebody’s back

meaning …Not necessarily yours. Call me cynical, but by the time your crappy code will unleash its sinister powers you’ll probably be in another company complaining about a different yet stinky legacy codebase. Time for a soundtrack: try this.

A little tragedy of the commons

If code stays around longer than the expected lifespan of developers there’s not much that we can do to prevent technical debt to flourish. It’s just another version of the well known tragedy of the commons, that economists know very well: people acting in their own interest will ultimately harm a greater good.

A system view

Thinking of a codebase as a part of a larger system, if independent agents aren’t there to stay, they won’t improve the system. I remember my days when I was a university student. I hated it. I probably hated every single exam. The way it was led, taught, and the way students were evaluated. But the thing I hated most was that the whole system was hopeless: nobody liked it, but nobody really put any effort in changing the system. Students were not there to stay, the winning strategy was to keep your mouth shut and pass the exam. It will be somebody else’s business, not ours.

Does it sound familiar?

If the time needed to be rewarded for an effort is longer than the expected time in the system, well …who cares?

Enter the Boy Scout Rule

Interestingly, Uncle Bob proposed a solution for that: the so-called Boy Scout Rule, which goes as follows:

always leave the campground cleaner than you’ve found it

I love it, for many reasons. It creates a sense of ethics and belonging. It provides a little quick reward: “I am doing something good here”. But the most interesting thing is that it turns a rational argument into an ethical and moral one. Boy Scouts do not rationally believe that every forest will be cleaned up if everybody starts behaving the way they do. But the feeling of doing something good and right, is a good one. And moral - together with the fear that Uncle Bob in person will one day see my code and throw me in the flames of hell as a consequence - will probably do a better job.

A higher duty

Who should care for the system? Who should ‘enforce’ (I hate this term) the boy scout rule? It’s pretty clear that this is the job of somebody who’s going to stay around for a longer time. Somebody that cares about the ecosystem, and sees the whole. Or at least tries to do that. Be it a CTO a Technical Lead or the CEO, depending on your company.

For example, working for a sustainable work environment with as little turnover as possible might be a really good strategy in the long term. If developers feel the workplace as their own workplace, continuos improvement policies might flourish and people might stay around long enough to see them working, creating a virtuous cycle.

No hope in the short term?

Here we are. Now you’re probably thinking that - since the battle for maintainable code is very hard to win I - am depressed enough to give up and let the barbarian hordes commit whatever they want.

Not yet, guys.

The thing is, good practices like TDD and continuous refactoring should repay themselves in the short term too. And they do, …if you know them well enough. Which means that you’re correctly accounting costs and benefits, including the cost of learning TDD and keeping it distinct from the cost of doing TDD.

Personally, I would be lost without TDD. But I am a different beast: despite all of my efforts, coding is not my primary activity: booking hotels and flights is. So TDD is a way to keep a codebase manageable even in scenarios with no continuous work on a project. I guess this goes for many open source projects too.

But the most interesting thing is that the best developers I know are all doing TDD. Not for religion, but because they get benefits in return. Not potential benefits, just-in-case style. Real benefits, in the short term.

Safety, confidence and speed.

Very good developers can also recognise different needs in different contexts. There may be cases when TDD is overkill or bringing little value: if all you need right now is showing a prototype, show a damn rails prototype right now! It’s no religion, it’s a continuous evaluation of return on investment and evaluating options. Which means that you can also be wrong: you may be too conservative and pay an insurance for a disaster that would not happen, or be taking some risks, just to end up smashing your face against a wall with your colleagues staring at you with a I-told-you-so look on their faces. That’s life, dude.

But in the land of continuous choices, being able to code only in one way, means being weak. The only safe spot, is being so good in TDD to know when not to use TDD. Aren’t you there yet? Good luck. There’s still people that go to a Japanese restaurant and ask for fork and spoon. But, you don’t want to be the one.

So please, stop raising arguments like “Yes, but if one day somebody wants to change the codebase”  because I won’t buy it. If you can’t provide short term benefits for TDD and refactoring, then you probably don’t know them well enough, yet. And at the same time I don’t want to be the one trading a sure expense, for a potential gain in the distant future.

We can do better than this... Reloaded

In the last Italian Agile Day I delivered an open presentation called "Possiamo fare di meglio" (this post's title is just the translation), where I raised some questions about the way we develop software. I then tried to summarize some of the things that emerged in the discussion here, but there are still many things that are bouncing in my head...

One thing that really disappoints me is the low quality of many software applications I have to deal with in my everyday life. Some days I really feel like I am surrounded by crap. Just to make it clear what I am talking about: here is an excerpt of what a home banking service is asking me every single operation I do.

Choose the desired account:

My Company Account (all the accounts)

And every time I think: "It's one account, it's just one account, it's always been just one account, so why on Earth you keep asking me this question every time! Couldn't you just take me straight to the account...".

Some days I just feel like surrender, some other I see how some application are doing a fantastic job, on platforms like iPhone or on the web, and I feel like there's still some hope for us.

There is still a lot to do

Despite all of our efforts, to introduce TDD and continuous integration, velocity based estimations and the like. There's a lot to do in other fields as well. Let me say it in another way: many companies are looking towards agile methodologies as a way to improve their development processes, in order to deliver software on time and on budget.

Doesn't sound that bad isn't it?

It does. It doesn't mention the quality of the product. ...Oh, yes, we forgot, we need also tests to reduce our defect ratio. Tests are good, software quality is good, but still doesn't address the whole point. We should deliver better products. Emphasis on the process itself would satisfy the managers' ancient need for schedules and predictable outcome, allowing teams to produce crap more efficiently, but would not produce better products unless some key points are specifically addressed.

The most neglected areas at the moment - at least from my point of observation - are:

1. user's involvment,
2. understanding the domain complexity,
3. lack of an overall perspective,
4. inefficient learning during the product development lifecycle.

The items are not completely separated, in fact they're slightly overlapping, but I think these are the areas where as software developers we can (and must) improve a lot.

In the next posts I'll dig deeper in these topics.

Shared the Agile Day Presentation

The Slideshare version of my "Possiamo fare di meglio" (we can do better than this) presentation, is now available (in Italian) here.

I did my best to add comments to the many pictures, sooner or later also the video version will be available.

Sharing information within organizations

Naresh Jain tweeted code reuse being still an unsolved problem within organizations some days ago. This made me think that this is a symptom of a more general problem within organizations, who do not account for information and knowledge transfer costs.

I’ve seen a few attempt to promote reuse within organizations, and they all failed because they assumed a non realistic communication model within the organization. In some cases the approach was completely naive: just because some code had been written to address a specific problem, everybody was supposed to know about this code. There's only one organization capable to achieve such level of coordination: it's called "The Borg". They have a collective intelligent and are continuously in sync, one and another.
This type of information sharing simply does not happen in nature.

A more sophisticated approach involved use of repositories and/or knowledge management system to promote information sharing in a common tool. I've seen most of these approach fail as well. Having a common repository to share the information is generally a good idea, but expectations on those systems were really high, and probably unreasonable.

The starting point to promote code reuse within an organization is to know that there is some code that already solves your problem. From a communication perspective we have Developer A, a potential writer of an information at time T0 who knows something about what he's currently doing. Some times later, or maybe simply somewhere else, we'll have Developer B with a similar problem, at time T1. To have a successful communication:

• A should have written something meaningful about the problem he solved
• B should have searched the repository for an existing solution
• B should have found what A wrote
• B should have recognized that what A said was relevant to his problem

I think most of these steps are inherently non-trivial, basically because there is a huge amount of arbitrary decisions in every step (including whether to do the step or not). Words used by A might be different from words used by B to search. Or the information might be filed under an arbitrary label. Basically, this is a decoupled asynchronous communication. There's no feedback possible, and every arbitrary decision is an obstacle between A and B.

I personally find structured systems for storing information particularly bad: they force users to choose where to file their data, and the way people label their knowledge varies a lot form person to person and from time to time. Non-structured information repositories, such as Wikis with a Google-like search tool are definitely better, but they don't solve the core of the problem.

Sometimes we can connect our specific problem to a more general case, for example a "shortest path problem" or a "traveling salesman". This is probably how we'd call the problem after we've found the solution. The way we describe the problem before might be a lot different. Knowing how to call the problem is already a step towards the solution. This means also that who describes the problem and who describes the solution might use different terms. making our asynchronous-remote-decoupled communication at high risk of being completely ineffective.

Effective communication about partially unknown problems needs a higher amount of feedback. That's what we do when reading the menu in an ethnic restaurant: or (closer to our example) when we get into a shop to buy things we've seen but we don't know the name. A clerk might have a gotcha and tell us "Ah! you meant this!" showing us what we wanted and telling us how it's called. The most effective way to communicate about unknown things is the one with more feedback: face to face conversation.

Unsurprisingly, organizations trying to push communication through structured systems are often the ones that put a lot of efforts in preventing effective face to face communications to happen. Well ... in fact preventing good things to happen is one thing organizations are good at. And effective communication is one of these fragile things which might happen in a good environment, but there's no way to force them to happen.

The Big Agile Practices Survey

Jurgen Appelo is is currently doing an interesting all-round survey about agile practices. The more people will participate, the more accurate the results. If anyone is interested, just follow the link.

Blogged with the Flock Browser

Toxic Code

During these days, the word toxic is often associated to investments hidden in some bank caveau, their risk have been underestimated and the result is a backlash with devastating effect.

I think many institutions have the same toxicity problem in their software, but still fail to admit it: key portions of their software act like small timebombs and are quietly undermining their businesses.

There are infinite examples of what I consider to be toxic code, here is a small checklist some might be familiar with:

• Useless code: it’s there but nobody uses it
• Untested code: it’s there, probably works but nobody wants to touch it
• Valueless code: Software which is not delivering any business value
• Annoying code: every change to the system takes more time than it’s reasonably necessary.
• Annoying software: the code is “working”, but in a way that makes things harder instead of easier. Users waste time every time they perform an operation.
• Leaking software: key portions of the application are not tested enough, some exceptions are unhanded resulting, in random untraced errors.
• Countdown software: software with a time-related flaw that will have a havoc effect at a given point in time.

• ...(add yours here)

Well... it’s basically the same idea of technical debt (and Ward Cunningham posted an excellent short video on that - thanks to Piergiuliano Bossi for pointing it) which should be familiar to any agile developer, and was created as an analogy to the financial world. But my worries are more related to the business side, of software rather than to its production process. Ideally, we should have 100% trust on our IT systems (don't laugh please... I said "ideally"), because every time some of them it’s not working there is a waste somewhere (be it time, money, or a customer abandoning the company).

Sometimes companies are aware of the real quality of their running software, and decisions to fix or improve specific portions of the running application are just the results of conscious strategic budget allocations. More often, this is not the case: like toxic assets, software with risky behavior is treated like normal production code, underestimating the associated risks and drawbacks.

Anyway, even when companies are taking into account the costs of a less-than-optimal solution, they’re generally considering only the direct effects of this choice. Something like “this feature might be more user friendly but it will cost us $xxx to rewrite it and the possible revenue for that will be only $yyy, so it’s not worth developing” which is reasonable, but doesn’t consider the cost of not developing it which is put on the user who is having a less-than-optimal user experience (well ...sometimes a miserable one) wasting time and/or money any time he uses the system. It's like polluting: if you don't count it, it's cheaper to do dirty things.

I am still intimately convinced that computers should make life simpler and not more complicated. And I am also convinced that they can.
For some reasons they don’t.

It's all in the mindset

I don’t know if it has something to do with the Shu-Ha-Ri theory that Alistair Cockburn uses to describe the three level of experience in the agile practices (In the Shu phase you practice and learn, in the Ha phase you start applying variations to the lessons learnt, in the Ri phase you do the right thing by instinct, because you know that’s he right thing to do), but there’s something in common between the most successful approaches to software development I’ve seen. It doesn’t necessarily have to do with agility, many of these teams or individuals were indeed applying agile practices, but I’ve realized that they were doing something deeper, and agile practices was only part of it. On the other hands I’ve seen so many people completely screwing up software development, regardless of the process in place, blaming waterfall first, and then RUP, and Scrum later on.

Technical excellence does make a lot of difference, but still I’ve seen really skilled people doing awful things, despite their qualities. What does still make a lot of difference is in he approach to software development, is something that is a combination of a quest for continuous improvement (something that we could maybe call lean, but that’s to close to be another label) and of desire to experiment and challenge what we already know.

Learning is a non-linear process. It’s not piling up bricks of knowledge. It’s challenging what we already know against new experiences. It’s experimenting different ways to do things, and analyze the outcome. It’s something different from reading books or following advices, pretending to be right just because somebody else said that. It’s dare to be wrong. It’s trying to do something different from the last time we did the same thing, because doing the same thing twice in the same way won’t teach us anything.

Put this approach in a person, or in a team, and you probably won’t need so many books. Books can provide you inspiration, for “proven” solutions, but proving a solution, is ultimately the key of our job.

The dark side of self-organizing teams

Agile methodologies focus on the notion of self-organizing teams, as a key to software development success. This works for a lot of reasons: talent is not constrained to follow a pre-defined process, the process is adaptive and tailored on the individuals and so on.

But it looks so beautiful and simple in books, and so hard to achieve in reality. One reason is that there are two main patterns for self-organizing team creation:

• being already on one
• spontaneously creating one

Penguins are self organizing. Males stay in the middle of the antarctic continent for months with an egg on their feet keeping each other warm, and discussing about politics and soccer. At springtime female penguins arrive from their shopping and fishing just in time to start feeding the new born baby penguins. Young penguins follow the group habits without questioning and the ritual goes on and on like this year after year.

Creating a new group involves discovering common passion, is just like starting playing baseball in a small town. Maybe 2 kids are just playing with a ball and a bat. Another one joins the two, same day same hour, then later on, they discover they’re enough to form a real team. There can be some key moments in the growing phase, but many times, the original vision is shared by the majority of the team.

What is really hard, is trying to transform a team who’s always been directed from above, into a self-organizing team. There are so many things that can get in the way, that is often better to start with a newly formed group instead of turning an existing team into a self organizing one.

...not necessarily good
But the notion of self organizing team is not necessarily good in itself. Self organizing teams can be pretty nasty, (Mafia, Al-Quaeda, Ku-Klux-Klan are all examples of self organizing teams, as this interesting post says). But the question is not only tied to team ethics, it’s also related to what the team can do to achieve its goals, which involves powers and responsibilities.

If you follow soccer a bit you’ll probably know the story of Antonio Cassano. One of the most talented Italian players, he played in AS Roma for a while. After some initial success, its role (not his skills) started being questioned. AS Roma sold Cassano to Real Madrid and right after that started an impressive record of 11 victories in a row. After the embarrassing parenthesis in Spain, now Cassano is doing fine again with Sampdoria. This is a clear example of how a team can improve as a team by getting rid of some talent.

Talent is hard to manage (and this could be the subject of a dedicated post), but the key point here is that the team might decide to drop team members that don’t share the same values with the rest of the team. You need to grant this right to the team, otherwise shared vision and behavior will never emerge, but you’ll also need to be prepared to the consequences, as a manager or as a team member. And consequences can be pretty nasty: like telling a colleague that he/she is not welcome (maybe with some “unless...”).
One key point is to be sure that the most influential team members have a positive influence on the team. If you can’t guarantee that, you’re probably doomed, right from the start.

So ...where’s the fun part?

I have the feeling of having killed somebody’s dream in my last post. And also of walking on the edge of being misinterpreted. I think I’ve got to blog about it again from a different angle.

I like developing software. I think it’s fun. Even if I like solving complicated puzzles, I think the best part of software development comes from working within a team. I have some friends working for the Toro Rosso F1 Team (being from the same town makes it easy), and I know how everybody felt after winning in Monza. This is the kind of feeling a team should work for. Not every team could be so lucky, but ...you got the point.

Are we just executors?
So, are we just implementing a specification? Nope. If that was the job, pick some MDA tool and have it do the job for you. I just asserted that coding is a lot less creative than many developers think. Finding a proven solution (hopefully a good one) is often a preferable alternative to an original one. And “original solutions” often degenerate in in-house frameworks lock-in.

But being mere executors just doesn’t work either. Following specifications is a waterfall, but it’s also a waste of talent. And such a scenario will eventually lead to low motivation, and talent loss within your company.

Creativity might slip in when the puzzle is more complex than usual, or when tempting something new. In this cases, you have to fight with the creativity tools: get away from the workstation, have a (time-boxed) brainstorming session, involving somebody else, be provocative, avoid censorship. The key is to recognize which type of problem you’re facing and to use the right tool for the job.

Learn and challenge
Another area of software development where you definitely need some creativity is interaction with users and stakeholders. Understanding the problem domain is an interesting activity, which will lead you in unexplored areas of knowledge. Understanding the way users are interacting with your application, and the reasons for that, might lead you to propose some features that might come in handy, or make the difference!

Software development teams are often a concentration of pretty smart minds. Pretending that the only focus is code, is wasting talent and, often, a mess.

How much Architecture is Enough?

During my London staying, I had also the chance to attend Simon Brown's in-the-brain event, which turned out to be more two-way than mine, with Simon softly facilitating a really open discussion. As it is often the case, interesting ideas emerged. Here are some of my side notes, mixed with some of my following ruminations.

Communicating the architecture

One of the biggest challenges seems to be finding the right way to effectively communicate the architecture. It turned out that architecture documents are not the best way to convey architecture information alternative and possibly more creative ways are needed, spanning from Wikis, to podcasts, to live sessions. Honestly, the Google Chrome Comics is a great way to deliver an architectural view. But maybe it's not a viable option for common teams.

Also, a prototype could bring so much information and is particularly efficient in edge situations, where the team is not properly trained or skilled and needs to produce some code anyway. Bad as it may sound, also cut&paste information propagation (the worst case scenario) might be preferable to paralysis or anarchy.

Championing the architecture

Without an active championing role, the architecture is at risk of being twisted or thwarted. Without an active role, an architectural specification is just a piece of paper.

Non Functional requirements

Managing non functional requirements seems to be a hot spot for many agile teams: requirements do not naturally show up as user stories, so their emergency has to be somehow facilitated. Also, automatic tests on nonfunctional requirements are necessary.

Strictly tying delivered software to business value might eventually put non functional requirements aside, but having a fully operating passed load test delivers both business value and confidence.

Resisting temptations

A pragmatic strategy to define if an occasional need needs to be transformed into an Architecture requirement is to count the number of times you've been hit by the need to refactor a given thing, with 'three' as a threshold for a rule of thumb. Before getting to the threshold one should "resist to the temptation" of refactoring the code to a more elegant solution.

Personal note: it's interesting to notice that this works only in teams where you have an efficient information flow within the team. If development process does not facilitate this sort of information exchange (es. no daily scrum meetings), your counting might reach '20' before the need for a given refactoring is communicated to the architect-in-charge (whatever that means).

Sustainable architectures

I also advocated the cause of "sustainable architectures": putting too much stuff on an architecture definition might result valueless if the team isn't able to grasp it. Sometimes it's better keep everything as simple as possible, or to step back and start with a less sophisticated solution, and invest in training or improving the team technical skills (or the architect communication ones) before building and obscure cathedral.

Deferrable architectures

A key point is that architecture is about things that are expensive to change in software applications. Interestingly, a too sophisticated architecture (architects love to create frameworks) is more frequent than the need to heavily refactor an existing one. Honestly, I think that in many cases, simply defining a clean layering architecture might be enough, to allow a clean solution of emerging architectural requirements.

Personal note: also, the cost of change has largely dropped: if we compare heavyweight frameworks that were popular some years ago, with what can be achieved with Spring and AOP in terms of pluggable, orthogonal requirements, it's clear that the scenario has changed radically, at least in the JEE landscape.

Should architects be part of the team?

It turned out that architects are sitting on the borderline of the team. There are architects that are definitely not in the team (and absolutely do not code), and some which are more willing to pair with developers, improving team skills and gaining feedback on the correctness of the architectural choices. At the end of the day, developers are architecture primary users, and, as an architect, I really would have my choices tested and challenged by a real feedback. On the other hand, an architect should preserve its ability to think long term, so can't be fully involved in coding, especially tasks which are on the critical path. Moreover, they might be external roles, hired on a short term basis, or with reduced availability, so being part of a fully involved Scrum team might be problematic. Doing Scrum by the book, the Team might ask for an external architecture support as long as the team thinks they need it.

Project start effect (personal note)

As a consultant, I might have a slightly biased perspective, but in many times, the architecture definition has just been the result of a "resource allocation incident". Sometimes, the team was just set and ready-to-code, so there wasn't really much time to have them wait for a good architecture specification to start coding. Sometimes, I joined a project when development had already started, so I had to impose an architecture over a (barely) existing another. Finally, some other projects would not even start before an "architectural view" documentation is developed, as this is part of the project budget negotiation by the management (SOA is a typical example of this case); so when the project starts you have more or less a "specification" already in place. In other words, the early days really influenced the whole project direction, and this rarely has to do with architecture, per se.

Did we get it all wrong?

I've got some thoughts bumping in my mind for quite a while, then somebody today on the Italian XP mailing list pointed me to this post by Jay Fields, urging me to blog about it. The main topic is the distinction between two different species of software professionals, hackers and developers. You might have faced this distinction with different names in the past ("The pragmatic programmer" is another name for the same god-like figure), but the main point is that hackers are brilliant in solving technical problems, while developers are better in interacting with stakeholders to better understand what the problem is. The bottom line is that hackers are at higher risk to develop brilliant solution to non-existing problems while developers are better in ensuring that they are trying to solve the real problem.

Breeding developers

Ok, we need both species. Are we good in breeding them? I'd probably say "Nope", at least for developers. From what I can recall, our school system is largely interested on the technical side of the software development professional training. So we're taught computational theory, operative systems, programming languages, algorithms and the like. Most of these disciplines are oriented towards an individual problem solving approach. We learnt that some sorting algorithms perform better than others, so do some data structures and architectures. Most of the exams were defined in terms of finding the solution to a given problem. This couldn't be further from the real world, because what is really difficult is to define and scope the problem. But to define and scope the problem you need to interact with more and more people, many of them without a strong technical background, which will tell you valueless information, lies, bullshit, crap, and …a tiny slice of the truth which you'll have to pick up and transform into the foundation of your application.

Unfortunately, this is an activity that involves a completely different part of the brain from the one used for coding. This involves social skills and the ability to successfully interact with different people. Involves the ability to start swimming (each time) in a completely different water, to learn about a problem domain we know just a little about. And …we were not prepared for it. Moreover, we were somehow chosen not to be good at it. I am consciously talking stereotype here, but the "computer guy" (or whatever we've been called when we were teenagers) is the kind of person that likes the It's-just-me-and-the-machine approach, or that doesn't find strange to spend a whole night awake playing World of Warcraft or trying to fix a bug. Somebody else might be having a beer in the meanwhile… I am not saying that computer geeks do not have social skills, what I am saying is that social skills are a largely underestimated component of what makes a good developer (you can still call it "social engineering" to attract hackers to the discipline, by the way), and that you can't expect to find these qualities so easily.

Tags: agile, software development

Photos from the saddest cubicle competition...

Wired published photos of the saddest cubicles sent by readers. And what you see is only a small part. of what you can actually get. What you, hear, feel and smell is what definitely makes your workplace special...

Tags: agile, Software Development

From crossover comes breakthrough

One of the biggest sins in software development industry is "knowledge hubris". Software engineers are "natural born specialists" (not all of them, but I guess that a vast majority could fit under this definition), people who have a deep knowledge of some technical aspects and have a tendency to "drill down" for a deeper knowledge of a specific topic. Experience helps in defining what's the better balance for the situation, enabling you to make decision based on a superficial knowledge (and maybe some expert's advice) instead of studying the whole stuff before taking any decision step. Given that software engineering is a profession that needs continuous refreshment of the acquired knowledge, and that the field is constantly enlarging, reading more books in a shorter time is probably the only viable option.

Many of us think this is enough.

I had a couple of loads delivered by Amazon in the last two weeks. So I have fresher books about Software Architecture, Project Management, Software Development Process, Software Security, Grails, Groovy, Ruby, SAP and so on. I looked at the pile and felt somewhat disappointed: "looks like I am merely changing the perspective on the same stuff", or maybe I become too much influenced by Amazon's recommendations. While browsing some books bibliography (in a childish attempt to score more than 50% books read) I realized that probably the more interesting perspective could come from the books I've never heard of. Agile movement brought a lot of external influences into the software development field: SCRUM introduced elements from process engineering (that were quite obvious for process engineers, if only anybody thought about asking…), Cockburn and DeMarco and Lister analyzed deeply the human factor which is predominant in software development, both in terms of self organization and team dynamics, bringing anthropology, communication science and psychology into play. After all, in software development the rough material is people, but most of us focus primarily on technology forgetting that, unlike technologies, there is no obsolescence in people.

Tags: Agile, Software Development

Videogames for the Development Team - Updated

This is just a slightly different version of an old post...

I’ve got this stuff bouncing in my head, right after reading this Joel Spolsky’s post trilogy about project management. You’ll realize that my head is pretty foggy lately…

The average developers attitude is best expressed by 1st person shooters, such as Quake. A developer has a dirty job to complete, and has not to be afraid to dig into a bloodbath to finish it. Collaborative teamwork is encouraged for better result, but in the end it’s you against the enemy.

Team leader’s attitude is best trained by strategy games such as Warcraft, or similar. You organize the teams, assign task, and make characters’ experience evolve, so they can perform more complex task. Controlling parallel activities is just the nature of the game.

A project manager perspective is almost the same that a game like SimCity can give. You don’t control characters anymore, but create conditions for them to do stuff, such as provide houses, roads, and so on. And watch interesting happen: if your city is a nice place to be, then people will be glad to join your city, otherwise they will leave.

You might be tempted to choose a different approach, Sims-like, but as Joel’s article explains pretty well, it’s just pretty fine grained to work effectively (to be honest, I always have flies over the garbage can...).

The perfect game for the DBA is Dungeon Keeper, well... you might guess that I am not a DBA, but the whole game is defined in an access provider perspective.

Ok, somebody might be curious about my favourite game… well, I’ve always been a Civilization’s fan, from version 1. But the one and only that really pleased my ego was Populous II, the moments I could send a tidal wave and watch poor innocent human beings drowning in a tidal wave… Does this mean something?

Tags: Software Development, Agile, Project Management

A nice puzzle for the ubiquitous language

Multi Lingual Domain

During the Domain Driven Design seminar I had the pleasure to attend in JAOO, a question arose from one of the participants, something like: "we have a domain model which is expressed in Spanish, but then the company bought an Italian one, and then the model had to be extended to the Italian scenario (which is obviously slightly different), and then to a British one…". The main problem is that a model should be expressed in a language. Countries like United States, Great Britain, Canada, India etc. are privileged because they seldom have to choose the language in which the model has to be expressed.

Our usual scenario instead is a mixture of English and Italian; getNome() and getCognome() are typical examples of this language mismatch. You can put some effort to rule out the situation, declaring an "all Italian" rule (which doesn't necessarily mean spaghetti-code) or an "all English", but don't expect a 100% compliance to it: you'll always end up with a mess. Technical terms (especially in banking and insurance industry) are scary enough in Italian so no developer would easily stick to an English definition of the term, but most of the analysts would agree with them (or use them as an excuse). Anyway, given that the domain model must be shared with the domain experts, most of the times you simply have no choice.

Defining the core domain

Given that language issue in non-English speaking countries is already an issue in a single nation scenario, how could the initial puzzle be solved? There is no real answer. Basically it depends a lot on the team special skills: if you have people with cross competencies, you probably could leverage this knowledge, to expand the core domain, including multi-lingual scenarios. If those people are not there to stay, you could be anyway putting the core domain at risk of being unmantainable.

Given that national domains will be subject to EU and national regulations we have also a two-level driver for change. EU regulations would push for a shared domain, while national one will call for separate ones. Assuming that you can't really predict where the change will happen (but EU regulations leave you more time to think than national ones) you are left with the basic question: "how much of the model is really shared?" the answer is going to be "a small portion" now but probably going to increase as long as a deeper knowledge of the system is shared. At the end of the day, the really tricky thing to do, is to evaluate the costs of sharing information versus the costs of maintaining duplicated code. One can invest on knowledge sharing (Wikis, mailing lists and so on) but I guess that if the developers don't "cross the boundaries" spontaneously to attack the duplicated component (or to translate it to English), forcing them to do so will be probably more expensive than keeping separate domains.

Language specific wrappers

If shared components are to be used, then language will be an issue again. With simple terms English would be fine. With more complex ones, then code readability might suffer. A possibility is to define language-dependent wrappers, whose only purpose is to embellish the original component, providing access to keywords in the language of the local domain. I personally prefer to use English terms and to write a multi-lingual javadoc (but alignment is risky) or wiki documentation. But in some circumstances this might help.

Tags: JAOO2007, DDD, Software development

Brief summary from JAOO 2007

Last year was my first time at JAOO, and I ended up doing a short buzzword-based review of the overall conference. This year I had slightly different feelings about it, but I think that keeping the same format will still be valuable.

• TDD: Lots of attention on the topic. Robert C. Martin held the opening keynote and a dedicated tutorial. An interesting – and controversial – issue popped out during James R. Coplien speech about "frictions" between a sound architecture and a full TDD approach.
  
• Web 2.0: whatever it means, there was a lot of hype about it. AJAX, interactivity, Ruby, Grails, and in general the possibilities of development where the world is your stage are definitely interesting, and we're still far from seeing the final destination. Really hot topic.
  
• Ruby: one year after, what looked a promising language really hit the mainstream. Most of the attendants were still on Java, but many of those were having a Ruby project going on. And they all were happy. There are still a few naïve elements, but they don't change the overall picture of a really successful language.
  
• Erlang: the language has been designed for parallel processing, and has truly brilliant ideas in it. Google bases its computational power on MapReduce, but also a lot of Social Software (more than I imagined) is putting strong effort on parallel computation as well.
  
• SCRUM: SCRUM trainings were completely sold out. SCRUM methodology itself deserved a whole – crowded – track.
  
• Agility: while only a few speeches explicitly talked about agile, still it is an interesting issue. Developers listened to other's developers trying to measure how agile they were an how they could improve. It's definitely a path and not a label, so talking about it still matters.
  
• Domain Driven Design and Intentional Software: from different angles they both try to tackle the cultural mismatch between Domain Expert and Domain Modelers. Eric Evans by presenting techniques and a clean conceptualization of the steps needed to have the modelers successfully interact with the experts. Charles Simonyi's Intentional Software takes the other way round, by presenting tools that allows domain expert to express the software in their own Domain Specific Language, sort of MDA without all of those meta- words. Both approaches have strengths and weaknesses: I am a DDD fan, but I'll keep focused on both.
  
• SOA: this acronym looked dead and buried. There was a SOA track in 2006, and this year only a few mentioned it. I have the feelings that SOA was – and still is – a great idea, but completely messed up by big vendors (or big buyers). Sort of what happened to UML some time ago…
  
• Software Architecture: I followed part of the track. I ended up having a broader vision of the overall possibilities (I've really been stuck with JEE for too long).
  
• Ethics: a lot of speeches focused on ethical aspects of the software development process. "The Pragmatic Programmer" was the reference book of the whole conference, giving a somewhat introspective feeling. Maybe we didn't learn that much about the new cool stuff, as much as what we've learnt about who we are and what our job is.
  
• Semantic Web: unfortunately it wasn't even mentioned.
  

After all, I was a little less excited than the year before, maybe because I missed the surprise effect, and also because some speeches were resounding the ones of the year before. There were no astonishing news on the stage, but we were not looking for the new messiah, so I guess this is pretty good.

Tags: JAOO2007, Software Development

A prophecy come true

While working for the Italian Ministry of Internal Affairs, on a social registry data exchange application, we faced the topic of validation rules applied to birth names according to the Italian law. I challenged the application submitting "boundary values" for names, while studying the law. So I discovered that Obi Wan Kenobi is a legal name while 2Kool, D3PO or C1P8 are not. Chewbacca is fine. Giulia and Julia are both allowed, while Juli@ is not. As always, anyway, the Italian law is open to different interpretations, cause it states that you can't give ridiculous or shameful names to your babies, but avoids to define what is exactly ridiculous or shameful.

Colleagues looked at me in strange ways (it often happens), but today I've got the proof that my concerns were right, somebody in China tried to call the son the name '@'… Read the Article (in Italian).

Tags: software development, Italy, humour

Doing Agile in Italy – part II

To my surprise, the previous post about the influences of job market regulations on agile development in Italy has been cited in an InfoQ article, and had a couple of interesting comments. This glamorous parenthesis happened while I was on holiday, and my Inbox was dangerously inflating. So, after getting back to work and diligently processed all of the issues waiting for me, here's the second part of my post.

The "too much process" rebound

Be it for our latin nature, be it the average size of projects and companies, I've never faced a situation in which "too much process" or "too much paper" was an issue. There can be a correlation with the Italian job market issues presented in my previous post, in the sense that structured processes definition were meant also as a way of protecting the project from turnover, by making everyone a "replaceable part" of the process. Agile practices lowered the process burden, and put the emphasis on individuals implicitly stating that the best way to handle turnover is to have no turnover at all, thus putting some effort in transforming the team in a nice place to work in.

The average company, has instead sort of a "raw" development process instead, and spoken words prevail on written ones. Given this scenario, I feel like I am a sort of "fake agilist", because many times my job is not to strive for a thinner process definition, but just set up a process, whatever it might be. I also think that one needs to start from something structured, to acquire degrees of freedom later: as an example, even if I fancy the "user story" concept a lot, I seldom use it in undisciplined contexts, and favor a more traditional use case approach. Once you master use cases and are able to distinguish the key parts of the specifications, you can benefit from a less structured and more distilled way to express them. Doing otherwise will probably result in a "blank paper syndrome".

Letting the kids play

Apart from careful people selection and the job market issues discussed here, the single most effective team motivator is probably the introduction of a new "cool" technology in the project. This approach is particularly effective in situations where the technology scenario is particularly flat. Once, I've found myself pushing for a technology change, and the only real reason for it was to have something new to "let the kids play", and to have a motivated team. Of course, there is a limit to the number of new concurrent technologies that a team can master simultaneously, but a "technically boring" project breeds low motivation, and turnover (so you have learning costs anyway) and productivity tends to flatten in the long term.

Tags: agile, software development, Italy

Doing Agile in Italy

During the latest JAOO conference, Alistair Cockburn deliberately provoked the audience stating that "we can't do agile in Europe" and this was due to EU regulations for contracts, which have to be filled with the requirements of the software to be developed by a contractor. This was a serious issue also, because in this case the law (unconsciously, I suppose, unless there is a powerful waterfall lobby in Brussels) poses a serious constraint on the development process. A contractor has to face the fact that requirement discovery in an agile fashion might drive the development team out of the officially signed project goal, some might go for a more ambiguous contract (but the customer has to agree on that, and this is not a piece of cake, especially with new customers), some others might require more paper to change the project scope (but this doesn't agile), the latest option is to take the risk and be "paperless" for a while.

Setting up an Italian development team

In Italy we have also some peculiarities related to the job market, which have some interesting effects on the way software is developed. I remember an old yet interesting article by Steve McConnell, dwelling on the notion of "problem programmers" meaning programmers with a negative impact on productivity. In other words programmers which are normally unable to finish assigned tasks, who need external help to finish the tasks, whose tasks needs extra cleanup work afterwards or which are sucking time an productivity to other members of the team. Put in this way this sounds somewhat scary, but it's not infrequent to meet such a beast. McConnell has only one solution to the problem: get rid of the problem programmer as soon as you can, which goes to a small set of options:

1. Resolve the causes for him/her being problematic
   
2. Move him/her to another team (hoping that this is a variation on option 1)
   
3. Fire him/her before the damage is too big
   

In Italy we can't fire. Job market regulations make firing an employee a very long nasty and complicated process, and 99% of the cases a non-viable option. I mean, in some places, you can't fire somebody even if you found him stealing (which puts the problem programmer in a much better light). The average Italian solution to the problem (there is always an Italian solution) is to go for temporary recruitment, hiring programmers from body rental companies, the third millennium version of mercenaries. Unfortunately this option has some drawbacks.

1. In many cases you have less control on who you are hiring. Even if I've seen pretty bad recruitment strategies in many places, being the one to choose who you are going to work with it's still a recommendable option.
   
2. You'll have less control on turnover issues: people might leave the company which hired them (and your project) for reasons which have nothing to do with you.
   
3. You have less control on motivation, you can't do much to improve it (at least on the salary side, you can do a lot, and probably more effectively on other aspects), you can do a lot to destroy it (with bureaucracy, holidays management, etc.).
   
4. Being hired by a third party is often a treat to motivation per se. One might ask "why doesn't this company want to actually hire me?" and feel they are just a replaceable part of the process, which helps them feel less guilty the day they'll decide to leave.
5. Last, but not least, the control chain sucks money. So the sub-employee is also usually sub-paid compared to the official counterparts.
   

So you generally end up with external teams with potentially lower motivation, or internal teams with people which tend to "seat" on a safe job, without being challenged for a while, and that need "something to do". I don't really want to end up arguing on the ongoing political debate which is going on in Italy these days on the topic, and please don't derive conclusions on my political orientation from this article. If you have to achieve high productivity from a software development team the best way is hire the best people, and get rid of the problematic ones. In Italy is hard to do both, so the net result is that you have to squeeze the best out of the existing team, which is sometimes …challenging.

Tags: agile, software development, Italy

Things they wouldn't teach you of in college - Part 2

In my previous post, I focused on some of the scope differences between software project assignment in the university, and real world projects, Pointing out that there is no maintenance phase. Well, it ca be if you fail the exam and have to evolve your personal assignment, ....but don't take it as an advice.

You are not alone
Well, time is not the greatest scope limitation you have to cross: being in a team can be far more challenging. One of the first thing you'll have to learn is sharing your code with your team, a practice which involves knowledge of SCM tools mechanics (first encounter with a CVS, Subversion or VSS is often a sort of initiation ceremony) and of the group mechanics as well. Something like "who breaks the build takes the blame" or "the last one to commit gets the conflict", but it's just the fact that the team as a whole carrying on the burden of the whole development. So you need coordination, and you need to agree with your teammates on the borders of your own activity.

Sharing the code also shows the symptoms of the "it works here!" syndrome, making it clear that you can't rely on your IDE (or on your memory) anymore, but you need more specific tools such as Ant or Maven to carry on the build process. The notion of repeatability, platform independence, environments and so on (have you ever heard the term "production environment" while at the university? I didn't).

A more subtle question is related to the notion of good code. Writing code just for yourself is close being an onanistic practice: good code is something somebody else finds useful. Which means that:

1. it works
2. easily
3. understandably

Code must work. This might sound trivial, but the point is that code should do what is expected to do. Expectations, unfortunately, could be quite non deterministic if left alone. So you gotta get a grip on what people expects your software to do, and to the average developer's expectation, which leads basically to one word (and one adjective): good documentation.

Documentation can make a lot on ease of use, but doesn't address point 2 completely. Software must be easy to use: doesn't matter if steps from 1 to 17 are thoroughly documented in MyFabulousPieceOfSoftware.howto, if you can achieve the same result in 3 steps, or one. The two things come together: as Vincent Massol points out in an old post about documentation driven development, documentation could be a good test for code and an inspiration for refactoring. If documenting takes too much it's probably because interfaces are far too complicated, or are not placed at the right level of abstraction. It's too easy to mess with interfaces and method signatures when we design both the client and the server class, for example 'cause we are stuck in the mud doing it, but we forget that we don't want classes that call our methods to be stuck in the same complexity that we are.

Understandability of the code refers to the internal mechanics of our systems. And to the beauty of the design. This is a tough lesson: a clean model is potentially useless if nobody else can understand it. It will lead to a cathedral in the desert, or - worse - to the code equivalent of a ghost town. Even if you are the best OO designer around, you can't write code that nobody else can maintain. Elegance in software can have dreadful effects, if nobody but you can appreciate it. Don't get me wrong, clean elegant software it's most of the time better than spaghetti code, but in a team, design elegance must stabilize on a level that can be reachable by a large enough number of teammates. To achieve this you can lower down your expectations, or raise team modeling skills. I tend to favor the second option as much as I can.

Tags: OOP, Software development, Coaching