DDD patterns as “elegant support”

I've got quite a challenging response to my article about context mapping on InfoQ, that forced me to re-think about the article, and re-express some concepts, because some statements could have been misleading. I quickly mentioned the “classical” or “tactical” DDD patterns, such as Aggregates and Repositories and it looked like they’re the key to successful design. Well, they do help a lot, but the key to successfully implement a domain model is to understand the domain, achieving a creative collaboration with the domain expert. Practically this produces a small, but sophisticated, domain model that deeply reflects our understanding of the domain and supports our future, business-driven, changes. It’s impressive to see how a well designed model can accommodate changes.

So, the whole point is to keep your model tidy, small and clean. Most of the tactical DDD patterns serve the goal of keeping the domain model clean. Classes like Factories or Repositories, belong in the domain layer, have a well defined responsibility, but I think their overall purpose is to allow for the cleanest possible programming style in Entities and Value Objects, where most of the domain behavior is coded. In a certain way, the whole system of patterns is a sort of necessary scaffolding to allow the domain model to thrive. Patterns are not part of discussion with users and domain experts (this is pretty obvious, but ...zealots are zealots), they’re just part of the implementation. They allow the model to be technology independent, or decoupled from other portions of the application, they allow the model to be easily tested. They allow for complexity to be managed in an elegant and scalable way.

Sometimes elegance might be a problem
There is some aesthetic quality in a well crafted domain model. Some developers really like what a DDD model will look like. Guess what? I like it too. But that might deviate our attention to what really matters. Sometimes elegance has no business value, or better, is perceived to have none, by non-developers.

• “We’re refactoring this architecture to make it more elegant”
• “I don’t pay you to write Armani-dressed code! I want code that works and I want it fast!”

Has any of us developers being seduced by the beauty of numbers in an Excel balance sheet from the finance department? I guess not. It’s just a different language. So, sometimes it’s better to express “elegance” as “reducing the cost of change”. Sounds a lot more like business, and a lot less like a disconnected geek spending the time tidying up the code (by the way also cleanup doesn’t really work). We’ll still implement elegantly, but that’s our job.

So if you’re enthusiastically embracing DDD and try to apply what you’ve read in the book, be careful not to transform yourself in a DDD pattern zealot. Their elegance might distract attention from the real goal: they’re helpful and often indispensable, but the real target is in the domain itself, and in it’s business value.

Video of my "Possiamo fare di meglio" speech is out.

Big thanks to Yuri Valentini and all the friends from the Bologna XP User Group, for editing and posting the video. Here's the link.

Shared the Agile Day Presentation

The Slideshare version of my "Possiamo fare di meglio" (we can do better than this) presentation, is now available (in Italian) here.

I did my best to add comments to the many pictures, sooner or later also the video version will be available.

Article on DDD Context Mapping is out on InfoQ

We've been talking about Context Mapping quite a lot in the last days, I was also writing a lot in the meanwhile. The article I wrote for InfoQ is finally live.
Thanks to Srini Penchikala for taking care of me and of my messy review process.

A crowdsourcing experiment

Like one year ago, I decided to give my talk at Italian Agile Day as an open session with roughly the same schema: 15-20 minutes of impact talk, and 30 minutes of open discussion. I like this type of sessions because it’s not a “I know the truth” approach (I don't), but rather a way to make a small portion of a possible truth emerge from anyone in the room. So, even if I had some answers in mind, I preferred to focus on the questions, leaving the answers eventually open. I’ve seen some open sessions, but it’s not the kind of things that you practice every week (unless you’re leading a talk show), so please forgive me if it anything went wrong. Personally, I think it went a lot better than the year before: the discussion flew naturally and touched many of the topics I wanted to touch.

See it in another way
Ok, now think of me like Dr.Evil, with a “masterplan”: I’ve managed to hire about 200 among the best IT professionals I know as consultants for one hour. Basically for free. That’s my crowdsourcing experiment. And I had the feeling of so many little things emerged. Maybe not so clearly, maybe not so well formed or well expressed, but enough to make me think a lot in the following days.

On the way home, and later in the weekend some thoughts were a little clearer. I guess my personal follow up is worth sharing: at least ...I owe it to the participants. :-)

We’re making products, not software
Software is what we know, what we like to write, what we learned to produce. Some of us are really good in writing software, or in making their team write good software. That’s only half of the story: from the user perspective we see a product, not code. In the discussion, many mentioned user interface or the user experience. Which is right. But it’s still trees, not the forest. If we start thinking about the overall product we might realize that we need to see the whole a lot better than we usually do. Scrum has a good shoot on this by emphasizing the role of the Product Owner, meaning that we have a single responsible person (which is a gift from the gods in many organizations) for the whole product. There’s a problem up there, because so many times the whole idea of product is flawed, and we’re just writing software. Guess what... with a high probability of being completely useless.

I’ve seen so many projects missing their goals or their potential, just because the responsible organization didn’t have a clear idea of the whole product not to see a pattern here.

Non technical integration is not so easy
Ok, so we should try to have an all-round view to our product. Including also non developers roles within our project scope. Interesting but tricky. Especially if the team boundaries do not match with the company boundaries, which might happen quite often in a cross-functional team. But software-only teams can control continuous integration to a very high degree of efficiency using TDD, Continuous Integration and SCM tools. But this is only a portion of the product. Other professional might follow a different approach, that has nothing to do with TDD (or maybe it has... only on a different time scale), and the perfection we strived to achieve becomes frustratingly fragile. I guess there’s no easy recipe here, we’ve got to fall back on mantras like “inspect and adapt”: solutions might be a lot different from team to team. After all we’re humans. But it’s funny to realize that we sometimes we want to “integrate early and often” but “only software artifacts, please”. Postponing integration with non-software components of your project is even riskier.

This is definitely a hot topic, especially if you approach it from an enterprise management perspective: enterprise 2.0 folks are talking about more flexible approaches to collaboration, and this might be at odds with the benefit of a jelled team.

Reading suggestions after IAD2009

I’ve found myself talking about many different books with many different persons during IAD2009, here’s a summary of the books I more or less explicitly suggested:

• Pragmatic Thinking and Learning - Andy Hunt: some things happen by chance, some don’t some others you could make it happen.
• Agile Software Development, the cooperative game - Alistair Cockburn: an eye-opener on what’s around the code.
• Agile Retrospectives - Esther Derby, Diana Larsen: so little small details can make the difference between a good retrospective and a useless bloodbath.
• Collaboration Explained - Jean Tabaka: sometimes there’s so much about the people that we don’t really know.
• Agile Project Management with Scrum - Ken Schwaber: it’s not software development, it’s product development.
• Don’t Make Me Think - Steve Krug: how many times are we willing to transform a service into a pain?
• The design of everyday things - Donald A. Norman: guys, I hate bad design.
• Domain Driven Design - Eric Evans: there’s much more to this book than it looks (and looks like there’s plenty of stuff already).
• Patterns of Enterprise Application Architecture - Martin Fowler: the reference book for current architectural patterns.
  
• Agile Project Management - Jim Highsmith: it’s not all about the software,but it works.
• Slack - Tom de Marco: oops seems like so many things your organization is doing are wrong.
• Peopleware - Tom de Marco, Timothy Lister: still incredibly useful, if you haven’t read it you’re just pretending to manage software projects.
• Getting Things Done - David Allen: everybody’s talking about GTD, maybe because there’s something true in that.
• Clean Code - Robert C.Martin: you won’t go anywhere if you don’t agree on this.

Hope these might be useful to some.

Back from the Italian Agile Day 2009

I’ve seen skilled and curious colleagues. I’ve seen nice ideas emerge from different perspectives (more on this soon). I’ve met old friends and nice persons which I didn’t know and that I am now honored to know.

All this doesn’t happen everyday and everywhere.

Thanks to Marco Abis that every year make this magic happen. Thanks to everybody in the staff and to all friends from the Bologna XP Users Group, once more fantastic. Thanks to all the speakers, able to tell real stories without hype or fluff. Thanks to all the participants to my “talk show” sharing interesting ideas and thoughts. Thanks to the heroes that risked asphyxia to follow my Domain Driven Design mini-workshop.

Thanks also to Roberto Bettazzoni that reminded me what it means to be “Romagnolo”.

It has been a great day.

Thanks to everybody made it possible.

Scrum Introduction with Peter Stevens

As a satellite event to the 2009 Italian Agile Day in Bologna, I'll be helping Peter Stevens in the one day Introduction to Scrum that'll take place in Bologna on November 19th, right before the IAD.

Peter is a seasoned Scrum trainer based in Switzerland, and helping companies in different countries. More info could be found on his blog Scrum Breakfast.

Peter speaks quite a good Italian, we thought that the best combination would be to keep the teaching part in English, and question time in Italian. But ...I guess this would be an emerging behaviour as well.

By the way, Agile Day participants will get a 20% discount on the full price.

JAX Italia 2009

Event agenda is getting crowded: in November I'll also be speaking at the first edition of the Italian JAX Conference, I'll be giving a talk about Domain Driven Design, and a full-day DDD seminar on Saturday 14th.

Some more detailed info here.

Italian Agile Day 2009

Registrations are open for the Italian Agile Day 2009, still in Bologna on November 20th.

I'll be giving a talk "Possiamo fare di meglio" (We can do better than this) and maybe a short workshop. Everybody interested in agile software development is warmly welcome. Hurry up!

Fast and Slim Enterprise 2.0

Next Friday - July 10th 2009 - I'll be participating to a public debate in Bologna (Palazzo Gnudi at 17.30) about the Enterprise 2.0 topic. Speaker panel looks promising, and the event is free :-)

Software house Gecod is hosting the event, more information on the official event page.

So ...what exactly is a context? - Part 1

One of the premises of Strategic Domain Driven Design is that in every non-trivial application there will be multiple models at play. This may sound controversial to some, but since a model is a representation of a portion of the problem domain aimed to serve a specific use, the need of multiple models promptly arises as our application grows in size and/or complexity.

The Ubiquitous Language plays many roles in DDD, among others it may be considered an integrity check for any model in play assuring that every role involved in software development, from domain experts to developers, shares the same language and agrees on the specific meaning of every term which is part of the ubiquitous language.

The Ubiquitous Language should be the only language used to express a model. Everybody in the team should be able to agree on every specific term without ambiguities.

Code is the primary expression of the model, other artifacts might be necessary along the way to capture requirements or portions of the design, but the only artifact that will be constantly in sync with the application behavior is code itself. Ensuring that every term is precisely defined and that is consistently used in every representation of the model greatly helps ensuring integrity of our application. Model integrity comes at a (reasonable) price: the model cannot be extended indefinitely, but can be shared, coherent and acknowledged only within a given Context.

So, ...what exactly is a context?

In Domain Driven Design, a Context is defined as

“The setting in which a word or a statement appears that determines its meaning”

which might sound rather ...vague, at first read. But I’ve learnt that Eric Evans always chooses carefully each word, especially in the key sentences. So the definition above describes exactly what a context is (at least in the Domain Driven Design context). This definition says nothing about the size, shape or other characteristics of a context, so they are likely to be consequences of the link between the word or the statement and the meaning.

But this definition calls for examples, here is a trivial one. The word Account has a very precise meaning in the banking context, and in a typical application, we would probably model it with a balance and an accountNumber attribute, and so on. But in the context of Web Applications, an Account is closely related to credentials, such as username, password, and so on, and we would tend to model it in a very different way.

Trying to force the two definition within a single class might result in awkward code, whose conceptual integrity has been undermined by the attempt to fulfill two distinct models within the same class. Admitting that the same term might be referred to two distinct concepts is crucial for model integrity.

Making the contexts explicit, we are making both models clearer and unambiguous.

The term Account does not have a single non ambiguous meaning, but in a given context the meaning is well defined.

Ok, this first example might be too trivial: the term Account refers to two different concepts, and simply calling them BankingAccount and LoginAccount might have solved the problem. A more subtle distinction might arise when the concepts is the same but but used in different ways, thus leading to different models.

Let’s suppose we’re working with an application managing payments. We’ll probably use this application to manage our Banking Accounts, keeping track of current balance and past operations. The model for our Banking Account might be a BankingAccount class like the one below.



Some PFM apps allow us to manage also payments, keeping a Payee Registry. In this scenario, a Payee is normally associated with one or more Banking Account, but in this case we won’t know anything of the internals nor we can issue any operation on those accounts. Does it make sense to model the Payee account with the BankingAccount class we’ve just defined?


Well... it does sound reasonable: it’s the same concept, I mean, in the real world our account and the payee’s one might even be in the same physical bank. Still, it doesn’t feel completely right: we are not supposed to issue any operation on the payee Banking Account, or to track anything on that. Even worse: doing so would probably be a bad mistake within our application.

In this case, we’re using the same concept in different ways, and this will lead us to different models. Even if the class is already defined in the application, a different context calls for a different model.

Again, name clashes might be solved in different ways at the code level, you can effectively partition the system by adding meaningful prefixes to the classes, or using packages, or in some other way. What matters now is that there are ambiguities, and a term can be effectively defined only within a given space.

So, a Context is a “space” where a given concept is precisely defined without ambiguities, so that it can be safely used in the Ubiquitous Language. If the same concept appears different times with different meanings within an application, we’re probably heading towards multiple Bounded Contexts within the same application.

In Domain Driven Design, a model is intended to serve a specific use, and nontrivial applications often tend to be used in different ways: trying to accommodate every use within a single model is a meaningless effort. It will end up corrupting your model with ill-defined or ambiguous term making the model you developed within a context less valuable. Contexts cannot be expanded indefinitely: Context Boundaries are needed to define which concepts are in and which ones are out, and to ensure conceptual integrity to the model within the context.

Back from the London DDD-Exchange

I just came back from London where Gojko Adzic and I led a 4-day full immersion Domain Driven Design class in Skills Matter HQ. Wasn’t it enough, on friday we had the first edition of DDD-Exchange, in the Crypt with quite a few interested participants.

First of all, I’ve got to thank everybody in Skills Matter for the fantastic job. Organization was incredibly efficient with wi-fi, food and beverages, real-time twitter coverage and quick publishing of the talks on the company website (mine is here). Despite a long line of sleepless nights working on the presentation and my terrible accent, looks like everything went fine. We had quite a few interesting talks, and I also participated to my first park bench panel, which turned out being quite interesting and open ended. Also, beers with some of the folks that came to attend the DDD-Exchange were even more open ended.

So, despite being incredibly tired, at the end I felt like I did something good, many people actually liked my presentation and we had many interesting discussions during the breaks.

By the way I uploaded my presentation on Slideshare, after making some small corrections and adding some notes to prevent possible misunderstandings.

Tags: DDD, Software Development, Agile, Project Management

Preparing the Friday DDD Event

Ok, this is probably more a twitter than a blog stuff, but despite the Domain Driven Design class that I am teaching together with Gojko Adzic, I am still up at night trying to refine my presentation for the DDD Exchange this friday in London.

It's growing in size, but time is fixed... hmm (sounds like some kind of deja-vu). We'll see...

Sharing information within organizations

Naresh Jain tweeted code reuse being still an unsolved problem within organizations some days ago. This made me think that this is a symptom of a more general problem within organizations, who do not account for information and knowledge transfer costs.

I’ve seen a few attempt to promote reuse within organizations, and they all failed because they assumed a non realistic communication model within the organization. In some cases the approach was completely naive: just because some code had been written to address a specific problem, everybody was supposed to know about this code. There's only one organization capable to achieve such level of coordination: it's called "The Borg". They have a collective intelligent and are continuously in sync, one and another.
This type of information sharing simply does not happen in nature.

A more sophisticated approach involved use of repositories and/or knowledge management system to promote information sharing in a common tool. I've seen most of these approach fail as well. Having a common repository to share the information is generally a good idea, but expectations on those systems were really high, and probably unreasonable.

The starting point to promote code reuse within an organization is to know that there is some code that already solves your problem. From a communication perspective we have Developer A, a potential writer of an information at time T0 who knows something about what he's currently doing. Some times later, or maybe simply somewhere else, we'll have Developer B with a similar problem, at time T1. To have a successful communication:

• A should have written something meaningful about the problem he solved
• B should have searched the repository for an existing solution
• B should have found what A wrote
• B should have recognized that what A said was relevant to his problem

I think most of these steps are inherently non-trivial, basically because there is a huge amount of arbitrary decisions in every step (including whether to do the step or not). Words used by A might be different from words used by B to search. Or the information might be filed under an arbitrary label. Basically, this is a decoupled asynchronous communication. There's no feedback possible, and every arbitrary decision is an obstacle between A and B.

I personally find structured systems for storing information particularly bad: they force users to choose where to file their data, and the way people label their knowledge varies a lot form person to person and from time to time. Non-structured information repositories, such as Wikis with a Google-like search tool are definitely better, but they don't solve the core of the problem.

Sometimes we can connect our specific problem to a more general case, for example a "shortest path problem" or a "traveling salesman". This is probably how we'd call the problem after we've found the solution. The way we describe the problem before might be a lot different. Knowing how to call the problem is already a step towards the solution. This means also that who describes the problem and who describes the solution might use different terms. making our asynchronous-remote-decoupled communication at high risk of being completely ineffective.

Effective communication about partially unknown problems needs a higher amount of feedback. That's what we do when reading the menu in an ethnic restaurant: or (closer to our example) when we get into a shop to buy things we've seen but we don't know the name. A clerk might have a gotcha and tell us "Ah! you meant this!" showing us what we wanted and telling us how it's called. The most effective way to communicate about unknown things is the one with more feedback: face to face conversation.

Unsurprisingly, organizations trying to push communication through structured systems are often the ones that put a lot of efforts in preventing effective face to face communications to happen. Well ... in fact preventing good things to happen is one thing organizations are good at. And effective communication is one of these fragile things which might happen in a good environment, but there's no way to force them to happen.

The Big Agile Practices Survey

Jurgen Appelo is is currently doing an interesting all-round survey about agile practices. The more people will participate, the more accurate the results. If anyone is interested, just follow the link.

Blogged with the Flock Browser

Kanban vs Scrum

Thanks to Jurgen Appelo, I've found this incredibly clear and useful article about Scrum and Kanban on Henrik Kniberg's blog.
The type of reading that makes things perfectly clear. :-)

Blogged with the Flock Browser

Join the Agile Revolution

Another nice agile certification.

The mechanics of continuous integration

I was reading this nice post about Continuous Integration, from Distilled Brilliance, by John Feminella (thanks to Marco Abis for pointing me to that, via twitter), and I have to say I wholeheartedly agree: setting up a CI dedicated machine is absolutely pointless if mechanics and disciplines are note attacked.

• Unless the teams move to a smaller increment policy, some of the advantages of CI won't be appreciated.
• To make consistent progress with small increments, TDD is your better friend.
• If the team is using a CI server, but team members don't quickly fix the build if it happens to be broken, then you're probably lost (I know this is rather obvious... but... )
• If the team is using a CI server but nobody checks the status before updating the local version of the code you're probably lost again.
• The team should be working on the trunk, and not on separate branches.

As John pointed out, it's much more a matter of discipline rather than of practice. And discipline takes time and effort. In this case it takes trust and coordination. The same kind you might find in a crowded bar at rush hour when many bartenders are communicating quickly and finding things exactly where they should be, by adhering to team standards (such as "she small dish calls for an espresso" while "the large one calls for a cappuccino").

It takes time to get there, but once you're there, the benefits are great.

Toxic Code

During these days, the word toxic is often associated to investments hidden in some bank caveau, their risk have been underestimated and the result is a backlash with devastating effect.

I think many institutions have the same toxicity problem in their software, but still fail to admit it: key portions of their software act like small timebombs and are quietly undermining their businesses.

There are infinite examples of what I consider to be toxic code, here is a small checklist some might be familiar with:

• Useless code: it’s there but nobody uses it
• Untested code: it’s there, probably works but nobody wants to touch it
• Valueless code: Software which is not delivering any business value
• Annoying code: every change to the system takes more time than it’s reasonably necessary.
• Annoying software: the code is “working”, but in a way that makes things harder instead of easier. Users waste time every time they perform an operation.
• Leaking software: key portions of the application are not tested enough, some exceptions are unhanded resulting, in random untraced errors.
• Countdown software: software with a time-related flaw that will have a havoc effect at a given point in time.

• ...(add yours here)

Well... it’s basically the same idea of technical debt (and Ward Cunningham posted an excellent short video on that - thanks to Piergiuliano Bossi for pointing it) which should be familiar to any agile developer, and was created as an analogy to the financial world. But my worries are more related to the business side, of software rather than to its production process. Ideally, we should have 100% trust on our IT systems (don't laugh please... I said "ideally"), because every time some of them it’s not working there is a waste somewhere (be it time, money, or a customer abandoning the company).

Sometimes companies are aware of the real quality of their running software, and decisions to fix or improve specific portions of the running application are just the results of conscious strategic budget allocations. More often, this is not the case: like toxic assets, software with risky behavior is treated like normal production code, underestimating the associated risks and drawbacks.

Anyway, even when companies are taking into account the costs of a less-than-optimal solution, they’re generally considering only the direct effects of this choice. Something like “this feature might be more user friendly but it will cost us $xxx to rewrite it and the possible revenue for that will be only $yyy, so it’s not worth developing” which is reasonable, but doesn’t consider the cost of not developing it which is put on the user who is having a less-than-optimal user experience (well ...sometimes a miserable one) wasting time and/or money any time he uses the system. It's like polluting: if you don't count it, it's cheaper to do dirty things.

I am still intimately convinced that computers should make life simpler and not more complicated. And I am also convinced that they can.
For some reasons they don’t.

SOA desperately needs DDD

Talking with some colleagues, involved in SOA projects, I often have the feeling of a big hole in the perception of what SOA is really meant for, and what is needed for SOA projects to be successful.

I do believe that SOA is a great idea, it just sounds like large scale common sense. But I also do believe that “large scale common sense” is an oxymoron. Many dysfunctions in SOA projects are closely related to what normally happens in large scale projects, and in many cases those dysfunctions are transforming the whole stuff a whole big loss of money.

A common mistake, is to focus primarily on the architectural aspects of a SOA implementation. Which sounds like a rather obvious thing to do, considering what the A of SOA stands for. Unfortunately, this approach often turns out being narrowed to large scale OOP, only with a different terminology. Services (and their underlying model) are not reusable the same way objects are (and you remember a bit of the times where “reuse” was the buzzword and OO was hype, you probably know that also object reuse turned out being a lot different from the premises).

Behind its interface, a service is implemented according to a specific model. SOA makes no assumptions about the model, and allows for different implementation strategies: so far so good. Still one of the primary drivers for SOA is the need to rationalize the enterprise landscape, by removing unnecessary duplications and enforcing reuse of enterprise level services (we’re still in the “large scale common sense” here). Too often, the attempt to rationalize the landscape goes too far, trying to involve also the model in the process. This is often linked to the way SOA is conceived and implemented within your organization, a policy like “every significant entity will have to be wrapped by a service” will possibly lead to an enterprise scale CRUD moloch.

Enter Strategic Domain Driven Design
The point is that a service might be a significant reusable entity throughout the enterprise, while a model might not. A model should be the optimal solution to one specific problem, valid within a specific context. The context must have boundaries to allow for an optimal solution. An enterprise-level model will rapidly become blurred and some key abstraction will start to serve too many different purposes.

What is a Customer within an enterprise? Can you really find a one-size-fits-all model to represent a customer within the many context that must be supported by your enterprise scale SOA? My take on that is ...nope. Well, you’ll always be able to find some trivial entity that can be the same throughout the whole enterprise, but the odds will be against you if you start looking to some non-trivial entities existing in multiple domains.

In strategic Domain Driven Design there are some key principles to address this situation.

• “a model serves a specific use”: a model is a tool, and to be perfectly shaped for a specific use, the use must be well defined. Like a tool, a model can’t be really effective if the same object is used for many different purposes. Also, an effective model must be kept small enough to be coherent and manageable by a single skilled development team.
• “a model lives within a well defined context”: context and their boundaries are really important to define a coherent model. There are entities that can be used in different context, but sharing the same entity is not necessarily the best way to address the problem. Often the drawbacks are heavies than the advantages.
• “there will always be multiple models”: despite this sounding rather obvious on a large scale SOA, many times a lot of effort is dedicated to try to fight this situation, with minimal chances to win.

A typical problem with SOA is that implicit communication costs are rarely accounted. Sharing the vision of a model within a development team already has a cost which can be kept small enough if the team size is reasonable. Having the same vision within a 5 persons team is feasible. Sharing the vision among 40 people (or more) from different consulting or body rental companies (which is a common scenario in large-scale SOA development) is pure utopia.

Greg Young interview about DDD State Transition

In this interview on InfoQ, Greg gives a very interesting perspective on how (and also why) to use Domain Driven Design to build a complex system.

Playing with Domain Specific Languages in Groovy and Grails

I’m just coming back from the Italian Java Day, where I talked about Domain Specific Languages on the Groovy - Grails platform. I’ve uploaded the reviewed version of the presentation on slideshare, with some extra notes, if you are interested.

The whole idea is related to the possibility of adopting multiple DSL simultaneously. I took some interesting hints from Guillaume Laforge, Graeme Rocher and Peter Backlund, for the DSL part, as well as some from Ninja Coding Monkey, when it came to integrate my DSL specific value Objects with Grails.

Groovy meta-programming offers tools for designing DSLs that were not possible in a static language such as Java. Operators overloading and coercition make it smooth to deal with numeric types including non-primitive types such as BigDecimal or BigInteger. Still, the most powerful tool is the possibility to re-define the reference operator ‘.’
allowing for more compact and explicit notation such as

100.EUR == 50.EUR * 2

for Money types.

However, this involves adding new behavior on the Number metaClass, which is not entirely correct in terms of object oriented responsibility assignment: it’s basically a new Money constructor that we’re attaching on the Number class instead. It works, it’s cool, but it’s definitely not the cleanest code on Earth.
Different DSLs have many points in common and their readability relies heavily on the same features, so a centralized coordination role turns out to be necessary. At the moment I am working on a prioritized DSL engine that allows pluggable fine-grained DSL to be activated separately in the same application, for example I can write something like:

1000000.EUR == 1000.k.EUR
1000.k.EUR == 1.M.EUR

but that’s only a small a part of the fun. More details in the next posts

It's all in the mindset

I don’t know if it has something to do with the Shu-Ha-Ri theory that Alistair Cockburn uses to describe the three level of experience in the agile practices (In the Shu phase you practice and learn, in the Ha phase you start applying variations to the lessons learnt, in the Ri phase you do the right thing by instinct, because you know that’s he right thing to do), but there’s something in common between the most successful approaches to software development I’ve seen. It doesn’t necessarily have to do with agility, many of these teams or individuals were indeed applying agile practices, but I’ve realized that they were doing something deeper, and agile practices was only part of it. On the other hands I’ve seen so many people completely screwing up software development, regardless of the process in place, blaming waterfall first, and then RUP, and Scrum later on.

Technical excellence does make a lot of difference, but still I’ve seen really skilled people doing awful things, despite their qualities. What does still make a lot of difference is in he approach to software development, is something that is a combination of a quest for continuous improvement (something that we could maybe call lean, but that’s to close to be another label) and of desire to experiment and challenge what we already know.

Learning is a non-linear process. It’s not piling up bricks of knowledge. It’s challenging what we already know against new experiences. It’s experimenting different ways to do things, and analyze the outcome. It’s something different from reading books or following advices, pretending to be right just because somebody else said that. It’s dare to be wrong. It’s trying to do something different from the last time we did the same thing, because doing the same thing twice in the same way won’t teach us anything.

Put this approach in a person, or in a team, and you probably won’t need so many books. Books can provide you inspiration, for “proven” solutions, but proving a solution, is ultimately the key of our job.

Subcontracting is a recipe for disaster

A friend of mine recently told me about a tricky situation he was trying to solve. A software company was supposed to develop an application, but they subcontracted development to another company. After a while, the project started going out of control: features were late, quality was poor, bugs were not fixed and release date slipped indefinitely. Things were so bad that lawyers started to warm-up.

As many of you, I had a sense of deja-vu, when hearing this story. The thing that struck me was that this type of project is doomed from the start! Why do people repeat the same mistakes over and over and over?

Money for nothing
Let’s dig into the scenario: customer asks Company A to produce some piece of software.
The price can be defined in many ways, but it’s normally related to a rough estimation. Anyway, Company A gets the contract. Even if they are not going to write the code, they want to be paid for the marketing, and everything that comes before the project starts, somebody will be paid to “manage” the project and some money will be allocated to cover project risk.

Budget splitting from Company A’s perpective.

Ideally, such project should provide a good ecosystem for communication between the customer and the developers, allowing developers to explore and to build consistently upon frequent customer feedback.

The supposed development ecosystem: project manager deals with high level issues, while detailed communication happens informally between the team and the customer

The subcontracting scenario
In a common subcontracting scenario, Company A decides to outsource development operations to Company B. However, since the original deal was between Customer and Company A, a new deal is necessary between Company A and Company B. Sometimes this deal is completely visible to the customer, but sometimes Company A is just pretending to develop the application while Company B is developing it behind the scenes.

Anyway, the new deal between Company A and Company B is based on a different project budget, since some costs (marketing, risk and project management) are accounted on Company A.

Now the project may be worth about 1/3 less than its original budget (the exact figure may vary according to greediness and other human factors) but Company B still needs to allocate budget for its own project management and risk coverage: nobody wants to be sued for somebody else’s mistake.

The project budget, now seen from Software Company B’s perspective.

Adding two separate management boxes around the project doesn’t provide any real value in return, the financial effect is that now the project is a low-budget project, so company B will probably start looking for the cheapest software development resources available.

One would expect that the extra money put in Project Management and Risk Coverage would turn into a perfectly managed, risk free project. Well... not exactly so.

By the way, this is often the schema which is applied in offshore software development, where the common belief is that developers are so cheap that you can compensate the extra costs related to offshoring with huge savings in product building. I am not going to dig into offshoring in this post, but it’s definitely not that simple.

Messin’up with things
Unfortunately, this type of two-levels management doesn’t only duplicate costs, it does more: it makes the whole process inefficient, and it’s actually effectively working against project success.

The subcontracting development ecosystem: formally the referring person is still belonging to Company A, but development is performed by Company B. The key communication channel is now obstructed.

• Communication flows from Customer to Company B through Company A. If Company A is the one to be paid, then all of the key discussion must pass through A, transforming the supposed Project Manager into a bottleneck.
• Indirect communication means delay, important information may rot in a mailbox, before being forwarded to the person associated with the solution.
• Indirect communication means also information loss. Some information must be understood, not only listened or read. Sometimes forwarding it’s not enough to ensure some key detail is not missing.
• Since the information is sensitive for two different contracts, all parties become more inclined to write key issues instead of using face-to-face communication. Some issues are then carefully evaluated and involve more negotiation on an official level. The entire process becomes then waterfallish: having three players, with higher risk and no effective direct communication, paper (specifications, change requests, and so on) becomes more important than face-to-face communication eventually slowing down the process.
• The customer role becomes awkward. The customer is the real customer, but for Company B, Company A is ‘the customer’. Unfortunately, the quality of information you can get from a customer who is not the user of the software it’s a lot lower. So the risk of doing the wrong thing it’s extremely higher.
• If Company A is still pretending to be the developing company, direct communication between Company B and the Customer might even be forbidden, making things even more awkward.
  

The overall effect of all these factors combined is often pure havoc. A project with a decent budget becomes staffed with inadequate people, in an ecosystem that obstructs good communication between the customer and the developers, and where management roles are duplicated adding costs and subtracting value.
Is there any hope?
In theory, there is one scenario where it could still work: it’s where Company B is so good at building software that its estimations are significantly smaller than Company A ones. But to put this scenario in the real world one should answer the question: “Why a company that is so good at building software needs marketing from another (and poorer) Software Company?”

As developers, we’re probably associating this situation with the decorator pattern, which is simply adding features (a more effective marketing) to some other (the development) without changing it. Well ...this is simply not the case with subcontracting. The act of subcontracting has a negative impact on development and should be avoided or limited. As a customer, it does make a lot of difference if the software company is actually developing the software or simply playing the role of a broker. Having seen many of these situations going straight to non-recoverable state, I really think that special care about these issues should be taken in the pre-contract negotiation phase to prevent unavoidable surprises.