Monday, November 18, 2013

Introducing Event Storming

In the past months I’ve spent some time experimenting with this weird thing. It started like a “Ouch I have no time to draw a precise UML diagram, let’s do this instead” then became a thing called Event-Based modelling workshop that I presented at Italian Agile Day 2012, I later had the chance to do more experiments in Belgium and Poland during Vaughn Vernon’s IDDD tour, and I gathered incredibly valuable feedbacks and insights. I managed to find a cooler name - EventStorming - just before the whole thing exploded in summer 2013. While I realised there was a lot of value in it, other practitioners (Mathias Verraes, Tom Janssen, Marco Heimeshoff, Yves Reynhout, Tomas Jaskula, Alessandro Colla, Andrea Balducci, Jef Claes, just to name a few) started exploring and playing with the format with amazing results, leading me to the conclusion that this is something more than “just another workshop format”.

What is Event Storming

Event Storming is a workshop format for quickly exploring complex business domains.
It is powerful: it has allowed me and many practitioners to come up with a comprehensive model of a complete business flow in hours instead of weeks.
It is engaging: the whole idea is to bring people with the questions and people who know the answer in the same room and to build a model together.
It is efficient: the resulting model is perfectly aligned with a Domain-Driven Design implementation style (particularly fitting an Event Sourcing approach), and allows for a quick determination of Context and Aggregate boundaries.
It is easy: the notation is ultra-simple. No complex UML that might cut off participants from the heart of the discussion.
It is fun: I always had a great time leading the workshops, people are energised and deliver more than they expected. The right questions arise, and the atmosphere is the right one.

How does it work

Here are the basic steps:
  • Invite the right people to the workshop. Ideally you’ll want a large meeting room with 6..8 people, with the right mixture of the ones who know the questions to ask (and which are curious to listen to the answer) and the ones who know the answers.
  • Provide unlimited modelling space. Too often complex problems are not properly analysed because there’s not enough space available to see the problem. Your problem is bigger than your whiteboard, so what? My solution is to hack the modelling space using whatever available (my favorite tool is an Ikea paper roll) to get rid of the space limitation.
  • Explore the domain starting from Domain Events. A Domain Event is something meaningful happened in the domain. It can be easily translated into software, but the real value here is that it could be quickly grasped from non technical people. An event might be the predecessor of the follower of another one. Place all of them onto your modelling surface (the convention is to use orange stickies for this purpose) according to a timeline.
  • Explore the origin of Domain Events. Some events are the direct consequence of a user action —> represent it as a Command using a blue sticky note. Others are the consequence of something happening in external systems or of the time passing, we’ll use a purple sticky note for them. In some other cases, we’ll have events that will be the direct consequence of some other events. We’ll simply place the two events close together.
  • Look for Aggregates. Instead of defining aggregates starting from the code, we’re taking an outside-in approach: the Aggregate is the portion of the system that receives commands and decides whether to execute them or not, thus producing a domain event.

Bonus targets

These are the basic steps of the original EventStorming format. However, you might spot some bonus goals along the way, if the discussion becomes hot. Here’s a list of the possible bonus targets which are worth considering as a rewarding detour from the standard route.
  • Exploring Subdomains: some domain experts will show more expertise in an area, leaving other portions of the domain to others. Different areas of responsibility map pretty well to different subdomains or portions of the pork if you’ve been exposed to my presentations in the past.
  • Exploring Bounded Contexts: during the discussion, some conflict areas might emerge. Developers and facilitators with a DDD mindset will spot different interpretations of terms, as a trigger for discussion around the meaning. This might be a good moment to draw the boundaries between the multiple consistent models that will coexist in your domain.
  • Sketching User Personas: when talking about commands, conversation tends to steer towards the context where the command is issued and the goals of the person triggering the action. This is valuable information, don’t blow it! You may expand the notation to include little yellow sticky notes as personas.
  • Sketching Key Acceptance Tests: if the discussion starts revolving around corner cases or ambiguous scenarios, there’s no better way to remove ambiguity than to define a clear acceptance test. Not all scenarios will need to be documented in such a way (I mean not in this workshop, mostly for time reasons) but if they’re a tie breaker in some areas there’s no reason not capture the emerging knowledge right now.
  • Sketching Key Read Model Artefacts: for some scenarios what the users see if far more important than what the system does. If a screen, table or graph particularly valuable  to a given user, just sketch it in a sticky and place it close to the command it is associated to.
Below an overview of all the possible the workshop steps, so far:
All the possible workshop steps
A better quality image could be seen here

Putting everything together is a lot of stuff. Just keep in mind that the goal of the workshop is to learn as much as possible in the shorts possible time. We invited key people to the workshop, and we don’t want to waste their time.
So, when it comes to these bonus goals, be ready to use time in the most efficient way: you get a valuable hint, just sketch it and place the corresponding sticky in the hotspot. Don’t  drive the discussion towards model completeness: the model is going to be huge, completing it might be of little value, or even sound scary for some participants.
Embracing incompleteness will make the workshop less boring and more fruitful.

Choosing the right format

As you noticed, there isn’t a single format for the workshop. In fact, the first steps are more or less the same, but the format may vary according to the roles of the participants, and the project scope.

Minimal Scope

I’ve found the results really valuable even when stopping with Domain Events, i.e. we did an EventStorming session in my company exploring the business flow of training classes we are running. Participants were not developers, so the goal was simply understanding the system by asking the right questions. And we did it in a couple of hours. At the end of the workshop, every new hire had a clear idea of what our company was supposed to do.
IMG 2171
(In this scenario we used a slightly different notation, that was more meaningful for the participants: Orange —> Events, Purple —> external systems, Blue —> external organisations or communities, Green —> Artefacts). 

When exploring for software development the result is even more powerful. Aggregates, Commands and Domain Events, all map pretty well into software artefacts. You might run the workshop to grasp the big picture really quickly and to provide a physical space where discussion around the flow happen.

Turning the model into code

DDD practitioners like this approach not only because it’s fun but also because the resulting model is a lot closer to what they need: this ain’t no data model. The resulting model is fully behavioural: there is no underlying data model that constraints the implementation against its very nature. If you’re oriented towards Command-Query Responsibility Segregation chances are that after trying the workshop you’ll be experiencing the urgency of offering me a beer.

You can also start coding almost immediately and validate the result of your exploration in code in a very short time. This is what most productive teams do, and the combination of discussion-based discovery with a Domain-Driven oriented implementation can deliver massive improvements.

You’ll be basically implementing the right thing faster.

How do I start doing EventStorming?

Running an experiment isn’t that difficult. All you need is:
  • a suitable room, quiet and large enough to contain the modelling surface (if the weather permits, outdoor modelling might be an option too, but wind might be a major blocker);
  • a writable surface, most likely an Ikea paper roll (codename Måla, you can find it in the kids area).
  • a LOT of sticky notes, in different flavours (the basic set is pale yellow rectangular stickies, plus orange, blue, and purple squared ones);
  • working markers, ideally one per participants plus backup;
  • some masking tape, just in case;
  • the right people.
  • a facilitator.
You might want to prototype the execution in a sandbox (some User Groups are perfect for that, they love experiments) before involving your Darth Vader-like boss. You might also want to join the growing community of practitioners and experimenters.
Or you might want to hire a facilitator, for a day or two, to run the workshop in your company, while providing guidance about the workshop itself and the further possible steps.

What’s next…

While practicing and experimenting with EventStorming, I realised that I’ve stumbled upon something bigger. I tried to tell the story of what happened in the past month in this presentation. I called Model Storming the more general approach I am now applying, with EventStorming being just the most notable implementation.
More about it will come soon. This stuff is too cool to keep it only for myself! ;-)

Friday, October 12, 2012

Root Cause Analysis stencil uploaded on Graffletopia Stencil

There are few OmniGraffle stencils which I use frequently during my consulting gigs, especially while gathering information for a change initiative, and in presentations also. One of my favorites is the one I just uploaded to Graffletopia for sharing.

Notation info

  • Observable Facts are ellipses (or bubbles). 
  • When using a tool for drawing like OmniGraffle, I take advantage of colors to display perceived severity. White is neutral, from yellow to red is bad. Black is disaster.
  • Border thickness maps to our perception of our ability to affect/change a phenomenon. A fact with no border is under our control, a fact with a thick border is something we cannot influence easily. A company policy which is generating an undesirable effect might be mapped like a constrained fact (but later, you might be in the position to influence it).
  • Unknown Areas are represented as clouds. There's something important there which we don't know yet.
  • Question marks are part of the stencil as well. I never know everything, but mapping the things that I don't know is often a good idea.
  • Since my primary use is to collect information for change at a system level, local solutions are part of the notation also.
  • Direct dependencies are represented by arrows. 
  • Dotted line arrows represent indirect dependencies. I'm less strict than some other more precise notation, I use the same dotted line for not-so-clearly-related facts and for relationships with delay. It's less precise, but at sketching time I probably don't have enough information to make a clear distinction anyway.

Some guidance

I have to admit I am driven by instinct more than a real formal process. However in drawing the map I have influences from Peter Senge ("The Fifth Discipline"), Craig Larman ("Scaling Lean & Agile Development" and from Eliyahu Goldratt (who described the Reality Tree in "It's not Luck!").

Collecting Data

I start collecting sparse data, from my observation. If I can draw direct links, I add them. If a fact has no clear explanation, I add a cloud or a question mark. Same goes of the causal effect link is no sufficient to imply the consequence. The fact that it's raining doesn't necessarily imply that I'll get wet. But if my car is broken, and I have no umbrella, and I have to go shopping for food, that's another story.

I don't need that much precision on the constraints. I like to capture the mood. If the team says there's nothing that it can be done about a policy, I'll draw it as a constrained fact. But I'll challenge that assumption later when it comes to use the drawing for collaborative problem solving,

Please feel free to share your comments. Hope you'll find it useful!



Monday, January 02, 2012

Setting up a DDD sample app

The guy who started it all is called Emanuele Del Bono. He’s a friend, a nice guy and a respected colleague. He participated in the first Domain-Driven Design class of my company, and also did a fantastic job in finding a suitable place for the class. He is also one of the guys that animated a discussion on the DDD-IT mailing list that urged me to write this posts, some time ago. So I should have been warned about the danger behind his simple request.
Some weeks ago, he complained about the lack of code example in the DDD-IT mailing list, and challenged the list to see if anybody dared to post the code of some aggregate. Innocent and legitimate request, ...or so it seemed. However, since he used the word “courage”, the one that kids use among themselves to do things they will regret, I accepted the challenge and started working on some code that could be shared, and found useful by the list.

Why should I regret it?

As I said, the request is both legitimate and innocent. What could be the drawbacks. I can point out two.

  • Time: I like to do things well, and sometimes I tend to get lost in meaningless details. So, for me, doing things right will take time. Given how busy I am, there’s a clear and present danger of never finishing the stuff.
  • Questions: there are many things that I don’t want to talk about when talking about Domain-Driven Design. Frameworks are one of these. I use some of them, but I can’t really say I love them. However, I don’t like questions like “how do you do this with Spring and Hibernate?” because you can probably do it in a smarter way that I can figure out. I think one of the values of DDD is to provide a conceptual background where these type of problems, decrease their importance to the point of being mere implementation details (even though they can make your time a hell if they don’t work exactly as expected). So my worry is to get too many questions focusing on the wrong side without really clarifying doubts on the part that matters most.
  • Choosing a domain

    I definitely need something different from Cargos. Since I know the training domain a little, and used it also in the recent DDD school camp, I decided to choose this domain. However, as will be clearer later, we’ll need precision in defining what this is really about. Simply knowing how things work in the training domain won’t be enough.

    Choosing a programming language

    Since colleagues were eager for code, the first thing to do was choosing a programming language to actually show some code. The obvious choice was to choose my favorite language and get along with that. Problem number one: I don’t have a favorite language. Problem number two: there are too many things in the code that could divert both writer and readers attention from the actual scope of the task.

  • Java: this has been my first choice, a little conservative I have to admit. However, it’s bee a while since I wrote some java code, and I stopped following Java trends after Oracle’s takeover. Last but not least, all of my local configuration needed an update.
  • Groovy: a lot more pleasant to work with than Java, but to make it really pleasant I had to deconstruct Grails a little, in ways that might look obscure to most readers, carefully selecting only some of the available features. Good topics for a blog post, but not for this app.
  • C#: though I’ve been recently playing with the platform, and I appreciated some features, I still don’t feel like it’s mine. Still too hard for me to distinguish good and bad stuff.
  • Ruby: I am relatively new to the language, and probably missing some key tips on the technology stack and on the programming style (“You still think like a Java guy” Paolo Perrotta once said, I hope I improved a little since then…).
  • Given this starting point, I started with Java. I thought I could set up quickly a skeleton architecture with Spring, like the ones I was used to work with.

    So I opened up the IDE and started coding.

    Wrong choice.

    After setting up an Aggregate an some satellite Services that did their dirty job, or faked it with a mock implementation, I finally realized that I was getting all wrong and was underestimating the task of writing a sample application. What was my problem?

    Ambiguity.

    The model I was building looked perfectly sound from a “modeling” perspective, but I was getting uncomfortable in the multiple roles of developer, analyst, and domain expert or product owner. Moreover, the “let’s show some code quickly” approach that I started with, didn’t really work. To define exactly what my aggregates were supposed to do, I needed a lot more precision, since a little business tweak in the underlying domain would have driven me in divergent modeling choices. I needed consistency and unity of vision in my domain, before I could implement it in my model.

    Well, that was no surprise. That’s exactly what DDD is for. And a common, simplified development cycle is:

    1. gather some information from the domain expert about features A, B and C;
    2. quickly implement it as a domain model, using DDD patterns;
    3. spot contradictions in the current understanding of the domain, that emerge from the model;
    4. refine the model, asking to the domain expert what he precisely meant when he talked about features A, B, and C.

    The only weird thing is that all the roles are in my head now. And to keep my sanity and stop swinging from one interpretation to another one of the same corner of the domain, I need to be a little more structured, and define exactly what I want from myself.

    Providing a little structure


    I want to model my aggregates around a problem, and I want this to be precisely defined. So, despite this being just an example, or maybe exactly for this reason, I can’t go for shortcuts. Let’s start with a user story and see where it lead us. 

    DDD Sample User Story

    My first problem was to find a good name for my role. I mean, my company is rather small, so there’s not much of a clean role separation. I could have put something like “as Me” and was probably going to be more correct, even if less portable.

    However, that’s defining the starting point, and the scope for the first iteration. But it’s not precise enough. Let’s nail it down with an acceptance test.
    Scenario: Publishing a new training
    Given a Catalog
    And a Training gourmet coding not yet published
    When I publish Training gourmet coding on Catalog
    Then Training gourmet coding is available on Catalog


    And try to make it work. Of course this means setting up Cucumber in the development environment, something I thought it wasn’t strictly necessary, but I realized I am getting addicted to.

    By the way, despite my previous experiences with Cucumber and Java, and the precious guidance by the secret ninjas cucumber scrolls, setting up again Cuke4Duke in my project didn’t work that smoothly. Looks like Maven got in the way. Again.

    No, please. Not that again.

    In a second, Maven destroyed my pleasure. And made me think that, even though I am still a rookie on Ruby, every second spent on solving problems on Ruby is a second spent learning, while the time spent on Java is probably wasted. So, after just a few minutes, I already dropped my platform and started again with Ruby, instead of Java. But this sample app is a sandbox, for learning and experimenting. Needs to be fun.

    Modeling question number one


    Do I really need a catalog? When thinking about managing trainings, a Catalog is the abstraction that naturally comes to mind. However when I started thinking about what I really wanted to do with this application, I couldn’t find any behavior for the catalog within the boundaries of my application. I said “boundaries” which means that I need to define scope more precisely. A little context map will do the job.
    Bounded Contexts plus ACLThe ultra-basic context map, with my application and an Anti-Corruption Layer separating it from the external website model


     The main idea, for now, is to manage the trainings locally and trigger a publishing action on an external website. I have no idea about how to do that, but I need to make it clear that I am not directly working on the catalog that is visible to a potential customer. I have an external website for that. Maybe some hosting service exposing APIs would do the job. But I don’t want to know that. Right now I just need to know that I won’t publish directly on that. I’ll hide the details behind some domain service and hide all the little dirty things needed to complete the job behind the curtain of a neat interface. More likely, I’ll pretend to do that, and shamelessly fake everything, but behind the same interface.

    Ruby, the architecture and me


    I hereby state loud and clear my ignorance about the architectural patterns in Ruby. I don’t know much about it, so whenever you see something that doesn’t quite fit the picture or that seems awkward, feel free to comment and correct me. The only thing I am almost sure about is that I don’t want to use Rails. Can’t say it won’t fit in the picture (I am not looking for a 100% DDD application, and I can figure out portions of the application that could be implemented according to that paradigm) but definitely Active Record is not what I want to start with, and definitely I don’t want a full framework stack. I want to add things little by little.

    Persistence framework


    After exploring a little, I’ve chosen Sequel. A lightweight persistence framework that can be used at two different levels of abstraction. I started with the very low level approach, so my repositories now look a little vintage. :-)

    Also DataMapper looked interesting. I’ll probably give it a try when trying to make the architecture a little more agnostic about persistence frameworks.

    The trickiest part so far has been separating persistence instructions from transaction management. In a typical DDD architecture, transaction management is an application layer concern, but this seems to be at odd with most of the examples regarding ORMs and persistence frameworks in general.

    What I like so far


    I really like the Ruby testing stack. Cucumber and Rspec are incredibly well supported by RubyMine, and coding is a pleasure in this area. Getting to all green is always a pleasure.

    What I don’t like so far


    Of course the database had his revenge after that presentation. SQLite rebelled and I wanted to test the app also with some real DMBS, but installing MySQL has been a little pain (well, the actual installation run smoothly ...it just didn’t work with my Ruby version). I had to switch to a different Ruby version and to reinstall it before having it work with my stack.

    There are also a few things that don’t feel right on the architecture. I missed some of the things Spring was doing behind the scenes, and that used to be my default paradigm so there is a high possibility that the whole stuff looks like a Spring based Java application in Ruby dress. Some patterns which are relevant in Java and C# (like Application Facade and Repository) probably have a more elegant alternative implementation in Ruby, and there are some Singletons around (forgive me) which are basically placeholder for a different implementation. In general wiring up is not as elegant as I would haver liked.

    Ok, but where is the code?


    I’ve started a project on GitHub. It’s called D3N. You can find it there. It's still naive, but I’ve tried to do things well and started to post also issues about the things I want to cover/experiment in the next weeks. At the end I am thinking more about a playground than a “sample app” (I hate this term).
    So far, there's not so much to see, but I've been writing a hell of a long post. Let's try to get more use cases implemented and more coverage of the actual evolution. As I said before, there’s a lot more in doing a DDD application than just the resulting code.

    As somebody once said: it’s the journey, not the destination.

    Wednesday, November 30, 2011

    Denial won't help you learning

    While preparing my talk (video) for the last Italian Agile Day conference, I spent some time retrospecting on my past projects. As people who attended know, this retrospective step ended up completely thwarting the original purpose of the speech: looking back in the past I discovered more than I expected.

    To be completely honest, some of the inspiration for the talk came from Alistair Cockburn closing keynote at JAOO 2007 , I rarely invent anything. Connecting the dots is my job.

    My key factors for successful learning

    (OMG, I wrote this title like I know this stuff... and I actually don’t) While retrospecting on my past projects, I didn’t focus on normal success conditions. I focused on the amount of relevant knowledge learnt during the project. This allowed me to look at the past with a different perspective, seeing things that I probably overlooked in the past. From the perspective of learning, the key factors that most influenced the outcome were:

    1. Team attitude
    2. Motivation
    3. Context.

    Team attitude is a tricky definition. Because it states 2 different things: that the attitude is relevant, and that I actually need a team. Of course I’ve learnt a lot also in projects where I was a lone consultant: I did some stuff, I solved some problems, I spent some time thinking about what I actually did. But the real massive amount of learning happened in projects where I had the chance to solve complex problems as a team.

    Interestingly I couldn’t find any connection to the role I was playing at that time. Didn’t really made a difference if I was a junior developer or a project manager, or a senior architect. Didn’t really made a difference if I was supposed to be managed or managing (I hate this definition, but that used to be the past). Didn’t really made a difference if I was supposed to learn from the colleagues or to teach them something.

    The one single thing that made a lot of difference was the attitude me and my colleagues shared in those difficult projects.

    Learning attitude patterns

    People made a lot of difference also in the business outcome of the project (yes, that boring on-time and on-budget part) but could give me from the very beginning a grasp about what I was going to expect. Let’s make some examples with somewhat stereotypical approaches.

    “I know what I am talking about” that the kind of sentence often heard in organizations. To me, it means “I don’t feel any need to improve”. I’ve met some person like this, but luckily they were not in my team. Sometimes they were part of the organization we were supposed to work for, almost always they were part of the problem. I don’t want hem in my team. Plain and simple.

    “That’s all clear to me” ... No. It isn’t. People that for some reasons pretend they’ve understood everything needed to solve the problem, are part of the problem as well. Quite often, under the surface they look for hints about what the boss said, and keep pretending that everything is fine. Walking timebombs.

    “I have no clue about what we’re going to do” Now we’re talking! The most effective colleagues I’ve worked with always started the project with fear. They called me out for a one on one talk, and with fearful eyes told me “I know absolutely nothing about the topic of this project”, and - believe me - their faces were even better when I answered honestly “I have no idea about it either”. But that conversation was honest, we defined a common ground, and we started learning collaboratively assuming that we didn’t know all the things and that we needed to share information to solve a common problem.

    Confront this profile with the previous two. Would you share information with those colleagues also? A lot less, ...they won’t need it anyway: they (pretended/assumed) that they already know the stuff. They basically shut themselves out from any possibility of collaborative learning. Ok, that’s basically Socrates statement: “The real wise man is the one that knows that he doesn’t know”. ...But not only.

    Focusing efforts in the right direction

    What’s striking me more and more, is that the attitude we had sometime towards learning new things as a team, was incredibly effective also because it was efficient. We din’t waste any time in pretending that we were better than we really were. We did have a massive amount of work to do, but we didn’t have any knowledge debt, and we didn’t spend a second studying things that people around me expect me to know but I don’t.

    Don’t you smell something familiar here? It smells like your dirty little secret to me. Make me think about movies where one of the characters did something wrong in the past and lives in the fear of being blackmailed... Ignoring a key skill, it’s not as bad as committing a crime, but the mechanics in our mind are not that different. Our brain start working in “I can’t let them discover my secret" mode, which is a dangerous slope: it makes you feel like “I am not the person they expect”.

    It doesn’t matter how bad is our secret, the mechanics are similar: shame and fear of humiliation (even if only at the coffee machine level) are powerful triggers, and would lead to the wrong behavior, and also to learn less in the long run. Because stress is preventing from effective learning, because one can’t enjoy collaborative learning, and because a lot of time won’t be dedicated to learning but just to cover-my-ass activities which are brain draining (how many brain cycles are you spending deleting compromising SMS from your phone or wondering if your girlfriend can actually see what you posted on that other girl's wall on Facebook?).

    To be honest, in a software project, your fault (if we want to call it like that) is really little. Telling that you’ve actually never understood OOP, or that you’ve never actually shipped anything to production isn’t a fault, it’s just a honest starting point. Stating where you are exactly and honestly right now would save you and your team a lot of troubles in the future.

    When is the moment of staffing the team for a new project. Now I know exactly what I want. I want people with the right learning attitude.

    Only knowledge gap?

    Not really. The more I look into the problem, the more I realize that the mechanics are the same whether it is knowledge or learning debt, technical debt or motivational debt. Failing to acknowledge where we really are only exacerbates the problem. But that’s a topic for a whole new post. I also didn’t say much about motivation and context... Stay tuned.

     

    Monday, March 07, 2011

    See the Forest and the trees with Kanban

    I uploaded the slides of my presentation about Kanban that I gave at last UGI Alt.Net Conf in Milan.

    Sunday, October 17, 2010