Queuing Theory

What is Queuing Theory and Why Is It Relevant?

Our goal in an Agile Project should be to create Business Value in the shortest possible time. In undertaking this goal we are attempting to move a backlog or "queue" of tasks through a process as quickly as possible. It is here that Queuing Theory becomes relevant.

The time taken from the moment a piece of work enters our work process until it exits (Done) is known as the "Lead Time". This is a valuable metric when using Kanban. Other valuable metrics are "Cycle Time" (the time taken to actually process the work item once work starts) and "Throughput" (the rate at which work flows through the system).

Some Relevant Guidelines From Queuing Theory

To minimise the cycle time we can look for certain problems or warning signs. The following problems can increase cycle times:

•Variability in rate of arrival, or in service time (the time taken to respond to or process a task), will increase the cycle time and the queue. Reducing variability will improve flow. Variability increases cycle time.
•The arrival of large batches leads to queues and associated inefficiencies. Large batches increase cycle time.
•Very high utilization results in no spare capacity for dealing with sudden incidents or unusual batches.

Consequently, when such incidents occur, there are problems associated with increased queues, task switching and associated inefficiencies. This greatly increases cycle time. High utilization increases cycle time.
To improve cycle times and reduce queues, target an even, stable rate of arrival. Restaurants, parking stations and night clubs achieve this by applying discounts or introducing offers at certain times of the day.

Other ways to help are: create smaller batches, create stable service times, and utilize parallelism where applicable (if a task can be split into 2 atomic tasks with no interdependencies then it can be shared between 2 people, thus halving the cycle time.)

Other Guidelines include:

•Decreasing variability early in the process is better than later. The first bottlenecked station dictates the rate to the rest of the system.
•High utilization can lead to problems that are worse than just high cycle times. A system where every station is working all the time is close to collapse (also known as Goldratt's maxim). If everyone is already working all the time, the system has no time to recover when its queue grows due to upstream variability.
•Engage early. Start processing as soon as a batch starts to arrive.
•Monitor the queue. Respond if the queue grows and understand what caused it to grow - respond to the cause as well.

Applying Queuing Theory To Agile Software Development - An Example

As an example, we can apply queuing theory to a common problem in testing:
Testers in many projects do little for the first 75% of the Sprints and are then overloaded in the last part of the sprint - they work long hours (often all night or weekends) to get the work done.
Queuing theory tells us that if you can't match servers to the load (increase the number of testers on demand), then you need to create a more stable arrival rate (the work associated with testing needs to be spread out more evenly).

Based on the guidelines above we need to:
1.Decrease variability. The load should not all arrive at the end of the sprint.
2.Decrease the size of individual units of work (Decrease Batch Size).
3.Engage with a unit of work early.
4.Monitor the queue and respond.

So, in terms of Queuing theory, our solution to the problem is for testers to:
1.Decrease Variability and engage earlier with tasks. Plan and write tests earlier, as part of requirements elaboration (based on acceptance criteria), and automate as you implement. Don't wait for the developer to complete development as this leads to a large, late batch arrival. Writing tests as part of requirements elaboration smooths out the work flow and addresses points 1 and 3 above.
2.Create Small Batch Sizes (work units). Plan for tasks to be small and testable. Don't plan for large user stories, break them into smaller chunks of functionality, ideally 4 hours – use the INVEST principle.
3.Monitor Your Queue. Look for people that are sitting idle because they're waiting for something to test. Understand why they are sitting idle.

References:

Lean Tools and Queuing Theory: http://css.dzone.com/articles/lean-tools-queuing-theory

Wikipedia on Queuing Theory: http://en.wikipedia.org/wiki/Queueing_theory

The Theory Of Constraints

Overview

In conjunction with Queuing Theory, the Theory of Constraints (ToC) presents some additional ideas for making improvements in the efficiency of the Software Development process.

The Theory of Constraints was introduced into management by "The Goal", a book written by Eli Goldratt in 1984.

The basic idea is that you can always find a slowest point (the area with the slowest throughput of work) or the "limiting constraint" in a process by finding out where there is a buildup of unfinished work. This slowest point defines the limit of the amount of work that can be put through the system.

This slowest point is where you need to concentrate your efforts for improvement - because improvements made anywhere else will not improve the rate at which work can be moved through the workflow.

For example, suppose two people are cleaning dishes in a restaurant, one washing dishes and the other drying. If the number of wet washed dishes keeps building up, then the constraint for the process is the person drying. Increasing the rate at which dishes get washed won't increase the number of dishes you are able to put on tables, because those dishes aren't going to be dry. If you want to increase the rate at which you can put dishes on tables you need to increase the rate at which dishes get dried, not the rate at which they get washed.

Applying The Theory to Agile

In more complex processes such as software development, it is sometimes more difficult to see where the constraint is. So look for people who are waiting for work that is coming from someone else. Usually, the "someone else" is a constraint, so a method should be found to improve the efficiency there.

In Agile, the focus on resolving the constraint would be to provide that person with extra training or get other members of the team to assist in the work by "Swarming".

The ToC is important because it tells us where to focus our efforts. For example, if we need to improve throughput in a project, it would be a waste to increase the developer's capacity to code stories from 90 points up to 100 points per sprint if the BAs could only supply 80 points of elaborated stories and the testers could only test 70 points of stories!

The "Limiting Constraint" is defined as the point at which the lowest throughput occurs - in this case the testers, who can only throughput 70 points of stories. Work that focusses to improve any other area will be wasted as our delivery of tested code would still be limited to 70 points per sprint. The first area of concern should be the limiting constraint.

NOTE: Removing the limiting constraint (in this case the testing process, at 70 points) simply moves the constraint to a new bottleneck (in this case it would move to the BAs at 80 points). The process of identifying and removing constraints is thus ongoing.

The Theory of Constraints can be applied to measures other than rate of work throughput. There may, for example be a limiting constraint on work quality.

The ToC may have implications in prioritizing which areas to improve during a Retrospective. If an area identified for improvement does not represent a constraint to the work by any measure (volume, quality, etc) then this area may not be a priority.

Root Cause Analysis For Agile

Overview

Agile typically uses Root Cause Analysis techniques drawn from LEAN.

Root Cause Analysis using the 5 Whys

The 5 Whys is a technique developed by Sakichi Toyoda at Toyota as part of the lean methodologies that were developed there. It is an iterative question-asking technique. The primary goal of the technique is to determine the root cause of a defect or problem.

The technique starts with a statement of the problem, then the application of 5 "whys", with each answer triggering the next "why".

The classic example given to illustrate this technique is:-

The problem: The vehicle will not start.

1.Why will the vehicle not start? - Because the battery is dead. (first why)
2.Why is the battery dead? - Because the alternator is not functioning. (second why)
3.Why is the alternator not functioning? - Because the alternator belt has broken. (third why)
4.Why is the belt broken? - Because the alternator belt is well beyond its useful service life but has not been replaced. (fourth why)
5.Why has the belt passed its useful service life but not been replaced? - Because the vehicle was not maintained according to the recommended service schedule. (fifth why, a root cause)

The use of 5 questions derives from an empirical observation that 5 is typically the number of iterations required to establish a Root Cause of a problem. In practice, the number may be greater or smaller than that.

Root Cause Analysis using Fishbone Diagrams

Note that a Root Cause may not be just one factor. In Complex Adaptive Systems it is common for the Root Cause to be an interaction of factors. The primary technique used to address this level of complexity is the fishbone (or Ishikawa) diagram although a tabular format listing causes can also be used. These tools allow for analysis to be branched in order to provide multiple root causes.

 Th Fishbone Diagram divides the causes into general categories (often Equipment, people, process, materials, environment and management) and iteratively explores each one in an approach similar to the 5 Whys to understand all of the factors contributing to a problem.

Note: The shapes to create these diagrams are available in Visio, in Business->Business Process->Cause and Effect Diagrams.











(Example from http://en.wikipedia.org/wiki/File:Ishikawa_Fishbone_Diagram.svg )

Mess Maps

Lean assumes some degree of linear flow. For very complex environments it may be necessary to go outside of the LEAN methodologies - a Mess Map may be required.

Adaptive Stance: "Experiment, Inspect and Adapt"

Our linear thinking struggles with decision-making in an uncertain situation. Committing to an unknown strategy that will be guided by an undefined series of "experiment, inspect and adapt" cycles is a challenge to most leaders.

This problem is increasingly recognised in military circles. The battlefield is an extreme example of a Complex Adaptive System, with modern communication and networking enabling enemy elements to communicate instantly, sharing information and changing tactics. For a military leader, the consequences of failing to adapt quickly are extreme.

A guideline to achieving an "Adaptive Stance" mindset for military decision making is laid out in "Decision-Making" by Lieutenant Colonel Mick Say and Lieutenant Colonel Ben Pronk:


The Adaptive Stance is built within a framework of a number of key personal qualities:

• Ambiguity tolerance. There are no simple solutions to complex problems, and attempts to remove ambiguity from a situation can be very dangerous. Every effort must therefore be made to resist the urge to over-simplify the complex. Again, one must accept that messiness and sense-making are key.
• Self-reflection through ever-present consideration of the questions: ‘How would I know if I was wrong about this?’ and ‘How much would it matter?’ This characteristic encapsulates an ‘ingrained habit of thoughtful self-reflection about the effectiveness of one’s beliefs, actions and decisions’. It echoes the requirement to treat one’s own ideas dispassionately, helps to combat confirmation bias, and primes the practitioner to be constantly on the lookout for ‘Question Four’ moments.
• Decriminalisation of being wrong, openness to learning and supporting others’ learning. If one accepts that it is virtually impossible to predict the outcomes of an interaction with a CAS, and the process of adaptation entails elements of trial and error, then it becomes completely naïve to expect ‘fail-safe business plans with defined outcomes’. Toleration of failure is an ‘essential aspect of experimental understanding'

Lean - What To Cheer and What To Fear

Overview

Although it began in car manufacturing, Lean has become an increasingly important influence within the Agile Community. However, Lean's origin in manufacturing is a mixed blessing. Although the manufacturing production-line heritage provides us with some valuable continuous improvement concepts as well as language and patterns that are familiar to upper management, this heritage also leads to a tendency for management to see software engineering as an oversimplified pattern - a simple, well-understood process of repeated steps.

What To Fear

In reality, producing a new piece of software is rarely as neat as the production-line manufacture of a car.

The key difference is that the software development process is intended to design and create a new piece of software, while the car manufacturing process is intended to produce identical copies of a car that has already been designed, prototyped, tested and had the bugs shaken out.

Once we have developed our software, copying it is a simple file copy process. The car production line is an analogue to our file copy process, not our software development process.

Comparing software development to car design is probably more accurate than comparing it to the production-line "copying" process. Even the car design comparison is imperfect as the specifications for a car are defined by expert Mechanical Engineers using clear, unambiguous Engineering terms and CAD/CAM techniques, while the specifications for our software business logic are defined by business people, using ambiguous business terms, and are rarely as clearly elicited as the CAD/CAM diagrams for a car.

A Kanban board may be able to capture a perfect representation of the car production process, but it is unlikely that this could be done for software development. The flow for one software module could be quite different to the flow for the next. Attempting to turn a Kanban board into a complete description of your process is likely to be a mistake.

The challenge is to take from Lean the language and processes that are of value, while remembering that there is a difference between creating a fleet of identical cars on a production line, and creating a unique software solution based on an ambiguous, emerging and changing business specification.

What To Cheer: What Lean Brings to Agile

1) Lean/Agile fusion brought us ScrumBan. Scrum's Sprints provide the cadence and development practices, but Lean's Kanban provides the flow.

2) Lean provides terminology and patterns that Executives understand. Explaining Agile to executives in a way that they can buy into is sometimes challenging. We can use Toyota's Lean processes as an example to help explain how and why Agile concepts work.

3) Lean has processes for looking at the big picture. Agile's focus on improving the quality of the product's codebase and delivery process can potentially lead to delivering the wrong product in an efficient way. Lean extends Agile's focus to include areas like Systems Thinking and Vantage. Consequently, Lean has helped us focus on building the right product.

4) Lean provides techniques for continuous improvement. Lean provides us with ideas and techniques for improving the efficiency of our process. This is where most of the Lean concepts are being applied in IT today. Value Stream Maps provide one of the key methods for waste elimination and process improvement.

Value Stream Mapping and Beyond

Overview

Value Stream Mapping (VSM) is a LEAN technique, used to map out process steps in order to both illustrate and understand:
•where work is occurring
•where delays or waste is occurring
•how much work of different types are in the system.

A Value Stream Map also identifies queues, wait times, waste, cycle times and other useful pieces of data. A Value Stream Map helps design visual representations of the work (e.g. Kanban Boards on the walls) and provides candidates for targeted improvement efforts.

What is a Value Stream?

A Value Stream identifies all the steps needed in order to make and deliver a specific product. In redesigning a Value Stream, the ideal result is a continuous smooth flow of valuable new features into production with no delays or waste anywhere in the stream.

The value stream is not just the development process, it includes everybody from the customer to operations and support engineers, and may even include marketing, sales and any other Business Unit that impacts on the stream of work.

Criticisms

The word stream implies that the work passes through a smooth, unbroken, and defined sequence of steps. The reality in a Complex Adaptive System may be quite different:

•When building a software product, specialised steps may be required for some parts of the system, but not others.
•For complicated products it is common for paths to change, and it may be rare for any two software modules to follow exactly the same path.
•In a system that is undergoing continuous change (due to retrospectives or other improvement processes) any process map may be outdated before it is completed.
Defining the workflow too precisely may lead to a rigid, unvarying process being applied where a looser, flexible approach may be more appropriate in order to allow for emergent and changing needs.

However, defining the workflow too loosely may reduce the value of mapping the process. The correct balance will depend on the intended purpose of the mapping.

If the system does not seem to be mappable as a stream of value that follows a defined sequence of steps, then Process Mapping may be more valuable (see below).

When is VSM Useful?

Use VSM:
•When looking for waste, or for areas that offer opportunities for process improvement. (Ask how much work went into this step, and how much value came out?)
•When you need to track one isolated flow in a complex series of chains (particularly if you want to experiment with a process change in one isolated area, with a limited, mapped and understood scope for impact).
•When the process can be mapped as a relatively simple chain of events.
If the process can't be mapped as a relatively simple chain of events then Value Stream Mapping may not be the ideal tool (see below for criticism and alternatives).

How To Run A VSM Session

To prepare a Value Stream Map:

1. Make sure the process is facilitated by an expert with a strong understanding of value stream maps.
2. Get feedback from at least one representative from all of the areas within the value stream being mapped.
3. Do not have your facilitator take notes and then prepare the map in isolation - involve the full team and create the initial Value Stream Map collaboratively
4. Create your initial value stream map using a pencil (you will need to make frequent corrections and changes) on a sheet of butcher's paper. 
5. Once you have the initial map, you can create an electronic version using Visio, or a similar tool..

Make sure you note the metrics needed to detect waste. For example, if you are seeking to improve cycle time, then for each step in the process note down how long the item spent at that station and how long it actually took to be processed. For example, it is common to find that a manager exists as a "gate" in a process - required to make a go/no go decision. Although the manager may be able to make the decision in a few minutes, most managers are busy, and may take days to get to the decision. So although the processing time was only a few minutes, the time at the station was days. Moving the decision to a lower level may make a dramatic difference to the total cycle time.

Once you have mapped the process and eliminated waste, your goal should be to further improve the process. Ask (from a customer perspective):

•What would the ideal flow of new features look like from a customer perspective?
•To make this happen, what would need to happen and in what order?
•What resources would need to be available to make that possible?
•How would roles and responsibilities need to be defined?
•What is the ideal flow?
•What is the best flow we can achieve?
•How can we improve to meet the ideal?
 These questions may suggest possible areas of improvement.

Note: Visio supports Value Stream Mapping shapes. You can find them in Business->Business Process->Value Stream Map Shapes.

An Alternative - Process Mapping

If the system is complex and the path is variable, then it may not be possible to map the process as a simple sequence of steps. In that case, use an alternative to Value Stream Maps, such as Process Mapping.

 Process Mapping is from the Systems Thinking side of Agile and is closely aligned with the Vantage method - so is very focussed on the point at which the customer interacts with the system.

In Process Mapping:

•Don’t ask the managers - they may know the ideal process, but they are often too removed from the work to know the special cases, alternate paths, or even perhaps the actual process.
•Don’t just send in a Business Analyst to talk to people - people are busy, and may even be wary of revealing the full horror of the process.
•Don’t talk to proxy customers or talk to a spokesperson for an area because, once again, you will only learn the ideal process.
•Don’t try to map the process in a room with butchers paper and sticky notes, bringing people in as required.
•Instead go into where the work is occurring, with a small team comprised mostly of peers of those who do the work.

Why peers? Because workers tend to tell their colleagues the truth about what’s really going on at work.

Study the work as a series of systems, viewed from the customer's point of view. Start at the point at which the customer first contacts the system and work out from there.

 Map the system as a series of varying interactions between subsystems. Define the customer's needs and document the sub-system interactions from the customer's viewpoint - explaining how each sub-system contributes to those needs. To improve the system, ask how could the work (mapped as a path through subsystems) be redesigned to support the customer's varying needs. How could the sub-systems be redesigned? How could the interactions be redesigned?

When is a Process Map Useful?

If you have one or more of these problems:
•Multiple entry points to processes
•Independent but interacting processes
•Workflow path is extremely variable, with few or no steps guaranteed
then a Process Map may be of use - either in addition to or instead of, a series of Value Stream Maps:

For more on Process Maps: http://nnphi.org/CMSuploads/MI-ProcessMappingOverview.pdf.

How Agile Servant Leadership can Incorporate Situational Leadership

Servant Leadership is an important concept in the context of Agile Leadership. This topic is a favorite among Agile practitioners - but seems to have become so entrenched that other qualities of Leadership are ignored.

Situational Leadership, for example, asserts that different situations/people require different leadership styles. In some situations a leader needs to take an inclusive, consensus building approach, while at other times a critical situation may require an unambiguous, dictatorial response from even the most benign leader. Similarly, when managing staff, some of your experts may require very little advice or attention, while other less experienced staff may require closer oversight and direction, not because they can't be trusted, but because they don't yet know what they are doing.

Situational Leadership is a subject close to my heart, so here is my slant on Servant Leadership and how it can be synthesised with Situational Leadership.

Although popularised by Agile evangelists, Servant Leadership is not a new concept - Lao-Tzu wrote about servant leadership in the fifth-century BC: “The highest type of ruler is one of whose existence the people are barely aware…. The Sage is self-effacing and scanty of words. When his task is accomplished and things have been completed, all the people say, ‘We ourselves have achieved it!’”

The "Servant Leader" concept is relatively straight forward: It asserts that the role of a leader is to define the team's goals, and then remove the impediments that the team encounters. Since the team (not the leader) is best placed to define what the impediments are, the leader spends most effort on being a "servant" to the team - removing the obstacles as they define them. In Agile, the concept is pushed a little further, in that the Development Team is Self-Organising and Self-Managing. Thus, the role of a Scrum Master is to:

• Remove impediments that the team identifies
• Facilitate meetings and processes that the team requires
• Shield the team from diversions and distractions
• Coach, encourage and support the team
• Help the team improve, collaborate, self-manage and self-direct

The Scrum Master has a complex role - but a large portion of it is being a servant to the Team's needs.

However Gint Grabauskas in his discussion on Servant Leadership (http://www.agilealliance.org/files/1913/5525/5359/ServantLeadershipAgileTeams.pdf) says:-

There is a time and place for autocratic behavior. If a building is burning and I am inside, I don’t want a servant leader guiding me to the door. I want an autocratic manager telling me exactly where to go and what to do to escape. Inevitably those times will come on a project. But a servant leader will have gained the trust and respect of her team so that when the occasional autocratic slant is needed, the team will willingly accept this “anomaly.”

 

So Agile leadership is not just about being a Servant to the team, it is about knowing when to be what kind of leader. Sometimes your team needs impediments removed, sometimes they need a vision to guide them, sometimes they need motivation, and sometimes they just need an autocratic decision about the direction they should take.

The leader is the person that the team looks to for these things - which is not necessarily the person that the Org Chart says it should be.

Responding to the Team's needs gives the Leader a difficult problem. How do you fulfil the more conventional requirement of Leadership - that of keeping the team moving, and moving in the right direction, if you are simply responding to the team's needs? This comes down to the first part of a Servant Leader's role - "Define the Team's Goals". But for a Servant Leader, it is not enough to simply define the goal in terms of outcome - you need to define the goal in terms of motivation. The Team needs to be motivated enough to be pulling you, rather than you pushing them. So you need to describe why the team should want to do this. The goal should normally be intrinsic - the joy of doing something unique is usually enough (though some extrinsic recognition of achievement never hurts). Maintaining this vision is important, it keeps a team motivated.

The Servant Leader's task is further complicated by the fact that your team is not really a discrete entity that can be treated as a uniform whole  - it is a series of individuals with different needs. The Leader that your senior programmer needs may be a laissez faire Servant Leader, while the Leader your inexperienced BA needs may be a more dictatorial leader who provides a great deal of direction and oversight. So you won't be treating people the same - which hardly seems fair.

How do you ensure that this unfairness doesn't lead to resentment? The answer is that you need to be Dictatorial in a Servant Leader way - don't thrust direction and oversight onto your junior BA. Instead make sure that the BA is aware of need, and requests it. Your BA will forgive you if you make a misjudgement and are overzealous in fulfilling that request, but you won't be forgiven if the oversight and direction was not requested.  

Summary An Agile Leader must understand that Leadership is not assigned to you by virtue of your job title, leadership is assigned to you by your team - earned as a result of showing that you can remove impediments, facilitate meetings and fulfil the other leader roles needed by your team. Once this trust is earned, teams will forgive errors - even forgiving moments of autocratic behavior, where a wrong call was made.

The key is this: In Servant Leadership, your leadership is guided by requests. Your team pushes you - you do not push them. This seems a fragile thread to hang your leadership on, but the thread is surprisingly robust - because people forgive your mistakes if they are directing the action. This works at both the team and individual level:
- If teams are kept in the loop and understands the need for an autocratic decision, they will request one from you. Consequently they will forgive occasional wrong decisions and bad calls when they are made.
- If individuals are led to understand the need to be treated differently, they will request it from you. Consequently they will forgive mistakes or missteps in the oversight process, In this way, you can adapt your leadership style according to the situation (Situational Leadership) using the techniques associated with Servant Leadership.

It is thus possible to synthesize Situational Leadership with Servant Leadership. This is not easy. It requires honesty, openness, and direct communication at both the team and individual level. Above all, it requires you to have the courage needed to trust your team.

Complexity and Ambiguity – Or Why An Organisation Is Dumber Than Its Component Parts

Two related bodies of knowledge, System Thinking and Complexity Theory, help us to understand the theoretical basis behind the success of Agile. Consequently, understanding these theories can help us to better apply Agile practices and behaviours.

I have been considering these areas recently. I open up some of my early thoughts for review:

Policy Paralysis: Unanticipated Interactions.

I recently worked for a very large IT company that was going through hard times. It came up with a raft of seemingly reasonable policies that proved insane when applied in combination. Here are two seemingly sensible policies in response to economic hard times:
1.        The company did not want to fire anybody unnecessarily, so it put in place a policy that said that if a local customer needed someone that we did not have locally, before we could hire new staff we had to check to see if the company had someone at some other location who was “on the bench” and in danger of being fired. If so, we would offer that person to the customer.
2.        The company needed to reduce costs, so it had a policy that said that we were not allowed to fly to a meeting – we should use telephones or video-conferencing instead. The only travel that would be approved by upper management was travel that the customer approved and paid for.

These seemingly innocent policies, when taken in combination with the existing policies and environment, drive the system into suicidal dysfunction. It works like this:

The salesperson (I’ll call him John) sees an opportunity to supply four Scrum Masters and a Project Manager to the customer for a 9 month contract in my city. John brings the opportunity to me. We don’t have any spare Scrum Masters or PMs in my city, so I implement the above Policy 1 and discover we have some spare people interstate. However the company has an older policy that states that for any interstate relocation of longer than 6 months but less than a year, the employee has the right to choose between a range of options including to be paid relocation costs for his whole family.

All very enlightened. But in view of the requirement in Policy 2, that the customer must explicitly approve and pay for all travel costs, this meant that we needed to directly pass that cost to the customer, rather than just building travel costs into our day rate.

So I told John that the customer would have to pay all travel and/or relocation costs as a defined line item, rather than just wrapping it up in an overhead built into the day rate.

Naturally, we have a policy that says that salespeople who don’t sell anything get fired. John knew that he couldn’t make the sale if he told the customer that they had to pay the relocation costs to move our employee’s families to this city for 9 months, particularly since there are plenty of very capable Scrum Masters and PMs in this city already.

So John did what he needed do to avoid the sack. He decided that how the staff would get here was a Delivery issue and therefore owned by the Delivery Team. He made the sale without discussing travel costs, and left it to Delivery to discuss relocation costs with the customer.

This broke the customer’s trust in us of course, and it will be some time before we do any business with them again.

Customers in general are reporting that we are “too hard to do business with”, and this invariably stems from requirements to meet multiple internal and contradictory policies – each of which seems individually perfectly sensible.

Did we try to change these policies? Of course we did. But each policy was issued by a different authority, and each authority was charged with meeting a different agenda. Each policy addressed the authority’s required agenda, so we had no hope of getting the issuing authority to change it.

Individually, every policy was perfectly sensible. It was only in combination that they were toxic.  
Which brings me to Systems Thinking…..

Systems Thinking Overview

Systems thinking is not one thing, but a set of habits and practices within a framework. It is based on the belief that, rather than study a system in isolation, a system is best understood by examining the context of the system and its interactions with other systems. Systems thinking focuses on the impacts of the interactions.

Systems Thinking seeks to understand how small catalytic events in one part of a system can trigger significant and unexpected changes in other parts of complex systems. An improvement in one area of a system can have a catastrophic impact on another area of the system - a phenomenon very familiar to Software Developers.


Russel Ackoff is usually considered to be one of the first to apply Systems Thinking to Management.
Ackoff is credited with a number of insights into Systems Thinking as it applies to management including:

·        A System is not a sum of the behaviour of its parts, it is the product of their interactions.
·        If we have a system of improvement that is directed at improving the parts individually and separately then you can be absolutely sure that the performance of the whole will not be improved.

Classifying Systems

Many different methods for classifying systems exist, each allowing us to consider the systems from a different perspective but Russell Ackoff proposed the following 4 classifications be applied to Systems Thinking, based on the level to which the system was able to implement choice (or purpose):

1.        Deterministic (or Mechanical) Systems. If individual parts of the system display no choices, and the system as a whole does not display choices it is a Deterministic System: A wind-up watch for example.

2.        Animated (or Organic) Systems. If parts do not display choices while the whole does, then it is an animated system: A person for example. These systems are also often called organic systems.

3.        Social Systems. If both parts and the whole display the ability to make choices it is a social system: An organisation for example.

4.        Ecological Systems. If some parts of the system do display choices but not the system as a whole then it is an ecological system: An island ecology or a business ecology would be examples.
(Classification is adapted from Ackoff on Leaders: http://frank543.homestead.com/Ackoff_on_Leaders.pdf )


Historically, Governments, Corporations, Business Units and Project Teams have been studied and run like mechanical systems - machines for delivering services (Government) or generating profit (Corporations).

More recently Governments and Corporations have been viewed as organic systems - an organism aiming for growth, with profit being one measure of growth. But neither of these models fit the reality. The reality is that Governments, Corporations, Business Units and Project Teams are examples of Social Systems - able to make decisions at each level of abstraction - and treating the people in these organisations as if they are mechanised parts of a mechanistic systems can lead to as many problems as treating the organisation itself as if it will display simple, mechanistic and predictable behavior.

Because complex organisations display decision making at multiple levels, and because these decisions can interact with each other and with the environment, large organisations display a very high level of complexity, with unexpected paths and interactions constantly emerging and changing. Because of these complex interactions, large organisations may behave in unpredictable - sometimes even irrational - ways.

Amoral Companies:  The Behaviour of the System is Not The Sum of its Parts

In such a complex environment, the interactions between the parts of a system will influence behaviour more than the actions of its individual parts because the emergent properties of the system derive from the interactions between the parts, not the individual actions or even the individual intentions of the individuals who make up those parts.

Emergent properties are defined as properties of the system but not of its individual parts. For instance, a corporation may be composed of people who do not display amoral or predatory behaviour, yet the corporation may display both amoral and predatory behaviour as an emergent property of the interaction between its individual parts within the context of its environment.

Systems need to adapt to their environment to survive. Any system (in our area of interest a system might be a corporation, a Business Unit, or even a project team) that freezes or chooses to go in the wrong direction in a changing environment will suffer from harm or even die. Yet for seemingly incomprehensible reasons, systems frequently choose to take such actions despite the good intentions of the people making up these systems.

(Have you ever asked one of these questions: “How come we have so many smart people in this nation, yet we have a foreign policy that seems to be mostly a collection of pointless wars, while our trade policy revolves around sending everything we have overseas, and then buying it back by going into debt?”  Or: “How come my company is full of smart people yet we are being slowly strangled by misguided policies while being driven headfirst towards certain destruction by a strategy that looks like it was produced as an April Fools joke by a departing work-experience student?)

Ackoff believed that the cause of these irrational and potentially lethal actions is generally found to be not just one problem or influence, but a set of interacting problems, goals and influences. Systems Thinking does not offer a “Fix”, it just tells you what to avoid. Once the interacting "mess" is identified, systems thinking tells you what not to do if you want to avoid further irrational actions.
Unfortunately, just identifying what to avoid is rarely an effective management technique.

Management of a system requires directing the system towards a goal, not just avoiding undesirable outcomes, thus there is a need for strategies for developing solutions in an environment of constant change and unpredictable interacting factors.

The Argument So Far: Unexpected interactions sometimes drive irrational and unwanted emergent behaviours in complex organisations. These interactions are often unpredicted, but can be understood and explained after they emerge. The Agile Methodology’s emphasis on flexibility and adaptation is well suited to this environment, and offers us an approach for dealing with it.

Complexity Theory

Complexity theory helps us:
·        classify the nature of the problems we face,
·        recognise which of these problems are of the complex type described above,
·        aids us in defining approaches that will help us direct our projects towards a goal, despite the constant, unexpected turbulence around us.

My Take on Complexity

In the Agile Community, Dave Snowden is probably one of the most influential shapers of our understanding of Complexity Theory.

The decision-making implications of Snowden's complex systems framework (http://en.wikipedia.org/wiki/Cynefin ) and his emphasis on the importance of running Experiments to help inform decisions and strategies in Complex environments has been influential in shaping the actions of many Agile practitioners in some of the most high-profile and difficult projects and environments.

In 2007 Snowden's conceptual framework around Complexity Theory was the cover feature in the Harvard Business Review (November 2007 issue - http://hbr.org/2007/11/a-leaders-framework-for-decision-making/). The paper was a Citations of Excellence winner as one of the 50 best papers published in 2007. It was also designated as the 2007 Best Practitioner-Oriented Paper in Organizational Behavior by the Organizational Behavior Division of the Academy of Management with the following comment:
"This paper introduces an important new perspective that has enormous future value, and does so in a clear way that shows it can be used. The article makes several significant contributions. First, and most importantly, it introduces complexity science to guide managers' thoughts and actions. Second, it applies this perspective to [] help leaders to sort out the wide variety of situations in which they must lead decisions. Third, it advises leaders concerning what actions they should take in response."

Snowden's Framework

The concepts behind Snowden's framework are deceptively simple. Snowden said that problems can be divided into 5 broad categories:
1. Simple problems. Simple is the domain of Best Practice. The problems are recurring and the Best Practice response is known.
Characteristics: Problems are well understood and solutions are evident. Solving problems requires minimal expertise. Many issues addressed by help desks fall into this category. They are handled via pre-written scripts.
Approach: Problems here are well known. The correct approach is to sense the situation, categorize it into a known bucket, and apply a well-known, and potentially scripted, solution.
2. Complicated problems. Complicated is the domain of "good practice". Problems are rarely repeated, so Best Practice has not been defined.
Characteristics: You have a general idea of the known unknowns — you likely know the questions you need to answer and how to obtain the answers. Assessing the situation requires expert knowledge to determine the appropriate course of action. Given enough time, you could reasonably identify known risk and devise a relatively accurate plan. Expertise is required, but the work is evolutionary, not revolutionary.
Approach: Sense the problem and analyse. Apply expert knowledge to assess the situation and determine a course of action. Execute the plan.
3. Complex Problems. Complex is the domain of emergent solutions. The cause-effect relationship is broken, with a given action rarely achieving precisely the same outcome twice.
Characteristics: There are unknown unknowns — you don’t even know the right questions to ask. Even beginning to understand the problem requires experimentation. The final solution is only apparent once discovered. In hindsight it seems obvious, but it was not apparent at the outset. No matter how much time you spend in analysis, it is not possible to identify the risks or accurately predict the solution or effort required to solve the problem.
Approach: Develop and experiment to gather more knowledge. Execute and evaluate. As you gather more knowledge, determine your next steps. Repeat as necessary, with the goal of moving your problem into the “Complicated” domain.
4. Chaotic Problems. Chaotic is the domain of novel solutions.
Characteristics: As the name implies, this is where things get a bit crazy. Things have gone off the rails and the immediate priority is containment. Example: Production defects. Your initial focus is to correct the problem and contain the issue. Your initial solution may not be the best, but as long as it works, it’s good enough. Once you’ve stopped the bleeding, you can take a breath and determine a real solution.
Approach: Triage. Once you have a measure of control, assess the situation and determine next steps. Take action to remediate or move your problem to another domain.
Example: Production defects.
5. Disorder. Disorder is the space of flux between problem types.
Characteristics: If you don’t know where you are, you’re in “Disorder.” Your first priority should be to identify your problem type and move to a known domain.
Approach: Gather information on what you know and identify what you don’t know. Get enough information to generate a plan to move to a more defined domain.

Where to turn:
·        If it’s simple, trust the manual
·        If it’s complicated, trust the experts
·        If it’s complex, trust the team’s experiments
·        If it’s chaotic, trust the leader

 

The Cynefin Framework

Moving Between Domains

The boundaries of these domains are not hard. Based on activity, situations can bounce between domains or live on the borderlands between two domains. A commonly observed pattern is that a failure to make correct decisions in the Simple domain can lead to unexpected interactions and tip a system very quickly into the Chaos domain. The key to recovery is to recognise that, if the situation has deteriorated into an area where the outcomes are unkown, then rapid experimentation may be needed to establish a path back to stability. Managers must be quick to recognise the point at which expert opinion is no longer applicable (we've never been here before). At that point the correct plan is to move from "Analyse and Decide" to "Execute and Evaluate" - take an action, then check to see if it worked and shape the recovery plan iteratively, rather than trying to define the entire recovery plan in advance. 

As an example, below is the framework as it applied to the various decisions leading up to a potential air disaster. It demonstrates that if Simple problems are handled incorrectly there is frequently a transition from Simple problems to Complicated problems, which then quickly transitions to a descent into Chaos. In this case an incorrect fuel calculation by the flight crew interacted unexpectedly with a technician's mistaken attempt to address a technical fault by disabling the fuel gauges.....

When, without warning, the fuel ran out and the engines flamed out at 41,000 ft, the pilots discovered that most of their control systems were inoperable and, on consulting the expert guides, they discovered that there was no documented procedure for attempting to fly a 767 with no engines, electrics, instruments or hydraulic controls. They were in the realm of Chaos, with no knowledge of what actions might produce their required outcomes. In order to succeed, they had to find a way to transition from Chaos to the Complex realm. Pilots usually work in the Complicated realm, so their framework for making decisions needed to change from analysis of the situation and reference to Operating Manuals ("Analyse and Decide") to experiment-guided decision making ("Execute and Evaluate"). In this case, the pilots successfully transitioned from Chaos into the Complex domain by attempting a series of experiments and evaluating the outcomes in order to generate the knowledge they needed. Using these experiments as a guide, they identified a chain of novel actions that worked and achieved a successful (survivable) landing of the aircraft.

 

The Cynefin Framework as Applied to the "Gimli Glider" - an Air Canada Boeing 767 that ran out of fuel half way through its flight.
(From http://emergingoptions.wordpress.com/2011/11/17/the-cynefin-framework-in-action-the-gimli-glider/ )

Conclusion

The net takeaway is that because the Agile methodology requires a lot of Just-In-Time decision making, you need to understand the type of problem so that you can pair your decision-making approach with the problem type. Cynefin provides a simple Framework that provides a guideline on how to match your decision-making against the situation.

When complex interactions make it difficult to identify a successful path forward for your project, use Agile Experiments to guide your journey.

More on Snowden's conceptual framework can be found here: http://cognitive-edge.com/library/more/video/introduction-to-the-cynefin-framework/


So how do you ensure that your team doesn't form a system that is dumber than its parts?

Policies and procedures are a solution appropriate to the Simple realm - they assume that the correct action is simply to read the script and carry out the action. Yet our work is mostly conducted within Complex, not Simple, problem domains.  So is it any surprise that the emergent interactions of numerous simple Policy documents within a complex environment can generate unexpected and frequently unwelcome outcomes?

It seems obvious to me that we need to depend more on sensible, guided, decisions and we need to rely less on inflexible, unpredictably-interacting processes and policies. Encouraging decisions guided by evidence, gained empirically within your specific problem space, rather than simply implementing blind policy is one way to ensure that you manage a team that is at least as smart as its parts.

The Boss’s Role in a High-Performing, Self-Managing Team

Welcome to your new team. The Team you have been given is brilliant – self-reliant, and successful. You are the guy in charge. Congratulations. But they are so brilliant that they don’t seem to need a manager… they don’t even need a leader…. so what is your role as the “Guy in Charge”?

I’ve been there. Here is my advice:

1. You do not manage a High-performing team, they self-manage.
But you do have management tasks:
·        Yes they mostly self-manage, but despite how it sounds, self-management doesn’t just happen. Some stuff needs to be scheduled or organised. For example - Schedule their planning sessions and facilitate the meetings. As a facilitator you are independent – you don’t get a vote. If you (rather than a team member) facilitate, this frees the individual team members to advance their views without concern that the team member who is facilitator is pushing the team in a particular direction. Similarly, schedule and facilitate the Retrospective and other meetings.
·        Find out what they need (ask them). Get it.
·        Find out what their impediments are (ask them). Fix it.

2. You do not lead a High Performance team – once they know where they are going, they will decide how to get there.
But you do have leadership tasks:
·        Set the tone. A high-performance team is positive. If the guy in charge is snarling at everyone, the tone of a team can decline quickly. Smile, greet everybody cheerfully in the morning, and speak about everybody with respect.
·        Set the direction. Yes, once they know where they are going, they don’t need you to lead them. But they NEED TO KNOW WHERE THEY ARE GOING! Place a clear vision in everybody’s head at the outset, and keep it there. This vision is not about requirements, it is about the intent behind the requirements. Your team should be able to answer questions about the intent: Why is this project good? Who is going to benefit? How? What is the business need?
·        Continuously improve the team’s working processes, tools and environment. Retrospectives and impediments should provide signposts towards making the project better and making life easier for everyone. Watch for the opportunities and act on them.
·        Continuously improve the team. Cross training, in-house training, mentoring (formal and informal). Make it happen.


3. So if you don’t manage them, and you don’t lead them…. What do you do with a high performance team?

You unleash them.
You have an unleashing role and it is this:

1. Motivate your team. At the outset place a picture in their head about why they want to do this. What is in it for them? Why is this project cool? Often it is just about being part of a great team that really rocks. If so, make sure the fact that your team rocks gets recognised – both within the team and externally. Then set them free. Make sure they know they are free.... but that is a story for another day.
2. Be a decision maker when your team needs a decision. How to decide (adapted from the Cynefin framework):

• If it’s simple, trust the manual
• If it’s complicated, trust the experts on your team 
• If it’s complex, trust the team’s experiments
• If it’s chaotic, trust yourself. Stay calm, but unleash hell if that is what it takes to bring things back from the brink. Then buy your team some breathing space and put them back in charge.

 

Why I Am Pushing BDD

Test Driven Development (TDD)

TDD, as usually practiced, focuses on writing automated Unit Tests. The developer writes the test so that it fails until the code performs the functionality that the test expects. First the coder writes the test, then the coder writes functionality that passes the test. Refactoring the code to improve structure and readability is commonly done as a final tidy-up step.The main focus of TDD is on testing the low-level functionality in single modules, classes or methods. Kent Beck wrote Test Driven Development: By Example (ISBN 978-0321146533) in 2002, and is credited with developing and promoting TDD as a practice.

Beck said “Test-Driven Development is a great excuse to think about the problem before you think about the solution”.

TDD is one of the core practices of Agile's XP Technical Practices and is frequently credited with being one of the reasons for the low defect density associated with Agile projects.

Criticism: 

1. David Heinemeier Hansson (Agile developer and creator of Ruby on Rails and Instiki Wiki) compares Agile's inflexible requirement for test-first development to religious fundamentalism. Hansson is a strong supporter of Automated Regression Testing, but questions the value of always writing the unit test first. He asserts that test-first development is leading to inelegant software design.

2. James O Coplien (often known as "Cope") is one of the founders of Agile and a leading software theoretician who offers a similar view - suggesting that "Most Unit Testing Is Waste". He points out that the massive computational complexity of testing every unit through every pathway renders the idea of 100% unit test coverage unachievable - thus calling into question the value of attempting such broad testing at the unit level. He is not opposed to the concept of writing tests to prove the functionality of code matches the requirements  - he simply questions the value of broad testing at the unit level. His advice is to focus most attention on Functional and Integration Tests. He provides this guidance:

• "Low-Risk Tests Have Low (potentially negative) Payoff "
• "So one question to ask about every test is: If this test fails, what business requirement is compromised? Most of the time, the answer is, "I don't know." If you don't know the value of the test, then the test theoretically could have zero business value. The test does have a cost: maintenance, computing time, administration, and so forth. That means the test could have net negative value." 
• "If you cannot tell how a unit test failure contributes to product risk, you should evaluate whether to throw the test away. There are better techniques to attack quality lapses in the absence of formal correctness criteria, such as exploratory testing and Monte Carlo techniques. (Those are great and I view them as being in a category separate from what I am addressing here.) Don’t use unit tests for such validation." 
• "there are some units and some tests for which there is a clear answer to the business value question. One such set of tests is regression tests; however, those rarely are written at the unit level but rather at the system level." 
• "if X has business value and you can text X with either a system test or a unit test, use a system test — context is everything."
• "Design a test with more care than you design the code."
• "Turn most unit tests into assertions."

Response to criticism: 

Kent Beck has mounted a spirited defence to these criticisms. Among other things, he responds that TDD is NOT intended to be universally applicable. In conversations with Martin Fowler (http://martinfowler.com/articles/is-tdd-dead/) he further points out that the much of the criticism is about granularity and extent of testing, but this is a decision that should come down to the preference of individual developers:

• "TDD puts an evolutionary pressure on a design, people have different preferences for the grain-size of how much is covered by their tests." 

Beck also makes the comment (with regard to the level at which tests are written and the utility of maintaining tests) that he would:

• "often write a system-y test, write some code to implement it, refactor a bit, and end up throwing away the initial test. Many people freak out at throwing away tests, but you should if they don't buy you anything" 

This certainly suggests that Kent Beck has a more pragmatic approach to TDD than some of the Agile evangelists who are promoting the practice and this guidance can perhaps be used in conjunction with the other comments above as a guide when trying to decide what tests to write and at what level the tests should be written.

Acceptance Test Driven Development (ATDD)

ATDD is a collaborative exercise that involves product owners, business analysts, testers, and developers. By defining the tests that need to pass before the functionality will be accepted, ATDD helps to ensure that all project members understand precisely what needs to be implemented. After it is implemented in an automated testing suite, ATDD continues to guarantee that the functionality works - providing regression testing after code or configuration changes.

So, in theory, TDD focuses on the low level and ATDD focuses on the high level, thus suggesting a clear separation. But in practice it simply isn't that clean. Many coders struggle with questions like what to test, where (at what level) tests should "live" and how to demonstrate to customers that a code-based Acceptance Test actually proves that the code is performing as defined in the requirements.

What is the Difference - TDD Vs ATDD

The concepts that drive Test-Driven Development are related to those that drive Acceptance Test-Driven Development (ATDD). Consequently, tests used for TDD can sometimes be compiled into ATDD tests, since each code unit implements a portion of the required functionality. However TDD is primarily a developer technique intended to create a robust unit of code (module, class or method). ATDD has a different goal - it is a communication tool between the customer, developer, and tester intended to ensure that the requirements are well-defined. TDD requires test automation. Although ATDD is usually automated this is not, strictly required.

Limitations of Conventional ATDD as a Communication Tool

ATDD is envisaged to be a collaborative exercise that involves product owners, business analysts, testers, and developers. In theory, these stakeholders work together to design ATDD tests that provide automated code-based traceability from requirements to functionality. But this is difficult to do in practice because the tests are specified and written in a programming language that is usually incomprehensible to non-technical stakeholders.

This opaqueness acts as a blocker to communication and makes the required collaboration with the PO and other non-technical stakeholders difficult for the developer and frustrating for the stakeholders. Can the PO agree that an automated test effectively addresses the required functionality, if the automated test is implemented using code that is opaque to the non-technical PO? How can the PO understand what is being tested if there is no shared and mutually comprehensible language for specifying the coded tests? Or to put it another way - how can we specify code and test behaviour in a way that is comprehensible enough to achieve customer sign-off, yet precise enough to be directly mapped to code? These questions led to a new approach, known as Behaviour Driven Development (BDD).

Behaviour Driven Development (BDD)

BDD was an effort pioneered by Dan North (http://dannorth.net/introducing-bdd/)  to clarify questions like

•What should I test?
•How do I derive tests from Acceptance Criteria?
•How can I make automated tests that are easy for my customer to understand?, and
•How will the customer confirm traceability from requirements to tests?
It provides a philosophical framework for approaching everything from eliciting requirements to defining tests.

BDD provides a shared and mutually comprehensible technique for specifying the tests, by linking the tests back to the Acceptance Criteria, and specifying that the Acceptance Criteria should be defined using scenarios in a "Given-When-Then" format (see the section on How To Write Conditions of Satisfaction and Acceptance Criteria). The Given-When-Then format supports the definition of scenarios in terms of:

•preconditions/context (Given),
•input (When)
•expected output (Then)
This style of scenario construction lends itself well to defining tests. The tests are easy to specify and after they are coded the test behaviour is easy to demonstrate to a non-technical PO. Since the test implements a scenario, it can be instantly grasped by the PO.

BDD links Acceptance Criteria to tests in a way that is both customer-focussed and customer-friendly.  BDD is usually done using very natural language, and often uses further tools to make it easy for non-technical clients to understand the tests - or even write them. "Slim" is an example of such a tool (or FitSharp in the .NET world). BDD is thus intended to allow much easier collaboration with non-technical stakeholders than TDD.

In line with the benefit-driven approach, BDD frequently changes the order in which the usual formula for story writing is framed. Instead of the usual formula:

As A…. <role>

I Want… <goal/desire>

So That… <receive benefit>

The new BDD order puts the benefit first:

In Order To… <receive benefit>

As A… <role>

I Want… <goal/desire>

But all this doesn't mean that BDD should replace TDD or ATDD. BDD can be used as a philosophy behind both TDD and ATDD, helping the coder decide what to test and which tests should live where.

Even more importantly the BDD "Given, When, Then" formula allows the conversation with the customer (PO or BA) to continue into the domain of acceptance tests - further validating the coder's understanding of the product while simultaneously further validating the customer's confidence that the code does what is expected.

Experiment Driven Development

Experiment Driven Development (EDD) is a relatively new concept in Agile.

The theory is that implementing TDD, ATDD or BDD only works if the assumptions about business benefits (usually made by the PO) are correct! EDD suggests that we test those assumptions with lighweight experiments before we commit a major coding effort to the concept.

EDD suggests that if the Product Owner fails to grasp the big picture, or makes a wrong assumption then TDD and BDD will simply drive optimizations in one area that negatively impact the business as a whole (in line with the predictions of Systems Thinking and Complexity Theory that unexpected outcomes are the norm in Complex Adaptive Systems). The EDD recommendations are in line with the concept of Agile Experiments.

To support the experimental nature of this type of development, EDD introduces a User Story formula for experimental stories:

We believe <this capability>

Will result in <this outcome>

We will know we have succeeded when <we see this measurable sign>

To implement this story formula, the first step is to develop a hypothesis concerning features that might provide business benefits, then construct a story around the hypothesis:

We believe <this capability>

• What functionality could you develop to test the hypothesis? Define a test for the product or service that you are intending to build.  

Will result in <this outcome>

• What is the expected outcome of your experiment? What specific business benefit do you expect to achieve by building this test functionality?

We will know we have succeeded when <we see this measurable sign>

• Based on the possible business benefits that the test functionality might achieve (remember that there are likely to be several), what key metrics (qualitative or quantitative) will you measure?

Example
An example of an EDD User Story might be:

We Believe That increasing the size, prominence and graphic nature of images of fire damage and casualties on our page

Will Result In earlier civilian evacuation prior to a fire

We Will Know We Have Succeeded When we see a 5% decrease in unprepared civilians staying in their homes.

Implementing EDD

In line with common Agile Experiment practice, EDD experiments should be lightweight, fast to implement (to gain feedback while it is still relevant) and targeted at areas of high value (either because there is a belief that the area will yield high business benefits, or because the area is believed to hold high-impact risks).

Conclusion and Further References

TDD and BDD focus on how to build the current Product, while EDD focusses on where to take the product next.

Experiment Driven Development

EDD Presentation

Agile Traceability - Moving from Stories to Tests in a Way That Keeps Stakeholders Engaged

  

Clarifying the requirements of a User Story is a problem of communication. The Development Team requires unambiguous requirements, defined in a form that translates easily into testable code. So developers would prefer requirements that are defined in a technical language such as UML. However the ultimate source of requirements is the Product Owner, who is rarely qualified to deliver requirements in a technically precise language.

The approach to communication outlined below provides one way to iteratively evolve requirements into technically precise and unambiguous specifications.

The Difference Between "Conditions of Satisfaction" and "Acceptance Criteria"

The Conditions of Satisfation (CoS) define the boundaries of a User Story and are usually written on the back of the story. These conditions are rarely detailed enough to fully define what needs to be coded (nor should they be) but will be used to start the conversation.

The outcome of the conversation should be more detailed Acceptance Criteria. The Acceptance Criteria are used to provide a testable way to confirm that a story is completed and working as intended. The CoS defines what the PO expects, while the Acceptance Criteria define how the CoS will be demonstrated.

• CoS capture the expectations of the PO - and more importantly the intent behind the story. They exist at various levels (Epic, story and frequently even Iteration).
• At the story level, the CoS are unlikely to completely define how the story should be coded. As always in the Agile process, this will be done at the last responsible moment (see "The Right Conversation at The Right Time" below).
• Acceptance Criteria define how the PO will confirm that a story has been implemented in line with expectations. Thus the acceptance criteria are a formal and less ambiguous definition of the CoS.

Incorporating Acceptance Criteria into your user stories and tasks has several benefits:

• they get the team to think through how a feature or piece of functionality will work from the user’s perspective
• they remove ambiguity from requirements
• they form the basis of tests that will confirm that a feature or piece of functionality is working and complete

How To Write Conditions of Satisfaction

A Condition of Satisfaction:

• Helps clarify objectives and intent. 
• Applies to a specific product backlog item
• Defines a condition that must be true for that product backlog item to be considered done.

For example, a user story such as, "As a user, I want to login so that I can use the site functions," might include these conditions of satisfaction:

• User starts logged out
• User is logged in only when proper credentials are provided
• User has access based on user role
• User is locked out after three failed attempts

These conditions are clearly not sufficient to code from, they are intended as a starting point for the conversation. It is rare to find that exhaustive acceptance criteria and tests were defined as part of story creation. The iteration down from Conditions of Satisfaction to Acceptance Criteria and tests usually occurs at or close to the last responsible moment. 

 

Why do we leave this final clarification until so late? Because in Agile we try to avoid wasted effort:

Sometimes we learned things from the last story that will help define Acceptance Criteria for the next story 

• Sometimes the PO decides not to do the story.
• Sometimes the PO decides to change the story
• Sometimes we discover in the course of doing a previous story that not all of the product owner’s conditions of satisfaction can be met, and we will need to negotiate or compromise - thus changing the Acceptance Criteria.

Some of the success of the Agile process comes down to minimising wasted effort by delaying effort until it is definitely needed. This waste minimisation is sometimes termed "Maximising the Work Not Done". 

 

The Right Conversation At The Right Time

Exploration of the product owner’s conditions of satisfaction is a highly iterative process. 

As an example: During Project Initiation (perhaps at a Discovery Workshop), the team and product owner will collaboratively explore the product owner’s Conditions of Satisfaction. These Conditions will go beyond scope to include schedule, budget, and quality. The team and product owner look for ways to meet all of the conditions of satisfaction. If no feasible approach can be identified, then the conditions of satisfaction must be renegotiated. For example, if schedule is the issue then the product owner may prefer a release in one month that includes one set of user stories rather than a release five months later that includes additional user stories.

Subsequent conversations may occur at various points. This iterative process allows you to have the right conversations at the right time. 

The best times for the team to be thinking about the detailed implementation of stories and tasks is just prior to implementing these stories, as this ensures that the understanding is still fresh in mind during coding. Consequently Acceptance Criteria are frequently written in conjunction with the PO during Backlog Refinement. Acceptance Tests may be written at the same time as Acceptance Criteria, or delayed until the moment that the task is actually picked up from the board, at which point the developer seeks to work in conjunction with a tester and PO or BA to define tests based on the Acceptance Criteria.  

How To Write Acceptance Criteria

Acceptance Criteria provide a communication tool allowing the PO, Tester and Coder to agree on an unambiguous definition of a task or story. This is a communication tool for crossing the boundary between technical and non-technical, so Acceptance Criteria must be written in simple language, but should be easily translated into automated tests.

Defining a simple scenario for each task is an easily understood way to achieve this goal, as scenarios are usually specific enough to provide the user action (input) and expected outcome (output) in a testable way while being clear enough for non-technical POs to understand.

The Given-When-Then formula is an approach that works well at both generating scenarios and providing a good basis for formulating the tests.

Using the Given-When-Then Formula to Write Acceptance Criteria

Although primarily used as a tool for helping to define Acceptance Criteria tests, the Given-When-Then formula is a template that can be used at every level from Epic down to task:

• (Given) some context
• (When) some action is carried out
• (Then) a particular set of observable consequences should obtain

 

An example (From Dan North: http://dannorth.net/introducing-bdd/ ):

• Given the account is in credit
• And the card is valid
• And the dispenser contains cash
• When the customer requests cash
• Then ensure the account is debited
• And ensure cash is dispensed
• And ensure the card is returned

Notice the use of “and” to connect multiple givens or multiple outcomes in a natural way.

BDD is intended to guide the writing of acceptance tests for a User Story or task. Tools such as JBehave, RSpec or Cucumber encourage use of this template.

 

If your Given-When-Then scenario is fine-grained enough to allow a definition of input and output, then you are ready to write your test.

How It Works

The system works like this:

1.        The initial Conditions of Satisfaction should have been written when the Story was created. The Conditions of Satisfaction are then refined and will probably be elaborated down to Acceptance Criteria during Backlog Refinement, in order to reduce ambiguity and thus enable the estimation of the story. Tests may be defined at this time.

2.        During the Stand-Up a developer claims one or more tasks (as always). At that point, if acceptance criteria have not been defined down to the test level then the coder agrees to meet with tester and (ideally) the PO or BA right after the Stand-Up to define the acceptance tests. The tester works with the coder and BA or Product Owner to write an appropriate scenario for each task (or more than one scenario, if needed, but keep the number to the minimum possible). This scenario should be simple enough to convert into the Given-When-Then form. When the scenario(s) have been written and converted into given-when-then test(s), the PO should confirm that this fully illustrates the task and this confirmation should be captured. (Note: Automated tests provide protection against unexpected code interactions or behaviors, however tests are also an overhead. As a guideline: At this stage, the coder should only code the tests required to demonstrate the agreed scenarios. This provides enough tests to provide complete requirements traceability. This traceability is important because if the application passes the automated tests, all requirements are demonstrated to be present and functioning as expected - an automated "Traceability Matrix".)

3.        The coder can choose to implement tests first (if the coder is using TDD or BDD) or write the code first and confirm function by implementing the agreed tests. As soon as the coder completes the task and tests the coder calls in the tester and any other interested party, such as Scrum Master and/or PO for a “Desk Check”, in which the code is shown with functionality running and passing the Acceptance Tests. The coder receives immediate feedback concerning whether the code is performing as expected from the tester, and other stakeholders present.

4.        The tester may then run further tests on the code (Edge conditions, Usability, Accessibility, etc) and then “sign off”.

5.        Final PO sign-off occurs, as always, at the end of the sprint after the review or showcase.

Rant: Do I Have To?

Do I Have To?
“Do I have to do this?” or “Do we have to do this?”

I go into rant mode whenever I hear this question. Normally it will only come from a new team member.

I have a standard answer:

“Errrrr… no. How can I make you? You are a head-kickingly good developer in a high-performance team. So what am I going to do? Fire you? I would need to be crazy. So maybe I should yell at you? Really loud? You know that I am crap at yelling at people…. So if I can’t fire you and I can’t yell at you…..  When you think about it, I have no power to compel you to do anything really, do I?"

I wait for that to sink in. Normally they don't answer, so I continue the rant.

"You can say ‘no’ to me any time you like. If you think this is the wrong thing to do, or not the best use of your time, you should definitely say ‘no’. If you drop it, someone else in your team will pick it up. So if you think this is not the right thing to do, don’t do it .”

To which the reply is usually “Errrr, well it’s not that this is the wrong thing to do…. It’s just errr”. And that is the end of it. I never hear the question again. I hear “yes” or “no” but never “Do I have to?”

I call this the “Fuck You David” lesson. I have no authority except whatever authority the team invests in me. The sooner they get their head around that the better.

It is a terrifying realization for the new team member, because suddenly he/she realises that he/she is in charge. The buck stops with them, not with me.

Team members can tell me “no” any time they like. They just mostly chose not to…. In fact, after a rant like that they generally choose not to even ask me if they “have to” :-)

Rant: Want a Happy Team? Put them in Control.

Are you happy?
A few months ago I was a fill-in Scrum Master for an established team. I relished the chance to work direct with a single team - the opportunities are getting fewer with each year that passes.  

The team was new to me, but they were experienced. The Stand-up was smooth, finishing in less than 11 minutes. So I did what I often do at the end of a meeting when there are a few minutes to spare.

“Are you happy?” I asked.

Silence.

Then someone said “That’s a bit of a loaded question isn’t it?”

Sadly, this response did not surprise me. Teams aren’t used to anybody asking about their happiness.

This is strange. Happy teams are generally more successful and more productive than unhappy teams – so even if they have not an altruistic or empathic bone in their body, leaders should care about their team's happiness for the productivity advantages alone. So why do they never ask?

“Actually, he means it”, said the only team member who had ever worked with me before.

“I do,” I said. “Seriously, are you guys happy doing this, or does this project suck?”

Stunned silence.

With 3 minutes left in my time-box for this meeting, I concluded that this approach wasn’t going anywhere, so I decided to finish with a monologue that might help them understand.

“OK,” I said, “We all know that projects have ups and downs, but the fact is that you guys are going to do a better job if I can rig things so that you are happy. If you wake up one morning and think ‘I don’t want to go to work.’ I’d like to know, and I’d like to know why." Then I took a stab in the dark - naming the most common reason that Development teams feel unhappy... "If you start to feel like you are just a small cog in a machine that is grinding you into the ground, and you feel like you have no control over the machine… we need to change that. Think of it as being like inversion of control – this team needs to control the machine. My job is to make sure you do control the machine. We need to control everything in our environment that impacts on your ability to do your job. We can’t afford to have you guys feel frustrated for even 1 second. I know this won’t be easy or fast but you tell me what needs to happen to control the machine, we will prioritise your list, plan an approach, and then I’ll do my level best to get it happening for you.”

Disbelieving silence. Then a small voice pointed out “But we are only small cogs in the machine. We don’t control anything, we're just the dev team.”

So the next day we made a list, and I started work. I felt like, having talked the talk, I'd better walk the walk. It felt like jumping off a cliff and hoping that I could figure out how to build a parachute on the way down.

The plan to take control wasn’t simple - but it was quick to implement. We had to change the concept of “team” to incorporate those directly upstream and downstream into our new “extended virtual team”. The plan involved inviting them to our Scrum of Scrums and aligning their vision statement with ours (pretty easy, since they didn’t have a vision statement). It turned out that the managers in charge of the other teams were strongly supportive of better communication and integration and happy to have their team members "get updates" from my team.

Our "updates" quickly turned into discussions and agreements about small workflow changes and process improvements, most of which we just slipped into place immediately. We felt no need to seek approval, so we didn't.  

Did it work? Are they a happier team for this? Three weeks later I left the team and went off to find my next problem. Their new Scrum Master tells me that they raised my loss as an impediment. He also tells me that they are holding him responsible for helping them control their environment – something that he is quite happy about :-)

So don’t be afraid to ask your team “Are you happy?” They might need some coaching on what such a "loaded question" means, but once they understand you are serious be prepared for things to change.....and be prepared to jump off that cliff.

Agile Experiments - Mitigating Risk, Maximising Benefits, and Guiding The Way

Overview

Complexity Theory tells us that the systems that we deal with (corporations, business units, development teams, and the interactions between them) are Complex Adaptive Systems.

In Complex Adaptive Systems, using experiments to guide decisions will frequently outperform using expert opinion.


A CAS is a system in which the components interact dynamically, frequently producing unexpected outcomes. (How often have you checked in and run new code, or sent off an email, or just asked a question, and then been surprised by the explosion that followed?)

These unexpected outcomes make predictions (and thus plans) unreliable, so Complexity Theory suggests that when dealing with a CAS and a substantial number of unknown factors, using experiments to guide decisions will frequently outperform using expert opinion.

Unexpected interactions are not always negative. In addition to reducing our risk profile, experiments allow us to identify unexpected opportunities.

The kind of uncertainties that we deal with exist at many levels, ranging from code behavior, through requirement elicitation, to organisational uncertainty:

• How will this new code module interact with the old code?
• Is there really a business case underlying this unclear user story?
• How will overseas users respond to this UX change?
• How will the governance process owner respond to this Agile planning approach?
• How will the Operations team respond when they are told that they need to support these new products with Automated Regression Testing frameworks built into the code?

Examples of experiments that might guide decision making include:

• Rapid Prototyping 
• Wireframes
• Walkthroughs
• Reconnaissance by coding 
• Buy the head of the Project Management Office (PMO) a coffee and run him through a new process over Afternoon Tea
• Implement a new process incrementally in one area of the organisation

Why Experiments?

About Complex Systems:

• A complex system has no repeating relationships between cause and effect. For example: Offering the Apple Mac in citrus colours was a big hit for Apple in the late 1990s - it is frequently credited with saving the company - but if they did it again today, would it work again? In complex systems, reliable repeatability simply cannot be expected - the same answer rarely works twice. To quote a lecturer in economics who was asked why his exam papers were identical every year: "Sure, the questions are always the same, but the correct answers change every year!"
• A complex system is highly sensitive to small interventions - the famous "Butterfly Effect". For example it is common for a tiny change to a piece of code or a configuration (such as a slight change to an IP number) to completely bring down a large software system.
• The success of an action in a Complex System cannot be assumed even if it achieves the intended goal, because unexpected side-effects are common and the down-side of the side-effects may outweigh the upside achieved by the intended goal. (Example: Changing the IP to move a Web server into the same domain as the database server may speed up access to the database - the intended goal - but moving the Web Server behind a firewall may render it unreachable - an unintended but lethal side-effect).

Hence, when dealing with complex systems there are benefits in experimentation. "Safe-fail Probes" are small-scale experiments that approach issues from different angles, in small and safe-to-fail ways. The intent of these probes is to approach issues in small, contained ways to allow emergent possibilities to become more visible. The emphasis is not on avoiding failure, but on monitoring the outcome and allowing ideas that are not useful to fail in small, contained ways, while the ideas that show desirable outcomes can be adopted and used more widely.  
See more on Safe-fail probes at: http://cognitive-edge.com/library/methods/safe-to-fail-probes/

The take-away lesson is this:

• The outcome of acting on a complex system is rarely predictable. There is no repeating relationships between cause and effect and the emergent behavior of a complex system is frequently unrelated to the actions of the component parts.
• A complex system is input-sensitive - small changes may produce large payoffs.
• Consequently complex systems sometimes offer big payoffs for small experiments.
• The precise outcome of any given experiment is hard to predict, so managing this process requires that the outcome of each experiment is monitored. Successful experiments are expanded, while unsuccessful experiments are terminated.

Specific examples of safe-to-fail experiments might be:

• At the developer level - adopting a new design pattern in a contained area of the project
• On a larger scale - introducing a new methodology in a small Business Unit

Monitor the outcome and consider what effects the intervention had (both outcome and side-effects). If the undesirable impacts outweigh the desirable effects this experiment is not a "failure", the observed outcome and side-effects will provide a guide towards evolving a path towards a more desirable outcome.

Note that the requirement for safe-to-fail has an important implication in Software Engineering: We need the capability to create test environments that are sufficiently "production like" to yield useful lessons, but sufficiently disconnected from the main codebase to ensure that side-effects of experiments have no impact. This leads to Agile practices such as "mocking" (http://en.wikipedia.org/wiki/Mock_object ).

When To Use Experiments?

As a guide, if either of the following is true, then you are looking at a candidate for experiments:

• You have identified an area that you want to exploit, but don't know how to exploit it. Keep the investment small until you have demonstrated that the upside of success is large, then focus on the identified upside. Areas such as automation and the use of coding patterns are examples of commonly identified areas where experiments might yield large benefits.
• You have a job to do, but you don't know how to do it and the environment is large and complex. If the area is large and volatile, with a number of unknowns or risks, then the focus is on a broader picture - minimising risk and maximising opportunity by iteratively refining your strategy for reaching your intended destination.  Experiments that seek to navigate the unknown should be small probes, run iteratively, with the learning from each probe guiding the direction of the next. For example, when seeking to implement an Organisational Transition, such as an Agile Transformation, run your experiments in cycles:

When navigating the unknown, (e.g. Performing an Organisational Transition), run experiments in cycles, implementing what works and discarding what does not work.

Suiting The Experiment Type To the Situation.

It is often said that Fleming's discovery of penicillin was an accident. This is not entirely true. Fortuitous perhaps, but not a complete accident.

How it happened: Fleming was experimenting with growing bacteria in culture plates. After going on holidays, Fleming came back to his lab to discover that a number of uncovered culture plates had been contaminated by a mould that blew in through a carelessly-opened window. Wherever the mould grew, the bacteria was absent. Fleming recognised the significance at once and collected and labelled the plates, and so began the study that would result in antibiotics - one of the greatest advances in health in the history of humanity. 

Accident? Fleming had been looking for agents that would kill bacteria. He was working in a lab that was equiped to detect these agents and most importantly, Fleming was working with the right mindset to recognise an agent when he saw one, even if it wasn't where and when he expected to see it. The contamination of the plates was fortuitous, but the right environment and mindset had been created and the experiments were running.

In deciding what type of experiment to run, ask yourself what upside am I trying to achieve? Define areas of focus where the cost can be small and the upside large.

Then make sure that your experimenters are equipped with not just the right equipment and information, but the right mindset. They should be looking for opportunities to achieve upside, not just attempting to achieve a particular outcome. 

Remember that you need to chose between two types of experiment:
1.        Experiments that focus on knowns - seeking to achieve an upside in a known high-value area (in Fleming's case - an agent that would kill bacteria was known to be a likely high-value discovery). 
2.        Experiments that seek to navigate the unkown - reducing risk and identifying areas that might yield high-value results. This type of experiment yields two things: Clarity of direction and opportunities for future exploration.
 
The approach to each is similar, but with some notable differences. 
If you are seeking to navigate the unknown:
1.        Gather domain experts. 
2.        Clarify what areas might be of interest. Where are your unknowns?  
3.        Once you have identified your areas, gather your testing/UX/experimentation experts. What is the best tool to use in each area? (Wireframes to clarify thinking? Rapid prototyping to prove which options might work? Reconnaissance by coding to find out which approach demonstrates acceptable performance? Combinations of some or all? Or something else entirely?)
4.        Put together a team and set them up to do the experiments.
5.        Make the timebox for each experiment clear. This is not open-ended investment.
6.        Make it clear that they are not just there to achieve a particular outcome, they are trying to learn lessons and identify risks and opportunities. 
7.        Design your first experiment.
8.        Implement the experiment.
9.        Once the experiment is done, identify the lessons. Have we achieved clarity of direction? Have risks/issues/opportunities for future exploration been identified?
10.        Retain the positive and implement it. Discard the negative and learn from it.
11.        Based on what you have learned, is there a business case for continuing to run experiments?
12.        If there is a business case, design your next experiment, and go to step 8. Repeat as required.

The Differences

• Experiments in an area known to be likely to yield upside will tend to focus on identifying rewards in the specific area.
• Experiments for navigating the unknown will tend to focus on identifying positive directions for further development or investigation and (equally importantly) directions that are unlikely to reward further investigation..

 Further References:
EDD: Experiment Driven Development  
David Clarke at LAST Conference 2014: Managing Risk Exploiting Chaos