Queuing Theory

What is Queuing Theory and Why Is It Relevant?

Our goal in an Agile Project should be to create Business Value in the shortest possible time. In undertaking this goal we are attempting to move a backlog or "queue" of tasks through a process as quickly as possible. It is here that Queuing Theory becomes relevant.

The time taken from the moment a piece of work enters our work process until it exits (Done) is known as the "Lead Time". This is a valuable metric when using Kanban. Other valuable metrics are "Cycle Time" (the time taken to actually process the work item once work starts) and "Throughput" (the rate at which work flows through the system).

Some Relevant Guidelines From Queuing Theory

To minimise the cycle time we can look for certain problems or warning signs. The following problems can increase cycle times:

•Variability in rate of arrival, or in service time (the time taken to respond to or process a task), will increase the cycle time and the queue. Reducing variability will improve flow. Variability increases cycle time.
•The arrival of large batches leads to queues and associated inefficiencies. Large batches increase cycle time.
•Very high utilization results in no spare capacity for dealing with sudden incidents or unusual batches.

Consequently, when such incidents occur, there are problems associated with increased queues, task switching and associated inefficiencies. This greatly increases cycle time. High utilization increases cycle time.
To improve cycle times and reduce queues, target an even, stable rate of arrival. Restaurants, parking stations and night clubs achieve this by applying discounts or introducing offers at certain times of the day.

Other ways to help are: create smaller batches, create stable service times, and utilize parallelism where applicable (if a task can be split into 2 atomic tasks with no interdependencies then it can be shared between 2 people, thus halving the cycle time.)

Other Guidelines include:

•Decreasing variability early in the process is better than later. The first bottlenecked station dictates the rate to the rest of the system.
•High utilization can lead to problems that are worse than just high cycle times. A system where every station is working all the time is close to collapse (also known as Goldratt's maxim). If everyone is already working all the time, the system has no time to recover when its queue grows due to upstream variability.
•Engage early. Start processing as soon as a batch starts to arrive.
•Monitor the queue. Respond if the queue grows and understand what caused it to grow - respond to the cause as well.

Applying Queuing Theory To Agile Software Development - An Example

As an example, we can apply queuing theory to a common problem in testing:
Testers in many projects do little for the first 75% of the Sprints and are then overloaded in the last part of the sprint - they work long hours (often all night or weekends) to get the work done.
Queuing theory tells us that if you can't match servers to the load (increase the number of testers on demand), then you need to create a more stable arrival rate (the work associated with testing needs to be spread out more evenly).

Based on the guidelines above we need to:
1.Decrease variability. The load should not all arrive at the end of the sprint.
2.Decrease the size of individual units of work (Decrease Batch Size).
3.Engage with a unit of work early.
4.Monitor the queue and respond.

So, in terms of Queuing theory, our solution to the problem is for testers to:
1.Decrease Variability and engage earlier with tasks. Plan and write tests earlier, as part of requirements elaboration (based on acceptance criteria), and automate as you implement. Don't wait for the developer to complete development as this leads to a large, late batch arrival. Writing tests as part of requirements elaboration smooths out the work flow and addresses points 1 and 3 above.
2.Create Small Batch Sizes (work units). Plan for tasks to be small and testable. Don't plan for large user stories, break them into smaller chunks of functionality, ideally 4 hours – use the INVEST principle.
3.Monitor Your Queue. Look for people that are sitting idle because they're waiting for something to test. Understand why they are sitting idle.

References:

Lean Tools and Queuing Theory: http://css.dzone.com/articles/lean-tools-queuing-theory

Wikipedia on Queuing Theory: http://en.wikipedia.org/wiki/Queueing_theory