I received a great question from one of my webinar attendees and this question has also received a bunch of attention from the Kanban community since this visualization first surfaced in the summer of 2012 ago at the Kanban Leadership Retreat in Austria.
How do we measure our maturity using the Kanban method?
I have used the technique that came out of Mayrhofen at #KLRAT as the basis for how I work with teams to monitor Kanban maturity. There isn’t a “standard” set of questions that the Kanban community uses in the creation of the kiveat chart. In general, it is being suggested that each team/coach comes up with a context-specific way of measuring and ranking maturity within the 6 practice groups that applies to “that team”. That visualization can then be used to monitor the anticipated growth for that team over the course of time. One thing that the Kanban community is cautious about though is “comparisons” between kiveat charts and assessments. Since each assessment is relatively subjective, comparisons should be avoided as it would be hard to compare and may potentially be misleading to the team.
That said, here is how I use the chart.
I have a set of questions I ask per category. On a scale of 1 to 5, 1 being “We do not do that”, 3 being “We do that”, 5 being “We could teach someone to do that”, I look for activities that would allow me to select one of those values for one of my questions. As an example:
Question: Identifying types of work
Observation: Does the team identify different types of work? Do they have User Stories, Bugs, Improvement Tasks, etc. described within their process? If they do not, they would be a 1. If they do, they would be a 3. I’d then look for evidence that they are capable of teaching work item type defining to another team, or that they could do so if required. If they could, they would be a 5.
I go through all of my questions for an axis of the chart and give them these rankings. I then take the average value of all of the answers and that is my data point on the kiveat chart. My visualization category currently has 10 questions in it, so if I get 35 total points/10 questions, I get a 3.5 visualization score on the kiveat chart.
My Limit WIP category currently has 5 questions, so 15 points/5 questions would give me a 3 Limit WIP ranking on the chart.
Following this pattern, I eventually end up with 6 axis on the kiveat chart, all ranked from 1 to 5 and this “coverage” can be used to describe the team’s Kanban maturity from my perspective.
David Anderson, in some of the slides I’ve seen him use and in talking to him, describes a progression of novice to experienced tactics within each category and each of his axis has different scales to represent the increasing maturity on that scale. He does arrive at the same kind of coverage visual and describes that coverage as an indicator of maturity from his perspective. My kiveat chart and David’s would not be comparable though and this is completely OK and encouraged. As I mentioned above, the current thoughts within the Kanban community are to discourage direct comparisons between these kiveat charts.
What kinds of questions would you use to measure you’re companies Kanban maturity?
If you’ve built a kanban system, or you’ve tried to put a WIP (work in progress) limit on a phase in your workflow, you’ve probably asked or been asked this question. And very often, then answer is “I don’t know. How about we try n.” where n is a guess. Usually an educated guess like:
- 2 x the number of developers
- 1.5 x the number of people on the team
- Number of people involved + 1
And these are all ok places to start if you have no data, but with a little data, we can stop guessing and set our WIP limits with some empirical information and at the same time start building a system that will satisfy one of the assumptions required for us to use Little’s Law properly. There are two things that we need to have in order to use this super easy method:
- Data about average time in state for work items
- CFD (cumulative flow diagram)
Ok, I guess we don’t need the CFD if we have the data, but it sure is nice to visualize this information. 😉 We do need to have some data about the way that work passes through our system and we need the data that would be required to create a CFD. For the purposes of this post, lets assume that we are capture the time in state for each work item. Entire time in the system is often called lead time. Time in between any two phases in the system can be cycle time but we’re interested in cycle times for a single state at a time as our objective is to determine the WIP limits for each column in our kanban system.
Let’s use a simple approach to measuring average time in state in days. On our simple kanban system above, we have a Ready State, Development state and a Done state. Each day, we count the # of items that have cross a state boundary and put those numbers on our CFD chart. After several weeks, we have enough data to start calculating a couple new metrics from our CFD.
With even just a couple weeks of data that visualizes how work moves through our system, we can now start measuring Average Arrival Rate (AAR) and Average Departure Rate (ADR) between any two states in our system. AAR and ADR are simply represented as the slope of a line. If we calculate the rise (x-axis) over the run (y-axis) values, we get the slope.
It is the relationship between the two values that is interesting to us and will allow us to more empirically set the WIP Limit values in the system. Based on our understanding of Little’s Law, we are striving for a average rate of divergence between the two of near 0.
A negative divergence suggests the WIP limit is to low and that we are under utilized.
A positive divergence rate suggests the WIP is too high and we are overburdened.
Since ADR (the rate at which we finish work) represents our current capability, the value of ADR should be considered a great candidate for the WIP limit for this state. With the the right WIP limit in place, AAR should match ADR and we will find an average divergence rate of 0. As your team’s capability changes, our divergence will go either positive or negative and will provide an indication of when our WIP limits should change and what they should change to.
And there you have it! When the rate of divergence between AAR and ADR is near zero, we know that our WIP limit is right and that we’re satisfying one of the assumptions required to make Little’s Law work for us!
Humans love pictures.
We have numerous sayings in our cultures about how powerful visualizations are.
“A picture is worth a thousand words“
“The drawing shows me at one glance what might be spread over ten pages in a book.”
“Every picture tells a story.”
One that the challenges with knowledge work is that for the most part, most of our work is invisible until it’s done. Only Neo can see bits and bytes which leaves the rest of us a little in the dark. If you’re building a building, or digging a hole in the ground, progress is obvious! For invisible work, progress is very difficult to monitor.
So we track work in some software system, but then we hide all of our knowledge in text and numbers! Our love of spreadsheets is epic in proportion. Status reports! MS Project files! All of this data!! For all intents and purposes, our work is still invisible! We often mistake this data for knowledge and understanding and the true insights are left hidden in the numbers.
On our project, we knew it would be very important to understand at a glance where we were in our workflow. We were doing something very novel for this organization and in order to mitigate the risk of heading too far in the wrong direction (which could be any direction at this point), we needed knowledge. We needed to quickly identify roadblocks to progress, which often included self-inflicted delays. We needed to engage with our sponsor in a meaningful manner.
So we started drawing pictures!
The most important “picture” that we have used has been our storyboard. This board is very important to us. It is a muster point for our daily stand-up meetings. It is a place on which to hang “adornments” that help us quickly assess our current state. It is a tool that promotes conversations. And it is always there, silently reminding us to be disciplined with efforts to visualize our work.
Our storyboard looks like pretty much like any other Agile teams storyboard in form, but this one is a unique work of art. Well, unique in that it is exactly what we need it to be at the moment. It represents our workflow. We have our WIP limits on it. And because we are fairly disciplined in maintaining the current state of all our work, it’s always up to date.
We also visualize our tasks and review these daily as well. TeamPulse has a story board as well as a task board for more granular visualizations.
We don’t have WIP limits enabled on the tasks boards, but again, the intent is to see work flowing through the system. And with an explicit policy that tasks should have a cycle time of 1 day, any time an item is discussed at 2 stand-up meetings, the team knows that the work isn’t progressing as expected and some sort of investigation and corrective action may need to take place.
Finally, we use TeamPulse dashboards to visualize our progress over the course of an iteration.
Everyday, we review the storyboards and the tasks boards during our stand-up, and then we review the dashboards to bring the entire conversation back to our progress versus our (the team’s) expectations for the iteration. These dashboards provide that high-level perspective that ties everything back together.
And all of this information is available on the web, to everyone on the project team, including the sponsor. There is no need to interpret numbers. There isn’t a report that needs to be compiled. And the information we want to present is immediately available in a very easy to consume and impactful format because a picture is worth a thousand words!
Going back to the first ALM Adoption Success Factor of Transparency, having something to be transparent with has been a key enabler of success for our team as we strive to deliver this project and continuously improve and get better as we do it.
- Success Factor 1 – Transparency
- Success Factor 2 – Visualize Your Work
- Coming soon – Success Factor 3 – Co-location
- Coming soon – Success Factor 4 – Invest In Your Tools
- Coming soon – Success Factor 5 – Retrospectives
I was recently at the Kanban Leadership Retreat in San Diego, hanging out with a bunch of great people from the community and talking all about Kanban for three days.
This is an un-conference format event so we end up talking about whatever the group is interested in talking about, and one of the best sessions that we had (amongst a group of great sessions) was a session on Little’s Law that was hosted by Dan Vacanti (@danvacanti). Dan has been talking about Little’s Law at conferences for the last few months and the group was really interested in hearing this presentation as well as the interactive questions and answer that would happen throughout. It was during the interactive part of the session that “it” finally clicked for me!
Up until now, I didn’t understand why I would need to use Little’s law. I don’t want to use this blog post to describe Little’s Law and besides, Daniel’s presentation does an excellent job of that.
I’m really dumbing this down but in short (Sorry Dan), Little’s Law provides two things of interest for us Kanban practitioners:
1. allows us to derive one of three important Kanban metrics when any two of them are available to use in the calculation
2. allows us to understand/predict the impact of changes on one of those values to the other two
On the first point, for people running their work in kanban systems, normally all of the values are available if you’re managing your work well enough. Approx. Avg. Lead-time, Avg. Work In Progress, Avg. Throughput are relatively easily tracked or counted, and they can all be seen in a Cumulative Flow Diagram. Given I can observe all three in our system; I don’t need Little’s Law to help with that.
Note* – For planning purposes, we can use this relationship to understand what numbers
we might need in the future. Eg. Calculating required WIP based on an anticipated throughput and a proven capability (lead time) to establish resourcing levels
The second point is really interesting to us but is also where we stumble upon the gotcha and the false knowledge that often occurs on teams when they are using Little’s Law. It was in the interactive session that I finally had my epiphany around this second point! It isn’t (usually) about calculating the missing number or predicting changes in the numbers. It is about ensuring that you CAN use Little’s Law at all!! That’s the tricky part and in my opinion, the really powerful thing I learned at KLRUS.
See, in order to use Little’s Law and have it work as you’d expect, there are a number of really important assumptions that have to be true before you can use Little’s Law and it is in making these assumptions true in our kanban systems that we achieve the true benefit!
The assumptions are (for the period of time that is under observation):
- All measurement units are consistent
- Conservation of Flow which assumes
- Average Arrival Rate == Average Departure Rate
- All work that enters the system flows through to completion and exits
- System is “stable”
- The average age of WIP is neither increasing or decreasing
- The total amount of WIP is roughly the same at the beginning and at the end
All Measurement Units are Consistent
This one is relatively easy. If you’re measuring in weeks for Throughput, your Lead time should be in weeks. If your WIP is a User Story, your throughput is in User Story. We can’t have WIP as User story and Throughput as Tasks. We can’t have Throughput as User Story/week and Lead Time as User Story/day.
Average Arrival Rate (AAR) == Average Departure Rate (ADR)
This one is a bit trickier, but not too bad. One of the assumptions required, as described by Dr. Little, is Conservation of Flow and this metric is one of the ways that we measure this in Kanban. And the tactic we use to manipulate this situation is our WIP limits. If we only accept new work into our system at the same rate that work leaves our system, we are stable enough for Little’s Law to work.
All Work That Enters the System Leaves the System
Work that enters our system and doesn’t exit in a predictable fashion will disrupt the Conservation of Flow. AAR and ADR can’t be equal if this happens frequently in the system. We do have to watch out for work being “created” within the system that could mask abnormal termination of work within the system.
Average Age of WIP is Constant
Little’s Law also requires a “Stable System”, which I’m not going to explore deeply in this article. One of the two metrics that we can use from our kanban system though to determine if our system is stable is the average age of WIP in the system. Over the period of time being observed, we want this average to be constant.
Total WIP at Beginning and End of Time Period are the Same
The second metric we use to determine system stability is WIP totals at the beginning and end of the time period under observation. If our WIP is trending up or down, we do not have a stable system and predictions that we get from Little’s Law will be suspect.
So what does this all mean…
So we have a law that we want to use, but in order to use it, we have to fulfill the required assumptions for the law to work. The required assumptions are going to cause us to create specific and explicit policies that will describe how our teams behave. The effects of these policies are varied, but you will see improvements in prioritization and pulling of work, queuing, swarming, workflow/system design, and WIP limits just to name a few in order to effectively strive to satisfy the assumptions.
The power of Little’s Law to Kanban teams is not its ability to predict WIP, Thoughput or Leadtime. The true power lies in its ability to influence team behavior with its underlying assumptions.
When Little’s Law is properly understood by Kanban teams, it will encourage those teams to strive to satisfy the Law’s assumptions which will result in improved predictability and a kanban system that can be “tweaked” to achieve whatever goal is required at the time.
Queues are all around us. Policies are everywhere.
Can we use the two together in a simple fashion to create business value?
Note* – Once you get really interested in Kanban, these are the kinds of observations and questions
that just randomly pop into your head, so beware. 😀
I was sitting on a plane and luckily had a window seat and I was watching the baggage handlers loading the luggage onto the plane. They were counting the bags and starting to load them onto the plane and I looked away for a bit. After a few minutes, I looked back and notice they were almost done and that is when I noticed the priority baggage tag on the last bags being loaded. This is where kanban took over my brain.
Could a simple policy that was implemented by baggage handlers and a Last In – First Out (LIFO) queuing technique manage the priority baggage solution? It seemed to me that this would be a simple tactic to provide a value-add service to customers that they might pay for.
I’m sure it would be a little bit more involved than that (bag check tagging bags, people loading the conveyer belts that take luggage the last mile would need to use a FIFO approach) but basically with a simple policy change (it would have to be an explicit policy) and an understanding of how queues work, the airlines have created business value out of thin air!
Are there any simple policies or queues that you could use to create business value for your organization?
I’m a consultant. I’ve been very fortunate to fall into this line of work and I really love what I do.
One of the hard parts of being a consultant though is “staying ahead of the curve”. A consultant is only as good as the skills he possesses and our industry is constantly (and successfully) striving to make our current knowledge and skillset obsolete.
So I do the things that most consultants do. I practice what I know and try new things. I read books. I surf the web. I have my favorite blogs. I use twitter lists to follow people who are aligned with particular interests of mine. And this has been a good tactic for keeping up.
At some point though, I started going to conferences and this is when my growth as a practitioner in my profession started to go to the next level.
I remember the first time I saw a industry luminary, Don Box. At the time, he was really ramping up SOAP, XML, Messaging, all that goodness of the early 2000s. The content of his talk was all available on the web or in books but that wasn’t what I really got from his presentation. In hindsight I’ve realized the true value I took from that experience was his passion for the topics and his stories! I came back from that conference pumped about those technologies and finding exciting new ways to use them! And boy is XML, SOAP and messaging exciting!! <smirk>
Fast forward a bit and now I’m a consultant who is trying to be better at helping individuals, teams and organizations be better and I’m doing this at an amazing time in our industry. Agile is going full steam ahead. Lean and Kanban are entering the process space and starting to make significant inroads. There is a lot to learn and discover, validate or refute. And this is where the thought-leaders come in.
This might sounds obvious, but thought-leaders are really interested in the things they are leading in! They are deeply interested in why “it” works or how they can apply “it” to a new problem. They want to break or validate their thoughts or have them refuted because they are passionate about making “it” better. It’s been my observation that they aren’t normally as interested in applying their specific knowledge to someone else’s problem like most of the rest of us.
It is this unique environment that thought-leaders create around themselves that presents an amazing learning opportunity to the rest of us.
Let me use an example that is fresh in my mind. I’m really interested in The Kanban Method. I think it has the potential to change the way that we all work. But there is a lot to learn and I also want to make sure I’m challenging the method and myself continuously. Enter David Anderson, Kanban thought-leader.
Now David is a really smart guy and he is passionate about his art. He’s a “center of gravity” in the Kanban universe right now. And the really cool thing about that is the attraction of all things Kanban, good and bad, to that center of gravity. If you put yourself in proximity to that center of gravity, virtually or physically, you will be exposed to so many ideas, good and bad, about Kanban. You’re also going to be exposed to the passion to evangelize and grow that David brings to the community.
When I became interested in Kanban I:
- Attended a leadership course with David
- Attended LSSC12 in Boston and Kanban Leadership Retreat in San Diego
- Started following a lot of people in the Kanban community on twitter, finding them because many of them followed David
- Started following blogs of people in the Kanban community, many found from being in the mix with that community
- Got involved in leading-edge questions and challenges with the method, many of which are being discussed by David
I had the same experience with Scrum. My perspective on Scrum was completely opened up by spending a couple days in a course with Ken Schwaber. His perspective on Scrum was so different than anyone else’s, he is very passionate about Scrum and it was such a great learning experience. I also started following @Scrumdotorg on twitter and recognized the value in finding an anchor within that community!
I had the same experience technically with CQRS, an application architectural design pattern. When I became interested in that, I started following thought-leaders in that space.
All of the “technologies” that we use have these centers of gravity that attract all of the good and bad. And there is so much learning to be gained from being in these environments!
So it’s my challenge to you today to go and find a center of gravity in a technology you are really interested in. Find a thought-leader on twitter or find their blog. Find a conference that they will be speaking at and attend. Ask them a question, or ask the community around them a question! Get involved with the communities that are swirling around these thought-leaders and take full advantage of the learning opportunities that will present themselves!
This is a story about my job, which I love.
During the day, I’m a mild-mannered process consultant (read: Lean/Agile evangelist) for Imaginet, a technology consulting firm that provides end-to-end software development services primarily specializing in Microsoft products and technologies. The really cool thing about Imaginet is that we are large enough to be able to help companies with projects that span the full lifecycle of solutions development, or Application Lifecycle Management as it is more well known. And these projects often involve helping organizations adopt, refine or improve their existing ALM processes.
That is the part that I get excited about. As a process consultant and change agent, I get to help companies take their development processes to the next level and build an organizational culture that is capable of continuing that growth.
It was in this context that our current client engaged us. They were going to build a team that was expected to build a mobile transportation solution that would help them manage a fleet of trucks. This client had never had a core competency in software development but realized that in order to realize their vision, they would need to become competent as there was no off-the-shelf solution that would fit their needs. They did have experience with traditionally managed knowledge work projects and really felt that this was not going to lead to a successful project. There were far too many unknowns and risks and they knew they would need to learn and adapt very quickly throughout the project.
So we all decided to adopt an Agile/Lean mindset and build a process that would help us deliver in this challenging situation.
This long preamble is really just setting the stage for me to share with you what I think were the 5 core principles that we used on this project to ensure that the mindset and methodology development worked and became institutionalized. I’m hoping that by showing you how they affected the way that we work and the benefits, you’ll be able to learn from us and see if they could work for you. This 5-part series won’t be as much about the methodology we ended up with as much as it is about how we ensured that the right methodology for the organization and project emerged and that we maximized our chances of success.
Success Factor 1 – Transparency
The first core principle that we followed was to provide as much transparency into everything that we did, the information that we had and the challenges that we faced. We were so focused on this transparency that it actually tended to border on continuous broadcasting.
In an environment where there is a lot of uncertainty and risk, we felt that it was crucial to our success that we get as much information as possible from as many sources as possible and to engage anyone who could help us succeed. And with a project sponsor who was taking a significant risk in building a new competency for his organization, it was important that he see how we were working, the challenges we faced, and that we were always looking to improve the way that we worked.
To that end, we used numerous tactics to provide as much information on how we were working at a glance to anyone who was interested. These tactics included:
- a big team room which encouraged information sharing and broadcasting
- digital storyboard (work management system) that was projected onto a screen at all times and accessible anytime from a browser
- daily stand-up meetings where anyone was invited to attend
- iteration planning and retrospective meetings every 2 weeks, anyone could attend and the outcomes of these meetings were left in plain sight for anyone to see
- team Skype account that would auto-answer (silently I might add) and project onto the screen, with a live video feed and multiple microphones in the room
- Skype accounts and webcams for all team members
- tons of whiteboards and large post-it notes that we write on continuously – every wall in the room (600+ sq. ft room) was covered in information and updated frequently
- large post-it notes on the wall outside the team room to notify anyone walking past in the hall
- email notification of significant events in the development process (e.g. broken builds, server outages)
And it is working!
Our sponsor, who sits across the hall, frequently comes into the room and sees exactly what the team is doing. Our openness has fostered a culture of trust between him and the team that allows both to ask questions, provide inputs, and make decisions for the good of the project together. And one of the really cool aspects of his interactions with us is that he very rarely comes in and asks us how we are doing or where the project is at. He already knows! He usually comes in asking about a specific item we are working on, or asking if he can help with one of our problems.
We never have to defend our methodology because we are constantly sharing it with everyone and allowing them to participate in its improvement. This is one of the hard parts of an Agile adoption project that is easily avoided by aggressively being transparent.
Another benefit of this level of openness is that when we have brought on a new team member, which has happened several times, they are immediately immersed in a LOT of project information. This ranges from work in progress, upcoming stories, improvement work items from the retrospective and any risks or challenges that we are actively facing. We also have numerous big PostIt notes that contain the values and principles that we have chosen to follow as we evolve our methodology. How we perceive value, prioritize and the core vision for the product is all up on the walls all the time.
We have received help from a variety of people within the organization who saw something that we had on our boards that they could help with. These helpful acts ranged from sharing critical bits of information that helped us make a much better decision to actually removing a roadblock or impediment from our current blockages. And without this level of transparency, there is a good chance that these helpful acts would have been delayed if they had even arrived at all.
Transparency has been, and will continue to be, a significant cultural aspect of our team that we feel helps drive our success.
Do you have any transparency tactics you’d like to share? Please do! Comments and thoughts are always encouraged!
p.s. I plan to enhance this post with pictures soon, but I’m under a deadline to get this out (thanks Dylan) so they’ll have to wait! Please check back in the next couple days.
p.p.s. This blog is being inspired by a competition between a bunch of us to see who fails the blogging frequency requirement first! Please check out the blogs of all these other awesome guys and encourage them to keep on blogging too!
- Success Factor 1 – Transparency
- Success Factor 2 – Visualize Your Work
- Coming soon – Success Factor 3 – Co-location
- Coming soon – Success Factor 4 – Invest In Your Tools
- Coming soon – Success Factor 5 – Retrospectives