Storing Secrets In Azure Guest Post on Canadian Developer Connection Blog

I was recently asked to contribute to the Microsoft Canadian Developer Connection blog and the post has gone live! This post details how to Store Your Secrets in an Azure Websites App Settings! Go check it out!

http://blogs.msdn.com/b/cdndevs/archive/2015/03/20/storing-your-secrets-in-azure-websites-app-settings.aspx

Calculating Lead Time for Work Items on TFS 201x

This is part 2 of a series of posts that show how we can model virtual kanban systems on TFS 201x.

If you didn’t read part 1, I would recommend that you start there.

Big gotcha with regard to this blog post

You cannot access the TFS data warehouse (which we will need to do)  using Visual Studio Online and as far as I know, there is no way to access the data stored there in any way yet. As such, this post will make the assumption that you are using TFS 201x on-premise and can access the TFS Data Warehouse.

Assumptions

Same assumptions from part 1 apply and I’ll re-iterate them here along with a few other assumptions.

  1. You are a TFS user and will understand the features that I’m using.
  2. You have a basic understanding of TFS Process Templates
  3. You have a basic understanding of the Iteration Path hierarchical work item field that is present in the various Process Templates
  4. A basic understanding of virtual kanban systems as they pertain to software development
  5. NEW – You have a basic understand of how TFS Work Item States work
  6. NEW – You have a working knowledge of Excel 2010+
    1. We’ll be connecting to databases and programming in VBA

Quick Review

Just as a quick review, here is the current state of our virtual kanban system implementation.

image

Remember that we are using iteration paths to represent the backlog (uncommitted work) and the kanban system and the only work item state we will be (really) interested in is “Done”.

Calculating Basic Lead Time Metrics

Using the movement of a work item into the kanban system iteration path, until it gets to a “Done” state, we can now calculate the lead time for that item. The easiest way I’ve seen to do these analysis, after your kanban system has been setup and running for a while, is to use Excel to consume and analyze TFS data warehouse data about work item changes.

Developer Ribbon in Excel

The first thing we need to do is turn on the Developer Ribbon in Excel.  After you’ve started Excel (I’m using 2013), right click on the ribbon and select the Customize the Ribbon option.

image

Turn on the Developer  tab by checking the box and pressing OK.

image

Prepping the Spreadsheet

For this exercise, I like to create three sheets in my Excel workbook.

  1. Parameters
  2. LeadTimeReport
  3. Data

image

We will refer to the names of these sheets in our Excel VBA code.

Getting Data From TFS

Excel has the functionality to connect to easily connect to SQL Server databases and that is exactly where TFS stores all of it’s data. TFS stores the data that we are interested in in a database that is called Tfs_Warehouse.

Navigate to the Data tab on your spreadsheet and put the caret on cell A1.

Now create a connection to Tfs_Warehouse by visiting the Data tab, clicking the From Other Sources button and selecting From SQL Server.

image

 

Enter the name of the database server that is the backend for your TFS installation and use the appropriate credentials (SSO or SQL Server account) in the connection dialog and click Next.

image

Note – You will need to able to access the server with either your domain account permissions or a SQL Server account that has permissions to access Tfs_Warehouse database.

 

Select the Tfs_Warehouse database on the server and the DimWorkItem table and click Next.

image

 

Change the name of the connection to something more friendly like LeadTimeConnection.odc and set the friendly name to something more descriptive and click Finish.

image

 

An Import Dialog will pop open once you have hit Finish. Select the Properties button on this dialog.

image

 

On the Properties dialog, click the Definition tab and then change the Command Type to SQL  and enter the SQL statement below in the Command Text textbox and press OK.

image

 

SELECT system_id, system_title, system_state, di.iterationpath, system_changeddate, System_WorkItemType
FROM dimworkitem dw
INNER JOIN dimiteration di
ON dw.iterationsk = di.iterationsk

 

Now press OK on the Import dialog and you should now see the Data sheet fill up with data from your TFS database.

image

VBA Magic to the Rescue

Now that we have our work item data coming, we need to do a couple things:

  1. Narrow it down to the time period under review
  2. Determine what started and finished within that time period
  3. Calculate the Lead Time for the items that completed during this period

Caveat – I’m not a VBA Script writer by trade! I’m fairly certain there could be better ways to write this code, but since writing elegant VBA code is not my objective, I’ll just get you to the lead times. If you have any suggestions on how to better code this out, I’d love to hear from you.

Create a VBA Module

On the Developer tab, click the Visual Basic button to get to the VBA IDE.

image

Once you are in the Visual Basic editor, create a new module for holding your code by right-clicking on the Module node of the tree and inserting a new Module.

image

Now we can just start entering all of the code from below.

Declare String Constants in the Module
Public paramSheet As String
Public reportSheet As String
Public dataSheet As String
Public connectionName As String
Public backlogIterationPath As String
Public kanbanSystemIterationPath As String
Public startState As String
Public endState As String

 

These strings (global) allow us to do set our sheet names and change the iteration paths that we use to model our kanban system.

Determine Vertical Range of Data
Private Function VerticalRange()

Sheets(dataSheet).Activate
Range("B1").Select

Dim height As Integer
height = 0

Do While ActiveCell.Value <> ""
    height = height + 1
    ActiveCell.Offset(1, 0).Activate
Loop
VerticalRange = height

End Function

This code needs to discover the length of the data set that we’re processing. This function does that.

Create Lead Time Generation Function
Private Sub GetLeadTime(ByVal rows)

Dim sourceRowCount As Integer
Dim targetRowCount As Integer
targetRowCount = 2

Dim sws As Worksheet
Dim tws As Worksheet
Set sws = ThisWorkbook.Sheets(dataSheet)
Set tws = ThisWorkbook.Sheets(reportSheet)

Dim workitemid, firstRow, lastRow, nextFirstRow As Integer
firstRow = 2

For sourceRowCount = 3 To rows
     workitemid = sws.Range("A" + CStr(firstRow)).Value
     Dim wiD As String
     wiD = sws.Range("A" + CStr(sourceRowCount)).Value
     
     If wiD = workitemid Then
        'skip if true – we are looking for the last row for the current work item
     Else
        lastRow = sourceRowCount - 1
        nextFirstRow = sourceRowCount
    
        Dim isUncommittedBacklog, isKanbanSystemBacklog, isUncommittedNew, isKanbanSystemNew, isDone As Boolean
        Dim uncommittedBacklog, kanbanSystemBacklog As String
        Dim uncommittedBacklogFound, kanbanSystemBacklogFound As Boolean
        Dim startDate, endDate As String

        For rowNum = firstRow To lastRow
            isUncommittedBacklog = InStr(1, sws.Range("D" + CStr(rowNum)).Value, backlogIterationPath, vbTextCompare) > 0
                If isUncommittedBacklog = True Then
                    isUncommittedNew = InStr(1, sws.Range("C" + CStr(rowNum)).Value, startState, vbTextCompare) > 0
                    
                    If isUncommittedNew = True Then
                       uncommittedBacklogFound = True
                    End If
                End If
            
            isKanbanSystemBacklog = InStr(1, sws.Range("D" + CStr(rowNum)).Value, kanbanSystemIterationPath, vbTextCompare) > 0
            
            If isKanbanSystemBacklog = True And uncommittedBacklogFound = True Then
                isKanbanSystemNew = InStr(1, sws.Range("C" + CStr(rowNum)).Value, startState, vbTextCompare) > 0
            
                If isKanbanSystemNew = True Then
                   kanbanSystemBacklogFound = True
                   startDate = sws.Range("E" + CStr(rowNum)).Value
                End If
            End If
                
            If kanbanSystemBacklogFound = True And uncommittedBacklogFound = True Then
                isDone = InStr(1, sws.Range("C" + CStr(rowNum)).Value, endState, vbTextCompare) > 0
                If isDone = True Then
                    endDate = sws.Range("E" + CStr(rowNum)).Value
                End If
            End If
        Next rowNum

        'Calculate Lead Time
        If IsDate(startDate) And IsDate(endDate) Then
            Dim leadTime As Integer
            leadTime = DateDiff("D", startDate, endDate)
            tws.Range("A" + CStr(targetRowCount)).Value = sws.Range("A" + CStr(lastRow)).Value
            tws.Range("B" + CStr(targetRowCount)).Value = sws.Range("B" + CStr(lastRow)).Value
            tws.Range("C" + CStr(targetRowCount)).Value = leadTime
            targetRowCount = targetRowCount + 1
            startDate = ""
            endDate = ""
        End If
        
        firstRow = nextFirstRow
        nextFirstRow = 0
    End If
Next sourceRowCount

End Sub

 

This code is pretty messy but isn’t actually very complicated. What happens in this function is that we go through each row of the source data set and look for the work item entering the kanban system iteration path. We then look for the work item getting to the done state.

If a Start Date and and End Date are discovered for the work item, we make an entry in the LeadTimeReport sheet with the calculated Lead Time. If we do not find those dates, no entry is made.

You can step through the code in the IDE using F8 to step through each line of the function.

Create Public Access and Initialization Function
Public Sub GenerateReport()

'constants in module
paramSheet = "Parameters"
reportSheet = "LeadTimeReport"
dataSheet = "Data"
connectionName = "LeadTimeConnection"
backlogIterationPath = "\Tailspin Toys\Iteration 1"
kanbanSystemIterationPath = "\Tailspin Toys\Iteration 1"
startState = "Active"
endState = "Closed"

Dim startPeriod, endPeriod As Date
startPeriod = ThisWorkbook.Sheets(paramSheet).startDatePicker.Value
endPeriod = ThisWorkbook.Sheets(paramSheet).endDatePicker.Value

Dim cmdText, periodDates As String
cmdText = ThisWorkbook.Connections(connectionName).OLEDBConnection.CommandText
periodDates = " Where system_changeddate between '" + CStr(startPeriod) + "' and '" + CStr(endPeriod) + "' and System_WorkItemType = 'User Story' Order by system_id , system_changedDate asc"

Dim location As Integer
location = InStr(1, cmdText, "where", vbTextCompare)

If location > 0 Then
    cmdText = Left(cmdText, location - 1)
End If
    
ThisWorkbook.Connections(connectionName).OLEDBConnection.CommandText = cmdText + periodDates
ThisWorkbook.Connections(connectionName).Refresh

Call GetLeadTime(VerticalRange)

End Sub

This code is the function that is going to be invoked by the button we put on our parameters sheet. It sets all of the global values for the module, acquires the start and end date of the period under observation, modifies the command text of the SQL query to use the dates in the query, 

Create UI

Now that the module has been created that does all of the data transformation work, we need a simple UI that allows us to pick a Start Date and an End Date for the period under observation and a button to kick the whole thing off.

After you have selected the Parameters worksheet, in cell  A4 type out “Start Date”. In cell A6 type out “End Date”. And finally on B3 put “Period under Observation”. These are labels for our UI.

Now on the Developer tab, click the Design Mode icon on the ribbon. Now we can Insert 2 Date Picker controls onto the worksheet. Click the Insert icon on the ribbon and then click the the very last icon on the bottom right.

image

In the More Controls dialog that comes up, select the Microsoft Date and Time Picker Control 6.0 (SP4) and then click OK. You will now be able to “draw” out the new control on the worksheet. Draw out the first control roughly on cell B4.  In the Excel Name Box, name the control startDatePicker.

image

Repeat the process for the second date time control and draw the second control roughly out on B6. In the Excel Name Box, name the control endDatePicker. These names are used in the script to find the range parameters. 

Now we need to add a button. Click the Insert icon again and select the first icon on the top left and draw out a button roughly on cell Here is what it should look like when complete.

image

Generating Lead Time Data

We should have completed all of the steps required to generate lead time data for the kanban system that we’ve model in Part 1 of this series of blog posts!

If you press the buttons and everything has been typed in correctly, you should see data populate the LeadTimeReport worksheet. This will look roughly like this.

image

You can format the first row of the LeadTimeReport sheet however you see fit. If you want to add more rows of labels, you will need to edit the scripts as they start adding lead time entries on row 2.

I’d like to point out that in this case, I’m using the Brian Keller ALM VM for Visual Studio 2013.4 and the Fabrikam Team Project inside of that VM for data. You will need to change the script to reflect the iteration paths that you are using your kanban system in TFS.

Final Thoughts

This has been a fairly long and detailed post and I hope that you’ve been able to follow it and get the lead time data generation working in your environment.

There are more sophisticated ways of generating this lead time data (programmatically more advanced applications). The reason I wanted to use Excel and Visual Basic was to allow anyone to generate lead time data with the basic tools that are available in a normal IT shop that uses TFS. If you’d like to expand on this approach and develop an application that does this (which I have already done elsewhere), I hope that you can use this example as a guide to what you’ll need to do to write your application.

Thanks and I hope to hear from you in the comments soon. Let me know if this worked for you or how I can make this example better.

Modeling your kanban system on TFS 201x using Iteration Paths

Microsoft has been making some great strides in adding features to Team Foundation Server (TFS) that allow for the modeling of the virtual kanban system that your teams might be using to manage their work. The web access kanban boards and the ability to manage WIP limits on the boards are two such features.

Kanban tooling is still relatively new to Microsoft and they are trying to adapt a product/approach that has historically been very well suited to Scrum and Traditional work management techniques, and so it isn’t much of a surprise that we need to tweak the way that we use TFS 201x to help us correctly model our virtual kanban systems. To that end, we need to take advantage of a few of the features in TFS to allow our teams to be more effective when using kanban.

This is the first in a series of blog posts detailing how we can model kanban systems on TFS.

Assumptions

I have several TFS related assumptions that I’m making as you read this post:

  1. You are a TFS user and will understand the features that I’m using.
  2. You have a basic understanding of TFS Process Templates
  3. You have a basic understanding of the Iteration Path hierarchical work item field that is present in the various Process Templates
  4. A basic understanding of virtual kanban systems as they pertain to software development

Iterations vs. Iteration-less Planning

The first challenge that we’ll encounter when using TFS is that there is no specific kanban process template, which is actually the right approach in my opinion. It would be tremendously difficult to create a one-size-fits-all kanban process template and I think we need to get into the habit of customizing our process templates if we’re going to be using kanban work management techniques. That said, we still need to be able to have an ability to plan in an iteration-less environment.

This is where we get a bit confusing with the features we use in TFS vs. our intent, but it will work for us.

TFS uses the Iteration Path structure to determine what is on the Product Backlog board in web access. The Product Backlog board is the current feature in TFS to visualize our kanban system so we would like it to represent things in a more traditional kanban way. In order to do this, we need to modify the Iteration Path structure and communicate with our team how it is intended to be used. This means that we need to communicate that we are using the TFS feature named Iteration Path for something other than an iteration as it would be commonly understood if we were using a Scrum work management approach.

Here we find a model of a simple kanban system.

image

We have an uncommitted backlog of options or ideas that we may or may not eventually work on. We have a Ready Queue which is the work we have committed to doing next, we have a Doing column that represents the actual development work and a Done column that represents that we have finished some piece of work.

Understanding Lead Time

One of the primary reasons that we need to doing all of this is so that we can calculate a lead time for our work items, which TFS 201x currently does not do for us and the way that the kanban boards are currently implemented, we cannot access the data that is captured by the kanban boards. (Product Group call out here: We want access to that data!) Lead-time is a basic and fundamental metric for measuring the performance of work through a kanban system and so we kind of need to be able to generate that information.

When you want to capture lead time data for work items, we need to identify the start and finish line for the work. Done is fairly universally recognized as the finish line, but the start line is a little harder to define sometimes. In this kanban system example, we are going to state that Uncommitted work is not a part of the lead time calculation but when we move work into the Ready Queue for the team to pull from next, the lead time period has been determined to have started.

image

One of the current challenges with TFS is that lead-time is perceived to have started when a card appears on the board and the cards appear on the board as soon as it is placed into the Product Backlog iteration.

Another challenge is that all work items (uncommitted or committed) may appear on the kanban boards if the work item is placed in the Iteration Path that has been identified as the Product Backlog in TFS.

To this end, with this simple workflow, we are going to use the Iteration Path feature and create an Uncommitted backlog and a Kanban System ‘backlog’.

Creating an Iteration Path that is our Uncommitted Backlog

One of the great aspects of TFS is the ease with which you can create or modify Iteration Paths. I’m going to be using Visual Studio Online for demonstration purposes. The Visual Studio Online web interface is the same as the on-premise Team Foundation Server web access interface, so if you have TFS on-premise, you can follow along just the same.

First, we need to go configure our Iteration Path structure. Once you have arrived at your Team Project page on Web Access, you can find the link to make those modifications on the right-hand side of the page.

image

For simplicity sake, I’ve hidden a couple iteration paths in the Released branch. For now, let’s look at the root node of our Iteration Path structure. Remember that the Iteration Path work item field is a hierarchical data type.

image

Let’s create an Uncommitted iteration path. Go a head and select the root node in your environment and create a path called “Uncommitted Backlog”.  Make sure you are on the root node of your Iteration Path.

image

image

In our situation, the dates don’t mean anything to us, but TFS 2013 presents the iterations in this screen ordered by Start Date so we’ll use the dates to “order” our Iteration Paths visually.

Note – In drop-downs on the work items and in Excel, the Iteration Path values will be ordered alphabetically.

Now let’s create our Kanban System Iteration Path. It should also be a child of the root node and the start date should be 1/2/2012 or one day after the uncommitted backlog’s start date if you chose a different date than I did.

image

You’re Iteration Path structure should now look like the image above.

There is one more Iteration Path node that we need to create in order for the Web Access Task board to work correctly as well.

On the Kanban System node, create a new child called “Committed”.

image

Awesome! We have now created an Iteration Path structure that models our virtual kanban system.

Hooking up the Web Access Boards

Now there are two more things that we need to do on the Iterations management screen and that is hook up the two boards we’ll be using in web access to the correct Iteration Path nodes.

First, you need to right-click on the Kanban System node and select the Set as team’s backlog iteration option.

image

This connects the Kanban System  node to the Backlog Kanban board.

image

Next, click the check-box on the Committed node.

image

This connects the Committed node to the Task Board in web access.

image

You might be asking yourself why we have to have the Committed node under the Kanban System node. Well, give it a try if you’d like, but currently, the team’s backlog iteration cannot be the checked iteration that will be shown as the task board.

Reviewing What We’ve Done

Now that we have our kanban system model and a Iteration Path structure that represents that system, we can show how they are intended to map together.

image

Because the current crop of TFS process templates don’t naturally support modeling virtual kanban systems, we are able to use process template agnostic features of TFS to model our kanban system.

Advantages

So why are we doing all of this you ask? There are several advantages to this particular approach that I like.

  1. Process Template Agnostic
    You can use this technique with any process template that uses the TFS Iteration Path work item field.
  2. Iteration Path based Lead Time boundaries
    Currently, most teams will try (want) to use the kanban board columns but because the data is not accessible, we can’t. Other teams will model their kanban system with Work Item States, but this tends to become a mess as Work Item Template modifications need to be done on a lot of work items and frequently. This is just a PITA, especially if you don’t have TFS administration skills in-house and rely on consultants to help with process template modifications.By using membership in an Iteration Path as the means to determine when something has entered (and/or left) a lead time boundary, we free ourselves from having to use work item states as a means of creating these lead time boundaries.
  3. Clean Up Our Queues
    One of the things we want is concise, useful queues (and queue visualizations) in any tool, including Excel, Visual Studio or the web access. Currently, web access will limit the # of work items it presents in the queues, but users of Excel or Team Explorer will get potentially long and confusing lists of things to review.  While our Uncommitted queue may be (and usually is) quite long, the daily working queues and the kanban system can be made smaller and more concise. This aids planning and decision making about what to prioritize and pull daily.
  4. Works on TFS 2010
    This technique will work on TFS 2010. We won’t get the nice web access boards, but as we’ll see in follow-up posts in this series, there is still value in using this technique to capture lead time for work items.

Disadvantages

There are a couple disadvantages. They include:

  1. Can’t determine cycle time easily
    Cycle time is the time between two arbitrary states in your kanban system. Using our example kanban system, I would say that cycle time of the Doing column might be 2 days, but the lead time of the entire system is 4 days. I don’t think that I would encourage my teams to use this Iteration Path technique to model individual columns.
  2. Non-Intuitive
    I don’t think that this technique is natural to the VS community at large. I think it works and is a novel way of modeling kanban on TFS, but there needs to be education about how the team (and upstream and downstream partners) are using TFS.

Final Thoughts

Hopefully this have given you some ideas on how to better model virtual kanban systems in TFS. Virtual kanban systems are a tremendously valuable means to manage work, and even though TFS doesn’t currently support them in a intuitive manner, we can still use many of the features in TFS to achieve our goals. You could also use these techniques in similar fashion in non-kanban environments, if you simply want to clean up or better manager your queues.

In the next part of this series, I will show how we take advantage of this technique to calculate lead-times from your work item data.

As always, I would love to hear your thoughts or answer your questions.

LeanKanban United Kingdom 2013 Video up

Hello everyone,

Just wanted to point out that the Lean Kanban  United Kingdom videos are all up! Check out my talk on Myths and Misconceptions about the Kanban Method in addition to all the other great talks from this conference.

http://www.youtube.com/watch?v=-9vLfEz4aqY&list=PLVsUnwOzPqiRZ2cnrzfKFnDEDTqL3pr5-&index=28

You can find a lot more information on the LeanKanban Conference series here.

http://conf.leankanban.com/

I hope you all have Happy Holidays and a Prosperous 2014!!

Cheers,
Dave

The Agile Manifesto is NOT a Buffet

I’ve recently stumbled upon two blog posts that got me thinking about agile adoptions and why they might fail (not in a safe way).

First there was this post that I saw because it resonated with a colleague and he shared it.

http://inline-lambda.blogspot.ca/2013/11/agile-b2b-and-beta-conundrum.html

I don’t know Guillaume and I’m sure he is a very nice guy, but the post struck me as from someone who didn’t understand the Agile Manifesto and was confusing it with the prescriptive guidance in the Scrum Guide or the guidance of The Lean Startup.

And then I saw a post from my friend Dylan Smith starting a blog series about “Why Does Agile Fail”.

As I was reading these two posts, I started asking myself Why are these authors making these statements? Why are they observing these effects? The way that I’ve understood the Agile Manifesto and its intent, you shouldn’t see these effects, at least not for very long. It struck me though that this is a story that I hear over and over again.

Then I had a light bulb moment.

We often talk about an “Agile Buffet” as a set of tactics or practices we can pull from and use as best fits within our organization. I like this. As a passionate Kanban Method practitioner, the idea that we’ll continuously discover problems opportunities for improvement and that we’d want to leverage past industrial learning is a core philosophy for me. But the Agile Manifesto is NOT a buffet. We should not pick and choose which of the values and principles that we will follow and aspire to. They are all necessary ways of thinking if we’re going to pull our industry out of the dark ages.

Now I am not suggesting we shouldn’t be finding the best way to, in context, live up to those values and principles. We should be very disciplined in What we pull from the Agile Buffet and When we pull the improvement. This is where a Kanban Method approach to continuous improvement is enormously beneficial. With a functioning kanban system, we have data that allows us to make informed, economically sound decisions about improvements and a mindset that we are going to improve incrementally and in an evolutionary manner. We are actually slaying a software development dragon with a 1-2 punch of Agile Manifesto thinking and Kanban Method thinking. We’ve got something guiding us towards what a successful software development team would behave like (Agile Manifesto) and we’ve got something showing us how to get to that goal in a humane, intelligent, and economic manner (Kanban Method).

 

Final Thoughts

I would like to encourage you to go read the Agile Manifesto again and to go read about The Kanban Method. They won’t tell you how to do things, but they are telling you what you should aspire to do. Now with that in mind, go and look at the Agile Buffet. Read the Scrum Guide. Read the kanban blue book as well as look at case studies from teams that have used the Kanban Method. This is your buffet of tactics that can help you solve your problems. Start pulling from this buffet of industrial learning. Develop a capability to understand what works for you and perhaps more importantly what doesn’t.

But please remember, these value systems are not a buffet and you don’t get to pick and choose from the values and principles they provide.

The Kanban Method Is A Vehicle – It Is Not A Destination

Wow! I just got back from #Agile2013 in Nashville and I have to say it was a great experience and my brain is overflowing with ideas! Which is always good for blogging. Open-mouthed smile

One of the things that occurred to me while I was there was the seeming surge in the competitiveness amongst methodology practitioners. There were numerous tweets and blogs, sessions and conversations all discussing the pros and cons of one method versus another.  Here are just a few things that I’ve seen… (FYI, I pick on everyone equally. Let’s call it an evolutionary fitness test.)

http://kenschwaber.wordpress.com/2013/08/06/unsafe-at-any-speed/

 

http://www.nofluffjuststuff.com/blog/alan_shalloway/2013/08/announcing_the_3rd_generation_of_lean_agile

http://www.netobjectives.com/blogs/why-net-objectives-supports-scaled-agile-framework-safe

 

http://www.djaa.com/kanban-alternative-path-agility

http://www.djaa.com/kanban-anti-safe-almost-decade-already

 

So we have a lot of posturing and positioning, and I actually don’t mind that. With noise comes interest, with interest and curiosity comes exploration and learning, and we can all grow. I hope though that by reading this blog and the other blogs and by exploring the topic, you gain a greater understanding of the items under discussion. It was with this intent that I had a really good conversation (or two) with Al Shalloway and Nayan Hajratwala about SAFe and the current thinking about The Kanban Method and Kanban/kanban in general.

So we were hearing the noise, exploring the enemy, probing for weaknesses and then trying to sway the enemy to our side. ;P

Today I had a realization about the experience. I learned about SAFe. I learned more about Scrum. I expanded my own understanding of The Kanban Method and kanban in general and I discovered more refinements to my thoughts when having this conversation! The Kanban Method, as David has been saying and said in the post above, is not the enemy of SAFe or Scrum or any other practices-based methodology that is a manifestation of the Agile Manifesto. It isn’t even comparable. It isn’t an Agile-practices based methodology at all, which is why David (and many of us at LKU) have been saying that The Kanban Method isn’t an Agile methodology! It is the vehicle that GETS you to Agile, wherever that may be for you! It is the vehicle that you use to encourage a collection of weird-acting individuals (people in an organization) to do something, such as adopt a bunch of agile-influenced tactics that might be described within any of the other practices-based methodologies!

Let me draw a picture!

image

We all start on these process adoption/organization transformation/improvement initiatives from a current state. Very often we can’t even describe the current state, but that is where we are. And very often, we can’t accurately describe the desired end-state, but let’s suggest for a moment that we can. So we know where we are (maybe) and we know where we want to go (maybe) but how to get there?!?! Some folks would have you believe you can jump there by telling the organization to put something in place.

image

I don’t mean to pick on Scrum. I like Scrum when it is the right fit. But it is frequently adopted blindly and prescriptively because it provides a lot of guidance on specific practices. But what if Scrum doesn’t fit? (Are there any DevOps people reading?) What if Scrum doesn’t help with the capability that you’re interested in? Or what if you’re problem is a scaling it out problem, which Scrum has troubles with. Good news! SAFe is right around the corner!! And virtual kanban systems! And Random Acts are there too!!

image

These are all methodologies with some great tactics to use for organization improvement right?! If you just tell everyone to do Scrum, SAFe, virtual kanban, or random stuff (heros, cowboys and smart people), we’ll achieve the improvements we’ve been struggling to get for the last 50 years in our industry!! But I heard this awesome comment from Linda Rising at #Agile2013 during her talk. I forgot to attribute it to her in my tweet though!

This is part of the problem with prescriptive approaches to process adoption. And unfortunately, Scrum and SAFe as described are easily prescribed. I’d even venture to state that the authors and evangelists of these methods want them prescribed. “If you do this, everything will be ok.” Of course they will not admit that they want you to blindly adopt all of the tactics, but in my opinion, the way those methodologies are described and evangelized, that is how it works out in the end.

And never mind that we still don’t really know if the methodology that we’ve selected will get us to the desired improvement given the context in which we are applying the methodology! Do the people in your organization want Scrum? SAFe? virtual kanban systems? Will they be able to pick up all the necessary practices? Apply them properly? Will the organization have the maturity to get through the hard parts? What if you’ve picked the wrong method? And what if the actual end-state option that you need isn’t the one you thought you needed at the start?

image

 

Has this happened to you? Have you seen this happen?

I want to reiterate at this point, I don’t think that the practices and tactics described in Scrum are bad. I don’t think SAFe tactics are inherently bad. I don’t think that “Kanban is so easy” crowd (proto-kanban or basic virtual kanban systems) are bad. It’s just going to be harder, longer, and more costly to get to an end-improvement than it should have been. You’re increasing the chance of your improvement initiative being interpreted as a failure, or being aborted early. Or leaving improvements undiscovered!

Another thing that I think is really interesting is that most of the practices-based methodologies describe themselves as an end-state. Like there is no more to learn and no more reason to grow in your context. What if the discovered end-state is a healthy, functioning Scrum implementation. Is that the end? Is there no where else to improve? What guidance does Scrum give you if you find an improvement that is counter to a described Scrum tactic? Or SAFe for that matter? Should we have end-states?

image

 

I think we are far better served trying to create an organizational capability to identify improvement opportunities and  use any/all available tactics that we think will move us towards the desired improvement, once we know what the specific problem is. I think we are better served by creating an organizational capability to understand the attributes of the work that we do (volume, frequency, size, complexity) and understand the capability of the organization to execute that work. And I think that we are better served by facilitating the creation of a kaizen culture within the organization that will continuously seek out opportunities to improve because then the people own the problem and the improvement!

I want a vehicle to get me to my currently planned destination!! I want to be able to make informed decisions and change the destination! I want all my peers to dynamically guide their team to the next improvement that is discovered in the context of doing the work! And I don’t want to limit my destinations! I want to know that I’m making progress!

So we know you’re organization is ‘here’! Sounds silly when you say it that way but even if you can’t describe it, you are where you are. And there is some value in describing where you are because it will make progress measurable and discrete instead of abstract or qualitative. And we know that we need to be… faster let’s say. It should be more elaborate than that but bear with me for a moment. Now we have to convince everyone involved to paddle in the same direction!

What can I use to do that? Will Unit testing help? Will team structures help? continuous delivery? project roll-up techniques? Will any of these specific practices help me create an environment that causes groups of people to understand the problem, understand the capability, formulate a plan to improve and then determine if the execution worked? I do not think so. Are they all good things? YES! Unequivocally! Are they good things for everyone? All the time? No.

The Kanban Method will foster these capabilities for you because that is what it is designed to do. It is not designed to make you code faster or with higher quality. It will help you understand the cost of poor quality and then allow you to experiment with quality improving tactics like Unit testing/TDD/ATDD/BDD/xDD. Will it help you deploy your application faster or sell more units of your product? No, you have to use tools or LeanStartup for those kinds of things! Will it help you create a kaizen culture within an organization? Yes, because that is what it is designed to do. Will The Kanban Method encourage you to adopt concrete practices from any other methodology? Yes. Can you use a SAFe tactic if it seems logical in your context? Yes! Scrum tactic? Yes! virtual kanban system? Absolutely!

Final Thoughts

I keep coming back to something that my friend Frank Vega has said numerous times.

“The greatest learning happens at the boundary of disagreement.”

It was because of the disagreement that Alan Shalloway and I have about how we should guide people that I had these conversations. It was because of the respect and curiosity that we both have that we continue to meet at this boundary.

It think it is important though to be clear and compare like things.

The Kanban Method is not an Agile Manifesto-inspired practices-based methodology like Scrum or SAFe. It is an approach that is intending to create a capability within organizations to do incremental, evolutionary improvement. It is a transition method that understands people and the various factors that cause people to disengage. It is a transition method that provides high-level guidance for the creation or adoption of specific practices in a situation where a  problem has been identified. It is a transition methodology that sees no end to the improvement cycle and as such, does not provide guidance on concrete tactics or the order in which you adopt concrete practices.

And because of the opportunistic nature of the culture created by a Kanban Method implementation, all the specific practices that are described by Scrum, SAFe, XP or RUP for that matter, are viable improvement options to specific problems that you may encounter on your journey of continuous improvement.

So get in your vehicle, start improving and go until you hit the horizon!

Them’s Fighting Word!

I should start this post with some qualifications. I’m an consultant for a company called Imaginet. We’re a full service IT consultancy. Imaginet is a LeanKanban University (LKU) Founding Member organization. I am Imaginet’s Accredited Kanban Training Program Advisory Board representative and I also sit on the LKU Management Board. I’m an Accredited Kanban Trainer (AKT) and a Kanban Coaching Professional (KCP) with LKU. That said, I’m going to try and be as unbiased in my responses as possible, but please bear with me if I rant a bit. :D

This week seems to have been a particularly interesting week in the volume of rhetoric and misinformation that is being spread about The Kanban Method. I’m not even sure where to start, but I’m compelled to say SOMETHING!

What is Kanban

I saw a tweet from Alan Shalloway (@alshalloway) talking about a great blog post from Joseph Hurtado (@josephhurtado) discussing What is Kanban? In this article, Joseph describes the current state of confusion in the industry around the word kanban and tries to start a movement to provide some clarity.

I’m the first to agree that there is a lot of confusion around the word kanban. In all of my webinars and presentations, I take a moment to provide some clarity on this issue. I think it’s very important that we all do this.

image

Link to webinar

Joseph goes on to add some definitions, which is great.

This is where I start to take exception to his blog post because he then goes on to start actually misinforming his readers. He states:

The problem is that David Anderson, LKU and some other Kanban Consultants have decided that there can be only one valid Kanban method for business or even IT, theirs. In their view if you do not follow exactly The Kanban Method, dare to call Kanban Agile or add any Lean or Agile practices to it you are wrong.

This is so blatantly wrong that I hope it is unbelievable to his readers, but I feel compelled to respond because Imaginet and I belong in the group “LKU and some other Kanban Consultants”.

Our understanding of The Kanban Method (as per wikipedia) is:

The Kanban method as formulated by David J. Anderson[1] [2] is an approach to incremental, evolutionary process and systems change for organizations.

I’ve heard David (@djaa_dja) speak many times defending this definition of The Kanban Method because as far as I know, he is the originator of the name and the definition attributed to it. That seems reasonable to me. And in those conversations and in David’s classes, I have often heard him embrace specific improvement tactics from other industry-specific methodologies like Scrum or XP. Many of the LKU Kanban practitioners that I have personally spoken with embrace many of these same improvement tactics. David even goes ones step farther than many “Agile” evangelists and embraces improvement tactics from traditional project management methodologies like PMI when they make sense in a context.

What I think Joseph (and others) are doing is trying to define the meaning of the word kanban in a manner that suits them and at the same time belittle David, LKU and it’s members, for trying to protect the definition of “The Kanban Method” which we also sometimes just shorten to Kanban. Joseph and Alan also seems disturbed that we often shorten The Kanban Method to simply Kanban.

There was a time when I thought somewhat like Joseph but in a Scrum context. I often thought that people, who said “You are not doing Scrum” when a team was doing some of the tactics that the Scrum Guide but not all of them, where wrong and mean. Of course we were doing Scrum, just not all of it. I then had a conversation with Ken Schwaber and he put it very simply and it made sense and my thinking change. Ken used the game of Chess as an analogy and said to me  (paraphrased) “If you use the rules of chess, you can say you are playing chess. But if you change the rules, are you still playing chess? No. The same goes with Scrum. If you follow the rules provided in the Scrum Guide, you are doing Scrum. If you deviate from the rules, you aren’t doing Scrum.” It seems to me that this is the same issue that Joseph is having an issue with, except we’re going one step further and using an abstract word (in this case) like kanban.

Joseph then goes on to create a new ‘thing’ which is great. Open Kanban. Sounds good until he describes it. And it starts to re-describe things that The Kanban Method has described. It describes some things that LKU Kanban practitioners have described. Which is honestly really flattering. As the saying goes, imitation is the sincerest form of flattery. The problem though is that now there is a competition. The Kanban Method vs. Open Kanban. And we’re back to rhetoric and bickering because honestly, we are all here to make money and grow our businesses.

To wrap up this portion, here are a bunch of thoughts that I hope counter some of the misinformation in Joseph’s article.

  • The Kanban Method has a definition. I don’t think we should really argue about that.
  • The Kanban Method has a singular purpose. Catalyze (encourage?) incremental and evolutionary process changes for organizations.
    • David freely gives credit to all of the people who have influenced his journey and his continuing growth in this space.
    • David strives to stimulate continuous improvement of The Kanban Method, especially by reaching out to the community of practitioners.
  • Whatever your process is or the process that emerges to help your organization create value, that is NOT The Kanban Method, but simply what The Kanban Method helped you create.
    • You’re process will likely evolve towards being flavoured by Agile or Lean thinking. They are both #reallygoodthings
  • The Kanban Method is not Scrum. It does not compete with Scrum.
  • The virtual kanban system that may be used to help your organization evolve WILL compete with Scrum. The Kanban Method does suggest the use of virtual kanban systems to better manage work. This doesn’t have to be viewed as a bad thing unless you are a Scrum evangelist who isn’t interested in new ideas. Scrum and Kanban both have noble goals.
    • Scrum has a virtual kanban system at the heart of its work management system. It just uses larger batches than most kanban practitioners like and couples cadences unnecessarily.
  • The virtual kanban system that may be used to help improve your organization will probably use some sort of kanban (visual signals, signboards)
  • The Kanban Method does not advocate the exclusion of any kind of improvement tactic for the process or the organization. It just doesn’t necessarily provide specific guidance, which is fine. You are MORE than welcome to pull in unit testing, team formation, automated builds, continuous integration, iterations, tooling improvments, etc. if the specific tactic is anticipated to improve the system.
  • The Kanban Method, as a methodology, won’t give you advice on a specific improvement tactic. It does not understand the context and it would be deemed “tampering” (and perhaps arrogant) to suggest improvements without information.
  • The Kanban Method does provide guidance on several high-level improvements tactics that will help you create an environment for your organization to improve.
    • These high-level tactics usually lack specific implementation details. Let’s call them meta-tactics that will allow for the creation of context-specific tactics.
  • Don’t let myths and misconceptions cloud your assessment and the opportunity to learn about The Kanban Method, or any of these other methodologies. Please go out, learn and make your own decisions.

I’d love to talk to you about this. I’ll be at Agile 2013 in Nashville next week. If you’re going to be there and you’d like to discuss any of this, I’d love to catch up with you. Please message/mention me at @agileramblings on twitter and we’ll figure out a way to get together over a pint or two.

Who is Estimating?

Part 3 of my Exploration of Estimation

In part 1, I posed the question of Why are you estimating! I hope that you thought about it and may have found an answer or two.

Part 2 explored What are we estimating. Usually we are asked to estimate effort (cost) which is then turned into a schedule (duration) which are two very different things.

The next question I’d to explore is Who is doing the estimating?

In my experience,  normally it is the technical people who will be doing the actually work (coding, sql, ux, etc) that are asked to provide an estimate. Developers are the best and most familiar example. And while at first glance this seems reasonable, when you start to dig into the entire workflow required to move an idea to reality in software, it begins to look a bit inadequate.

 

image

 

As we can see in the workflow, there are several phases of life that a feature/user story/use case goes from to be brought to life, only one of which is development. Often, there are many tasks across elaboration/analysis, development,  quality assurance, user acceptance and IT/Ops just to name a few common phases. Are developers performing all of these tasks? Do they take into account the effort required in all of the other disciplines when they do their estimates?

So why is it that developers are usually the only ones estimating effort?

 

image

Surely developers aren’t the only one working on your projects!! (Don’t get me wrong. I was a developer growing up, so I’ve definitely thought that the software development workflow revolved around me!) But this is a team sport, and without the business analysts, QA professionals, IT professionals and what ever other role is in your delivery workflow, we may have a difficult if not impossible time getting our software out the door!

 

image

 

So as a part of your estimation process, how do you understand the effort and/or time required to get work through the entire delivery process?  Do you take into account all of the phases of life, and hand-offs, and interactions in your delivery process? Do you get all of the people involved in the estimation process? In fact, I’ve seen (not often) teams involve a tester in the estimation efforts, but very infrequently do I see analysts, DBAs or IT/Ops folks involved in the process, even when they’re role is critical to the delivery of the product!

So who is involved in YOUR estimation process? Do you have all the people you should have involved?

And are you asking yourself this question in conjunction with the other two questions I’ve already asked you about? Hmmm….. I hope you’re all thinking about all of these questions! I hope to present a couple more questions in the next week and then start to tell you how I think we can start make some real changes in the way that we do all of this estimation stuff!!

What Are You Estimating

Part 2 of my Exploration of Estimation

In part 1, I posed the question of Why are you estimating! I hope that you thought about it and may have found an answer or two.

The next question I’d love for you to think about is What are you estimating?

I was just in a meeting that was discussing a RFP for a client. This is pretty common in the consulting industry where we see a Request For Proposal and in that, we need to define the scope of work that we can address, align our skills and services to deliver  that scope of work and then provide an estimated cost for that service. Seems kind of normal. But then we are also sometimes required to provide an estimated calendar duration for the project. So in essence, we are being asked to provide 2 estimates. We then present our guess (educated) at how much work needs to be done, and here is our guess (educated) about how long it will take the team to execute on that vision.

So when the developers were being asked to provide estimates to the work in the RFP, what do you think were they estimating?

I haven’t had a chance to ask them yet, but based on my experience (as both a developer and a person being asked to provide project duration estimate), the development team is probably estimating effort and not duration.

So who estimates duration then? How do we get duration from effort? And if we stacking a guess (how fast we deliver) on top of a guess (effort), are we getting a meaningful expected duration value?

If you go back to my rant on estimation (which I will reference often in this exploration series) you will see how most organizations, in my experience, do mental and mathematic gymnastics to come up with an expected duration value.

My belief is that we are not getting a useful duration value. But everyone treats it as useful, starts building elaborate project plans and organizational activities around this duration value, and then we are all shocked when we have to change request the project, drop scope, or scramble like crazy (and usually with questionable quality) in the latter half of the project when it becomes painfully obvious that we’re not going to make our original duration estimate. Either way, the initial setting of expectations based on the layer of several guesses seems to be a Bad Thing™.

So I’ll leave you with this final question, what are you estimating?

p.s. I will explore better ways soon! Just bear with me for a bit longer! Open-mouthed smile

Why Are You Estimating

Part 1 of my Estimation Series

A while I go, I wrote up a bit of a rant on a crazy estimation scenario that was presented to me by a colleague. If you didn’t get a chance to read it, you can find it here. We’ll call that rant Part .5 of this series of blog posts on estimation.

Since then, I’ve seen a bit of a swell on twitter of the #noestimates hashtag and a lot of discussion on topic. I was recently saw a tweet from Allan Shalloway that prompted a bit of a conversation:

image

(click imagine to see original tweet and conversation)

As you might have guessed, my thinking is more aligned with the #noestimates crowd and I’ll elaborate that over the coming months here, but this tweet started a conversation that I’ve had before.

Why do you estimate?

If I’m going to encourage you to stop estimating, it’s really important that we understand what I’m encouraging you to stop vs. what you might actually be doing.

As the conversation continued, Allan stated:

image

…which made wonder if we’re confusing prioritization with sizing, which is actually quite common.

So I won’t re-post the entire twitter conversation here, but I’ll start to share my thoughts on the why of estimate.

Intent

When I engage in an “estimating” exercise with a team, I very often try and restate the activity as a sizing exercise. For me, this starts us down the road of using the language that more accurately describes the end goal of our exercise. We want, to the best of our ability, describe how “big” we think this work item is. I’d like the team to use their past experience and compare the current work item (requirement, user story, etc.) against similar work items they’ve worked on before. This is a relative sizing technique that is used all the time by other agile methodologies such as Scrum or XP. You’ll often here the term “yesterday’s weather” in the same conversation about agile estimation techniques. The best predictor of today’s weather is yesterday’s. The team’s recent experience is the best predictor of its near-term current capabilities.

To me, the really important point of this exercise is sizing! Not scheduling. Not prioritizing. Those are completely different activities! The product owner/business representative on the  team will tell you, perhaps with size as a decision input, what is next and when they might hope for it to be done. I generally expect them to help the team constrain the story, but the delivery team are the ones that are responsible for determine the size of the story.

So in my mind, the intent of the “estimation” exercise is to allow the whole delivery team to determine the relative size of the story compared to work items they have done in the past.

Why?

So why are we asking delivery teams (Development, Ops, anyone who is creating value) to size their work items? This is where I think all of the estimation craziness and problems start.

We need to provide some sort of indication of the anticipated financial impact to the organization making the request of this work item. We also need to provide information that would allow us to set a delivery timeframe expectation. And the next level of this problem is to take 100s (or 1000s) of these work items and provide the same financial and delivery timeframe expectations. This is, for most teams, really really hard.

And I will state that it is my belief that size is not the primary factor in determining cost or delivery timeframe. Will a bigger story cost more? Usually, yes, but not always. Will a bigger story take longer? Usually, yes, but not always. What if to delivery a story, a team has no external dependencies? What if it does? What if the story is big, but a really well known problem? Unknown problem? What if the team is missing a team member when it finally pulls the story from the backlog? What if the team composition has improved?  There are so many things that can change the cost and expected delivery timeframe in addition to the size of the story that are very difficult to predict.

So in my mind, we are trying to size our work so that we can help business decision makers make the best decision for the organizations economic well being.

Final Thoughts

I’ll leave you with a couple homework questions. What is the intent of your team’s estimation exercises? Why is your team performing those exercises?

Part 2 will explore my approach to providing business decision makers with better information so that a team (business and delivery groups) can make better decisions on behalf of our organizations.

@AgileRamblings on Twitter

Follow

Get every new post delivered to your Inbox.

Join 32 other followers