A KALEIDOSCOPE OF NOISE

Remember playing with kaleidoscopes as a kid? Turn the end, the colors and parts would shuffle and change, leaving it up to your imagination to decide what you saw.

‘I kind of see a turtle now.’ Twist. ‘That is a crazy snowflake.’ Twist. ‘I have no idea what that is, so I am going to pretend it is a dragon.’

I thought my kaleidoscope days were over. Yet here we are and we get to see things like this:

Various tools used for ‘software delivery improvements’ - found via google image search

These tools are supposed to give companies insight into what is happening in their organization. With these guaranteed epiphanies, leaders will then magically know where to focus on for improvement.

One problem (of many) with these radars / kaleidoscopes is that they create noise. Depending on what the viewer ‘wants to see’, they will see it. ‘I told you we needed to do better planning.’ ‘See, I told you we have tech debt.’ Twist, see something else. Now you get a platform team!

If that noise - the ambiguity of the interpretation of the information - was the only problem, we could consider them marginally harmless.

But these Kaleidoscopes of Noise - and there are no shortage of them - have an immense cost. Teams, which are already swamped with too much work, are asked to spend hours filling out forms to feed this data. Time is then spent analyzing the input. Dollars are spent on software. Delay keeps on happening. All to tell most people what is deeply evident already.

The astute reader will notice I used the plural of kaleidoscope. Why is that? Well, most companies need multiple tools and assessments to tell them what to do. Because every area with purchasing power wants their own story. More kaleidoscopes, more noise. More dollars wasted, more time lost, and more teams saying ‘just let us do our work.’

Kaleidoscopes of Noise Feed Improvement Theater

One role of management and leadership is to understand how the work works - what is making it difficult for people to get their work done. If you need a tool to tell you that…if you honestly don’t know and instead of just asking or, I don’t know, working with your teams, then perhaps you should reconsider just what it is that you do.

Talk to your teams, understand the work. If you have to spend money on a tool, feel free to send a check my way and I will mail you some spiral art from a kindergarten class. You can pass it around and tell your story, and it will save your teams a bunch of pain.

You Can’t Question your way to a better Architecture

A Baseball Story

My son plays high school baseball and he hopes to continue playing in college.

When he was young - just starting to play - I coached his team.  At first, as an assistant coach (keeping 6 and 7 year olds on the field can be challenging!) - then as a head coach with a partner, then as a head coach for the team.

The team progressed and did great!  Of course…youth sports are pretty volatile, so some of those wins might just be luck of the draw.  When the team was playing their age 9 season, they won, I believe, 6 out of 8 tournaments they were in.  More importantly, they were having fun.  The players were learning the game and their skills were improving.  The coaches and I - all parents - had, at best, played some high school baseball, and some years ago at that.

When the team entered their age 12 season, it was evident that the team was skilled.  They had developed beyond my ability to coach them.  Did I want to stay with the team?  Of course!  All of the coaches bonded with the players over the years, we had very little turnover, the parents were all great - why would we want to stop coaching?

My staying on wouldn’t have helped the team.  I didn’t know how to take them to the next level.  The team moved to a new organization, with new more experienced coaches.  They immediately upped the skills of the individuals as well as the team as a whole.  

What does this have to do with, well, anything?

I frequently hear and see coaches in the tech space make comments like “well, you don’t have to know how to code, all you have to do is ask the right question.’’

That is wrong.  

You might know how to play the game - you know the rules, how to organize the team, you might even win some - but if you don’t have the skills to take the team to the next level, you are doing them a disservice.

Some teams need some structure. They need to understand how to work together as a team, need to understand how their product works, maybe even talk about ways to plan.  

I have yet to see a coach ‘ask the right question’ that helped teams fix their architecture, fix their tests, or improve their product.  

Turns out, sometimes you need to know when to hand the team over to someone more skilled.

no nonsense agile podcast

We recently had the pleasure of chatting with Murray Robinson and Shane Gibson about Dojos, building organizational capabilities, and driving organizational change.

Listen here.

Dojo Coaching for Building Technical Skills - InfoQ Podcast

We were guests on Shane Hastie’s Engineering Culture podcast.

Topics include our latest book, “Coaching for Learning”, the dojo model, roles in the dojo, and key principles for creating effective learning experiences.

Engineering Culture by InfoQ · Dojo Coaching for Building Technical Skills

A transcript is available on the InfoQ site.

Seven principles for creating effective, Immersive learning experiences

This post is an excerpt from our recently published book, Coaching for Learning: The Art and Practice.

These seven principles are at the heart of how dojos and other, similarly immersive learning experiences work.

1. FOCUS ON LEARNING OVER DELIVERY

Plenty of organizations run the risk of calling a dojo an accelerator or using similar velocity-related words, and that’s simply not the right language for what you’re doing. A dojo isn’t a place to go faster. Instead, you measure learning and the impacts of learning. Success is defined as accomplishing learning goals, not delivering an output. If a team does an experiment to foster learning, and that experiment fails, it’s not viewed as a failure by the team because they will have learned from it. There is more learning gained in a failed experiment than in a successful one.

2. HELP TEAMS LEARN COLLABORATIVELY

We are very much for teams learning together collaboratively— helping and supporting each other while figuring out how to work together. People on teams often learn as much from one another as they do from the coaches, multiplying the impact of learning in the dojo. (You can thank your mirror neurons for that.) Dojos are effective in spreading knowledge that already exists within teams, whether it’s the knowledge about the code itself, the business domain, the problem space, or the architecture, to name a few examples.

3. GROUND LEARNING IN THE CONTEXT OF REAL-WORLD WORK

It’s critically important that learning in the dojo isn’t theoretical—it must be done in the context of participants’ real work. This corresponds to Bloom’s taxonomy of the brain, which ranks higher-order doing tasks such as creating, evaluating, and analyzing above book tasks such as simply understanding or remembering. In addition, learning must include all the real-world constraints that go along with the team doing their work—from technology constraints, to organizational constraints, to political constraints. You want all those constraints to be there. What good is it if when the team leaves the dojo, they must operate in that real world of constraints, and we haven’t taught them how to do that?

4. MAKE LEARNING HOLISTIC AND SPAN MULTIPLE PRACTICES

We encourage teams to take a systems thinking approach to their capability improvement efforts. When teams focus on improving a single capability, they run the risk of missing out on improvement opportunities that could be more impactful. What’s worse, they also run the risk of sub-optimizing their product delivery which could lead to an overall decrease in their performance.

Learning in a dojo is holistic. Teams learn how to focus on the capability improvements that will have the most impact. They see how practices relate to each other. For example, they understand how DevOps practices help them get feedback on product ideas faster instead of thinking of DevOps as only a tool for removing errors from the delivery process.

Remember: the neurons that fire together wire together. It’s not as simple as one practice—it’s about how the practice impacts other parts of the value stream. Seeing how practices relate to each other helps build new connections and learning within the brain.

5. PROVIDE ASSESSMENT, FEEDBACK, AND COACHING

It’s critical for learners to receive frequent feedback as early as possible in the process. When you’re learning, you can’t assess yourself. So anyone in a learning environment needs a coach who doesn’t just teach, but who assesses progress and provides feedback for improvement in real time.

Dojo coaches have multiple skills, including assessing where a team is at, providing feedback to teams as they’re learning or improving practices, and coaching teams to be more effective. They’re not just facilitators—they are people with deep skills, practitioners who are themselves capable of doing the work that they’re coaching teams on doing.

6. PROVIDE SUFFICIENT TIME FOR REPETITIVE PRACTICE

Dojos are about practicing just as much as they are about learning. Remember that the word “dojo” comes from Japanese martial arts—it’s “the place of the way.” The place where both learning and practice occur.

It’s not enough for teams to intellectually understand a practice or an improvement to the way they currently work. They must practice what they learn to make it stick—and the practice needs to be repeated. Remember, your teams are battling the Ebbinghaus Forgetting Curve. By taking advantage of the spaced repetition learning technique we introduced in Chapter 1, you can strengthen neural connections and solidify what people have learned.

In the age of Zoom, people often want to cut short this aspect of practice. There’s no reason why a coach can’t log off Zoom while the team stays on—working together and practicing what the coach has gone over with them. In fact, we believe if coaches are there all the time, they risk becoming a crutch for the team. Dojo coaches should never become a crutch— stepping away for a while is beneficial for the team’s growth.

7. ENSURE LEARNER SAFETY

For a dojo to be effective, there must be learner safety, both for the team and the individuals in it. People must feel they can experiment and learn from their mistakes, as well as from their successes, without fear of retribution. They must feel their boss and the other members of their team support them, that they have their backs. And they must be comfortable in the physical (or virtual) space provided for the dojo.

In a virtual world, ensuring learner safety is especially challenging. When you go virtual, and especially when you go global, you may have a hard time making sure everyone feels safe. You can’t tell if a team member turned off their camera because they’re generally more comfortable being off-camera, or because they’re uncomfortable right now in this particular situation, or simply because they needed to step away from their computer for a minute.

You can purchase Coaching for Learning here.

Dojos and Coaching for Learning Beyond Facilitation - Mob Mentality Show

We enjoyed the conversation we had with Chris Lucian and Austin Chadwick on their Mob Mentality Show.

Chris Lucian and Austin Chadwick discuss all things #agile and product development from a #MobProgramming perspective. What would happen if your team kept working on its current production tasks, but switched the top priority to learning while doing so? What if this flip would not only include learning facilitation excellence but also would include coaching, technical, and product excellence as well? What would the short term impact be? What effects would be seen in the mid to long term? Join Chris and Austin as they have a fantastic discussion with Joel Tosi and Dion Stewart on "Dojos and Coaching for Learning Beyond Facilitation." After Joel and Doin describe these immersive learning environments in production, they dive into how remote dojos have led to more ensemble programming. Then, for both sports and tech, they cover the many facets of coaching side-by-side and in-context. Lastly, they explore what a post-"Agile" world looks like for them.

Jess Brock is Speaking about Dojos and agility.us is Giving Away a Dojo Experience

Jess Brock is speaking on dojos at an upcoming event on September 21st. More details can be found here: https://agility.us/spark-dojo/spark-dojo-event/

We’re always happy to see the dojo model spreading and the community of practitioners growing.

We saw Jess speak at the Agile Alliance conference over the summer and encourage you to attend her talk on the 21st.

And agility.us is so confident in the value their dojos provide they’re giving away a dojo experience for free.

Anti-Patterns to Avoid When Growing Your Dojo

Dojos are a significant investment in people, time, and learning. It can be tempting to shortcut the team’s experience in the dojo when faced with organizational pressures.

Let’s look at three common anti-patterns when scaling your dojo.

Shortening the Learning Experience

To get more teams through the dojo, some organizations have shortened the learning experience. What was originally intended as a six-week experience for the team gets shortened to two weeks. Other times the six-week duration is maintained, but teams only commit to two hours a day or a few days a week.  

What gets lost? 

The team loses repetition and feedback opportunities. Practicing new skills over and over with feedback from a coach allows us to refine our learning. Practice enough and we can master our craft. Shortcut those repetitions and the learning doesn’t stick.

The team misses out on meaningful connections between practices. In the dojo, we often find teams connecting practices together to form a capability which is greater than the sum of its parts. For example, a team learns that setting up a continuous delivery pipeline that supports A/B testing can help them with product discovery. If learning is too narrow, these connections don’t occur. 

The team mindset shifts from experimental learning to delivery. Condensed time frames often cause the team focus to shift from learning to delivering (e.g., finish the pipeline, finish automating the tests, or finish creating the lambdas). 

Overloading Your Coaching Staff

Organizations sometimes overload coaches with too many concurrent teams in an attempt to get help to more teams sooner. Dojos require a mix of product and technical coaching, and as a result, we need two coaches per team in most dojos. Using a staggered approach, it is possible for a coach to balance up to two teams at a time. 

What happens?

The learning between teams and coaches becomes formulaic. A key part of the success in dojos is the spontaneous nature of the learning that occurs between coaches and teams. When coaches aren’t with the team "in the moment", the learning becomes less interesting. “Schedule time with me this afternoon.” “I can come work with your team on Tuesday.” Meaningful learning requires two-way focus and commitment from coaches and teams. Without this commitment, learning eventually stalls.

The relationship between the coach and the team suffers. When done right, coaches and teams should feel like peers on a learning journey. This relationship slips into an acquaintance relationship without the necessary time and investment from the coach. Communication levels drop, and the team eventually stops asking the coach for help. The team walks away with mixed feelings from their dojo experience.  

Growing Coaches Too Quickly

Most organizations want to staff up their dojos quickly, but lack the internal capability to do so. 

A real constraint on the number of teams you can coach in your dojo is the number of qualified coaches you have available to work with teams. This is clear prior to forming a dojo, but even more so once there is ample demand. While coaching in the dojo is not completely different from other types of coaching, it requires a specific skill set and understanding of how coaching for learning differs from coaching for delivery. 

What is the impact?

The team experience is degraded without qualified coaches. When organizations try to shortcut coaching expertise, the team experience inevitably suffers. For example, a team that requires both product and technical coaching would not feel satisfied without a suitable technical coach in their dojo. Team learning goals go unmet and outcomes fall short. Yes, you get more teams initially through your dojo, but at what cost? 

New coaches become dogmatic and close-minded. Many new coaches take time to internalize dojo coaching. New coaches initially copy and replay what they see after watching more senior coaches in practice. They ask questions because they saw the other coach ask them. Coaching in the moment requires more than rote memorization. It requires observation and awareness that only comes through experience. When we circumvent that experience, we get dogmatic coaches and inferior team learning.

So What To Do?

It is great that organizations want to scale their dojos. It is important to keep the following in mind when scaling your dojo:

The dojo approach is highly effective at helping teams build long-term learning habits. Shortening the learning experience can negatively influence the quality and sustainability of your team outcomes. Don’t weaken those outcomes in an attempt to help more teams sooner. 

A large part of the success of dojos has been because of the connections teams make with coaches on their learning journey. Overloading your coaching staff can reduce meaningful learning and produce mixed results for your organization. Do not sacrifice team and coach connections for a false sense of going faster.

Qualified coaches are a real constraint and require time to internalize. Growing coaches too quickly can impact your team experience and stunt your internal coach development. Don’t shortcut your internal coaching growth needs in order to reduce reliance on external coaching.  

Scale your dojos intentionally, focus on the biggest impacts for your organization, and create an experience that teams won’t mind waiting for.

How Do We Scale the Dojo?

When launching a new dojo, we start with a few pilot teams. There’s an intentional pause after the pilots so we can assess what’s working, what adjustments we want to make, and how we can be most effective in the future. Often, after demonstrating success with the pilot teams, a question that comes up is “How do we scale the dojo?”  

The first step in answering that question is to make sure you understand what is meant by “scaling”. Once you’ve done that, you can explore ways of answering the question.

“Scale” - You Keep Using That Word… 

In our experience, these are the three most common meanings behind the question “How do we scale the dojo?”.

1. Get more teams through the dojo faster. This is the most common meaning behind the question. The logic goes like this - if the dojo can have a positive impact on 4 teams at a time, how can we scale up to 8 teams at a time (or some other multiple). The goal is to get more teams through the dojo over a given time span. There are always constraints in one form or another when the goal is to scale the dojo linearly. Those constraints include budget, availability of skilled coaches, available space (not a problem with remote/virtual dojos), and scheduling around teams’ projects and commitments.

2. Get “more” results without additional investment. Here, there’s a desire to get more results without increasing budgets, number of coaches, space, etc. Another way of saying this is the organization is looking for exponential returns. When the scaling question is asked this way, it’s often accompanied by suggestions for documenting best practices, creating playbooks that can be executed by less experienced coaches, reducing the time teams spend in the dojo, and reverting back to traditional forms of training (e.g., shorter workshops). These suggestions often risk diminishing the effectiveness of the dojo and are often at odds with the underlying principles that separate dojos from traditional training.

3. Coordination of dependencies across teams. This is the least interesting form of ‘scaling.’ Sometimes, the organization wants to bring multiple teams on a project into the dojo together to help manage the dependencies between the teams. In the worse cases, the dojo becomes another method for doing cross-team project management. We discourage organizations for using the dojo in this manner. The exception to this is when the organization wants to use the dojo to help eliminate the dependencies between the teams. For example, the primary goal for the teams might be to learn how to refactor a monolithic application into microservices. If the primary goal is learning, that’s an effective use of the dojo. If the primary goal is project management, it’s best to pass on those teams.

How Can You Achieve Your Organization’s Scaling Goals?

Now that we’ve established common meanings behind the question of scaling, let’s talk about strategies for meeting the goals behind the question.

With linear scaling (getting more teams through in a given time span), the strategy is simple. You need additional investment where you’re constrained (budget, number of skilled coaches, etc.). While it may not be easy to work through these constraints, the strategy for linear scaling is obvious. 

Here are some approaches that may not be as obvious:

●      Leverage improvements with platform teams. We always encourage organizations to consider bringing internal platform teams into the Dojo first. If you have teams building frameworks and tooling for the cloud, microservices, infrastructure as code, continuous delivery pipelines, UI frameworks, etc. you can achieve a multiplier effect without increasing your investment. This is an effective way of achieving exponential scaling. Improvements to the capabilities of your platform teams can lead to improvements across all teams using those platforms.

●      Learn from teams and fix organizational constraints. As you’re working with teams in the dojo, you will identify constraints that are outside the teams’ ability to fix. Examples include wait times for rubber stamp approvals before a team can deploy to lower environments, restrictions around who can access information and who can talk to whom, waiting two weeks to have a new cloud environment available, etc. Leverage learning from working with individual teams to identify constraints that are likely affecting many (or even all) of your teams. Address those constraints and you’ve got another exponential return on your investment in the dojo.

●      Halve the problem. This is a great aphorism we learned from Woody Zuill. When working with a problem ask yourself, “is there a way we can cut this problem in half?” Often there is. For example, if you’re looking for ways to scale your dojo linearly you’re probably assuming you have to get all of your teams through the dojo. Is that a valid assumption? What if you identified the potential impact of working with your teams and focused on the top half of them in terms of impact? Do you need to work with all of your teams?

Revisiting Linear Scaling

OK, maybe you read our comments above about linear scaling and you’re saying to yourself, “Really, the best you can offer is tell us to address the constraints?” In the near term, yes, that’s what we’re saying. However, there are some things you can do in the longer term if you have the patience for them.

●      Grow coaches. Successful organizations grow their own coaches. The most common constraint when trying to scale linearly is a lack of skilled coaches. But here is the beauty of it - your organization already has people who can become effective coaches. It requires patience. At the first Dojo Consortium conference, Kent Beck talked about how he’s always considering how he can set up the next stage of the propagation of an idea. Part of that includes identifying people who can explain an idea to the next person. Or, Kent might say, “We’re doing this again in six weeks. Can you come in and help us teach it?”

●      Embrace Social Learning Beyond the Dojo - The most effective dojos we we’ve been part encourage social learning that extends outside the dojo. Part of a team’s dojo experience might involve learning how to use tools like Slack. We’ve also helped organizations set up internal communities of practice and meetup types of events. In addition, the dojo could also sponsor internal conferences that encourage social learning and help foster networking connections within the organization.

When faced with the question “How do we scale the dojo?”, first seek to understand the context in which the question is being asked. Make sure you understand the motivation and meaning behind the question. From there, you can decide what strategies you want to adopt.

In our next post, we’ll talk about some of the anti-patterns we’ve seen when attempting to scale the dojo. 

Reflecting Back on the First Dojo Consortium Event

Last week, we were proud to be a small part of the Dojo Consortium Event. From the humble, insightful speakers, the engaging question and thoughts shared by attendees, to the general energy at the conference, the event felt like something special.

Our Top Takeaways:

The Dojo is a Thing

We had a hunch this was the case, but let’s make it official - dojos are a thing. When there are over 30 (large) organizations working on dojos, a sold out conference a month ahead of time, and even industry thought leaders getting excited about the dojo approach to learning - there is something there. Our challenge as a community is how do we stay true to our first principles around learning and not lose sight of that. One excellent open space topic even brought this forward.

The Dojo Community is Vibrant and Open

Attendees were challenged right from the get go - get uncomfortable; share not only your successes but your challenges; explore your learnings and known unknowns. And the attendees were up to the challenge. We had 18 diverse lightning talks from the community - ranging in topics from measuring your dojo impact, to new ideas for dojos, to coaching skills. And that does not even address the open space topics. Our favorite open space topics (in no particular order):

• Project to Product discussion - great to see this becoming more real

• Frameworks discussion - the realization that dojo coaches are succeeding through engagement, not through frameworks

• Community discussion on what is next - It was great to see the community come together to define what is next for the consortium

Common Dojo Challenges Aren’t Necessarily Dojo Specific

We heard a few common themes around challenges for dojos - and challenges that perhaps aren’t just dojo challenges.

Remote teams are challenging for many organizations, especially those trying to start dojos. While we personally have not seen fully-distributed teams work well in the dojo, there were discussions around what might work. We are interested in experimenting with these ideas as long as we continue to follow the core principles of what a dojo is.

Finding coaches is a challenge. Similar to working with remote teams, finding good coaches is not just a dojo problem. However, strong coaches are essential for running a successful dojo. Without good coaches, you run the risk of teams having bad experiences that will result in a lack of learning and any significant change.

Coaches Never Stop Growing

We had many wonderful speakers. Thematically we heard the same thing - great coaches never stop learning. One particular great comment came from the man Kent Beck himself when he said “The best thing I can do as a coach for a team is be the best me I can be that day.” Beautiful and simple.

We have many more fond memories that we will cherish. But… we’re already on to thinking about the next event! We are honored to have been asked to coordinate the next event. While we and the community set a pretty high bar with the first event, we promise the next event will continue to challenge us and push us all forward.

If you joined us last week, what were your takeaways? What is next for you? If you weren’t able to attend, what would you like to see in the next event?

Lessons Learned From 5 Years in the Dojo - Part 2 - Focus on the Value Stream

In our last post we discussed how Dojos require a sound overarching strategy. In this post we talk about why Dojos provide the most value when they address the whole value stream.

Dojos Need to Address the Whole Value Stream

What Do We Mean by Whole Value Stream?

Your company offers products and services that deliver value to your customers. How you deliver that value - from your initial product ideas through delivery of products that evolve from those ideas - that's the value stream. A value stream includes how you identify opportunities and problems, generate product ideas, develop and release those products, and ends with your customer receiving value from your product.

Here is how we view the value stream:

How we see the value stream

Why Dojos Need to Address the Whole Value Stream

Several years ago we helped a large retailer start their Dojo. Initially, their Dojo was focused on helping teams learn DevOps practices. As we suspected going in, this focus was less than optimal. DevOps intentionally focuses on the subset of the value stream starting with code commit to product release. Focusing on improving that subset of the value stream often delivers significant gains, however, for many teams the biggest constraints and challenges exist upstream - before code commit.

The first teams that went through that Dojo improved how they built products, but for some teams the adoption and usage of their products was low. The primary constraint wasn’t their engineering skills - it was their lack of product knowledge.

We guided the organization on how to implement product discovery and product framing practices and started up-skilling teams in these practices in the Dojo. The practices helped teams vet product ideas before they invested in writing code. When they did write code, the contextual knowledge they had about the product improved design, architecture, and testing.

In addition to teaching specific product discovery and product framing practices, we coached the teams to adopt a “product thinking” mindset. We drove planning and design discussions with questions like “Does this get us closer to solving the root problem?” and “What else do we need to learn to solve the problem?”

Another situation we’ve run into was one where the delivery team was completely separated from the design team. Design was outsourced to a third party. Delivery team members could talk to the designers only every few weeks. Early in delivery, the team started raising questions about the design. They found gaps and inconsistencies. We challenged them to question whether or not they should continue delivering anything until these questions were resolved. Sadly, there was a lot of pressure on the delivery team to keep delivering. The constraints in their value stream had nothing to do with delivery.

Teams often come into the Dojo to improve their testing practices. This often involves automating test cases that are currently run manually. Teams sometimes get overwhelmed when learning to automate tests with a legacy codebase. The codebases often require complicated setup of test data and there are a large number of test permutations. We use product framing to help narrow the focus to find the most valuable tests to invest in automating. To do this, we need to understand the product in depth - again, we need to go upstream to get context.

Addressing the whole value stream is important. When we do Dojo readiness assessments with organizations this is one of the key aspects we explore before starting a Dojo. We need to be on the same page about what aspects of the value stream the Dojo will address and how we how we are measuring success for improving the value stream. For some organizations, the success of the Dojo might be based on improving the subset of the value stream limited to engineering skills, but this is usually not the case.

In our experience, the most successful and impactful Dojos address the whole value stream. Improvements have more impact and there is a direct connection between those improvements and the skills teams learn in the Dojo.

If your Dojo isn’t addressing the whole value stream, what can you do to help move it in that direction?

In the next post we’ll focus on the theme of learning over delivery.

lessons learned from five years in the dojo - part 1

Having helped organizations with Dojos for five years, we felt it was the right time to share what we’ve learned so far. In this series of blog posts, we want to offer you our “best tips” for starting your own Dojo or for improving your existing Dojo. We’ll wrap up with our thoughts on where Dojos are going next.

Without further ado…

Dojos Need to Support a Strategy

We talk with many organizations excited about starting a Dojo. The concept of teams learning new skills while building products is enticing and practical. Excitement is great, but Dojos work best when there is an overarching strategy the Dojos serve. Without a strategy, Dojos can flounder.

For example, an organization invested in moving from a project model to a product model would leverage that desired outcome as their strategy for the Dojo to support. The strategy frames the purpose for the Dojo. The Dojo is there to help teams understand how to work in a product model and learn the skills they don’t already have required to work in that model.

Another strategy we often see is leveraging Dojos to adopt DevOps. While this is more narrow than we recommend, it is still a nice frame for what purpose the Dojo is serving. (We prefer to see Dojos address as much of the product development value stream as possible. We’ll cover this in our next post in this series.) A “DevOps Dojo” would focus on helping teams learn how to build continuous delivery pipelines and automate infrastructure setup while foregoing other skills like product discovery practices.

A third example is using a Dojo to help the organization migrate applications to the cloud. This is an interesting start, but for the Dojo to truly be effective the strategy should be clear on how teams should migrate their applications. Will teams refactor applications for the cloud, move them in a “lift and shift” model, or follow a “re-platforming” model? If it’s a combination of those approaches, what are the criteria for determining which migration model for an individual application? And what is more important - having teams leave the Dojo with deep knowledge of the cloud or getting their applications into the cloud with sufficient knowledge of how to support them? Knowing the answer to these questions is critical if you want to use your Dojo to drive toward specific outcomes.

Starting from a sound strategy is key. It provides the following benefits:

• Teams understand the value of why they should participate in the Dojo

• The skills and topics taught in the Dojo are well-defined

• Growing coaches is easier because coaches can focus on specific skills

• Measuring the success and impact of the Dojo is easier since you can  measure outcomes against the strategy

The strategy your Dojo supports should be clear and easily stated. If the strategy is nebulous or complicated, your Dojo will struggle to provide value to the rest of the organization.

What strategy is your Dojo supporting?

Be on the lookout for our next topic in this series: why Dojos need to address the entire value stream.

Reflections & Learnings from Continuous Learning Event

Last week, we co-hosted an event with Mark Graban. We had the pleasure of spending two days with forty individuals focused on looking at continuous learning and improvement.

We previously posted on why we were excited about the event. Now that the event has passed, we wanted to share our "A-ha" moments.

Seeing and Hearing the Andon Cord

The Toyota Andon Cord is famous in the lean and agile communities. The Andon cord is a cord that anyone on the manufacturing line can pull to "stop the line" to correct a production problem. Two powerful realizations surprised us about the Andon cord. First, the frequency of the pulls was much higher than we expected. Second, the sound played when somebody pulls the cord is cheerful music! We were expecting a sound more like an alarm. The cheerful sound celebrates finding and fixing an issue before it becomes costly.

I compare this to what we commonly experience in software development. When we encounter an issue we tend to focus more on keeping delivery going and not on "stopping the line." (How many of you work with teams where the whole team stops current work to address a build failure?) We want to meet deadlines and don‘t discuss small friction in real time. The easiest time to fix this friction is when it's happening. What if we celebrated discovering issues earlier as a way of fostering continual improvement? Dojos focus on this through early single piece flow.

Suppliers on Site

We imagine the numbers are public since we were on a public tour of Toyota, but it was astonishing that of the workforce at the plant we visited, over half were from suppliers. Yes, they were there supplying (manufacturing and fixing) the parts that Toyota uses. Read that again. Over half of the workforce on site were suppliers. With this setup, parts are literally arriving "Just in Time." There is no risk of shipping delays from suppliers, no need for heavyweight logistical inventory management, and no need for constant negotiations over contracts devoid of the context of work in progress. And it makes Toyota have a beautiful symphony of construction.

In our world, how possible is it to have suppliers be part of our construction? The organizations we work with have an advantage over Toyota. Most times the  "suppliers" are employees. Yet we often confuse activity for progress. Everyone is moving as fast as they can. They must stay busy! This comes at the cost of delivering value. This is an easily solvable problem. Dojos help magnify this constraint.

Experimenting, Learning, and Safety

We also visited Garrison Brothers distillery to see how they apply continuous learning and improvement. We had a wonderful conversation with their head distiller. He shared stories of how they experiment with improving their product and develop new variations (including a honey infused bourbon).

He shared one example of a bourbon he wanted to make with a certain taste profile - his experiment. Guess what? It didn’t work. He had to tell the owner that a large batch ($50K in potential sales) wasn’t up to snuff and that it would have to be set aside to evaporate. Which they did. Because it was the right thing to do. They don't release products they are not proud of.

And they continue to experiment in an environment where it's safe to be wrong. Otherwise, they know they won't get better.

A beautiful story for a small distillery.

You Can't Teach Everything...and Shouldn't

Our group also asked the head distiller what he teaches the other distillers and what kind of consistency he expects from them in the distillation process.

His answer was more profound than our words will capture, but let us share the key thoughts.

He can teach the process - the ratios, the mixes, the times, the temperatures - but he can’t teach the taste. And he doesn’t want to because everyone’s pallet is different. Some character comes from the ‘white dog’ (raw whiskey), but the majority of the flavor comes from the aging process. He learns the characteristics of how the other distillers' bourbon ages. This knowledge can help him later on if he is blending barrels. But - he isn’t looking to change anyone’s taste - or teach them his taste.

That comment tied directly into the session that we led on Thursday around tacit vs explicit knowledge. (We'll explore this topic in more depth in an upcoming post.) Dojos acknowledge both types of knowledge and coaches work to foster knowledge creation both tacitly and explicitly. 

AND MANY MORE…

Both tours were great experiences. The attendees also shared their stories and knowledge. The topics were numerous - lack of stress even with low tak time, the visible enjoyment in work, lack of visible goals while still delivering quality with volunteers, differences between volunteering to work and being "driven" to work, respect for people… and so much more. And sharing the experience with the diverse group of attendees elevated the quality of the event.

If you joined us, please share some of your key learnings. If you missed out, we are already thinking about the next event that combines experiences with sharing and learning from each other. Stay tuned!

Mark’s Thoughts On The Event

Mark Graban also shared his thoughts of the event on his blog:

https://www.leanblog.org/2018/10/highlights-from-our-lean-event-tour-at-toyota/

Continuous Learning - Talking, Improving, and Learning across Domains

We’re excited to be partnering with Mark Graban in presenting a new “conference” on creating learning organizations. Mark is and is an engaging speaker. We look forward to learning from him ourselves (see his book Measures of Success).

The event is on September 26th & 27th. As we write this, there are only 5 seats left. More information and registration details are available here.

Why are we so excited about this conference?

It’s about Learning & Improvement

We’ve been working in the agile, lean, and DevOps spaces for years. Over the last five years, we’ve found our niche helping organizations embrace learning and continuous improvement. This conference is about that topic. It’s not about the process or technology. Instead, it focuses on these questions - how do we create spaces where people are engaged and continuously improving? How do we make it so that learning isn't a special one-off event? How do we make people’s lives better (isn't that what it is all about)?

The Conference is Hands On and Experiential

The conference is different by design. Mark, Dion, and Joel will have a small amount of content to deliver. Attendees will be experiencing events and sharing insights with each other rather than attending prepared sessions typical of most conferences.

We’re starting with a tour of a Toyota plant and seeing how improvement is a way of life there. We'll then visit a craft whiskey distillery that applies lean startup principles and we’ll experience a real world “beer game”. That’s just the first day!

The second day has more experience-based learning with Mark, Dion, and Joel leading short sessions in the morning. We’ll then have open space where we learn and share with each other.

This conference will be different.

Diverse Attendees

Principles of learning and improvement are not isolated to any one domain. The attendees who’ve already registered come from a variety of domains (financial, legal, government, retail, and IT). The diverse domains, backgrounds, and experiences people are bringing will enhance this experience even further.

The Concept Excites People

The idea for this conference started months ago with the three of us asking “Wouldn't something like this be cool”? But ideas are cheap, and you don't really know if they are interesting until you get feedback. So, we decided to try this conference out as a little experiment. We started by asking people if the concept was interesting. We quickly heard a resounding “Yes''. That was nice, but once it came to commit the time and money would there still be that level of interest? There was. And the feedback we have received tells us people are excited.

These are some of the many reasons we hope to see you there. Join us September 26th & 27th. Come learn and experience with us and Mark! Register here

See you in San Antonio!

Dion & Joel

Growing Coaches in the Dojo

Skilled coaches are critical to the success of any Dojo. The specific skills needed will vary. A Dojo focused on DevOps requires one set of coaching skills. A Dojo focused on agile and product discovery capabilities needs different coaching skills.

Staffing a Dojo with coaches can be a challenge. There’s an abundance of agile coaches but many of them know only process. The Dojo can be an effective place for improving development processes. But, the investment required to run a Dojo should return a bigger payoff. Most organizations want to improve engineering and product discovery practices. Coaches who can help teams improve these skills are hard to find. You may need to hire skilled engineers and product managers and develop their coaching skills.

Here are four ways we help grow coaching skills in the Dojo.

Observing, Pairing, Leading

We onboard new coaches following an “observe, pair, lead” progression towards competency.

As soon as possible, new coaches observe other coaches working with teams. During breaks and at the end of coaching sessions we have debriefing conversations. The coach leading the session will ask the observing coach—what did you see, hear, and observe? What might you have done differently if you were guiding the session? What questions do you have about the way I coached? The coach guiding the session will then explain what they saw, heard, and observed. They explain the choices they made in coaching the team.

After a few “observing” sessions, new coaches pair with an experienced coach. They might take turns guiding the team or the new coach may take the lead. The more experienced coach focuses on keeping things on track. She will step in if the new coach “gets stuck” or has a question. The pair of coaches will continue having debriefing conversations during breaks and at the end of the session.

Once the new coach has paired on a specific practice a few times, they will then lead the practice with an experienced coach observing. The experienced coach may jump in if the coach leading the session starts struggling. However, they are there mainly to observe and offer guidance during the debriefing conversations.

This is an effective way of onboarding new coaches. It’s impossible to replicate the experiential learning that comes from working with teams through training.

Running Simulations

We help coaches grow their skills by practicing in simulated scenarios.

In a simulation, coaches and Dojo staff will roleplay various team roles. Those include manager, developer, testers, designer, product owner, customer, etc. A coach will guide the "team" through a session while the rest of us play out behaviors the coach might meet. This is useful for practicing problematic behaviors - a manager who wants to control everything the team does, a product owner who insists on delivery of a feature faster than the delivery team’s estimate, or an entire team with a weak and nebulous vision for their product.

We use the debriefing conversations described above to review the simulation once it’s finished. We talk through what we saw, heard, and observed, what were effective ways of guiding the group (or ineffective), and other ideas for handling what came up during the simulation.

Practicing Teaching

One of the most effective tools we use for developing coaches is having the coaches teach in practice training sessions.

For example, before teams come into the Dojo a coach leads them through a Dojo Chartering session. We teach coaches how to do Chartering and then they observe a few Chartering sessions. The next step in developing their skills may be having them teach the rest of the coaches how to do Chartering.

This is also an effective way for the coaches to stay in sync.

Reflecting on the Role of Coaching

Periodically, the coaches will get together and reflect on what it means to be a coach.

In a coaching workshop we gave recently, we discussed the various roles coaches play in the Dojo. We had an interesting discussion about the timing and triggers for moving from one role to another. For example, when do you move from a teaching role to more of a partner role as teams are adopting new skills?

One of the new coaches started the discussion by asking if there were standard coaching roles and how you knew when to adopt each role. You could use questions coaches have as opportunities to bring the coaching group together to discuss the questions, share information with each other, and develop coaching skills.

Skilled coaches are critical to the success of any Dojo. Experimenting with new ways of developing coaching skills is part of running a Dojo. Just as we ask teams to adopt a culture of continuous learning and experimentation we do the same inside the Dojo. We’ll continue to share new techniques for developing coaches. What are the ways you develop coaching skills?

Six Reasons Why We Do Two-and-a-Half-Day Sprints in the Dojo

When we’re explaining the Dojo model to people who are unfamiliar with it, we often get a lot of head nods and other indicators that show people understand the model. They see how it’s different from other approaches to learning. Sadly, most of us have had experience with inadequate training programs. Many of us have worked in organizations who’ve sent hundreds of people through certification courses only to find the desired outcome doesn’t stick or never materializes in the first place. For us, the Dojo model just “makes sense”.

One aspect of the model that does raise eyebrows is the two-and-a-half-day sprints. People often question – “Why two and a half days?”, “Isn’t that too short?”, and “What are the advantages of sprints that long?” (Some even go so far as to say “That can’t possibly work.”)

Here are six reasons why two-and-a-half-day sprints are effective in the Dojo.

1. Repetition Fosters Learning

At our current client, Dojo challenges are six weeks long. Setting the sprint length to two and a half days results in teams completing twelve sprints during their challenge. If we set the sprint length to be two weeks, a common sprint length, the team would only complete three sprints during their challenge.

The two-and-a-half-day sprint format is a vehicle for us to foster learning through repetition. Teams using Scrum, for example, are getting frequent practice with the four core Scrum events (Sprint Planning, Daily Scrum, Sprint Review, and Sprint Retrospective). We’ve found that this interval of repetition helps the practices stick. Teams get better at them than they do if they practice them less frequently.

If a team is working on learning how to improve their CD pipeline, deliver microservices, or deploy to the cloud they also have opportunities for repetitive practice resulting from the two-and-a-half-day format.

Repetitive practice helps new learning stick.

2. Practice Getting to Done

Many of the teams we work with who are presumably already “doing agile” struggle with finishing stories by the end of the sprint. Often times Sprint Planning doesn’t produce a realistic plan, teams don’t allow enough time for testing, or they’re simply too optimistic about how much they can accomplish. All of these things contribute to teams carrying stories over stories from sprint to sprint.   

Ron Jeffries advocates for shorter sprints when teams are struggling to finish stories by the end of the sprint.

If you can’t get everything “done” (see above) at your current Sprint length, consider shortening your Sprint. If it’s two weeks, try one week. If you can’t get anything done in one week, try one day.

We’ve seen significant improvements in teams’ abilities to plan and execute a set of stories within a sprint over their six-week Dojo challenges. They come in acknowledging they often don’t complete stories every sprint and leave with the ability to “get to done” on a regular cadence. Often, it takes multiple sprints in the Dojo to make this change but we’ve got twelve opportunities to improve.

3. Teams (Finally) Learn How to Break Stories Down

“Instead of how big, we should be asking is it too big?” – David Hussman.

When it comes to planning sprints in the Dojo, we think it’s more valuable to ask “Can we get this story done in two and a half days?” than it is to ask “How big is this story?”.

Our friend, David Hussman, first got us moving away from focusing on “estimating” to focusing on “sizing”. Put another way, instead of a team getting bogged down in estimation meetings, the team focuses on breaking down stories until they can be done in two and a half days. For us, learning how to do this is significantly more important than learning how to get better at estimating.

Even when teams are delivering using Kanban, or they’re advocates of “no estimates” and use other techniques for predicting delivery timelines, breaking down stories so they can be delivered incrementally is an essential skill.

(For approaches to breaking down stories see this post.)

4. Learning is Supported by a Margin of Safety

There’s a lot of buzz these days about providing safe environments for teams to fail. Modern Agile holds safety as one of its core tenets. DevOps culture teaches us that when we have a problem with our systems we shouldn’t blame individuals, but instead look at the system itself.

Too often, teams are asked to learn while delivering. The message is clear – you have to learn agile/lean methods, or new technology, or new engineering practices – but it can’t impact the delivery timeline. Sadly, this often happens while delivering important or even mission-critical products. This is hardly an environment conducive to learning.

Failure is critical for learning. We learn from our failures and the stories that get created and retold about those failures. Do you remember any stories off the top of your head about a system that was designed so well there were never any outages? You might, but we’re guessing you know a lot more stories about system failures, what caused those outages, and what you learned from them.

In the Dojo, it’s critical to have an environment where failure is not only tolerated but where failure is understood to be a natural part of the cycle of knowledge creation. Without an opportunity for failure, learning becomes stifled. As part of the onboarding process we make sure stakeholders understand that delivery will slow down somewhat to make space for small failures and to make the Dojo a safe place for learning.

Small failures are the key. The two-and-a-half-day sprints provide a safe environment for learning. Failures at any level, whether they are product or technology related, can be corrected quickly. An investment of two-and-a-half-days is usually not very costly to the team.

At the same time, we want to provide opportunities for feedback, reflection on the part of the learner, and immersion back in the work in a timely manner in an environment where the learner feels safe. Two-and-a-half-day sprints help create this feeling of safety.

5. Frequent Retrospectives Lead to the Proper Mindset

Another common problem we run into with teams is they’ve stopped doing retrospectives and as a result have stopped focusing on continuous improvement. When we ask them why we’re told the retrospectives don’t have any value because they keep repeating the same problems over and over.

In these cases, we ask the teams to resume doing retrospectives for the duration of their Dojo challenge. During the retrospective, we ask the team to pick one item to work on improving over the next two-and-a-half days. In cases where the team identifies impediments that are largely beyond their control we ask them to focus on making one improvement within their control.

Sometimes the same idea for improvement will be carried over and continues to be worked on from one sprint to the next. But our experience in the Dojo has been that teams adopt a mindset of continuous improvement more easily than when they’re working within a two-week sprint cadence. The frequency of inspection and coming up with ideas for improvement helps the principle of continuous improvement stick.

We’re not saying every team should adopt retrospectives more frequently than on two-week sprint boundaries. What we are saying is that establishing the mindset is what’s important here.

6. There are More Opportunities to Ask “What is the next best investment in learning?”

An essential skill for Dojo coaches is knowing when to play the role of teacher and when to play the role of a guide who reflects behavior and ideas back to the team. The two-and-a-half-day sprint format provides a natural cadence to move back and forth between those roles.

One question we frequently pose to teams during Sprint Planning is “What’s our next best investment in learning?” The answers vary depending on the teams’ unique challenge goals.

In some cases, teams are focused on learning design thinking and product discovery so the discussion could be about coming up with ways to test a hypothesis about our product. Or, the discussion could be about pivoting to a new product idea entirely.

Other teams might have challenges focused more on learning new technology or engineering practices. For those teams the discussion might center around what stories to complete that will foster the desired learning.

Again, the frequency of stopping to reflect on this question every two and a half days fosters a mindset of thinking about product development and learning as a series of investments.

The six reasons are related and reinforce each other. Together, they help create an environment that supports learning and knowledge creation.

If you have experience with two-and-a-half-day sprints or if you have questions about running two-and-a-half-day sprints we’d love to hear from you.

Measuring Impact In The Dojo

Last month at Agile Day Chicago, I (Joel) had the pleasure of listening to Mark Graban speak about separating signal from noise in our measurements. Mark referenced Process Behavior Charts, a technique described in the book Understanding Variation: The Key to Managing Chaos by Donald J. Wheeler. This simple tool helps us look at metrics over time and understand the difference between naturally occurring variations and signals, or variation in the metrics representing real changes. Wheeler calls both of these (signal and noise) “The Voice of the Process,” with the key being able to distinguish between the two. Signals can be indicators that a desired change is manifesting, or they can be indicators that something is wrong and requires further investigation.

We immediately saw the value in being able to separate signal from noise when evaluating the types of metrics we’re capturing in the Dojo that we talked about in our last post. We both grabbed copies of the book, devoured it quickly, and started brainstorming on applications for Process Behavior charts.

Let's look at an example of how to use Process Behavior Charts in the Dojo.

BEFORE YOU START

This may sound obvious, but before you start any measurement think about the questions you want to answer or the decisions you want to make with the data you’ll collect.

In the Dojo, we help teams shift from a project to product mindset. We focus on delivering specific outcomes, not simply more features . When delivering a new feature the obvious question is – did the feature have the desired outcome?

THE SCENARIO

Imagine yourself in this type of situation…

We’re working with a team and we’re helping them move from a project model to a product model. In the past, the team cranked out features based on stakeholders’ wishes and success was simply judged on whether the features were delivered or not. We’re helping the team shift to judging success on whether outcomes are achieved or not.

We’re also working with the stakeholders and there’s resistance to moving to a product model because there’s fear around empowering the teams to make product decisions. New features are already queued up for delivery. Before we give the team more ownership of the product, the stakeholders want delivery of some of the features in the queue.

We can use this as a coaching opportunity.

The stakeholders believe the next feature in the queue will lead to more sales - more conversions of customers. The team delivers the feature. Now we need to see if we achieved the desired outcome.

Our first step is to establish a baseline using historical data. Luckily, we’re already capturing conversion rates and for the 10 days prior to the introduction of the new feature the numbers look like this:

Sample Data Set

Then we look at the data for the next 10 days. On Day 11, we have 14 conversions. Success, right? But on day 12, we have 4 conversions. Certain failure?

Here’s the full set of data for the next 10 days:

Sample Data Set, Next 10 Days After Feature Introduced. Did It Matter?

Overall, it looks better, right? The average number of conversions have increased from 6.1 to 7.9. The stakeholders who pushed for the new feature shout “success!”

PROCESS BEHAVIOR CHARTS

Given a system that is reasonably stable, a Process Behavior Chart shows you what values the system will produce without interference. In our case, that means what values we can expect without introducing the new feature. Let's create a process behavior chart for our example and see if our new feature made a difference.

First Step - Chart Your Data In A Time Series and Mark the Average

Plotting Daily Conversions, Average Marked In Dotted Red Line

What does this show us? Well, not much. Roughly half of our points are below average and half are above average (some might call that the definition of average).

Second Step - Calculate the Moving Range Average

Our next step is to calculate the average change from day to day. Our day to day changes would be 2, 4, 4, 2, 6, 3, 2, 5, 3 for an average change of 3.4. All this means is that on average, we see a change in the number of conversions day to day of about 3. If we were to plot the number of changes in conversion day to day, we would see roughly half above and half below - again, the definition of average.

Third Step - Calculate The Upper And Lower Bounds

To calculate the upper and lower bounds, you take the moving range average and multiply it by 2.66. Why 2.66? Great question - and it is well covered in Don Wheeler's book. In brief, you could calculate out the standard deviation and look at 3 sigma, but 2.66 is faster, easier to remember, and ultimately tells the same story.

We take our moving range average of 3.4 and multiply it by 2.66 giving us 9.044. What does this number mean? It means that with normal variance (the Voice of the Process), we can expect conversions to fluctuate 9.044 above or below our average number of conversions which was 6.

To put it more clearly, without any intervention or new features added, we should expect between 0 and 15 conversions per day - and that would be completely normal.

Let's visualize this data. We add our upper and lower bounds to our chart for our first 10 days. It now looks like this:

Data With Process Control Limits Applied. UPC - Upper Process Control, LPC - Lower Process Control. NOTE - Since the LPC is actually -3, we use 0 since a negative is not possible

Fourth Step - Introduce Change & Continue To Measure

We have established the upper and lower bounds of what we can expect to happen. We know that after the feature was introduced, our conversion numbers looked better. Remember, the average went up almost 30% (from 6.1 to 7.9) - so that is success, right?

We extend our chart and look to see if the change actually made a difference.

Conversion Chart With Upper And Lower Process Controls. Note - Average, UPC, and LPC Were Not Updated With New Data Points To Prove The Next 10 Days Fell Within Previous Dataset Limits

Our average for the next 10 days was higher, but looking at what we could normally expect the system to produce, all of the conversions were within the expected range. In essence, the feature we delivered did not create a meaningful impact to our conversions.

Note, we’re not saying that nothing could be learned from delivering the new feature. The point we’re making is that prior to delivering the feature we assumed it would lead to an increase in conversions. Using a Process Behavior Chart we were able to show our assumption was invalid.

Now we can continue the conversation with the stakeholders around empowering the team to improve the product. Maybe now they'll be more open to listening to what the team thinks will lead to an increase in conversions.

MORE USES FOR PROCESS Behavior CHARTS

We like using this visual display of data to help us concretely answer questions focused on whether or not our actions are leading to the intended outcomes. For example, we are experimenting with Process Behavior Charts to measure the impact of teaching new engineering and DevOps practices in the Dojo.

REMEMBER - MEASURE IMPACTS to the WHOLE Value Stream

Process Behavior Charts can be powerful, but they require that you ask the right questions, collect the right data, AND and take the right perspective. Using a Process Behavor Chart to prove a change is beneficial to one part of the value stream (e.g., the “Dev” group) while not taking into consideration the impact to another group (e.g., the “Ops” group) would be missing the point. Consider the complete value stream when you are looking at these charts.

FURTHER READING

For more information on these charts, as well as the math behind them and what other trends in data are significant, we recommend the following:

• Understanding Variation - The Key To Managing Chaos; Don Wheeler

• Lean Blog - Mark Graban, in particular this post on home runs in the World Series

• Process Behavior Charts (also called Shewhart Charts) – this article talks about various patterns that are statistically significant

If you found this helpful and you adopt Process Behavior Charts, please let us know how you are using them and what you are discovering.

Dojo Metrics - Moving From What is Easy to capture to What Matters

A fair question to ask when starting a Dojo (or any initiative for that matter) is “how do we know this is working?” Invariably, right on the heels of that question somebody always brings up the idea of capturing metrics. Then they turn to us and say “What are the right metrics for the Dojo?”.

The best metrics provide insights that help us take action to improve the current situation. In the case of a new initiative like a Dojo, that action might be making a decision to continue the initiative, modify it, or end it.

Sadly, metrics are often arbitrary or they tell an incomplete story. Single metrics fail to capture the interplay and tradeoffs between different metrics. We’ve heard many stories of how organizations optimizing for one metric created detrimental results overall. (We’re looking at you, capacity utilization.)

how do we measure the effectiveness of the Dojo?

The primary goal of the Dojo is to foster learning. We need to measure the effectiveness of that learning and ultimately, we need to measure the economic impact that learning has on the organization. But it’s not learning at any cost. We’re aligned with Don Reinertsen on this point.

In product development, neither failure, nor success, nor knowledge creation, nor learning is intrinsically good. In product development our measure of “goodness” is economic: does the activity help us make money? In product development we create value by generating valuable information efficiently. Of course, it is true that success and failure affect the efficiency with which we generate information, but in a more complex way than you may realize. It is also true that learning and knowledge sometimes have economic value; but this value does not arise simply because learning and knowledge are intrinsically “good.” Creating information, resolving uncertainty, and generating new learning only improve economic outcomes when cost of creating this learning is less than its benefit."

Don Reinertsen - "The Four Impostors: Success, Failure, Knowledge Creation, and Learning"

Reinertsen stresses the need to generate information efficiently. This is easy to understand when thinking in terms of generating information that helps you make decisions about your product. For example, it’s a fairly straightforward exercise to determine the costs for generating information by running low-fi, paper prototype tests that answer the question “should we include this feature or not?”

It’s also easy to understand how you might measure the effectiveness of knowledge creation when helping teams make improvements on their continuous delivery pipelines. We can calculate the cost of learning DevOps practices and compare that to expenses saved by automating manual processes.

What’s not as easy to understand is how to measure the impact of learning cloud native architecture or micro services - or something even more nebulous, like product thinking and the impact of learning a design practice like personas.

We would expect the impact of these learnings to result in lower development costs, decreased cycle times, and increased revenues resulting from better market fit for our products. But – there is a high degree of uncertainty as to the level of impact these learnings are going to have on the organization. (Again, hat tip to Don Reinertsen. His post about looking at the economics of technical debt influences our thinking here.)

In addition, during a team’s tenure in the Dojo it’s quite probable that their productivity will decrease as the team is creating new knowledge and incorporating new practices. The team's investment in learning carries a cost.

Ultimately, we need to understand the impact the Dojo has on lifecycle profits. That impact will often occur after a team has left the Dojo.

We have started organizing metrics in the Dojo into three groups. Our goal is to help orient stakeholders, leaders, and teams around what actions these metrics will help them take. We also want to help them understand the level of effort required to collect the metrics and the timeframes in which they will be available.

Three Categories of Metrics for the Dojo

Simple To Capture - Organizational Reach

These metrics simply show the amount of “touch” the Dojo has.

Examples include:

• Number of teams going through the Dojo

• Total number of attendees

• Number of Programs / Portfolios touched

Astute readers may critically call these “vanity metrics” and they would not be wrong. These metrics do not equate to impact. They don’t help us answer the questions “Were the right teams involved?”, “Did the amount of learning that happened justify the investment?”, or “How much learning stuck?”

However, these metrics are simple to collect and can be used as leading indicators once we have metrics on the economic impact the Dojo has on teams. For many organizations, these metrics are important because they imply value as the Dojo is being bootstrapped, even though they don't prove it. They are metrics everyone is comfortable with.

Harder To Capture – Directional/Team Based Improvements

Metrics in this category are more important than the previous category in the sense that these metrics look at the directional impact of learning in the Dojo and how that learning is impacting teams.

Examples include:

• Number of automated tests

• SQALE code quality index

• Percentage reduction in defects

• Cycle time reduction to deliver a product increment

• Velocity / Story count (with the obvious caveat that these can be easily gamed)

Again, these metrics are far from perfect. The testing related metrics do not prove the right tests were written (or the right code for that matter). Metrics showing products were built faster don’t shed any light on whether those products should have been built in the first place (what if nobody buys them?).

What these metrics do show is the incorporation of product delivery practices that are being taught in the Dojo - practices that our experience and the experiences of other organizations have shown to have a positive impact on lifecycle profits. These metrics can be collected with agile project management software, SonarQube, Hygieia, or other comparable tools.

When we use these types of metrics we need to have a baseline. It’s helpful to have data for teams for two to three months prior to when they enter the Dojo. We don’t always have this baseline, however, and in some cases the best we can do during a team’s tenure in the Dojo is help them establish the baseline. Obviously, we want to track these metrics for teams after they’ve left the Dojo to see how well new practices are sticking.

Difficult To Capture – Impact/Economic Improvements

Metrics in this group are challenging - not only to collect but also because using them to drive action challenges the way many organizations work. These are the metrics that force us to look at the question “Is this initiative having a positive economic impact on the organization?”

Examples include:

• Increase in sales conversion

• Cycle time reduction for a delivery with impact (not just delivery, but a delivery that mattered)

• Systematic cost reductions (not silo optimizations that may have detrimental effects in other areas)

• Savings resulting from killing bad product ideas early in the discovery/delivery cycle

Metrics like these can prove initiatives like the Dojo are having a positive impact on lifecycle profits. These metrics will be substantially harder to collect. We need to collect data for a much longer period of time. We need to align with the finance department in our organizations. And, we need whole product communities aligned around a shared understanding of what successful outcomes look like. In addition, we need to understand how to separate real signals of change from noise. (This post has more on that topic.)

Ultimately, this last category of metrics is what matters. This is where the Dojo shines. We work with teams to teach the practices, thinking, and communication strategies that will have an impact on lifecycle profits.

This is an ongoing area of improvement for us. This is what we are currently practicing. These categories of metrics are helping foster conversations, understanding of what knowledge individual metrics can provide, and the value of investing in the Dojo.

Empowering & Enabling Responsibility

Empowering teams is a topic the DevOps and Agile communities frequently talk about. But it is easier said than done. Here is one simple approach to empowering teams you can do right now.

But first a little background...

Responsibility vs Accountability

We frequently work with leaders who are new to DevOps. We ask them straightforward questions such as - “Why are you interested in DevOps?” We often hear answers along the lines of “We want to make teams more accountable for their actions.” When we dig in a bit further, we learn this is not actually what they mean. What they are trying to say is that they want to empower teams with responsibility for their own work.

What’s the difference between accountability and responsibility? Look up the definitions and you might find yourself going back and forth between them endlessly. It’s as if you are trying to navigate through an M.C. Escher drawing.

For us, being accountable means you’re answerable for something. In its worse form, a leader makes the team answerable for an outcome that is beyond their control to influence. It can come from command-and-control style of leadership. If you’ve ever been held accountable for meeting a goal without the ability to influence how to accomplish it, you know what we mean.

Being responsible means you have the competency, authority, and the correct understanding of the desired outcome so that you can deliver that outcome as you see fit.

When we discuss this topic with leadership, we often use Christopher Avery's work around the responsibility process. It’s an effective conversation starter that helps shift the focus away from accountability toward responsibility.

With that context out of the way, let's look at the one simple thing you can do to empower teams.

Decision Rings

The image above is something you can refer to with your leaders, coaches, and teams. In this example, the center circle is the team, the next outer circle is their manager(s), the next outer circle is domain experts from the business, and the outermost circle is some executive leadership.

The rings represent different levels in an organization. We use them to help frame discussions when asking “Who can make this decision?” The decision making structure in your organization may be different.  The decision making structure may also change depending upon the question at hand.

Let's look at an example. Imagine your product team is working on the goal of increasing sales by delivering promotional content in banner ads. After starting to work on that goal, the team uncovers a better way of improving sales with promotions that has nothing to do with banner ads. Who makes the decision on what to deliver?

First, we make sure we agree where that decision is made today – and we’re not talking about where it’s “officially” made according to some policy. We’re talking about where it’s actually made given the messy, often political nature of decision making within organizations.

Next, we ask “what information would need to be made available or what competency would need to be developed to move that decision inward?” We can then get to work making any necessary changes. Or, we can move the decision-making authority inward if no changes are necessary.

In the above example, we might say “Right now, the business team needs to make that decision. For the product team to be able to make that decision, we would need to provide them more information on the organization’s strategic goals.”

We have just started exploring using this technique in the dojos and it is driving productive conversations. It is not a silver bullet, but it’s a nice simple visual that starts conversations driving empowerment and growing responsibility.

Try it out. Let us know how it works for you.

Technical Debt - Learning in (and With!) the Face of Debt

Technical Debt has a few definitions ranging from 'the previous person's bad code' to 'the shortcuts taken to hit a deadline' to my favorite - Technical Debt is 'the gap in the code between what we knew when we started building our product and what we know now'.

It's easy to look at a codebase with no automated tests, high cyclomatic complexity, and a manual build process and say 'look - Technical Debt'. It is more challenging to work with a team implementing new features where previous technical choices are making it costly to improve the product. In other words, the code is not maneuverable (Nygard).

These situations happen frequently in the dojo. Let's look at a couple examples.

THE UNTESTED 20-YEAR OLD MONOLITH

Imagine a 20+ year old code base, modified every year by the lowest cost outsourcing firm the organization could find. Now imagine you inherited that codebase with such delights as 900 line constructors that directly connect to databases, establishing pools and locks.  An agile coach could (and had) come into that team and said “You need to get to 70% code coverage for your unit tests”. The team laughed because crying was too obvious.

When teams are in this kind of bind, entering the dojo isn't just about learning how to write unit tests. The team already knew how to write unit tests and was eager to do so. The challenge was finding a strategy to attack the beast. And when deadlines were hitting, it was easier to keep adding code and crossing your fingers.

We did a few things to address the problem. First was getting the team together to identify and discuss quick wins they all wanted to knock out. We did some light brainstorming and affinity mapping which led to a new set of lightweight design changes - all in about an hour or two - and the team had a nice shortlist of changes to start working on. The list would take a week to complete and would be worked on concurrently with new work. It was a small set of changes, but it gave the team a few wins to build upon.

Next, we came up with strategies for attacking the technical debt while working on new stories. One strategy we used quite a bit was using a test-driven approach, focusing on the design of the tests themselves. We’d have a quick discussion around the tests – “where should the responsibility of the tests lie”? With that in mind we’d ask – “what is the delta between where those responsibilities should be and where they'd be given our current design? Can we get there now?”

For some of the stories, we could implement our desired changes immediately. For others, we couldn't. In those cases, we kept a visual list of the spots we wanted to work on. We also created higher-level tests that we could implement immediately with the knowledge that we would remove those tests once we had implemented lower-level tests.

THE MULTI-TEAM MATRIXED DESIGN

Another team was similar in that they had a lack of testing in place, but their challenges were slightly different. This team was formed as an organization’s first attempt at creating 'product teams.' The codebase was gnarly and had contributions from multiple teams over multiple years without much knowledge sharing.

Enter the dojo. The team was doing a little product exploration around a new feature using impact mapping. They came up with a few ideas that would have more impact for the customer than the new feature as it was originally defined. Excellent, right? Not quite. While these ideas were great and the team agreed they were better approaches to the problem, the technology would not allow the better ideas to be built. The way the product was designed made it difficult for required data to be accessed when needed, resulting in unnecessary extra steps. Streamlining a separate process would break downstream services. And so on...

Teaching ideas like parallel change can help in this space. But the real value here came from the whole team learning the cost of technology decisions together, and working together to learn approaches to attack technical debt.

When items like this arise, first and foremost it is still a good thing. Now we can start quantifying design problems and communicating the opportunity cost of technical debt. In this example, we could calculate the cost of the options we could not deliver. The learning expanded beyond the team and became organizational learning.

And in the dojo – we don’t simply teach approaches for tackling technical debt and send the on their way – we help the teams work through it.

What have you done to help teams with technical debt?