Scrum – the hard parts 2: sprint harder

In my previous post, I provided an overview of the Scrum framework and some of the key benefits and challenges teams face when implementing it. As a full-stack developer who has worked on many Scrum teams over the years, I‘ve learned that the path to Scrum mastery is lined with obstacles. It takes dedication and perseverance to achieve a steady cadence of successful sprints.

In this post, I‘ll share more of my experiences and lessons learned sprinting on full-stack Scrum teams. We‘ll examine some of the most common impediments that cause sprints to stumble, and explore proven strategies to avoid them. I‘ll dive into technical practices that can help your team reach peak performance. By the end, you‘ll walk away with an upgraded toolbelt to help your team continuously improve and deliver more value to customers.

Scrum by the numbers

Before we jump into the details, let‘s take a quick look at some statistics that underscore the challenges and opportunities of Scrum:

• According to the 2020 Scrum Master Trends Report by Scrum.org, 81% of Scrum Masters say their teams operate at or above 50% capacity, but only 55% say their teams reliably deliver working software at the end of each sprint.

• The 13th Annual State of Agile Report by CollabNet VersionOne found that the top three challenges experienced by agile teams are culture clashes, general organizational resistance to change, and inadequate management support and sponsorship.

• On a more positive note, the same report found that 52% of respondents say most or all of their teams are using agile practices. Clearly, Scrum and agile are here to stay.

Tales from the sprint trenches

One of the hardest-fought battles in my journey as a full-stack developer practicing Scrum has been overcoming the tendency to overcommit in sprints. It‘s an easy trap to fall into, especially when you‘re on a high-performing team hungry to deliver as much value as possible.

In one memorable sprint, my team had a stretch goal of delivering a major new feature that would wow our customers. We had a clear product vision, well-groomed user stories, and all the technical pieces in place to bring it to life. What we underestimated was the sheer volume of edge cases and unanticipated complexities we would encounter once we started building.

Midway through the sprint, it became painfully clear that we had bitten off way more than we could chew. We started burning the midnight oil, cutting corners on code quality, and deferring testing just to have a shot at finishing all our stories. Team morale tanked and the cracks started showing in our daily stand-ups.

In the end, we limped across the finish line with a working feature, but the victory was hollow. The code was riddled with technical debt, the team was exhausted, and the product increment was not something we were proud to demo to stakeholders. We had fallen victim to the siren song of overcommitment, and it came at a high price.

The lesson I learned from that experience is that sustainable sprint success requires a commitment to ruthless scope management and relentless splitting of user stories. It‘s far better to deliver a small set of high-quality, well-tested stories than to take on an ambitious sprint backlog that leaves the team scrambling to finish.

As Ron Jeffries, one of the creators of Extreme Programming, eloquently put it: "Stories get easier to implement when you split them until they are ridiculously easy. Every story has to be completed in a short fraction of a sprint." Take it from someone who learned this lesson the hard way – embrace the art of splitting stories and defend your sprint scope like your life depends on it.

Technical practices for Scrum success

In my experience, the Scrum teams that consistently deliver the most value are those that invest in solid technical practices. Scrum provides a great framework for managing the flow of work, but it‘s not prescriptive about engineering practices. Adopting a core set of techniques rooted in Extreme Programming can help your team achieve a higher level of performance.

Continuous integration and test automation

Continuous integration (CI) is the practice of merging code changes to a shared mainline branch frequently, usually several times a day. It enables teams to detect integration issues earlier and reduces the risk of nasty merge conflicts. Automated tests are run against every commit to provide rapid feedback on code quality.

Research has shown that teams that adopt CI produce higher-quality code. A study published in the journal of Empirical Software Engineering found that open source projects using CI fixed bugs 16% faster and released twice as often compared to projects not using CI. Another case study from Microsoft reported that teams practicing CI had a 90% reduction in build and test time.

Setting up a robust CI pipeline takes effort, but it‘s well worth the investment. Some popular CI tools include Jenkins, CircleCI, and Travis CI. Couple your CI process with a suite of automated unit tests, integration tests, and functional UI tests to catch regressions quickly. Aim for test coverage of at least 80%.

Pair programming and code reviews

Another powerful practice for keeping code quality high is pair programming. It‘s a technique where two developers work together at a single workstation, taking turns "driving" the keyboard. The navigator reviews each line of code as it‘s typed, catching bugs and discussing design improvements in real-time.

Pair programming has been shown to produce higher-quality code with fewer defects. A study by Cockburn and Williams found that code written by pairs had 15% fewer defects than code written solo. Pairs also tend to complete work faster and have higher morale.

If pair programming full-time is not realistic, consider doing it for the most complex or high-risk parts of your codebase. You can also supplement it with lightweight code reviews using pull requests on GitHub or similar tools. The key is maintaining a continuous feedback loop on code quality.

Refactoring and addressing technical debt

Refactoring is the process of restructuring code without changing its external behavior. It‘s an essential practice for keeping codebases clean, maintainable, and free of technical debt. Teams that prioritize refactoring are able to sustain a faster pace of development over the long run.

One approach I‘ve found effective is to dedicate a portion of each sprint, say 20%, to refactoring work. This could involve tackling a major refactoring project or knocking out lots of small cleanup tasks. The key is making it a regular habit.

Another technique is to do "boy scout" refactoring, leaving the code a little cleaner than you found it each time you touch it. Encourage developers to refactor as they go, rather than waiting for a big bang refactoring sprint.

Metrics for guiding continuous improvement

Tracking metrics is an essential part of the Scrum framework‘s inspect-and-adapt cycle. By regularly measuring key aspects of your process and outcomes, you can identify areas for improvement and validate that your experiments are working.

Some of the most useful metrics I‘ve found for Scrum teams include:

• Sprint velocity: The number of story points completed each sprint. Track it over time to see patterns in your team‘s capacity. Be careful not to use velocity as a weapon to pressure teams into committing to more work.

• Cycle time: The time it takes a story to move from "in progress" to "done." Shorter cycle times indicate a healthier, more efficient process. Use cycle time to identify bottlenecks in your workflow.

• Defect escape rate: The number of bugs that slip through to production. A high defect escape rate suggests issues with your quality assurance process. Aim to catch and fix defects as early as possible.

• Code coverage: The percentage of your codebase covered by automated tests. Higher coverage means less risk of undetected bugs. Track coverage over time to see if your test suite is keeping up with new code.

• Happiness index: A qualitative measure of team morale and satisfaction. Use anonymous surveys or mood boards to track happiness sprint over sprint. Low happiness is a leading indicator of deeper issues to address.

The specific metrics you track will depend on your team‘s context and goals. The important thing is using them as a tool for empirical process control. Share metrics with the whole team and discuss them in retrospectives. Use them to drive experiments and celebrate successes.

Eyes on the horizon

Scrum is a simple framework, but it‘s far from easy to master. It takes dedication, discipline, and a commitment to continuous improvement to reap its full benefits. There will always be bumps in the road, but focus on progress over perfection.

As a fellow traveler on the Scrum journey, my parting advice is to stay curious and never stop learning. Regularly read books and articles on Scrum, attend agile conferences, and participate in your local agile meetup groups. Adopt a beginner‘s mind and be open to trying new practices.

Most importantly, foster a culture of psychological safety on your team. Encourage everyone to speak up about what‘s working and what‘s not. Create space to have the difficult conversations about your process and constantly look for opportunities to improve.

Keep sprinting, my fellow agilistas, and may your burndown charts forever slope downward. The road to Scrum mastery is long, but it‘s a journey well worth taking.