The Acceptance Criteria for Writing Acceptance Criteria

In the world of agile software development, acceptance criteria serve as the bedrock of shared understanding between the business and the development team. Well-crafted acceptance criteria not only define when a user story is considered complete but also align everyone involved on what "done" looks like. However, many teams struggle with writing effective acceptance criteria, leading to misunderstandings, wasted effort, and suboptimal outcomes.

As a seasoned full-stack developer and professional coder, I‘ve witnessed the transformative power of well-written acceptance criteria firsthand. In this in-depth guide, I‘ll share industry best practices, expert insights, and real-world examples to help you master the art of crafting acceptance criteria that drive clarity, quality, and customer satisfaction. Whether you‘re a product owner, developer, or tester, this article will equip you with the knowledge and skills to take your acceptance criteria game to the next level.

The State of Acceptance Criteria in Agile Teams

Before diving into the nitty-gritty of writing acceptance criteria, let‘s take a step back and examine the current state of acceptance criteria usage in the industry. According to a recent survey by VersionOne, 86% of agile teams regularly use acceptance criteria when developing software. However, the mere presence of acceptance criteria doesn‘t guarantee their effectiveness.

In the same survey, 64% of respondents cited poorly defined acceptance criteria as a significant challenge hindering their agile practices. This highlights the critical importance of not just having acceptance criteria but ensuring they are well-crafted and fit for purpose.

Acceptance Criteria Usage	Percentage
Teams regularly using acceptance criteria	86%
Teams citing poorly defined criteria as a challenge	64%

Source: VersionOne 14th Annual State of Agile Report

Anatomy of Effective Acceptance Criteria

So, what separates mediocre acceptance criteria from great ones? Based on my experience and industry research, effective acceptance criteria exhibit the following key characteristics:

1. Testable and Unambiguous

Each acceptance criterion should be clear, concise, and testable. Testers should be able to objectively verify whether the criterion has been met without room for interpretation. Avoid vague or subjective language that can lead to confusion and disagreements.

Consider the following example:

Poor: The search results page should load quickly.
Better: The search results page should load within 2 seconds for 95% of requests under a load of 1000 concurrent users.

The improved version provides a measurable and unambiguous standard that everyone can agree on.

2. Focused on User Value

Acceptance criteria should prioritize the end-user perspective and business value. They should describe the observable outcomes that matter to the customer rather than technical implementation details.

Poor: The system should use a Redis cache to store session data.
Better: The user should remain logged in even if they close and reopen their browser within a 30-minute window.

By focusing on the user-facing behavior, the revised criterion allows the development team flexibility in how they implement the solution while still meeting the core requirement.

3. Jointly Authored and Refined

Effective acceptance criteria emerge from collaboration between the product owner, developers, and testers. Each brings a unique perspective that contributes to a well-rounded set of criteria.

Liz Keogh, a prominent agile coach and Behavior-Driven Development (BDD) practitioner, emphasizes the importance of the "Three Amigos" approach:

"The Three Amigos refers to the primary perspectives to examine an increment of work before, during, and after development. Those perspectives are: Business – what problem are we trying to solve? Development – how might we build a solution to solve that problem? Testing – what about this, what could possibly happen?" – Liz Keogh

By involving representatives from each perspective in the creation and refinement of acceptance criteria, teams can identify gaps, edge cases, and potential misunderstandings early on.

4. Concise yet Comprehensive

Striking the right balance between detail and brevity is crucial for acceptance criteria. Aim to capture the essential elements of the requirement without getting bogged down in minutiae.

As a rule of thumb, most user stories should have between 3 to 8 acceptance criteria. If you find yourself consistently writing more than that, it may indicate that your user stories are too complex and should be split into smaller, more focused pieces.

Mike Cohn, one of the pioneers of the user story format, provides the following guidance:

"I‘ve heard an estimate that about 1 in 10 stories will need more than the usual number of acceptance criteria[…] Keep the acceptance criteria focused on the core of the story, the heart of the behavior, and the value it delivers." – Mike Cohn

Remember, acceptance criteria are not a substitute for detailed specifications or test cases. Their purpose is to define the boundaries of the requirement and provide a shared understanding of what "done" means in the context of the user story.

Formatting Acceptance Criteria

There are two common formats teams use for writing acceptance criteria: scenario-oriented and rule-oriented.

Scenario-Oriented (Given/When/Then)

Scenario-oriented acceptance criteria use a structured Given/When/Then format derived from Behavior-Driven Development (BDD). This format is particularly useful for specifying behavior that spans multiple user interactions or system responses.

Example:

Given a customer has items in their shopping cart
And they are not logged in
When they click the "Checkout" button
Then they are prompted to either log in or continue as a guest

The Given/When/Then syntax helps keep the criteria focused on the user‘s perspective and the observable outcomes.

Rule-Oriented

Rule-oriented acceptance criteria, on the other hand, are best suited for specifying discrete rules or validations that don‘t necessarily involve a sequence of steps.

Example:

• The email field should be mandatory
• The password must be at least 8 characters long and include at least one uppercase letter, one lowercase letter, and one number
• If the entered email is not in a valid format, display an error message "Please enter a valid email address"

Rule-oriented criteria are often easier to read and understand at a glance, making them a good choice for simpler requirements.

Ultimately, the choice of format depends on your team‘s preference and the nature of the user story. Some teams even use a hybrid approach, combining both formats as needed.

Collaborative Acceptance Criteria Creation

Writing acceptance criteria should be a collaborative effort involving the product owner, developers, and testers. Each brings a unique perspective that enhances the quality and completeness of the criteria.

The Role of the Product Owner

As the voice of the customer and the primary stakeholder, the product owner is responsible for defining the high-level acceptance criteria that capture the business requirements and user expectations. They should focus on the "what" rather than the "how" and prioritize criteria based on business value.

However, the product owner should not work in isolation. They should actively seek input and feedback from the development team to ensure the criteria are technically feasible, consistent with the system‘s architecture, and aligned with the team‘s Definition of Done.

The Role of the Development Team

Developers play a crucial role in refining and elaborating the acceptance criteria provided by the product owner. They bring technical expertise to the table and can identify edge cases, dependencies, and potential implementation challenges that the product owner might not be aware of.

Developers should ask clarifying questions, suggest improvements, and propose alternative solutions that can better meet the intent of the criteria. They should also push back on criteria that are overly prescriptive or unnecessarily constrain the implementation.

The Role of Testers

Testers are the guardians of quality and the final arbiters of whether a user story meets its acceptance criteria. They should be involved in the creation and review of acceptance criteria to ensure they are testable, unambiguous, and complete.

Testers can contribute valuable insights based on their experience with the system and their knowledge of common pitfalls and edge cases. They can also help identify missing criteria or scenarios that need to be addressed.

Collaborative Techniques

Here are some proven techniques for fostering collaboration and shared ownership of acceptance criteria:

• Three Amigos Meetings: Conduct regular meetings with representatives from the business (product owner), development, and testing to review upcoming user stories and define acceptance criteria collaboratively.

• Definition of Ready (DoR): Establish a shared Definition of Ready that includes well-defined acceptance criteria as a prerequisite for a user story to be considered ready for development. Use this DoR as a checklist to ensure consistent quality and completeness.

• Peer Reviews: Encourage peer reviews of draft acceptance criteria within the product owner and business analyst community. Having fresh eyes review the criteria can help identify gaps, inconsistencies, or ambiguities that might have been missed.

• Acceptance Criteria Workshops: Conduct dedicated workshops or training sessions on writing effective acceptance criteria. Invite experienced practitioners or agile coaches to share best practices and provide hands-on guidance to the team.

By involving all relevant stakeholders in the creation and refinement of acceptance criteria, teams can leverage diverse perspectives, catch issues early, and build a shared understanding of what success looks like.

Measuring and Improving Acceptance Criteria Quality

To continuously improve the quality and effectiveness of your acceptance criteria, it‘s essential to measure their impact and seek feedback from the team. Here are some key metrics and practices to consider:

Acceptance Criteria Quality Metrics

• Acceptance Criteria Coverage: Track the percentage of user stories that have well-defined acceptance criteria. Aim for 100% coverage to ensure consistent quality across the board.

• Acceptance Criteria Defect Density: Monitor the number of defects or issues related to incomplete, ambiguous, or incorrect acceptance criteria. A high defect density indicates a need for improvement in the criteria definition process.

• User Story Cycle Time: Measure the average time it takes for a user story to move from "in progress" to "done." Well-written acceptance criteria should contribute to faster cycle times by providing clarity and reducing rework.

Retrospectives and Feedback

Regularly conduct retrospectives focused on acceptance criteria to gather feedback from the team and identify areas for improvement. Ask questions like:

• What worked well with our acceptance criteria this sprint?
• What challenges or confusion did we encounter related to acceptance criteria?
• How can we improve our acceptance criteria definition process?

Encourage open and honest feedback from all team members and use the insights to refine your approach iteratively.

Continuous Learning and Skill Development

Invest in ongoing training and skill development for product owners, developers, and testers in the art of writing effective acceptance criteria. Provide resources, workshops, and mentoring opportunities to help team members hone their skills and stay up-to-date with industry best practices.

Acceptance Criteria and Agile Practices

Well-crafted acceptance criteria play a vital role in enabling and supporting other agile practices. Let‘s explore a few key examples:

Acceptance Criteria and Test-Driven Development (TDD)

In Test-Driven Development (TDD), developers write automated tests based on the acceptance criteria before writing the implementation code. This approach helps ensure that the code meets the specified requirements and prevents scope creep.

By having clear and testable acceptance criteria, developers can create more focused and effective automated tests. The acceptance criteria act as a guide for what to test and help maintain a tight feedback loop between requirements and implementation.

Acceptance Criteria and Continuous Integration (CI)

Continuous Integration (CI) is a practice where developers integrate their code changes frequently, often multiple times a day, into a shared repository. Each integration is verified by an automated build and test process to detect and address issues early.

Well-defined acceptance criteria contribute to the success of CI by providing a clear benchmark for what constitutes a passing build. Automated tests based on the acceptance criteria can be run as part of the CI pipeline, giving fast feedback on whether the integrated code meets the specified requirements.

Acceptance Criteria and Automated Testing

Acceptance criteria form the basis for automated testing at various levels, including unit testing, integration testing, and acceptance testing. By translating the acceptance criteria into executable test cases, teams can verify the functionality and quality of the system continuously and catch regressions early.

Tools like Cucumber and SpecFlow allow teams to write acceptance tests in a human-readable format using the Given/When/Then syntax. These tests serve as living documentation of the system‘s behavior and help maintain alignment between the requirements and the implementation.

Acceptance Criteria and User Story Splitting

Well-crafted acceptance criteria can also aid in user story splitting, a technique used to break down large or complex stories into smaller, more manageable pieces. By focusing on the acceptance criteria, teams can identify natural breakpoints and split stories based on individual criterion.

For example, consider a user story for a search functionality with the following acceptance criteria:

1. Users can search for products by keyword
2. Search results are displayed in a paginated list
3. Users can filter search results by category and price range
4. Search results can be sorted by relevance, price, or rating

If this story is too large to complete in a single sprint, it can be split into smaller stories based on the acceptance criteria:

• Story 1: Users can search for products by keyword and see the results in a paginated list
• Story 2: Users can filter search results by category and price range
• Story 3: Search results can be sorted by relevance, price, or rating

By splitting stories based on acceptance criteria, teams can maintain a clear focus and deliver incremental value to users.

Acceptance Criteria and the Definition of Done

Acceptance criteria are closely tied to the Definition of Done (DoD), which is a shared understanding of what it means for a user story to be considered complete. The DoD typically includes criteria that apply to all user stories, such as code reviews, unit testing, and documentation.

While the Definition of Done provides a baseline, acceptance criteria complement it by specifying the unique requirements for a particular user story. A user story can only be considered truly done when it meets both the general DoD criteria and the specific acceptance criteria defined for that story.

Criteria	Definition of Done	Acceptance Criteria
Applies to	All user stories	Individual user story
Includes	Code review, unit testing, documentation	Specific functional and non-functional requirements
Assessed by	Development team	Product owner and testers
Focuses on	Technical quality and completeness	Business value and user satisfaction

Comparison of Definition of Done and Acceptance Criteria

By using acceptance criteria in conjunction with a well-defined DoD, teams can ensure a consistent level of quality and completeness across all user stories.

Conclusion

In summary, writing effective acceptance criteria is a critical skill for agile teams looking to deliver high-quality software that meets user needs. By following best practices and fostering collaboration between the product owner, developers, and testers, teams can create acceptance criteria that provide clarity, focus, and a shared understanding of what success looks like.

Remember to keep your acceptance criteria testable, focused on user value, jointly authored, and concise yet comprehensive. Use formats like Given/When/Then or rule-oriented criteria based on your team‘s preferences and the nature of the user story.

Continuously measure and improve the quality of your acceptance criteria through metrics, retrospectives, and ongoing skill development. Leverage acceptance criteria to enable practices like Test-Driven Development, Continuous Integration, and automated testing.

By mastering the art of writing acceptance criteria, you‘ll be well-equipped to deliver software that delights users, meets business objectives, and stands the test of time.