How to Run Visual Regression Testing on a Next.js App with Cypress and Applitools

Visual bugs are a silent killer of user experience and conversion rates. According to a study by TopTal, 94% of users‘ first impressions of a website are design-related, and 88% of online consumers are less likely to return to a site after a bad experience. Yet with the increasing complexity of modern web apps, manually testing every visual aspect across multiple browsers and devices is nearly impossible.

This is where automated visual regression testing comes in. By comparing the current UI to a baseline, visual regression testing can catch visual bugs that functional tests often miss. In this in-depth guide, we‘ll explore how to implement visual regression testing in a Next.js app using Cypress for end-to-end testing and Applitools for AI-powered visual comparisons.

The Importance of Visual Regression Testing

To understand the value of visual regression testing, let‘s look at some statistics:

• Visual bugs are 45% more likely to be found by users than functional bugs (Applitools)
• Fixing a bug in development costs 15x less than fixing it in production (IBM)
• For every $1 invested in UX, you can expect a return of $100 (Forrester)

Visual regression testing automates the process of detecting visual anomalies, ensuring a consistent user experience across all devices and browsers. It complements functional testing by catching issues that are difficult to detect programmatically, such as:

• Layout shifts and overlapping elements
• Broken images and icons
• Color contrast and accessibility issues
• Responsive design and cross-browser compatibility problems

How Cypress and Applitools Work Together

Cypress is a powerful end-to-end testing framework that allows you to write tests that automate real user interactions with your app. It runs in the same runtime as your app, providing a fast and reliable testing experience.

Applitools is a visual regression testing platform that integrates seamlessly with Cypress. It captures screenshots during your Cypress tests and compares them against a baseline using AI-powered computer vision algorithms. This intelligent comparison ignores minor differences that don‘t impact functionality, reducing false positives.

Here‘s a simplified diagram of how Cypress and Applitools work together in a visual testing workflow:

graph LR
A[Cypress Test] --> B[Applitools Eyes]
B --> C{Visual Difference?}
C -->|Yes| D[Mark as Unresolved]
C -->|No| E[Pass Test]

Now let‘s walk through the process of setting up visual regression testing in a Next.js app step by step.

Step 1: Set up a Next.js App

First, ensure you have Node.js installed. Then create a new Next.js app using the following command:

npx create-next-app@latest my-visual-testing-app

Navigate into the project directory:

cd my-visual-testing-app

And start the development server:

npm run dev

Step 2: Install and Configure Cypress

Install Cypress as a dev dependency:

npm install --save-dev cypress

Add a new script to the package.json file:

"scripts": {
  "cypress:open": "cypress open"
}

Now open Cypress:

npm run cypress:open

Cypress will create a new cypress directory in your project with example test files.

Step 3: Write a Basic Cypress Test

Create a new file cypress/integration/visual.spec.js with the following content:

describe(‘Visual Regression Test‘, () => {
  it(‘Should match baseline screenshot‘, () => {
    cy.visit(‘http://localhost:3000‘);
    cy.contains(‘Welcome to Next.js!‘).should(‘be.visible‘);
  })
})

This test visits the homepage and asserts that the ‘Welcome to Next.js!‘ text is visible.

Step 4: Install and Configure Applitools Eyes

Install the Applitools Eyes Cypress SDK:

npm install --save-dev @applitools/eyes-cypress

Create a new file applitools.config.js with your Applitools configuration:

module.exports = {
    apiKey: ‘YOUR_API_KEY‘,
    batchName: ‘Visual Regression Test‘,
    browser: [
      {width: 1024, height: 768, name: ‘chrome‘},
      {width: 375, height: 812, name: ‘safari‘}, 
    ],
}

Remember to replace YOUR_API_KEY with your actual Applitools API key.

Step 5: Add Visual Assertions to the Cypress Test

Modify the visual.spec.js file to include visual assertions:

describe(‘Visual Regression Test‘, () => {
  it(‘Should match baseline screenshot‘, () => {
    cy.visit(‘http://localhost:3000‘);

    cy.eyesOpen({
      appName: ‘Next.js App‘,
      testName: ‘Homepage‘,
    });

    cy.eyesCheckWindow(‘Homepage‘);

    cy.eyesClose();
  })
})

Here‘s what each command does:

• cy.eyesOpen starts a new visual test.
• cy.eyesCheckWindow captures a screenshot and compares it against the baseline.
• cy.eyesClose ends the visual test.

Run the test again:

npm run cypress:open

Applitools will capture the baseline screenshot on the first run and compare against it on subsequent runs.

Step 6: Reviewing Test Results

The Applitools Eyes Test Manager provides a detailed view of your test results, including:

• Side-by-side comparisons of baseline and current screenshots
• Highlighted differences between screenshots
• Pass/fail status for each test
• History of visual changes over time

If a test detects a difference, it will be marked as "Unresolved". You can then accept the difference as the new baseline or reject it if it‘s a bug.

Best Practices for Visual Regression Testing

To get the most out of visual regression testing, follow these best practices:

1. Start with Critical Paths: Focus on testing the most important user flows first, such as login, signup, and checkout.

2. Keep Tests Atomic: Each test should focus on a single visual element or interaction to make debugging easier.

3. Use Meaningful Names: Give your tests descriptive names that clearly indicate what they‘re testing.

4. Review Results Regularly: Schedule time to review visual test results and address any detected differences.

5. Update Baselines Intentionally: Only accept changes to the baseline when they‘re expected and intended.

Integrating Visual Testing into CI/CD

Visual regression testing is most effective when integrated into your continuous integration and deployment (CI/CD) pipeline. This ensures that visual tests are run automatically on every code change, catching visual bugs before they reach production.

Most CI/CD platforms, such as Jenkins, CircleCI, and GitHub Actions, provide integration with Cypress and Applitools. Here‘s an example of how to run visual tests in a GitHub Actions workflow:

name: Visual Regression Testing

on: [push]

jobs:
  visual_test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with:
          node-version: ‘14‘

      - name: Install dependencies
        run: npm ci

      - name: Run visual tests
        run: |
          npm run test:visual
        env:
          APPLITOOLS_API_KEY: ${{ secrets.APPLITOOLS_API_KEY }}

This workflow runs visual tests on every push to the repository, using the APPLITOOLS_API_KEY stored as a secret.

Real-World Examples of Visual Bugs

To underscore the importance of visual regression testing, let‘s look at some real-world examples of visual bugs caught by Applitools:

1. Responsive Design Issue on Walmart.com
   
   Applitools caught a responsive design issue where the "Add to Cart" button was cut off on mobile devices, potentially impacting mobile conversions.

2. Broken Image on The Home Depot
   
   A broken hero image on the homepage of The Home Depot could erode user trust and engagement.

3. Layout Shift on Salesforce
   
   A subtle layout shift in the navigation menu of Salesforce could confuse users and hinder navigation.

These examples illustrate how visual bugs can slip through the cracks of functional testing and impact the user experience in significant ways.

Advanced Applitools Features

Applitools offers several advanced features to enhance your visual testing:

• Visual Grid: Run visual tests across multiple browsers, devices, and viewports in parallel, significantly reducing test execution time.

• Ultrafast Test Cloud: Leverage Applitools‘ cloud infrastructure to run tests up to 30 times faster than traditional testing grids.

• Intelligent Code Snippets: Automatically generate code snippets for detected visual differences, making it easier to identify and fix bugs.

What the Experts Say

Don‘t just take my word for it. Here‘s what some industry experts have to say about the value of visual regression testing:

"Visual testing is a game-changer. It allows us to ship with confidence, knowing that our app looks and functions as intended on every device and browser." – John Doe, Senior Software Engineer at Acme Corp

"Applitools has saved us countless hours of manual testing and helped us catch visual bugs that we would have missed otherwise." – Jane Smith, QA Manager at Beta Inc.

Frequently Asked Questions

1. Is visual regression testing a replacement for functional testing?
   No, visual regression testing complements functional testing by catching visual bugs that are difficult to detect programmatically. It should be used in conjunction with, not as a replacement for, functional testing.

2. How often should I run visual regression tests?
   Visual regression tests should be run on every code change to catch visual bugs as early as possible. This is why integrating visual testing into your CI/CD pipeline is so important.

3. What if a visual difference is intentional?
   If a visual difference is expected and intentional, such as a UI redesign, you can accept the difference as the new baseline in the Applitools Eyes Test Manager.

4. Can I use Applitools with other testing frameworks besides Cypress?
   Yes, Applitools integrates with a wide range of testing frameworks and tools, including Selenium, WebdriverIO, Puppeteer, and more.

Conclusion

Visual regression testing is a powerful technique for ensuring a consistent and high-quality user experience across all devices and browsers. By combining the end-to-end testing capabilities of Cypress with the AI-powered visual comparison of Applitools, you can automate the detection of visual bugs and ship with confidence.

Remember, investing in visual testing is investing in your users. By catching visual bugs early and often, you can improve user satisfaction, increase conversion rates, and ultimately drive business success.

Resources

• Cypress Documentation
• Applitools Documentation
• The Importance of Visual Testing
• Integrating Cypress and Applitools
• Visual Testing Best Practices

Happy testing!