How to Setup Continuous Deployment to AWS S3 using CircleCI in 30 Minutes

In modern software development, the ability to rapidly and reliably deploy new features and fixes is a critical competitive advantage. Continuous deployment, a practice in which every change that passes automated tests is automatically deployed to production, has emerged as a key enabler of this agility. In fact, a recent survey by CircleCI found that teams who have adopted continuous deployment deploy code 45% more frequently and have a 3x lower change failure rate compared to those who haven‘t (source).

In this guide, I‘ll walk you through setting up continuous deployment for a static website to AWS S3 using CircleCI. By leveraging the power of these tools, you‘ll be able to automate your entire deployment process, from code commit to production release. Whether you‘re a seasoned developer looking to optimize your workflow or a beginner just getting started with automation, this guide will provide you with a solid foundation.

Prerequisites

Before diving in, you‘ll need to have the following set up:

1. An AWS account (sign up for free here)
2. A CircleCI account (sign up for free here)
3. A GitHub account and repository with a static website you want to deploy

If you don‘t have an existing project to use, feel free to fork the sample project I‘ve created here.

Step 1: Connect CircleCI to Your GitHub Repository

The first step is to add your GitHub repository to CircleCI so it can monitor for new commits and trigger builds:

1. Log in to CircleCI and navigate to the ‘Projects‘ page
2. Click ‘Set Up Project‘ next to your repository
3. Select ‘Fastest‘ to use the default configuration and click ‘Set Up Project‘

CircleCI will now attempt to run a build but will fail because we haven‘t provided a configuration file yet. That‘s expected at this stage – we‘ll add our custom configuration in a later step.

Step 2: Create an S3 Bucket

Next, we need to create an S3 bucket to host our static website files:

1. Log in to the AWS Management Console and navigate to the S3 service
2. Click ‘Create Bucket‘ and enter a globally unique name for your bucket
3. Select a region that‘s closest to your target users for optimal performance
4. In the ‘Set permissions‘ section, uncheck ‘Block all public access‘ – we‘ll configure more granular permissions later
5. Click ‘Create bucket‘ to finish

Now let‘s enable static website hosting on the bucket:

1. Select your newly created bucket from the list and go to the ‘Properties‘ tab
2. Scroll down to the ‘Static website hosting‘ card and click ‘Edit‘
3. Select ‘Enable‘ and enter ‘index.html‘ as the Index document
4. Note down the ‘Endpoint‘ URL provided – this is where your website will be accessible
5. Click ‘Save changes‘

At this point, your S3 bucket is configured to host a static website. However, if you visit the provided endpoint URL, you‘ll get an ‘Access Denied‘ error. This is because, by default, S3 blocks public access to your files. We‘ll adjust the bucket policy to allow public read access in a later step.

Step 3: Configure IAM Permissions

For CircleCI to be able to deploy to your S3 bucket, it needs to be granted the necessary permissions. The best practice is to create a dedicated IAM user with programmatic access keys:

1. In the AWS Management Console, navigate to the IAM (Identity and Access Management) service
2. Go to the ‘Users‘ section and click ‘Add user‘
3. Provide a name for the user (e.g., ‘circleci-deploy‘), select ‘Programmatic access‘, and click ‘Next‘
4. In the permissions section, select ‘Attach existing policies directly‘ and choose ‘AmazonS3FullAccess‘
   • Note: For production use, you‘d want to follow the principle of least privilege and only grant the specific permissions needed (e.g., s3:PutObject, s3:GetObject, s3:ListBucket, etc.)
5. Skip the ‘Tags‘ section and click ‘Next: Review‘
6. Review the details and click ‘Create user‘
7. Copy the ‘Access key ID‘ and ‘Secret access key‘ – we‘ll use these in the next step to configure CircleCI

Now let‘s adjust the S3 bucket policy to allow public read access:

1. Go back to the S3 service and select your bucket
2. Go to the ‘Permissions‘ tab and find the ‘Bucket Policy‘ section
3. Click ‘Edit‘ and paste the following policy, replacing your-bucket-name with your actual bucket name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow", 
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::your-bucket-name/*"
        }
    ]
}

4. Click ‘Save changes‘

Your S3 bucket will now allow public read access to the objects inside it, which is necessary for serving a public website.

Step 4: Add Environment Variables in CircleCI

To securely provide your AWS access keys to CircleCI without exposing them in your code, we‘ll use environment variables:

1. In CircleCI, go to your project‘s settings (clicking the gear icon)
2. On the left sidebar, click on ‘Environment Variables‘
3. Click ‘Add Variable‘ and enter AWS_ACCESS_KEY_ID as the name and paste your access key ID from step 3
4. Click ‘Add Variable‘ again, enter AWS_SECRET_ACCESS_KEY as the name, and paste your secret access key
5. Your access keys are now securely stored and will be accessible to your build jobs

Using environment variables is a security best practice as it prevents your sensitive credentials from being exposed in logs or build artifacts.

Step 5: Add a CircleCI Config File

Now it‘s time to define our build and deployment process by adding a CircleCI configuration file to our project:

1. In your project repository, create a new directory called .circleci
2. Inside this directory, create a file named config.yml
3. Paste the following configuration:

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/node:lts
        auth:
          username: mydockerhub-user  # Replace with your Docker Hub username
          password: $DOCKERHUB_PASSWORD  # Replace with your Docker Hub password
    steps:
      - checkout
      - run:
          name: Install AWS CLI
          command: |
            curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
            unzip awscliv2.zip
            sudo ./aws/install      
      - run:
          name: Deploy to S3
          command: |
            aws s3 sync . s3://your-bucket-name --delete

workflows:
  build-and-deploy:
    jobs:
      - build:
          filters:
            branches:
              only: main

Let‘s break down what this configuration does:

• version: 2.1: Specifies the CircleCI configuration version
• jobs: Defines the jobs that will be run. In this case, we have a single job named build
  • docker: Specifies the Docker image to use for the job. Here, we‘re using the latest Node.js image
  • auth: Provides authentication details for private Docker images (replace with your own credentials)
  • steps: Lists the steps to be executed in this job
    • checkout: Checks out our code from the repository
    • run: Executes specified commands
      • First, we install the AWS CLI to enable interaction with AWS services
      • Then, we use the aws s3 sync command to sync our files to our S3 bucket. Replace your-bucket-name with your actual bucket name
• workflows: Defines a workflow named build-and-deploy that will run the build job on pushes to the main branch

4. Commit the config file and push it to your GitHub repository

Step 6: Test Your Deployment!

It‘s time for the exciting part – let‘s see our automated deployment in action:

1. Make a change to your website code (e.g., update some text in your index.html file)
2. Commit your changes and push to the main branch on GitHub
3. Navigate to the ‘Pipelines‘ page in CircleCI to see your workflow kick off
4. Once the build job completes successfully, visit your website‘s URL (the endpoint from step 2)
5. You should see your changes reflected on the live website!

Congratulations! You now have a fully automated continuous deployment pipeline. From now on, any changes you push to the main branch will be automatically deployed to your S3 bucket and reflected on your website.

Additional Considerations

While this setup provides a solid foundation, there are additional considerations to keep in mind for a production environment:

• Custom Domain: Use a custom domain name for your website instead of the default S3 endpoint URL. You can set this up using Amazon Route 53.
• Cache Control: Implement proper cache control headers for your static assets to optimize performance and reduce load on your servers.
• Versioning: Enable versioning on your S3 bucket to preserve previous versions of your files and enable easy rollbacks if needed.
• Testing: Incorporate automated testing into your CircleCI workflow to catch bugs and ensure the quality of your deployments.
• Approval Workflows: For critical projects, consider adding manual approval steps to your workflow using CircleCI‘s ‘Approvals‘ feature.
• Monitoring: Set up monitoring for your website using tools like AWS CloudWatch or Datadog to track performance and quickly identify issues.

Measuring Success

To gauge the effectiveness of your continuous deployment pipeline, it‘s important to track key metrics:

• Deployment Frequency: Track how often you‘re deploying new changes to production. Continuous deployment should enable more frequent, smaller releases.
• Lead Time: Measure the time it takes from starting work on a new feature to deploying it to production. Automation should significantly reduce this.
• Change Failure Rate: Monitor the percentage of deployments that result in failures or need to be rolled back. A successful CD pipeline should minimize this.
• Time to Recovery: In the event of a failure, track how long it takes to roll back or fix the issue. Rapid recovery is critical for minimizing downtime.

By tracking these metrics over time, you can demonstrate the value of your continuous deployment efforts and identify areas for further optimization.

Conclusion

In this guide, we walked through setting up a continuous deployment pipeline for a static website hosted on AWS S3 using CircleCI. By automating the deployment process, you can release new features and fixes faster and with more confidence.

Remember, continuous deployment is not just about tooling – it‘s a mindset shift that prioritizes automation, rapid feedback, and incremental improvements. By embracing these principles, you can deliver value to your users faster and stay ahead in an increasingly competitive landscape.

I encourage you to continue exploring and optimizing your deployment workflows. Experiment with new tools and techniques, and don‘t be afraid to iterate and improve over time. With the right practices in place, you‘ll be amazed at how much more productive and efficient your team can be.

Additional Resources

To dive deeper into continuous deployment and related topics, check out these resources:

• CircleCI Docs: Deployment – Official documentation on deploying with CircleCI
• Atomic Deployments with AWS S3 and CircleCI – Blog post on advanced deployment techniques
• AWS Whitepaper: practicing continuous integration and continuous delivery on AWS – In-depth guide on CI/CD best practices with AWS
• Accelerate: The Science of Lean Software and DevOps – Book on the key metrics and practices that drive high performance in software delivery

Happy deploying!