Cron Job AWS Lambda Functions Tutorial – How to Schedule Tasks

Cron jobs are a staple in the world of computing and system administration. They allow you to schedule scripts and programs to automatically run at designated times – incredibly useful for automating repetitive tasks like backups, system maintenance, data processing, and more.

Traditionally, cron jobs run directly on a server or virtual machine. But what if you want the power of cron scheduling without having to manage the underlying infrastructure? That‘s where serverless computing and AWS Lambda functions come in.

In this in-depth tutorial, we‘ll walk through how to use AWS Lambda functions as a fully managed, serverless alternative to cron jobs. You‘ll learn how to define, code, and deploy Lambda functions that are triggered on a schedule to perform tasks like starting and stopping EC2 instances. Let‘s jump in!

A Quick Primer on AWS Lambda

Before we get into the nuts and bolts of using Lambda for scheduled tasks, let‘s make sure we‘re on the same page about what Lambda is and how it works.

AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers. You simply upload your code as individual "Lambda functions", and AWS takes care of running and scaling your code with high availability.

A key aspect of Lambda is that functions only run when triggered by an event, such as an HTTP request, database update, file upload to S3, or a scheduled time. This event-driven model, along with the automatic scaling and the fact that you only pay for the compute time your code actually consumes, makes Lambda very cost-effective and efficient for many workloads.

Setting Up the Demo

To illustrate using Lambda for scheduled cron-like tasks, we‘ll walk through a concrete example of using Lambda functions to start and stop EC2 instances on a daily schedule. We‘ll use the AWS Serverless Application Model (SAM) to define and deploy the Lambda functions as infrastructure as code.

Here are the prerequisites and steps to get the demo set up:

1. Make sure you have an AWS account. You can sign up for the free tier if you don‘t have one already.

2. Create one or more EC2 instances that you want to schedule starting and stopping. Make note of the instance IDs.

3. Install and configure the AWS SAM CLI on your local machine or AWS Cloud9 IDE. See the AWS docs for instructions.

4. Clone the demo project code from the GitHub repo. In the project, you‘ll find a template.yaml file that defines the Lambda functions and a cron/ directory with the function code.

5. Open the template.yaml file and find the Environment section for each function. Replace the placeholder instance IDs with the actual IDs of your EC2 instances.

6. From the project root, run sam deploy --guided. Follow the prompts to confirm the deployment details. SAM will package and deploy the Lambda functions to your AWS account.

That‘s it for setup! Let‘s dive into the meat of the demo and look at how the Lambda functions are defined and implemented.

Defining the Lambda Functions

Open up the template.yaml file in the demo project. This is where we use AWS SAM to define the Lambda functions and their triggers, permissions, and configuration.

Let‘s break down the key parts of one of the function definitions:

StartInstanceFunction:
  Type: AWS::Serverless::Function
  Properties:
    Handler: cron/handler.startInstance
    Runtime: nodejs12.x
    CodeUri: cron/
    Environment:
      Variables:
        INSTANCE_IDS: !Ref InstanceIds
    Policies:
      - Statement:
          - Effect: Allow
            Action:
              - ec2:StartInstances
            Resource: "*" 
    Events:
      StartInstances:
        Type: Schedule
        Properties:
          Schedule: cron(0 8 * * ? *)

• StartInstanceFunction is the logical name of the Lambda function within the SAM template.

• The Handler property specifies the code module and function name to execute, in the format of path/to/module.functionName.

• Runtime defines the language runtime for the function. Here we‘re using Node.js 12.x, but Lambda supports many other languages like Python, Java, Go, and more.

• CodeUri points to the directory containing the function code.

• The Environment section is where we define environment variables that will be accessible to the function code. In this case, we‘re passing in the EC2 instance IDs to start/stop.

• Under Policies, we grant the necessary permissions for the Lambda function to make EC2 API calls. The function needs permission to start (or stop) instances to fulfill its purpose. Note that the "*" means any EC2 instance – in a real application you‘d want to lock this down to specific instance ARNs.

• Lastly and most importantly, the Events section is where we define the trigger for the Lambda function. Here we‘re using a Schedule event to trigger the function on a cron schedule. The cron expression cron(0 8 * * ? *) translates to every day at 8:00am UTC.

There is a second Lambda function in the template named StopInstanceFunction that is largely the same, except it grants permission to stop instead of start instances and triggers on a cron schedule of every day at 6:00pm UTC.

Writing the Function Code

Now that we have the Lambda functions defined in the SAM template, let‘s look at the actual code that will execute when the functions are triggered.

Both the startInstance and stopInstance function code is found in the cron/handler.js file:

const AWS = require(‘aws-sdk‘);
const ec2 = new AWS.EC2();

module.exports.startInstance = async (event, context) => {
  const params = {
    InstanceIds: process.env.INSTANCE_IDS.split(‘,‘)
  };

  try {
    await ec2.startInstances(params).promise();
    console.log(`Started instances: ${params.InstanceIds.join(‘, ‘)}`);
  } catch (error) {
    console.error(error);
  }
};

module.exports.stopInstance = async (event, context) => {
  const params = {
    InstanceIds: process.env.INSTANCE_IDS.split(‘,‘)
  };

  try {
    await ec2.stopInstances(params).promise();
    console.log(`Stopped instances: ${params.InstanceIds.join(‘, ‘)}`);
  } catch (error) {
    console.error(error);
  }
};

The code is fairly straightforward:

1. We require the aws-sdk module and create an EC2 client instance. The AWS SDK comes pre-installed in the Lambda Node.js runtime.

2. The startInstance and stopInstance exported functions are what get called when the respective Lambda functions are triggered on their cron schedules.

3. We grab the comma-separated list of instance IDs from the INSTANCE_IDS environment variable (which is set based on the SAM template), and split it into an array.

4. Using the EC2 SDK client, we call either startInstances or stopInstances depending on the function, passing the array of instance IDs.

5. We log a success message with the affected instances, or log any error that occurred.

And that‘s really all there is to it! When deployed, the Lambda functions will trigger on their defined schedules and start/stop the specified EC2 instances accordingly.

Testing and Verifying

After deploying the Lambda functions with AWS SAM, you can test that they are working as expected by either waiting for the scheduled trigger times, or by manually invoking them via the Lambda console or AWS CLI.

To manually invoke via the CLI, you can use the aws lambda invoke command:

aws lambda invoke --function-name my-function-name output.txt

Replace my-function-name with the actual name of your deployed Lambda function (StartInstanceFunction or StopInstanceFunction).

You can then check the EC2 console to verify that the instances started or stopped as expected. You can also view the Lambda function logs in CloudWatch Logs to see the logged success or error messages.

Cleaning Up

To avoid unnecessary costs, make sure to clean up the resources created for this demo when you‘re done:

1. Go to the CloudFormation console and delete the stack created by the SAM deployment. This will delete the Lambda functions and associated resources.

2. If you created EC2 instances specifically for the demo, go to the EC2 console and terminate them.

Further Considerations

While this demo illustrates the basic concept of using Lambda for scheduled tasks, there are several other factors to consider for real-world usage:

• Permissions and Security: Make sure your Lambda functions are following the principle of least privilege and are only granted the specific permissions needed to fulfill their purpose. Avoid using overly broad resource statements like "*".

• Error Handling and Retries: Consider what should happen if your Lambda function encounters an error. Do you want it to retry? How many times? What kind of alerting do you need?

• Idempotency: Scheduled Lambda functions may run more than once for a given trigger in rare cases. Make sure your functions are idempotent and can handle being executed multiple times without adverse effects.

• Logging and Monitoring: Take advantage of CloudWatch Logs and Metrics to log output from your functions and monitor their health and performance. Set up alarms for error rates or anomalous behavior.

• Concurrency Limits: Be aware of any concurrency limits on your account or functions that could throttle your scheduled executions if you have a large number of functions running at the same time.

• Time Zone: Note that the cron schedule is in UTC. If you want your functions to run on a schedule in a different time zone, you‘ll need to adjust accordingly.

• Cost: While Lambda has a generous free tier, be aware of the potential costs of your scheduled functions based on number of executions, execution duration, memory used, and data transfer.

Conclusion

AWS Lambda functions triggered by scheduled events are a powerful and cost-effective way to automate tasks in a serverless fashion. By leveraging infrastructure as code tools like AWS SAM, you can easily define, deploy, and manage your scheduled jobs.

From simple use cases like starting and stopping EC2 instances to more complex workflows like data aggregation and report generation, Lambda provides a flexible and scalable platform for a wide variety of scheduled workloads.

The next time you find yourself spinning up a cron job on a server, consider if it could be a good fit for a scheduled Lambda function instead!