Cloud with DJ

Scenario

Consider your DevOps team using a development lab for application testing, where AWS EC2 instances in the lab run continuously. Your management raises concerns about the escalating costs, as these instances operate 24/7 even when the team isn’t working.

Solution

The task is clear: implement an automated solution that provides the necessary CI/CD environment while reducing unnecessary expenses.

To address this, we can automate instance management by using a Python script combined with AWS Lambda. The script will target EC2 instances tagged as “Dev,” stopping them outside of active work hours.

Prerequisites

• Python installed.
• AWS CLI installed
• AWS account with necessary IAM permissions.
• Knowledge of AWS Lambda, EC2, and Boto3 documentation

Step 1: Define the Scope

Since costs are primarily associated with development environments, we’ll focus on EC2 instances tagged “Dev” in our solution. This tag will enable our Python script to identify and manage only the relevant resources.

Below is a snapshot of the EC2 fleet used by our developers.

Step 2: Write the Python Script

Our Python script uses Boto3, the AWS SDK for Python, to interact with AWS services.

Script Flow:

• Imports Boto3 to interact with AWS.
• Creates an EC2 client for a specified region
• Fetches a list of all running instances.
• Filters for instances tagged “Dev” and stops them.
• Outputs the IDs of the stopped instances for verification.

Script Breakdown:

• Imports necessary modules.
• Identifies “Dev” instances using specific tag criteria.
• Stops the instances and confirms successful execution.

Before all of this, you should make sure your AWS is configured.

Step 3: Test the Script

• Run the script manually to verify it’s stopping only the desired “Dev” instances. You’ll see confirmation of which instances were stopped, displayed both in the console and the EC2 dashboard.

Verify from the console:

• Run the script again. Based on our script you should get an error message.

Step 4: Automate with AWS Lambda

It’s time to fully automate this process. Make sure we don’t need a human to run the python script.

First, create a Lambda function with your Python script.

• In the Lambda console, create a new function with Python as the runtime.

• Paste your Python script into the function’s code editor and save.

Second, configure AWS EventBridge to trigger this Lambda function on a schedule, such as weekdays at 7 p.m. after the team’s working hours.

• Use EventBridge to set a cron job to trigger the Lambda function according to your desired schedule.
• From our Lambda Function, copy the function ARN and Click “Add Trigger”

• Trigger has been created.

With Lambda and EventBridge, the script will run automatically, stopping the “Dev” instances on schedule, eliminating human error and further controlling costs.

Step 5: Test the Lambda Function

• Lastly, it’s time to test our Lambda function. To do so, let’s ensure that we have at least ONE Dev Instance running in our EC2 Instance Dashboard.

• Run the Lambda function to confirm it performs as expected. Check the EC2 dashboard to ensure that only “Dev” instances were stopped.

Challenge:

After testing lambda, I encountered this error message:

The error indicates that the Lambda function does not have permission to stop EC2 instances. The IAM role attached to the function does not have the necessary permissions to allow the ec2:StopInstances action on the specified instance.

My Solution:

If you don’t have permissions to do the following changes, contact your AWS Administrator. In my case, this what I did:

• Go to the IAM console and select the ‘Lambdaec2describerole’ role.
• Click on ‘Add permissions’ and then ‘Create inline policy’.

• In the JSON editor, add the following policy:

• Click ‘Review policy’ and give it a name like ‘AllowStopEC2Instance’.
• Click ‘Create policy’ to attach the new inline policy to the ‘Lambdaec2describerole’ role.
• Re-test your Lambda function, everything will work fine.

If you haven’t experienced this issue, it shouldn’t be a concern.

With this automated solution, you’ve successfully managed the cost of your CI/CD environment, ensuring it serves the team’s needs without unnecessary expense or oversight. This approach minimizes manual intervention, optimizes performance, and keeps your DevOps processes streamlined.