NavyaDevops
Automating Security Group Updates with Dynamic Prefix Lists using Lambda and EventBridge Scheduler

Automating Security Group Updates with Dynamic Prefix Lists using Lambda and EventBridge Scheduler

Navya A's photo
Navya A
·

7 min read

In today's cloud-centric environment, security is paramount. As organizations leverage various services and APIs provided by external entities, ensuring that only authorized access is allowed becomes increasingly challenging. One such scenario involves dynamically managing the ingress rules of security groups in AWS to permit access from specific IP addresses. In this blog post, we'll explore how to automate the updating of security group rules by dynamically adding prefix lists using AWS Lambda and EventBridge Scheduler.

Introduction to the Problem

Consider a scenario where your organization relies on services from a third-party provider like Stripe or Github, which regularly updates its IP addresses for webhook communications. To maintain a secure infrastructure, you need to ensure that your AWS security group allows traffic only from these IP addresses.

Traditionally, this process involves manual intervention, requiring someone to update the security group rules each time the third-party provider changes its IP addresses. However, with the power of automation provided by AWS Lambda and EventBridge Scheduler, we can streamline this process and ensure that our security group rules are always up-to-date.

Solution Overview

We'll design a solution that consists of the following components:

  1. AWS Lambda Function: This function will be responsible for fetching the latest IP addresses from the third-party provider, creating a new prefix list, updating the security group to reference the new prefix list, and deleting the old prefix list if it exists.

  2. EventBridge Scheduler: We'll configure a recurring rule in EventBridge Scheduler to trigger our Lambda function at a specified interval. In our case, we'll schedule it to run every night at midnight.

  3. AWS IAM Roles: To grant the necessary permissions for our Lambda function to interact with AWS services, we'll create IAM roles with the appropriate policies

Implementation Details

Let's delve into the implementation details of each component:

1. AWS Lambda Function

Our Lambda function will be written in Python and will perform the following steps:

· Fetch the latest IP addresses from the third-party provider (in this case, Stripe) by making an HTTP request to their endpoint.

· Create a new prefix list with the fetched IP addresses.

· Update the security group to reference the new prefix list.

· Delete the old prefix list if it exists.

You can find the complete code for the Lambda function here

Create a Layer in the Lambda:

Increase the Execution Timeout of the lambda:

Go inside the lambda Function >>> Configuration >>> Increase the Timeout

Lambda Function Code

import json
import requests
import boto3
from botocore.exceptions import ClientError
ec2 = boto3.client('ec2')
def lambda_handler(event, context):
    security_group_id = 'YOUR SECURITY GROUP ID'
    response = requests.get('https://stripe.com/files/ips/ips_webhooks.json')
    ips_data = response.json()
    webhook_ips = ips_data['WEBHOOKS']
    cidr_blocks = [f"{ip}/32" for ip in webhook_ips]
    # Delete previous prefix list and detach from security group if exists
    delete_and_detach_prefix_list('StripeIPs', security_group_id)
    # Create a new prefix list
    prefix_list_id = create_prefix_list('StripeIPs', cidr_blocks)
    # Update security group to reference the new prefix list
    update_security_group(security_group_id, prefix_list_id)
    return {
        'statusCode': 200,
        'body': json.dumps('The security group ingress rules have been successfully updated with the latest Webhook IP addresses of Stripe services')
    }
def create_prefix_list(prefix_list_name, cidr_blocks):
    try:
        response = ec2.create_managed_prefix_list(
            PrefixListName=prefix_list_name,
            AddressFamily='IPv4',
            Entries=[{'Cidr': cidr, 'Description': 'Stripe IP'} for cidr in cidr_blocks],
            MaxEntries=len(cidr_blocks)
        )
        return response['PrefixList']['PrefixListId']
    except ClientError as e:
        print(f"Error creating prefix list: {str(e)}")
        raise e
def delete_and_detach_prefix_list(prefix_list_name, security_group_id):
    try:
        response = ec2.describe_managed_prefix_lists(
            Filters=[
                {'Name': 'prefix-list-name', 'Values': [prefix_list_name]}
            ]
        )
        prefix_lists = response.get('PrefixLists', [])
        if prefix_lists:
            old_prefix_list_id = prefix_lists[0]['PrefixListId']
            # Detach the old prefix list from the security group
            detach_prefix_list_from_security_group(security_group_id, old_prefix_list_id)
            # Delete the old prefix list
            ec2.delete_managed_prefix_list(PrefixListId=old_prefix_list_id)
    except ClientError as e:
        print(f"Error deleting old prefix list: {str(e)}")
        raise e
def detach_prefix_list_from_security_group(security_group_id, prefix_list_id):
    try:
        ec2.revoke_security_group_ingress(
            GroupId=security_group_id,
            IpPermissions=[
                {
                    'PrefixListIds': [{'PrefixListId': prefix_list_id}],
                    'IpProtocol': 'tcp',
                    'FromPort': 80,
                    'ToPort': 80,
                },
                {
                    'PrefixListIds': [{'PrefixListId': prefix_list_id}],
                    'IpProtocol': 'tcp',
                    'FromPort': 443,
                    'ToPort': 443,
                }
            ]
        )
    except ClientError as e:
        print(f"Error detaching prefix list from security group: {str(e)}")
        raise e
def update_security_group(security_group_id, prefix_list_id):
    try:
        ec2.authorize_security_group_ingress(
            GroupId=security_group_id,
            IpPermissions=[
                {
                    'PrefixListIds': [{'PrefixListId': prefix_list_id}],
                    'IpProtocol': 'tcp',
                    'FromPort': 80,
                    'ToPort': 80,
                },
                {
                    'PrefixListIds': [{'PrefixListId': prefix_list_id}],
                    'IpProtocol': 'tcp',
                    'FromPort': 443,
                    'ToPort': 443,
                }
            ]
        )
    except ClientError as e:
        print(f"Error updating security group: {str(e)}")
        raise e

Remember to replace placeholders like SECURITY_GROUP_ID with actual values from your AWS environment. With this setup, you can ensure that your AWS security group rules are always aligned with the latest IP addresses provided by your third-party services, enhancing the security of your infrastructure.

Explanation:

  1. Importing Libraries: The function starts by importing necessary libraries: json for JSON handling, requests for making HTTP requests, boto3 for AWS SDK, and ClientError from botocore.exceptions for handling AWS client errors.

  2. Initializing EC2 Client: An EC2 client is initialized using boto3.client('ec2'). This client enables interaction with EC2 services such as managing security groups and prefix lists.

  3. Lambda Handler Function: The lambda_handler function is the entry point for the Lambda execution. It receives two parameters, event and context, although they are not used in this specific implementation.

  4. Fetching IP Addresses: The Lambda function makes an HTTP GET request to Stripe's endpoint to fetch the latest IP addresses for Webhook services. The fetched data is then parsed as JSON.

  5. Converting IP Addresses: The fetched IP addresses are converted to CIDR notation (x.x.x.x/32) and stored in cidr_blocks.

  6. Deleting and Detaching Previous Prefix List: Any existing prefix list with the name 'StripeIPs' is deleted and detached from the specified security group to ensure a clean slate.

  7. Creating New Prefix List: A new prefix list named 'StripeIPs' is created using the fetched CIDR blocks.

  8. Updating Security Group: The security group specified by security_group_id is updated to reference the new prefix list created in the previous step. In this case, the security group allows inbound traffic on TCP ports 80 and 443 from the IP addresses specified in the prefix list.

  9. Return Response: Finally, the function returns a response indicating the successful update of the security group rules.

2. EventBridge Scheduler

We'll set up a rule in EventBridge Scheduler to trigger our Lambda function at a specified schedule. We'll configure it to run every night at midnight using the cron expression 0 0 * ? *.

3. AWS IAM Roles

We need to create IAM roles with the following policies to grant our Lambda function the necessary permissions:

· Lambda Execution Policy: Grants permissions to execute Lambda functions and interact with EC2 security group rules.

· EC2 Management Policy: Provides permissions for managing prefix lists, security groups, and related resources.

You can find the detailed IAM policies here.

IAM Roles

Lambda Execution Policy

This policy allows the Lambda function to execute and interact with EC2 security group rules.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:REGION:ACCOUNT NUMBER:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:REGION:ACCOUNT NUMBER:log-group:/aws/lambda/LAMBDA_FUNCTION_NAME:*"
            ]
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "*",
                "*"
            ],
            "Resource": [
                "arn:aws:ec2:REGION:ACCOUNT NUMBER:security-group-rule/*",
                "arn:aws:ec2:REGION:ACCOUNT NUMBER:security-group/SECURITY_GROUP"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "ec2:DescribeSecurityGroupRules",
            "Resource": "*"
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateManagedPrefixList",
                "ec2:ModifyManagedPrefixList",
                "ec2:DeleteManagedPrefixList",
                "ec2:DescribeManagedPrefixLists"
            ],
            "Resource": "*"
        }
    ]
}

EC2 Management Policy

This policy provides permissions for managing prefix lists, security groups, and related resources.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeSecurityGroups",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:DescribePrefixLists",
                "ec2:CreatePrefixList",
                "ec2:DeletePrefixList",
                "ec2:ModifyPrefixList"
            ],
            "Resource": "*"
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:CreateManagedPrefixList",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "ec2:AuthorizeSecurityGroupIngress",
            "Resource": "*"
        }
    ]
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:DescribePrefixLists",
            "Resource": "*"
        }
    ]
}

Replace REGION, ACCOUNT_ID, and YOUR_FUNCTION_NAME with the appropriate values for your AWS environment.

Putting It All Together

Once we have set up our Lambda function, EventBridge Scheduler rule, and IAM roles, the automation workflow will function as follows:

  1. EventBridge Scheduler triggers the Lambda function at the scheduled time (every night at midnight).

  2. The Lambda function fetches the latest IP addresses from the third-party provider.

  3. It creates a new prefix list and updates the security group to reference this new list.

  4. If an old prefix list with the same name exists, it is deleted.

  5. The security group now allows traffic only from the latest IP addresses provided by the third-party provider.

Conclusion

In this blog post, we have explored how to automate the updating of AWS security group rules using Lambda and EventBridge Scheduler. By leveraging these AWS services, we can ensure that our infrastructure remains secure and up-to-date without manual intervention. This approach not only saves time and effort but also reduces the risk of human error.

By implementing similar automation workflows, organizations can enhance their security posture and adapt to changes in external services seamlessly. As the cloud landscape continues to evolve, embracing automation becomes essential for maintaining a robust and secure infrastructure.

Did you find this article valuable?

Support NavyaDevops by becoming a sponsor. Any amount is appreciated!

AWSDevopsKubernetesAWS Certified Solutions Architect AssociateAmazon Web Servicestechnology