media server logo
AWS Instance

How to set up CloudFront on AWS and optimize data transfer

What is CloudFront?

CloudFront in is a CDN (content delivery network) that belongs to Amazon Web Services. CDNs are primarily used for caching (which is what we are going to do), to handle network load or to act like an additional security layer.

If CloudFront is being used and a user requests a webpage or some content, the request is routed to one of Amazon’s 300+ edge server locations. If the edge server already has the resource cached, it’s served to the client. If the resource isn’t on the edge server, it makes a request to a larger edge server called a regional edge cache, if it’s not there, the request goes to the origin server that houses the application. The edge server then saves a copy of the response locally so it can handle the next request without reaching out to the origin server. This reduces load on the origin server, so the instance that is housing the application can remain small. This also reduces latency for clients by moving commonly requested resources closer to the requestor, which is especially valuable for streaming.

Origin Shield

CloudFront Origin Shield is an additional layer in the CloudFront caching infrastructure that helps to minimize your origin’s load, improve its availability, and reduce its operating costs. Origin Shield’s pricing is also based upon the region (or availability zone) its being used in.

All requests to your origin server go through this caching layer, so users accessing different edge servers in different locations can be served the same cached content.

Read more : Origin Shield

When CloudFront would be helpful for streaming

CloudFront is great for optimizing HTTP/HTTPS traffic. This means, it’ll be helpful for those who stream over the HLS protocol.

Unfortunately, CloudFront (at least at the moment) can not proxy UDP traffic. This means it can not optimize SRT or RTMP streaming.

But for those who use HLS, CloudFront provides benefits that are worth looking into :

  • Low latency
  • Data transfer savings
  • Option to get 30% data transfer discount

Be aware of the costs

In this tutorial we are not going to focus on optimizing CloudFront costs, because this topic warrants its own article. And this tutorial is already enormous.

However, here we provide the necessary links for you to get informed about the pricing.

If this is your first experience with AWS, start here

In order to properly follow the steps described in this tutorial, you need to have an account on AWS platform and a EC2 instance with an assigned Elastic IP address.

If none of this makes sense, please read these two tutorials first :

This tutorial consists of three essential parts :

  • Creating Amazon CloudFront distribution
  • Creating a Lambda function in order to automatically update security groups for Amazon CloudFront IP ranges
  • Configuring your Lambda function’s trigger using AWS Command Line Interface (CLI)


Make sure you know how to finish your work and Stop or Terminate the resources you’ve used upon completion of your tasks, as AWS and similar cloud platforms charge for the time when these resources are “Running”.

Technically speaking, if your instance or any other resource is active, it means you are still taking this resource from AWS pool. AWS does not check whether you are actually doing something with the resource.

If you have not stopped the resource and it is running— you are using it and you are being charged accordingly.

Always make sure to stop or delete your instances and other resources when they are no longer needed.

This applies to Instances, CloudFront distributions, Elastic IPs and everything you are renting on AWS — make sure you turn it off when its not needed, so you won’t pay for something you didn’t actually use.

We put this warning in the beginning of our tutorial, so that nobody will miss this information.

Be careful with what is running in your account. Make it a habit to turn everything off when your mission is done.

Some resources cost very small bits of money and end up around $4 per month, some resources can add up to hundreds of $ in a matter of days.

Now that you are warned and prepared, let’s begin.

  1. Log into EC2 Console

2. First, make sure your instance has an associated Elastic IP address.
We are going to link our CloudFront distribution to Public IPv4 DNS. Public IPv4 DNS will change every time you launch your instance after pausing it if you do not use Elastic IP.

If you don’t know how to assign Elastic IP to your instance, please check out our tutorial : How to set up Elastic IP on AWS

Your instance does not have to be active while you are setting up if it has Elastic IP.

Copy Public Ipv4 DNS of your instance.
Keep it in your clipboard for now, we’ll use it very soon.

3. Open the Amazon CloudFront page or search for it directly in the console.

Click “Create a CloudFront distribution”

4. In the page that opens, we are going to fill in a few fields and change some settings. Please leave the settings we are not mentioning here unchanged, unless you fully understand what they do.

Origin domain
Paste Public IPv4 DNS address (the one you’ve copied).

Enable Origin Shield
Select “Yes”

Origin Shield Region
If the region where you’ve launched your instance is in the drop-down list, then select it.
If it’s not on the list, please use the table we provide on the screenshot below to understand which region to pick.

5. “Viewer” section

Viewer protocol policy
Select “Redirect HTTP to HTTPS”

Allowed HTTP methods

“Cache key and origin requests” section

Select Cache policy and origin request policy (recommended)

  • Under Cache policy
    select CachingOptimized
  • Under Origin request policy — optional
    Select AllViewer

Scroll down to the end of the page.
Click “Create distribution”

6. On the next page you’ll see the domain name. Your server will be available on this domain via CloudFront.
In the Last modified column you will see the Deploying status. Please wait until it changes to the date of the last setting.

Disabling / Deleting distribution

As always, a gentle reminder to disable/delete your distribution once you no longer need it.

Keep in mind, you can only delete an already disabled distribution.

To disable a distribution, select your distribution and click “Disable”

To delete your disabled distribution, select it and click “Delete”

Ta-da! First part of the tutorial is finished.

Creating a Lambda function to automatically update security groups for Amazon CloudFront IP ranges

What is Lambda

Amazon Lambda lets you run code without the need to obtain or manage servers. You pay only for the compute time you consume — there is no charge when your code is not running. You can upload your code and Lambda will take care of everything required to run and scale your code with high availability.

You can set up your code to automatically trigger from other Amazon Web Services services or call it directly from any web or mobile app — and this is exactly what we are going to use Lambda for.

Learn more :


So, now we are going to create a Lambda function that will update security groups for our instances in the event if any change of the range of available IP addresses occurs. And all in order to keep CloudFront working smoothly.

Before we dive in, let’s talk about why do we need to update CloudFront IP ranges. Here’s a little summary from the original AWS article :

For CloudFront to access an origin (the source of the content behind CloudFront), the origin has to be publicly available and reachable.

Amazon Simple Storage Service (Amazon S3) origins provide a feature called Origin Access Identity, which blocks public access to selected buckets, making them accessible only through CloudFront. When you use CloudFront to secure your web applications, it’s important to ensure that only CloudFront can access your origin (such as Amazon EC2) and any direct access to origin is restricted.

AWS publishes the IP ranges in JSON format for CloudFront and other AWS services. If your origin is an Elastic Load Balancer or an Amazon EC2 instance, you can use VPC security groups to allow only CloudFront IP ranges to access your applications. The IP ranges in the list are separated by service and Region, and you must specify only the IP ranges that correspond to CloudFront.

The IP ranges that AWS publishes change frequently and without an automated solution, you would need to retrieve this document frequently to understand the current IP ranges for CloudFront. Frequent polling is inefficient because there is no notice of when the IP ranges change, and if these IP ranges aren’t modified immediately, your client might see 504 errors when they access CloudFront. Additionally, there are numerous IP ranges for each service, performing the change manually isn’t an efficient way of updating these ranges.

Read original article on AWS : Automatically update security groups for Amazon CloudFront IP ranges using AWS Lambda

In short

We need to update the range of IP addresses in order for Cloud Front work properly, because otherwise the needed IP addresses would not be able to access our origin and thus will not be able to provide access to end users.

Now let’s begin.

Creating the IAM policy for your role

The first thing you need to do is create a Lambda function execution role and policy. Lambda function uses execution role to access or create AWS resources.

1. Open Identity and Access Management (IAM) Console
The account used must have administrator rights.

Go to Policies

Click “Create Policy”

2 . Open the JSON tab

Copy the code below and paste it into the JSON tab

  "Version": "2012-10-17",
  "Statement": [
      "Sid": "CloudWatchPermissions",
      "Effect": "Allow",
      "Action": [
      "Resource": "arn:aws:logs:*:*:*"
      "Sid": "EC2Permissions",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags",  // <-- Added comma here
      "Resource": "*"

Click “Next : Tags”

4. Click “Next : Review”

5 . On the third screen

Create a name for your policy (it should be specific and recognizable), for example LambdaExecRolePolicy-UpdateSecurityGroupsForCloudFront.

Click “Create policy”

6. Go to Roles

Click “Create role”

7. Select “AWS Service”

Use case : Lambda

Click “Next”

8. Find the name of the policy you’ve just created (in our case the name is LambdaExecRolePolicy-UpdateSecurityGroupsForCloudFront) and tick the checkbox next to it

Click “Next”

9. Role name :
Create a name for your role.
Name has to be specific and recognizable, in our case the name is LambdaExecRole-UpdateSecurityGroupsForCloudFront.

Scroll down to the end of the page.

Click “Create role”

Creating a Lambda function

Now that we’ve prepared our role and and our policy, we are going to create a Lambda function that will execute the policy.

1. Change your region to N.Virginia
Even if your instance in not in N.Virginia (or not even in the United States), you need to use N.Virginia, because it’s the default region.
Please do not worry, we will assign our function to our desired region a little later down the line.

2. Open the Lambda console
Or use the search function to find it.

3. In the Lambda console, go to Functions
Click “Create function”

4. In the next page that opens, select “Author from scratch”

Function name
Create a name for your function.
As always, the name should be specific and recognizable.
In our example the name is UpdateSecurityGroupsForCloudFront.

select Python 3.8

Expand “Change default execution role” settings

Select Use an existing role

Existing role
Select the name of the role you’ve created

Click “Create function”

5. In the next page that opens, switch to the Code tab

Copy the code below and paste it into the lambda_function tab


import boto3
import hashlib
import json
import logging
import urllib.request
import os
import random
import string

INGRESS_PORTS = os.getenv('PORTS', "80").split(",")
VPC_ID = os.getenv('VPC_ID', "")
REGION = os.getenv('REGION', "us-east-1")

def lambda_handler(event, context):
    global NRANGES
    # Set up logging
    if len(logging.getLogger().handlers) > 0:
    # Set the environment variable DEBUG to 'true' if you want verbose debug details in CloudWatch Logs.
        if os.environ['DEBUG'] == 'true':
    except KeyError:
    # SNS message notification event when the ip ranges document is rotated
    message = json.loads(event['Records'][0]['Sns']['Message'])
    ip_ranges = json.loads(get_ip_groups_json(message['url'], message['md5']))
    cf_ranges = get_ranges_for_service(ip_ranges, SERVICE)
    # Number of security group rules required as per the total range count
    NRANGES = len(cf_ranges) * len(INGRESS_PORTS)
    # Update SGs with the new ranges

def update_security_groups(new_ranges):
    global VPC_ID
    # Creating ec2 boto3 client
    client = boto3.client('ec2', region_name=REGION)
    if VPC_ID == "":
        result = client.describe_vpcs(Filters=[{'Name': 'isDefault', 'Values': ['true']}, ])
        VPC_ID = result["Vpcs"][0]['VpcId']
    # To number of SGs to update
    rangeToUpdate = get_security_groups_for_update(client, True)
    if len(rangeToUpdate) == 0:
        logging.warning('No groups to {}'.format("update"))
        update_security_group(client, rangeToUpdate, new_ranges)

def update_security_group(client, rangeToUpdate, new_ranges):
    old_prefixes = list()
    to_revoke = {}
    to_add = list()
    final_add = {}
    total = 0
    for each_grp in rangeToUpdate['SecurityGroups']:
        to_revoke[each_grp['GroupId']] = set()
        # If there are any existing ranges in the SG, compare and add it to the revoke list if necessary
        to_revoke_sg = 0
        if len(each_grp['IpPermissions']) > 0:
            for permission in each_grp['IpPermissions']:
                for range in permission['IpRanges']:
                    cidr = range['CidrIp']
                    if new_ranges.count(cidr) == 0:
                        to_revoke_sg += 1
            remain_rules = NRULES - (len(each_grp['IpPermissions'][0]['IpRanges']) * len(INGRESS_PORTS)) + to_revoke_sg
  "Total number of rules available in " + each_grp['GroupId'] + " are " + str(remain_rules)))
            final_add[each_grp['GroupId']] = remain_rules
            total += remain_rules
            final_add[each_grp['GroupId']] = NRULES
            total += NRULES
    # Compares and identifies the new range to add from the service ranges list
    for range in new_ranges:
        if old_prefixes.count(range) == 0:
            to_add.append({'CidrIp': range})
  " Range to be added: " + range))
    count = 0
    for group in to_revoke:
        if len(to_revoke[group]) > 0:
            count += len(to_revoke[group])
  "Rules that have to be revoked for  " + str(to_revoke[group])))
            revoke_permissions(client, group, to_revoke[group])
  "No rules were identified to be revoked in the security group " + group))"Total number of rules to be revoked in all the security groups are " + str(count * len(INGRESS_PORTS))))"Total number of rules to be added " + str(len(to_add) * len(INGRESS_PORTS))))"Rules to add " + str(to_add)))
    dynamic_rule_add(client, final_add, to_add, total)

def dynamic_rule_add(client, final_add, to_add, total):
    random_str = ''.join(random.choices(string.ascii_uppercase + string.digits, k=3))
    if total < (len(to_add) * len(INGRESS_PORTS)):
        security_group = client.create_security_group(
            Description=NAME + "-" + random_str,
            GroupName=NAME + "-" + random_str,
        all_sgs = list(final_add.keys())
        response = client.describe_network_interfaces(
                    'Name': 'group-id',
                    'Values': all_sgs
        final_add[security_group['GroupId']] = NRULES
        all_sgs = list(final_add.keys())
        for each_eni in response['NetworkInterfaces']:
            response = client.modify_network_interface_attribute(
    for each_grp in final_add:
        num_accomodate = final_add[each_grp] // len(INGRESS_PORTS)
        remain_per_grp = final_add[each_grp] % len(INGRESS_PORTS)"Number of rules can security group " + each_grp + " accommodate: " + str(
            num_accomodate * len(INGRESS_PORTS))))
        for each_proto in INGRESS_PORTS:
            permission = {'ToPort': int(each_proto), 'FromPort': int(each_proto), 'IpProtocol': 'tcp'}
            add_params = {
                'ToPort': permission['ToPort'],
                'FromPort': permission['FromPort'],
                'IpRanges': to_add[0:num_accomodate],
                'IpProtocol': permission['IpProtocol']
            client.authorize_security_group_ingress(GroupId=each_grp, IpPermissions=[add_params])
  "Modified " + str(len(to_add[0:num_accomodate])) + " rules on security group " + each_grp +
                          " for the port " + each_proto))
            to_add = to_add[num_accomodate:]

def revoke_permissions(client, group, to_revoke):
    # Revoked rules in each SG for every port number
    for each_proto in INGRESS_PORTS:
        permission = {'ToPort': int(each_proto), 'FromPort': int(each_proto), 'IpProtocol': 'tcp'}
        revoke_params = {
            'ToPort': permission['ToPort'],
            'FromPort': permission['FromPort'],
            'IpRanges': [{'CidrIp': iprange} for iprange in to_revoke],
            'IpProtocol': permission['IpProtocol']
        client.revoke_security_group_ingress(GroupId=group, IpPermissions=[revoke_params])"Revoked " + str(len(to_revoke)) + " rules from the security group " + group + " with port " +
                      each_proto))"Ranges revoked from the security group " + group + " are: " + str(to_revoke)))

def create_security_groups(client, response):
    num_sgs = len(response['SecurityGroups'])'Found ' + str(num_sgs) + ' security groups'))
    total_sgs_required = NRANGES // NRULES
    if NRANGES % NRULES > 0:
        total_sgs_required += 1'Total number of security groups required to add all the rules: ' + str(total_sgs_required)))
    to_create_sgs = 0
    if num_sgs < total_sgs_required:
        to_create_sgs = total_sgs_required - num_sgs'Total number of security groups to be created: ' + str(to_create_sgs)))
    # Creates SGs based on the total number of rules that are required to be added
    created_sgs = []
    for sg in range(to_create_sgs):
        random_str = ''.join(random.choices(string.ascii_uppercase + string.digits, k=3))
        security_group = client.create_security_group(
            Description=NAME + "-" + random_str,
            GroupName=NAME + "-" + random_str,
        response = client.create_tags(Resources=created_sgs,
                                          'Key': 'PREFIX_NAME',
                                          'Value': NAME,
                                      ], )
    return get_security_groups_for_update(client)

def get_security_groups_for_update(client, create=False):
    filters = [
        {'Name': "tag-key", 'Values': ['PREFIX_NAME']},
        {'Name': "tag-value", 'Values': [NAME]},
        {'Name': "vpc-id", 'Values': [VPC_ID]}
    # Extracting specific security groups with tags
    response = client.describe_security_groups(Filters=filters)
    # Return list of all security groups if none to be created
    if create == False:
        return response
        return create_security_groups(client, response)

def get_ip_groups_json(url, expected_hash):"Updating from " + url)
    response = urllib.request.urlopen(url)
    ip_json =
    m = hashlib.md5()
    hash = m.hexdigest()
    if hash != expected_hash:
        raise Exception('MD5 Mismatch: got ' + hash + ' expected ' + expected_hash)
    return ip_json

def get_ranges_for_service(ranges, service):
    service_ranges = list()
    for prefix in ranges['prefixes']:
        if prefix['service'] == service:
            service_ranges.append(prefix['ip_prefix'])'Found ' + service + ' ranges: ' + str(len(service_ranges))))
    return service_ranges

In case if the code does not work due to issues with the formatting, use this link to copy it :

Click “Deploy”

6. Switch to the Configuration tab

Go to General configuration

Click “Edit”

7. Change Timeout to 10 seconds

Click “Save”

8. Go to Environment variables

Click “Edit”

9. Now we are going to assign our function to our desired region.

Click “Add environment variable”

10. Fill in the fields


Value : AWS id of your desired region.
To find out the correct id, please refer to the table.
The table is scrollable. The data you need is in the Region column.

In our example the region name is Europe (Stockholm) and the region id is eu-north-1.

Click “Save”

Make sure that the end result looks like this

11. Ta-da! We’ve finished creating our function, now we are going to test it.

Switch to the Test tab

Select “Create new event”

Event name
Create a name. In our example the name is TriggerSNS

Event sharing settings : Private

Copy and paste provided code into the Event JSON field

"Records": [
"EventVersion": "1.0",
"EventSubscriptionArn": "arn:aws:sns:EXAMPLE",
"EventSource": "aws:sns",
"Sns": {
"SignatureVersion": "1",
"Timestamp": "1970-01-01T00:00:00.000Z",
"Signature": "EXAMPLE",
"SigningCertUrl": "EXAMPLE",
"MessageId": "95df01b4-ee98-5cb9-9903-4c221d41eb5e",
"Message": "{\"create-time\": \"yyyy-mm-ddThh:mm:ss+00:00\", \"synctoken\": \"0123456789\", \"md5\": \"7fd59f5c7f5cf643036cbd4443ad3e4b\", \"url\": \"\"}",
"Type": "Notification",
"UnsubscribeUrl": "EXAMPLE",
"TopicArn": "arn:aws:sns:EXAMPLE",
"Subject": "TestInvoke"

Click “Save”
Click “Test”

12. You’ll see an error message. This is fine, don’t worry, it’s supposed to be like this.

Click “Details” to see the contents of the error message

Copy the first MD5 hash from the error message.

13. Now we are going to edit the test event again.

Click “Format JSON”

Replace the MD5 hash value with the one you’ve just copied.

In our example we will change MD5 value in the event sample
from 7fd59f5c7f5cf643036cbd4443ad3e4b
to 13da9963cffcd98ba3b0d94c02ee1444

Click “Save changes”

Then click “Test”

Your function will now run successfully.

14. Congratulations! The function is completely done, now we are going to see what it has done.

Open the EC2 Console
Make sure you have your desired region selected (in our case it’s Stockholm)

Go to Security Groups

You should see new security groups created by the function. New security groups can be identified in the EC2 console by the name AUTOUPDATE_smth.

The function creates these security groups and puts maximum possible number of rules into each group.

15. Now we are going to assign our new security groups to our instance.

Go to Instances and select your instance
Click Actions > Security > Change security groups

16. On the page that opens

Use the search bar to add your new security groups. Click “Add security group” after adding each group.

Then click “Save”

17. Go back to EC2 Console
Go to “Instances”
Select your instance and click the “Security” tab below
Click the security group name that does NOT have AUTOUPDATE in its name.

You’ll get to an overview page of the security group.
Click Actions > Edit inbound rules

Here we’ll have to delete some rules and to create some new ones instead.

Delete rules for 22 and 80 Port range. Use the “Delete” button on the right.

Create rules for Port range 1935 and 1945. Use the “Add rule” button at the bottom.

The rules have to look like this :
1 — Custom UDP ; Port range 1935 ;
2 — Custom TCP ; Port range 1945 ;

Deleting Lambda function

As always, a gentle reminder : delete your function when you finish your work and no longer need it.

To delete your function, simply select it and then click Actions > Delete

Tada! Congratulations, second part of this tutorial about the Lambda function has ended. Now on to the last one.

Configuring your Lambda function’s trigger using AWS Command Line Interface (CLI)

What is AWS CLI?

The AWS Command Line Interface (AWS CLI) is an open source tool that enables interacting with AWS services using commands in your command-line shell.

Installing AWS CLI

Installation process for AWS CLI slightly differs, depending on the operation system you have.

Please read this manual, it contains instructions for all of the popular operation systems (Windows, MacOS, Linux) :
Installing or updating the latest version of the AWS CLI

If you have Windows, installing AWS CLI might be more confusing, so I decided to add a video tutorial by “Be A Better Dev” channel:
How to install and configure the AWS CLI on Windows 10

There are also videos explaining how to install CLI on MacOS and Linux and you can look them up too.

Getting access to AWS services via CLI

Now that we’ve installed AWS CLI, we have to create access keys to be able to send commands to AWS.

1. On AWS, click onto your username and select Security credentials

2. In the page that opens, we are going to generate Access Key. We will use this Key to allow CLI access our AWS services.

Expand the Access keys (access key ID and secret access key) block

Click “Create New Access Key”

In the popup window click “Download Key File”

You’ll download a .CSV file. Keep it somewhere safe. If you lose it, you’ll have to delete your key and generate a new one.

3. Open the Terminal on your PC and type aws configure command.

This command is going to prompt you for four pieces of information.

  • Access key ID
  • Secret access key
  • AWS Region
  • Output format

The following example shows sample values. Replace them with your own values.

$ aws configure 
AWS Secret Access Key [None]:wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY Default region name [None]: us-west-2
Default output format [None]: json

For more details about AWS CLI configuration and this particular command, read Configuration basics.

4. Next, we need to connect our Lambda function with the SNS (Simple Notification Service). To do that, first, we need to get ARN of our Lambda function. ARN means Amazon Resource Name. Learn more about ARN : Resources and conditions for Lambda actions

Open your Lambda function

Click “Copy ARN” or copy it directly

5. Now we are going to run a command to get a SNS ARN of our Lambda function.

In the provided code replace Lambda ARN with the ARN you’ve just copied from your Lambda function.

aws sns subscribe --topic-arn "arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged" --region us-east-1 --protocol lambda --notification-endpoint "Lambda ARN"

Then run the command in your Terminal

6.Now we are going to give the SNS permissions to call the Lambda function. The following command also adds a Lambda trigger.

In the provided command, replace the Lambda ARN with your ARN copied from the Lambda function.

aws lambda add-permission --function-name "Lambda ARN" --statement-id lambda-sns-trigger --region us-east-1 --action lambda:InvokeFunction --principal --source-arn "arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged"

Run the command in your Terminal

When AWS changes any of the IP address ranges in a document, an SNS notification is sent and your Lambda function is activated. This Lambda function checks for changed ranges in the document and effectively updates the IP address ranges in existing security groups.

In addition, the feature dynamically scales and creates additional security groups if the number of IP address ranges for CloudFront increases in the future. Any newly created security groups are automatically attached to the network interface to which the previous security groups are attached to avoid service interruption.

This is the end of our first tutorial about AWS CloudFront.
In the future we are going to add more tutorials about CloudFront settings and features. Stay tuned.

If you have any questions, please contact us at: [email protected]
We will respond to you within 48 hours.
Happy streaming!

Related articles :
How to launch Callaba Cloud Live Streaming
How to start streaming in OBS Studio over the SRT Protocol
How to multi-stream from OBS Studio to Twitch, Youtube and Facebook
How to multi-stream from Wirecast to Twitch, Youtube & Facebook
How to set up geo-distributed routing of video streams using SRT protocol and Callaba Cloud