Serve Django Static and Media Files in Production

Serving static and media files from the same server as your Django project during production can slow down site rendering and image loading time. 

This tutorial will provide instructions on how to securely serve Django static files and Django media files using Amazon S3 and Amazon CloudFront.

To begin, we will cover the AWS S3 and AWS CloudFront setup. Then we'll move on to configuring Django file storage in an S3 bucket. Finally, we will call the static and media files from the CloudFront domain.  

If you have not used static and media files in Django, check out the articles Manage Django Static Files and How to use Django Models before continuing with this article.

 

NOTE: These configurations can be done before the deployment of your site. 

 

---

 

Amazon Web Services - Setup

Create an AWS account

Create an Amazon Web Services account at https://aws.amazon.com/free/. You may need to enter your billing and credit card information to create the account but you will not be charged unless you exceed the storage limit of the free tier. 

The free tier provides access to over 60 AWS products including the IAM, S3, and CloudFront Services used below. 

If you already have an AWS account, log in.

GIF

 

Add AWS S3 permissions to the user using the IAM service

Before you can do anything else, you need to give the user the correct permissions to access and create S3 buckets, the storage system that will hold your Django static and media files in production. 

Once logged to your free AWS account, you will be brought to the AWS Management Console:

1. Find the IAM service to manage users' access to AWS resources. The IAM service can be found (a) by typing in IAM in the Find Services search bar or (b) finding it listed under the Security, Identify, & Compliance section under All Services.
2. Click on the IAM link and you will be brought to the IAM dashboard.
3. Then click on the Users tab located on the left side menu.
4. Click on the user name you want to have access to the S3 bucket.
5. Click the "Add permissions" button.
6. Select the "Attach existing policies directly" option and type in "AmazonS3FullAccess".
7. Select the policy "AmazonS3FullAccess".
8. Then click the "Next: Review" button at the bottom of the page.
9. The Permissions summary page will appear and click the "Add permissions" button at the bottom of the page.

You will then be redirected to the User summary page with "AmazonS3FullAccess" listed under Permission policies. 

GIF

 

 

---

 

Django AWS S3 Bucket

AWS S3 or Amazon Simple Storage Service provides object and file storage through their web service interface. The service is built on a scalable storage infrastructure that only charges based on the space used.

S3 buckets are what hold the actual objects and folder in AWS S3, so you need to create an S3 bucket for your Django static and media files. 

 

Create an S3 bucket for your Django static & media files

1. Return to the AWS Management Console.
2. Find the S3 service for scalable cloud storage. The S3 service can be found (a) by typing in S3 in the Find Services search bar or (b) finding it listed under the Storage section under All Services.
3. Click on the S3 link. 
4. Once on the S3 services page, click the "Create bucket" button at the top of the page.
5. Under General configuration, name the bucket something related to your website so it is easily identifiable. 
6. For the region, select your region of choice.
7. Leave everything else as the default.
8. Under Bucket settings for Block Public Access, make sure "Block all public access" is selected. This will restrict access to the Django static and Django media files uploaded to the S3 bucket. 

GIF

 

Add CORS permissions to the S3 bucket for the Django Admin stylesheets

When all of your Django static files are uploaded to your S3 bucket, the Django admin stylesheets and files are also included in this. That means that instead of them being served from the deployed domain, they will come from the CloudFront domain. This will cause a CORS or a cross-origin resource sharing error in production and cause them not to be loaded.

To prevent this:

1. Select the bucket and go to the Permissions tab.
2. Select the button "CORS configuration" and add the code below.
3. Replace the AllowedOrigin with your production domain.
4. Save the configuration.

Note: The CORS configuration in the new S3 console must be JSON.

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "PUT",
            "POST",
            "DELETE"
        ],
        "AllowedOrigins": [
            "http://www.your-domain.com"
        ],
        "ExposeHeaders": []
    },
    {
        "AllowedHeaders": [],
        "AllowedMethods": [
            "GET"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": []
    }
]

 

Again, be sure to save the changes. 

 

---

 

Connect Django S3 bucket to a CloudFront domain

CloudFront is a content delivery network that can speed up the delivery of images and videos. The service generates custom domains that can connect to and serve content stored in the private S3 bucket.

 

Create a CloudFront for your AWS S3 bucket

1. Return to the AWS Management Console.
2. Find the CloudFront service for scalable cloud storage. The CloudFront service can be found (a) by typing in CloudFront in the Find Services search bar or (b) finding it listed under the Networking & Content Delivery section under All Services.
3. Click on the CloudFront link to be brought to the CloudFront Distributions page. 
4. Under the Distributions tab, click the "Create Distribution" button.
5. Select the Web distribution delivery method for your content.

GIF

 

Under Origin Settings:

1. Select the name of your S3 bucket for Origin Domain Name.
2. Then select "Yes" for Restrict Bucket Access.
3. For Origin Access Identify, select "Create a New Identify". The Comment field should then auto-populate with an access identity based on your S3 bucket name. If your CloudFront distribution was created beforehand, you can select "Use an Existing Identity" and select your Identify from the dropdown menu (see image below).
4. For Grant Read Permissions on Bucket select "Yes".

Keep everything else as the default in this section.

 

---

 

Configure your CloudFront domain to accept CORS

You have already added the CORS configurations to the S3 bucket, now you need to whitelist the headers in the S3 bucket for CloudFront. 

 

Whitelist S3 bucket for Django static files

Under Default Cache Behavior Settings:

1. Change Allowed HTTP Methods to "GET, HEAD, OPTIONS".
2. Select "Whitelist" for Cache Based on Selected Request Headers.
3. Click "Use legacy cache settings" for the Cache and origin request settings option
4. Now, under the new option Cache Based on Selected Request Headers, select "Whitelist" from the dropdown menu.
5. For Whitelist Headers type "Access-Control-Allow-Origin" and click the "Add Custom >>" button for it to appear as whitelisted.

Keep everything else as the default.

Click "Create Distribution" at the bottom of the page to create the CloudFront distribution.

 

The distribution will then be created and start deploying. This will take a few minutes.

While you wait, you can go to the next step and begin to connect your Django project to your S3 bucket.

Take note of the Domain Name given to the distribution. You will need to add this to your Django project in a few minutes.

 

---

 

Connect Django project to your AWS account

The first step in connecting the Django project with an Amazon S3 bucket is to create a set of access keys that will allow you to connect your AWS account to your Django project. 

 

Create a set of AWS access keys

1. Click on your user name at the top of the page and in the drop-down menu select "My Security Credentials".
2. Find "Access keys for CLI, SDK, & API access".
3. Then click on the "Create access key" button. 

You will then see a popup that states "Your new access key is now available" with two keys listed below. The first is the Access key ID and the second is the Secret access key.

1. Click the "Download .csv file" button and save the access keys in a safe location on your computer. You can only view the secret key once in this popup so make sure to download the .csv file before closing the menu. 

If you already have a set of access keys, there is no need to create another set. If you've misplaced old access keys, create a new set.

 

GIF

 

---

 

Django S3 Storage

Next, you need to download two packages, Django storages and boto3. These packages will make it easier to upload and connect to your AWS S3 bucket and CloudFront distribution.

 

Install Django-storages boto3

macOS Terminal

(env)User-Macbook:mysite user$ pip install boto3

(env)User-Macbook:mysite user$ pip install django-storages

Windows Command Prompt

(env) C:\Users\Owner\desktop\code\env\mysite>pip install boto3

(env) C:\Users\Owner\desktop\code\env\mysite>pip install django-storages

Open the CLI with your Django project and install the following Python packages: pip install boto3 and pip install django-storages. 

 

Add Django storages to installed apps

env > mysite > main > settings.py

# Application definition

INSTALLED_APPS = [
    'main.apps.MainConfig',
    ...
    ...
    'storages',
]

Once the installations are complete, open settings.py and add Django storages to INSTALLED_APPS. 

 

---

 

Django static file storage

Now you need to change the Django static URL so the files uploaded will no longer upload to the Django static folder in the project but the S3 bucket made for the Django static files.

 

Django static URL in development

env > mysite > mysite > settings.py

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.1/howto/static-files/

# STATIC_URL = '/static/'

# STATICFILES_DIRS = [
#     os.path.join(BASE_DIR, 'static'),
# ]

Comment out the original STATIC_URL and STATICFILES_DIRS.

 

Change the Django static URL to the Django S3boto3storage

env > mysite > mysite > settings.py

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.1/howto/static-files/

# STATIC_URL = '/static/'

# STATICFILES_DIRS = [
#     os.path.join(BASE_DIR, 'static'),
# ]

AWS_ACCESS_KEY_ID = 'your-access-key-id'
AWS_SECRET_ACCESS_KEY = 'your-secret-access-key'

AWS_STORAGE_BUCKET_NAME = 'your-S3-bucket-name'
AWS_S3_CUSTOM_DOMAIN = 'your-cloudfront-domain.com'

AWS_LOCATION = 'static'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'static'),
]

Then add the AWS configuration settings to the bottom of settings.py. The four configurations below need to be replaced with your own AWS credentials and information.

1. Add your AWS Access key ID
2. Add your Secret access key
3. Add your AWS storage bucket name
4. Add your AWS S3 custom domain. This is the auto-generated CloudFront domain see in the AWS CloudFront Distributions table.

The next two configurations do not need to be changed.

1. Add the AWS location. In this case, it is a file called 'static'.
2. Add the static files storage location. This must be 'storages.backends.s3boto3.S3Boto3Storage'.
3. Update the static URL to be the AWS S3 custom domain and the AWS upload location.

 

As you can see, there is sensitive configuration information, like your AWS access keys, located in this file. We highly recommend using Python Decouple to separate and secure this information.

 

Django static URL for production

env > mysite > mysite > settings.py

...

STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)

...

The static URL domain (i.e. the location where all of the static files will not be uploaded) is now https://your-cloudfront-domain.com/your-S3-bucket-name/. 

 

---

 

Django collect static files

With all of the configurations complete, it's time to gather all of the Django static files and upload them to the S3 bucket.

 

Collect the static files

macOS Terminal

(env)User-Macbook:mysite user$ python manage.py collectstatic

You have requested to collect static files at the destination
location as specified in your settings:

    ...

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes

Windows Command Prompt

(env)C:\Users\Owner\desktop\code\env\mysite> py manage.py collectstatic

You have requested to collect static files at the destination
location as specified in your settings:

    ...

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes

Open the CLI again and run collectstatic to gather and upload the static files to the S3 bucket specified in the settings.

Type yes when prompted.

Then wait for all of the static files to be uploaded to the S3 bucket. 

 

View Django static folder in the S3 bucket

If you are interested, you can go back to your AWS S3 console and click on the S3 bucket you connected to this Django project. Within the bucket, there should be a folder called static that contains all of the Django static files previously located on your project.

 

Django template static URL

As for the Django HTML templates within your project, none of the static URLs need to be changed. They are still called using the template tag {% static 'img/example.png' %}. That's because the static folder in the S3 bucket works just like your local static folder. Only now the folder does not take up any server space. 

 

View the Django static files served from the S3 bucket

To see your static images loading in the browser, run your local server. 

If you inspect the image in the browser, you will notice that it is serving from https://your-cloudfront-domain.com/static/img/example.png. 

 

---

 

Serving Django media files in production

Having Django media files uploaded directly to the S3 bucket is also a good idea given that they also take up unnecessary space on your production server.

But there is no need to add this file to your project if you are not uploading media files to the S3 bucket. 

 

Create a file for Django media files

env > mysite > mysite > (New File) storage_backends.py

#storage_backends.py

If you have Django media files in your project, add a new file called storage_backends.py to the same folder containing settings.py.

 

Import S3Boto3Storage 

env > mysite > mysite > storage_backends.py

#storage_backends.py


from storages.backends.s3boto3 import S3Boto3Storage

Import s3Boto3Storage configurations at the top of the page.

 

Add the media upload location

env > mysite > mysite > (New File) storage_backends.py

from storages.backends.s3boto3 import S3Boto3Storage

class MediaStorage(S3Boto3Storage):
    location = 'media'
    file_overwrite = True

Then create a media storage class. Specify the Django media location, the name of the folder in the S3 bucket, and add a file overwrite to prevent duplicate uploads. Save the file before moving on. 

Other classes and locations can be added as needed.

 

---

 

Django media settings

Now you need to connect this new file to the existing S3 bucket settings. Again, there is no need to add these configurations if you are not using media files. 

 

Django media root

env > mysite > mysite > settings.py

# MEDIA_URL = '/media/'

# MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

First, comment out the previous Django media root and Django media URL in the settings. These were for development only.

 

Django default file storage for production

env > mysite > mysite > settings.py

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.1/howto/static-files/

# STATIC_URL = '/static/'

# MEDIA_URL = '/media/'

# MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

AWS_ACCESS_KEY_ID = 'your-access-key-id'
AWS_SECRET_ACCESS_KEY = 'your-secret-access-key'

AWS_STORAGE_BUCKET_NAME = 'your-S3-bucket'
AWS_S3_CUSTOM_DOMAIN = 'your-cloudfront-domain'

AWS_LOCATION = 'static'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

DEFAULT_FILE_STORAGE = 'mysite.storage_backends.MediaStorage' #the media storage configurations

STATIC_URL = '/static/'

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'static'),
]

Then add the media file storage configurations to the settings (i.e. 'mysite.storage_backends.MediaStorage').

 

Delete/comment out the if statement in mysite > urls.py

"""mysite URL Configuration

The `urlpatterns` list routes URLs to views. For more information please see:
    https://docs.djangoproject.com/en/2.1/topics/http/urls/
Examples:
Function views
    1. Add an import:  from my_app import views
    2. Add a URL to urlpatterns:  path('', views.home, name='home')
Class-based views
    1. Add an import:  from other_app.views import Home
    2. Add a URL to urlpatterns:  path('', Home.as_view(), name='home')
Including another URLconf
    1. Import the include() function: from django.urls import include, path
    2. Add a URL to urlpatterns:  path('blog/', include('blog.urls'))
"""
from django.contrib import admin
from django.urls import path, include
# from django.conf import settings 
# from django.conf.urls.static import static 

urlpatterns = [
    path('', include ('main.urls')),
    path('admin/', admin.site.urls),
]


# if settings.DEBUG: #add this
#     urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Before you can run the server again, you need to go into the mysite > urls.py and delete or comment out the if statement for media uploads.

If you never configured your Django project for media uploads you can skip this step.

 

---

 

Django upload media to S3 bucket

It's time to see if the media upload works correctly.

 

Upload Media files to the S3 bucket

Go to your Django media upload location and select a media file to upload.

Upload a file and it should now be uploaded to your S3 bucket rather than the media folder in your project's root directory. 

 

View the Django media folder in the S3 bucket

To double-check that the file uploaded to the S3 bucket, go to the AWS S3 console and click on the S3 bucket you connected to your Django media files. There should be a folder called media with the uploaded file. 

 

Django template media URL

If you upload media files via a Django model, your template media URL will remain <img src="{{object.image_field.url}}">. If you are unsure how to call your image in a template, visit How to use Django Models.

Else, call the image directly using the full URL, https://your-CloudFront.com/media/images/example.png, as your image source. 

 

View the full image address using Developer Tools

To see your media images rendered in the browser, run your local server. 

 

If you inspect the image in the browser, you will notice that it is serving from https://your-CloudFront.com/media/images/example.png. 

 

---

 

Django S3 Storage - Troubleshooting 

If Django static files not working:

1. Check that your AWS configuration settings are correct.
2. Check that you spelled the name of your S3 bucket correctly in your Django settings. 
3. Make sure you added the CORS permission to both the S3 bucket and whitelisted the headers in the CloudFront distribution.

 

If Django media files not working:

1. Check to make sure your storage_backends.py file is spelled correctly.
2. Make sure you added the default file storage to the settings.
3. Be sure to delete or comment out the if statement related to the media uploads in the urls.py file. It is not needed for production.