From Server to Serverless: A Timeline and Tutorial

Sept. 2, 2020, 2:58 p.m.

Amazon Web Services is a behemoth powering companies from Fortune 500 corporations to highly valued startups. However, how exactly did this happen? What were the actual problems cloud services providers solved and why is serverless computing getting so much attention? Let's start with a brief timeline to answer these questions. Then we'll jump into a simple serverless tutorial to create an API.

Server to Serverless Timeline

The 1990s:

With the internet still in its infancy in the 1990s, building web applications presented numerous obstacles. Back then, developers needed to manage their own physical servers and spend time upgrading and configuring hardware. Without enough hardware, websites would crash yet too much hardware meant increased cost and maintenance. Scaling hardware up for increased traffic was also wasteful since most of the time servers were not at full capacity. Given these issues, sales and marketing teams inevitably felt hamstrung by hardware limitations. Looking back, it was pretty difficult to run a software company successfully. If any hardware went down, that was the company's responsibility.

The 2000s:

At this point, a few major innovations came along. To alleviate hardware limitations, Amazon Web Services introduced EC2 instances, virtual servers that could add or remove capacity with the click of a button. This meant developers no longer needed to actually touch a physical server, instead, they could focus more time on delivering better features for their end-users. Other new AWS services such as S3 for file storage and SQS for scheduling tasks continued to reduce the number of time developers spent creating infrastructure. Yet inefficiencies still existed. Developers still needed to spend time setting up proxies to distribute traffic for load balancing and companies continued to pay for servers that rarely reached full capacity. Lastly, programmers needed to maintain an OS and manage application software, a time-consuming albeit necessary task.

The 2010s- Now:

With a developed suite of microservices(S3, SQS, SNS), a glue was still needed to connect everything. To accomplish this, AWS launched AWS Lambda, a Functions-as-a-Service platform where programmers can write code in the AWS console and connect to different microservices without having to worry about scaling or server capacity. Perhaps more importantly, charges only apply per execution of a function making the platform exceedingly affordable.

Of course, there are plenty of other benefits, and if interested in learning about switching from Django to Serverless, check out this awesome article by Nicolò Gasparini on Medium. Compared to the Django framework, Serverless offers the ability to decouple services and reduce code as much as possible, effectively making a web application only as big as necessary.

Serverless Tutorial

Alright, so now that you have a basic understanding of why Serverless computing is so popular, let's get started with a simple tutorial to introduce some of the topics we just mentioned, such as decoupled microservices.

If you don't have an AWS account, create one first. In this tutorial, we will first set up a project using the AWS console. Then, we'll use a third-party tool called the Serverless Framework to make our code more shareable and manageable (instead of using the AWS console). For a list of other third-party tools, visit https://aws.amazon.com/serverless/developer-tools/.

AWS Console:

After creating an account, visit the AWS API Gateway. This service acts as a management tool between client and backend services. In other words, it receives API calls, aggregates the different services required to fulfill the call, and returns the result to the client. For our purposes, this is where we match a Lambda function to a given request.

In the API gateway dashboard
- Choose "Build" under REST API (not the private REST API)
  - An example API will appear, feel free to review the YAML file but switch to "New API"
  - Under Settings, name the API "Demo API" and select "Create API"
- Click the "Actions" dropdown and select "Create Resource". Think of creating a resource as adding a URL path if you are familiar with Django
  - Enter "Congrats" as the Resource Name
  - The Resource Path should auto-fill with "congrats"
  - Ignore the "Proxy Resource" and "Enable API Gateway CORS" checkboxes for the sake of this demo. The former allows a Lambda function to handle routing and the latter allows you to make API requests with your browser
  - Click "Create Resource"
- The resource needs a method to create an actual endpoint. Under "Actions" select "Create Method"
  - In the dropdown under the "/congrats" resource, select "GET"
  - Now we need to connect to a Lambda function, however, we have not created one yet. We'll leave this page and return once we have a function.

Under "Services" in the navbar, type in and select "Lambda"
- Select "Create Function" in the top-right corner
- Keep "Author from scratch" checked
- Name the function "congrats" just to stay consistent with our API endpoint
- Select "Python 3.7" as the runtime. Obviously, feel free to select the runtime you are familiar with
- Under "Permissions", select "Create a new role with basic Lambda permissions" if not already selected
- Click "Create Function"
- After creating a function, scroll down on the page to the "Function code"
- The code bears some resemblance to a Django view
- Edit the function as demonstrated. Feel free to change the text to a different message:
  - ```
  import json
  
  
  def lambda_handler(event, context):
       # TODO implement
       return { 'statusCode': 200, 'body': json.dumps('Congrats on setting up your endpoint') }
```

- - Click "Save"
Return to the API Gateway dashboard, select your the API we created, and click the "GET" request method to return to where we once were
- Enter the name of your Lambda function. For example, we used the name "congrats"
- Click "Save" and confirm permissions by clicking "OK"
- Let the page reload
- Let's test our new function
  - Select "Test". Do not worry about query strings or headers, click "Test" at the bottom
  - You should see the response we added to our function as such:
  - ```
  {
    "statusCode": 200,
    "body": "\"Congrats on setting up your endpoint\""
  }
```
- Congrats!

Awesome, we just finished manually setting up an API endpoint with Lambda functions. Obviously, there are some shortcomings to this approach. Since we used the AWS console, it's difficult to share our code, allow team members to edit, and it's easy to make small errors.

So how can we collaborate with other users or experiment on our own branch, such as with a Github repository? Fortunately, third-party tools such as the Serverless Framework solve all of our concerns. By installing the framework on our local device, we can create our Lambda functions using our favorite text editor, share our work on a Github repository, and easily deploy to AWS with a list of powerful commands. Now that we know how to do things manually, let's get started with the Serverless Framework.

Serverless Framework:

Let's set up our project. Create a new folder to hold your project and change into that directory.
- ```
mkdir demo
cd demo
```

- Visit the official documentation to download the framework. There are various package managers you can use depending on your OS. We chose to use npm since we already had Node.js installed.
  - ```
  npm install -g serverless
```

- Once installed, enter the following command to create a serverless project. Note: --template represents the name of an available template. In this case, we specify the aws-python3 template so we can have a similar runtime as our previous example.
- ```
serverless create --template aws-python3
```

Open your favorite code editor and select the demo folder we created for the serverless project
- You should notice three files: serveless.yml which contains configurations details for deployment, handler.py for writing functions that will become Lambda functions, and .gitignore to ignore certain files when pushing to a Github repository
- You'll also notice that there is already a function in handler.py. Let's keep the name "hello" but simplify the function so it closely resembles the function we wrote in the AWS console
- ```
import json


def hello(event, context):
    response = {
        "statusCode": 200,
        "body": json.dumps("Congrats on deploying with the serverless framework")
    }

    return response
```
- Now, let's edit serverless.yml to add a URL endpoint and specify the HTTP request method just as we did before, except the name of our function and path will be "hello" instead of "congrats"
- ```
functions:
  hello:
    handler: handler.hello
    events:
      - http:
          path: /hello
          method: get
```
To begin deployment, we need to create a serverless account. Back in your command line enter:
- ```
sls login
```
- Your browser should open. Next, create an account, we recommended using Google authentication
- Provide a username and keep the default "myapp" app name
- When prompted, choose the new dashboard layout. Find your app and select settings under the three-dot icon. Next click "ci/cd" and under "connect AWS" click "connect".
- The AWS console will already have a stack specifically for the serverless framework. Simply check the box below to grant access to create roles and continue with the default settings
- After a minute or two, the stack setup will be complete and back in the serverless dashboard, your account should automatically be connected
Return to serverless.yml and add your app name and user name above service as such:
- ```
app: myapp
org: yourusername
service: demo
```
Make sure the files you edited are saved and deploy via the serverless command:
- ```
serverless deploy
```

- This will check if credentials are valid and use the ARN role we just created for deployment to AWS
- It may take a few minutes
Upon completion open your browser and copy and paste the endpoint under service completion:
- ```
endpoints:
  GET - https://lcqk33wu05.execute-api.us-east-1.amazonaws.com/dev/hello
```
- You should observe the following in your browser:
To remove most of the resources we just created enter the following command. However, you may need to visit the AWS Cloudformation page to remove all of the stack we previously created when granting access to our serverless account
```
serverless remove
```

Thanks for reading. If interested in learning how to deploy a serverless Django Application, instead of using an AWS ElasticBeanstalk or a Linode, check out our article on Django and Zappa.

Related Projects