Using Environment Variables with the Serverless Framework

As developers we all know that it's best practice to keep configuration outside of the application. This serves two purposes:

  1. It makes it easier to deploy the application into different environments (stages) like development, staging and production because only the configuration needs to be modified.
  2. It allows you to keep your production credentials from being committed into version control. This is particularly important in organisations where developers don't have access to production.

Prior to Serverless 1.2 you needed to use a plugin and the dotenv package as AWS Lambda didn't include support environment variables. This changed a few weeks ago and Serverless 1.2 quickly followed suit by adding support for environment variables into the core.

Setting environment variables with Serverless 1.2 is as simple an adding an environment key to either your function or the provider section inside your server.yml. For example:

provider:
  name: aws
  environment:
    VAR_1: "something"
    VAR_2: "something"

functions:
  myfunction:
    environment:
      VAR_2: "something"

Environment variables in the provider section are set for all functions. This is useful for settings like API keys or a table names that every function inside your service needs to access. Environment variables set in the function settings only apply to that function.

If you set the same variable in both places the function's environment variable will override the provider environment variable. This is ideal when you have an environment variable which needs one value for most functions but a slightly different value for one or two functions.

Inside your application environment variables are accessed exactly the same way you normally access your environment variables.

For NodeJS (Javascript) use:

process.env.MY_VAR;

For Python use:

os.environ['MY_VAR']

While you can set environment variables directly inside your serverless.yml this doesn't solve the problem of allowing different values for each stage or keeping your secrets outside of version control. There are two approaches you can use to for this.

Regardless of the approach you use you will want to add a custom variable to your serverless.yml file so that you have an easy way to reference the current stage.

custom:
  stage: "${opt:stage, self:provider.stage}"

For personal projects I prefer to have one environment file. I find it easier than managing multiple environment files and it's really not an issue if I have production credentials.

Start by creating an env.yml file. In that file you want one key for each environment with all of the environment variables for that environment set below it. Example:

dev:
  MY_VAR: "the dev value"

prod:
  MY_VAR: "the prod value"

You can now set your environment to use all of the keys from your env.yml file that are below the key for your current stage.

environment: ${file(env.yml):${self:custom.stage}}

If you don't understand that last line then read referencing variables in other files from the Serverless documentation.

Now that you have per stage environment variables working you may realize that some environment variables have defaults that don't change from one stage to the next. Copying these to every stage is laborious and error prone. Most developers would prefer to have a section with common environment variable settings that are only overridden when the stage requires a different value.

The YAML syntax supports this.

Add a new default_env section to your env.yml and place your default environment variables below it. You can then tell YAML to include those in your stage specific environment variables. Here's an example that demonstrates default values, overriding default values and adding additional stage specific variables.

default_env: &default_env
  SCOPE: "read"
  TABLE: "apple"

dev:
  <<: *default_env
  TABLE: "banana"
  PASSWORD: "my-password"

The end result would be like typing in

dev:
  SCOPE: "read"
  TABLE: "banana"
  PASSWORD: "my-password"

Lastly you will probably want to add env.yml to your .gitignore file so that it doesn't accidentally get committed to version control.

You can also use different files for each stage. I'm not going to cover this in as much details as I prefer the single env.yml approach.

Like the single environment file solution you will want to add a custom variable to your serverless.yml file so that you have an easy way to reference the current stage.

custom:
  stage: "${opt:stage, self:provider.stage}"

Now create an env-STAGE.yml file for each stage you have. For example: env-dev.yml or env-prod.yml. Inside those files place the environment variables for that stage.

MY_VAR: "the value"

Then set your environment variables using all the values in the file for your current stage.

environment: ${file(env-${self:custom.stage}.yml)}

Lastly you will want to add env-*.yml to your .gitignore file so that they don't accidentally get committed to version control.

When you deploy your service your should now see the environment variables in the AWS console. If you've followed these instructions and your environment variables aren't set then

  1. Check that you're using Serverless 1.2 or later.
  2. Try deploying the entire service and not just the function
  3. Check that your keys or file names match the current stage you're deploying to. I've found that Serverless will happily deploy my functions without a warning or error if the files or keys didn't exist.