How to Prevent Hard-Coded Secrets

Instead of writing a secret directly into the source code, you should define an alternative mechanism for obtaining the secret at the places where it is used, such as using environment variables or local files not under version control, or relaying to an external secret vault (aka secret manager). These are the most common options:

Environment Variables or Configuration files

Using SCM Secrets

GitHub

GitLab

Using Cloud Secret Management Services

AWS Secrets Manager

Azure Key Vault

Google Cloud Secret Manager

Using a Third-Party Secret Vault

HashiCorp Vault

CyberArk Conjur

Environment Variables or Configuration files

Taking secrets from environment variables or configuration files works for any programming language and operating system.

Environment variables are not hard-coded, but they should be given the value somewhere. Application code and scripts may read the environment variable, but environment variables must be set before the application or script runs.

With local files, you may need to enforce that the exclude patterns in .gitignore or .dockerignore configurations are properly excluding the secret-holding files

With environment variables, it is more difficult to accidentally leak secrets, but be aware that the software may write environment variables for debugging to standard output/error, potentially disclosing the value in publicly available logs.

A popular way to setup environment variables is to load them from an .env file, but remember: that file should never be under version control.

A template.env file containing only the variable names with empty/blank values, and comments explaining which is the secret and how it is used, is a good technique for documenting the secrets needed and for inventory of the secrets used.

If that innocuous template.env file is under version control and .env in .gitignore, users can copy template.env to .env, and then edit the git-ignored .env with the real secrets.

The following are examples for how to get a secret from .env file for each ecosystem. Replace the name of the environment variable SECRET_VAR and <path> to the .env file accordingly.

Code examples below are illustrative and simplified, with no error handling and configuration.

JavaScript

var dotenv = require('dotenv');
dotenv.config({ path: '<path>/.env' });

const secret = process.env.SECRET_VAR;

// alternative: load map instead of setting env vars
const secrets = {}
dotenv.config({ path: '<path>/.env', processEnv: secrets });
const secret = secrets.SECRET_VAR;

See dotenv for full details.

Java

Go

C#

Using SCM Secrets

Collaboration platforms (aka Source Code Management Systems, SCM) and CI/CD tools often provide Secret Management, so CI/CD pipelines may get the secret securely.

GitHub

Secrets in GitHub are variables set in an organization, repository, or repository environment, available to use in GitHub Actions workflows.

For secrets stored at the organization-level, access policies control which repositories can use organization secrets. Organization-level secrets let secrets be shared between multiple repositories, which reduces the need for creating duplicate secrets. Updating an organization secret in one location also ensures that the change takes effect in all repository workflows that use that secret.

For secrets stored at the environment level, you can enable required reviewers to control access to the secrets. A workflow job cannot access environment secrets until approval is granted by required approvers.

Once a secret is registered, it can be referenced in a CI/CD workflow using a {{ secret.SECRET }} expression. But if possible, do not pass the secret value to the command to be executed. The command should read the environment variable instead. In the following example, a secret named API_KEY is passed to the workflow step in the environment variable API_KEY, but its value is then hard-coded in the command line, so it will be visible in the process table:

steps:
  - name: Hello world action
    shell: bash
    env: # pass the secret as environment variable
      API_KEY: ${{ secrets.API_KEY }}

    run: |
      # Not recommended! ps will show the clear-text secret
      my-command --key="$API_KEY" ...
      # my-command must read environment variable API_KEY

Although GitHub Actions automatically redacts ("obfuscates") the contents of all GitHub secrets that are printed to workflow logs, this is not fail-proof. Only a limited set of secrets are recognized.

Read creating secrets for a repository, for organization, or for environment for details on how to register a secret for GitHub Actions at a given scope.

GitLab

GitLab provides CI/CD Variables as a convenient wau to store and reuse data in a CI/CD pipeline, but they can be exposed by accidental pipeline misconfiguration.

GitLab provides support for external secret management providers:

• Vault by HashiCorp.

• Google Cloud Secret Manager.

• Azure Key Vault.

After configuring a vault server, you may use vault secrets in a GitLab CI job:

job_using_vault:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://vault.example.com # use your own
  secrets:
    SECRET:
      # translates to secret `ops/data/production/db`, field `password`
      vault: production/db/password@ops
      file: false
      token: $VAULT_ID_TOKEN

This stores the value of the secret fetched from the vault into the SECRET variable.

Read Using external secrets in CI for full details.

Using Cloud Secret Management Services

AWS Secrets Manager

AWS Secrets Manager is the secret vault service in Amazon Web Services platform. The following are examples of how to use the official libraries for getting a secret using different programming languages:

JavaScript

 import {
  GetSecretValueCommand,
  SecretsManagerClient,
} from "@aws-sdk/client-secrets-manager";

export const getSecretValue = async (secretName: string) => {
  const client = new SecretsManagerClient();
  const response = await client.send(
    new GetSecretValueCommand({
      SecretId: secretName,
    }),
  );
  console.log(response);

  if (response.SecretString) {
    return response.SecretString;
  } else if (response.SecretBinary) {
    return response.SecretBinary;
  }
};

let secret = await get_secret("SECRET")

See AWS SDK for JavaScript v3 for further details.

Python

Java

Go

C#

Azure Key Vault

The Azure Key Vault is the secrets management service in Azure. The following shows how to retrieve a secret using the official libraries for some popular languages.

JavaScript

The following shows how to use the @azure/keyvault-secrets package.

const { SecretClient } = require("@azure/keyvault-secrets");
const { DefaultAzureCredential } = require("@azure/identity");

const keyVaultName = "your-key-vault-name";
const kvUri = `https://${keyVaultName}.vault.azure.net`;
const credential = new DefaultAzureCredential();
const client = new SecretClient(kvUri, credential);

const secret = await client.getSecret("SECRET");

Python

Java

Go

C#

Google Cloud Secret Manager

Secret Manager is Google Cloud’s storage system for API keys, passwords, certificates, and other sensitive data.

In the following, <project_id> represents your Google Cloud Project ID, and SECRET is the name of the secret to fetch. It is assumed that the latest version of the secret is fetched. Examples can be found in the Quickstart page.

JavaScript

The @google-cloud/secret-manager is the official JavaScript library for Secret Manager. The following example shows how to fetch a secret.

const { SecretManagerServiceClient } = require('@google-cloud/secret-manager').v1;
const client = new SecretManagerServiceClient();

async function accessSecretVersion() {
  const [version] = await client.accessSecretVersion({
    name: 'projects/<project_id>/secrets/SECRET/versions/latest',
  });

  return version.payload.data.toString('utf8');
}

const secret = await accessSecretVersion();

Python

Java

Go

C#

Using a Third-Party Secret Vault

HashiCorp Vault

HashiCorp Vault is a centralized secrets management system that provides secure storage of sensitive information, such as password, API keys, access tokens or cryptographic keys, encrypted in transit and at rest. It permits dynamic generation for temporary, on-demand credentials, and advanced features like automated key rotation and leasing/renewal of secrets, plus some built-in support for secret revocation.

In what follows, we assume a mount point of "secret" and a vault path of "SECRET", and the secret is stored under key "value".

JavaScript

See node-vault, unofficial NPM package for Vault.

const options = {
  apiVersion: "v1",
  endpoint: process.env.VAULT_URL,
  token: process.env.VAULT_TOKEN
};
const vault = require("node-vault")(options);

async function getSecret() {
    const result = await vault.read("secret/SECRET");
    return result.data.data.value;
}

var secret = getSecret();

Python

Java

Go

C#

CyberArk Conjur

CyberArk Conjur is an open-source security tool for managing secrets and credentials in modern IT environments.

The following show how to fetch a secret from CyberArk Conjur for popular programming languages. The environment variables CONJUR_APPLIANCE_URL, CONJUR_ACCOUNT, CONJUR_USERNAME and CONJUR_APIKEY contain the configuration needed to authenticate for fetching the secret.

Go to Client Libraries or https://docs.cyberark.com/conjur-open-source/Latest/en/Content/Developer/lp_REST_API.htm?tocpath=Developer%7CREST%C2%A0APIs for further detail.

Java

The following code uses the official conjur-api-java library.

import com.cyberark.conjur.api.Conjur;

String conjurUrl = System.getenv("CONJUR_APPLIANCE_URL");
String conjurAccount = System.getenv("CONJUR_ACCOUNT");
String conjurUsername = System.getenv("CONJUR_USERNAME");
String conjurApikey = System.getenv("CONJUR_APIKEY");

var credentials = new Credentials(conjurUsername, conjurApikey);
var conjur = new Conjur(credentials);
String secret = conjur.variables().retrieveSecret("SECRET");

JavaScript

Python

Go

C#