How to Prevent Hard-Coded Secrets

Instead of writing a secret directly into the source code, you should define an alternative mechanism for obtaining the secret at the places where it is used, such as using environment variables or local files not under version control, or relaying to an external secret vault (aka secret manager). These are the most common options:

Environment Variables or Configuration files

Taking secrets from environment variables or configuration files works for any programming language and operating system.

Environment variables are not hard-coded, but they should be given the value somewhere. Application code and scripts may read the environment variable, but environment variables must be set before the application or script runs.

With local files, you may need to enforce that the exclude patterns in .gitignore or .dockerignore configurations are properly excluding the secret-holding files

With environment variables, it is more difficult to accidentally leak secrets, but be aware that the software may write environment variables for debugging to standard output/error, potentially disclosing the value in publicly available logs.

A popular way to setup environment variables is to load them from an .env file, but remember: that file should never be under version control.

A template.env file containing only the variable names with empty/blank values, and comments explaining which is the secret and how it is used, is a good technique for documenting the secrets needed and for inventory of the secrets used.

If that innocuous template.env file is under version control and .env in .gitignore, users can copy template.env to .env, and then edit the git-ignored .env with the real secrets.

The following are examples for how to get a secret from .env file for each ecosystem. Replace the name of the environment variable SECRET_VAR and <path> to the .env file accordingly.

Code examples below are illustrative and simplified, with no error handling and configuration.

var dotenv = require('dotenv');
dotenv.config({ path: '<path>/.env' });

const secret = process.env.SECRET_VAR;

// alternative: load map instead of setting env vars
const secrets = {}
dotenv.config({ path: '<path>/.env', processEnv: secrets });
const secret = secrets.SECRET_VAR;

See dotenv for full details.

// See io.github.cdimascio:dotenv-java in Maven Central
Dotenv dotenv = Dotenv.configure()
  .directory("<path>").filename("env")
  .load();
String secret = dotenv.get("SECRET_VAR");
//

See dotenv-java for full details.

import (
  "os"
  "github.com/joho/godotenv"
)

func my_func() {
  err := godotenv.Load("<path>/.env")
  if err != nil { ... }
  secret := os.Getenv("SECRET_VAR")

  // alternative: load map instead of setting env vars
  reader := getRemoteFile("<path>/.env")
  secrets, err := godotenv.Parse(reader)
  secret := secrets["SECRET_VAR"]
}

See GoDotEnv for full details.

using dotenv.net;
using System;

DotEnv.Load(options: new DotEnvOptions(envFilePaths: new[] {"<path>/.env"}));
string secret = Environment.GetEnvironmentVariable("SECRET_VAR");

// alternative: load map instead of setting env vars
var secrets = DotEnv.Read(options: new DotEnvOptions(envFilePaths: new[] {"<path>/.env"}));
string secret = secrets["SECRET_VAR"];

See dotenv.net for full details.

Using SCM Secrets

Collaboration platforms (aka Source Code Management Systems, SCM) and CI/CD tools often provide Secret Management, so CI/CD pipelines may get the secret securely.

GitHub

Secrets in GitHub are variables set in an organization, repository, or repository environment, available to use in GitHub Actions workflows.

For secrets stored at the organization-level, access policies control which repositories can use organization secrets. Organization-level secrets let secrets be shared between multiple repositories, which reduces the need for creating duplicate secrets. Updating an organization secret in one location also ensures that the change takes effect in all repository workflows that use that secret.

For secrets stored at the environment level, you can enable required reviewers to control access to the secrets. A workflow job cannot access environment secrets until approval is granted by required approvers.

Once a secret is registered, it can be referenced in a CI/CD workflow using a {{ secret.SECRET }} expression. But if possible, do not pass the secret value to the command to be executed. The command should read the environment variable instead. In the following example, a secret named API_KEY is passed to the workflow step in the environment variable API_KEY, but its value is then hard-coded in the command line, so it will be visible in the process table:

steps:
  - name: Hello world action
    shell: bash
    env: # pass the secret as environment variable
      API_KEY: ${{ secrets.API_KEY }}

    run: |
      # Not recommended! ps will show the clear-text secret
      my-command --key="$API_KEY" ...
      # my-command must read environment variable API_KEY

Although GitHub Actions automatically redacts ("obfuscates") the contents of all GitHub secrets that are printed to workflow logs, this is not fail-proof. Only a limited set of secrets are recognized.

Read creating secrets for a repository, for organization, or for environment for details on how to register a secret for GitHub Actions at a given scope.

GitLab

GitLab provides CI/CD Variables as a convenient wau to store and reuse data in a CI/CD pipeline, but they can be exposed by accidental pipeline misconfiguration.

GitLab provides support for external secret management providers:

After configuring a vault server, you may use vault secrets in a GitLab CI job:

job_using_vault:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://vault.example.com # use your own
  secrets:
    SECRET:
      # translates to secret `ops/data/production/db`, field `password`
      vault: production/db/password@ops
      file: false
      token: $VAULT_ID_TOKEN

This stores the value of the secret fetched from the vault into the SECRET variable.

Read Using external secrets in CI for full details.

Using Cloud Secret Management Services

AWS Secrets Manager

AWS Secrets Manager is the secret vault service in Amazon Web Services platform. The following are examples of how to use the official libraries for getting a secret using different programming languages:

 import {
  GetSecretValueCommand,
  SecretsManagerClient,
} from "@aws-sdk/client-secrets-manager";

export const getSecretValue = async (secretName: string) => {
  const client = new SecretsManagerClient();
  const response = await client.send(
    new GetSecretValueCommand({
      SecretId: secretName,
    }),
  );
  console.log(response);

  if (response.SecretString) {
    return response.SecretString;
  } else if (response.SecretBinary) {
    return response.SecretBinary;
  }
};

let secret = await get_secret("SECRET")

See AWS SDK for JavaScript v3 for further details.

The following uses Boto3, the official Python interface maintained by AWS.

import boto3

client = boto3.client('secretsmanager')
secret = client.get_secret_value('SECRET')['SecretString']

See SecretsManager client for further details.

The AWS Secrets Manager Java caching client is the official Java library for accessing AWS Secrets Manager

import com.amazonaws.secretsmanager.caching.SecretCache;

SecretCache cache  = new SecretCache();
String secret  = cache.getSecretString("SECRET");

The secretsmanager package provides the official API client, operations, and parameter types for AWS Secrets Manager, using the AWS SDK for Go.

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/secretsmanager"
)

cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion(your_region))
if err != nil { ... }
client := secretsmanager.NewFromConfig(cfg)
input := &secretsmanager.GetSecretValueInput{ SecretId: aws.String(secretName), }
result, err := svc.GetSecretValue(input)
if err != nil { ... }
secret := *result.SecretString

using Amazon.SecretsManager.Extensions.Caching;

var client = new AmazonSecretsManagerClient();
var response = await GetSecretAsync(client, "SECRET");
if (response is not null && response.SecretString is not null)
{
    string secret = response.SecretString;
}
...

private static async Task<GetSecretValueResponse>
  GetSecretAsync( IAmazonSecretsManager client, string secretName )
{
    var request = new GetSecretValueRequest()
    {
        SecretId = secretName,
        VersionStage = "AWSCURRENT", // the default
    };

    return await client.GetSecretValueAsync(request);
}

For full details, proceed with Get a Secrets Manager secret value using the .NET AWS SDK.

Azure Key Vault

The Azure Key Vault is the secrets management service in Azure. The following shows how to retrieve a secret using the official libraries for some popular languages.

The following shows how to use the @azure/keyvault-secrets package.

const { SecretClient } = require("@azure/keyvault-secrets");
const { DefaultAzureCredential } = require("@azure/identity");

const keyVaultName = "your-key-vault-name";
const kvUri = `https://${keyVaultName}.vault.azure.net`;
const credential = new DefaultAzureCredential();
const client = new SecretClient(kvUri, credential);

const secret = await client.getSecret("SECRET");

The azure-keyvault-secrets is the official Python library for accessing the Key Vault.

from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient

key_vault_name = "your-key-vault-name"
kv_uri = f"https://{key_vault_name}.vault.azure.net"

credential = DefaultAzureCredential()
client = SecretClient(vault_url=kv_uri, credential=credential)

secret = client.get_secret('SECRET').value

The following shows how to fetch a secret from Key Vault using the com.azure:azure-security-keyvault-secrets library.

String keyVaultName = "your-key-vault-name";
String kvUri = "https://" + keyVaultName + ".vault.azure.net";
SecretClient client = new SecretClientBuilder()
    .vaultUrl(kvUri)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildClient();
String secret = client.getSecret("SECRET").getValue();

Read azure-security-keyvault-secrets documentation for full details.

The following fetches a secret from Key Vault using Azure Key Vault Secrets client module for Go.

import (
    "context"
    "fmt"
    "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
    "github.com/Azure/azure-sdk-for-go/sdk/keyvault/azsecrets"
)

keyVaultName := "your-key-vault-name"
kvUri := fmt.Sprintf("https://%s.vault.azure.net", keyVaultName)

cred, err := azidentity.NewDefaultAzureCredential(nil)
if err != nil {...}

client, err := azsecrets.NewClient(kvUri, cred, nil)
if err != nil {...}

secretResp, err := client.GetSecret(context.Background(), secretName, nil)
if err != nil {...}
secret = *secretResp.Value

The following shows how to fetch a secret using the Azure Key Vault secret client library for .NET.

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

string keyVaultName = "your-key-vault-name";
string kvUri = $"https://{keyVaultName}.vault.azure.net";
var client = new SecretClient(new Uri(kvUri), new DefaultAzureCredential());
var secret = client.GetSecret("SECRET").Value;

Google Cloud Secret Manager

Secret Manager is Google Cloud’s storage system for API keys, passwords, certificates, and other sensitive data.

In the following, <project_id> represents your Google Cloud Project ID, and SECRET is the name of the secret to fetch. It is assumed that the latest version of the secret is fetched. Examples can be found in the Quickstart page.

The @google-cloud/secret-manager is the official JavaScript library for Secret Manager. The following example shows how to fetch a secret.

const { SecretManagerServiceClient } = require('@google-cloud/secret-manager').v1;
const client = new SecretManagerServiceClient();

async function accessSecretVersion() {
  const [version] = await client.accessSecretVersion({
    name: 'projects/<project_id>/secrets/SECRET/versions/latest',
  });

  return version.payload.data.toString('utf8');
}

const secret = await accessSecretVersion();

See Python Client for Secret Manager.

from google.cloud import secretmanager

def access_secret():
    client = secretmanager.SecretManagerServiceClient()
    name = "projects/<project_id>/secrets/SECRET/versions/latest"

    response = client.access_secret_version(request={"name": name})
    return response.payload.data.decode("UTF-8")

# Usage
secret = access_secret()

See Google Secret Management Client for Java.

import com.google.cloud.secretmanager.v1.SecretManagerServiceClient;
import com.google.cloud.secretmanager.v1.AccessSecretVersionRequest;
import com.google.cloud.secretmanager.v1.AccessSecretVersionResponse;

public static String accessSecret() {
    String secretName = "projects/<project_id>/secrets/SECRET/versions/latest";

    try (var client = SecretManagerServiceClient.create()) {
        var request = AccessSecretVersionRequest.newBuilder()
            .setName(secretName)
            .build();
        var response = client.accessSecretVersion(request);
        return response.getPayload().getData().toStringUtf8();

    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

String secret = accessSecret();

See Secret Manager API for Go.

import (
    "context"
    "fmt"
    "log"

    secretmanager "cloud.google.com/go/secretmanager/apiv1"
    secretmanagerpb "google.golang.org/genproto/googleapis/cloud/secretmanager/v1"
)

func accessSecret() (string, error) {
    ctx := context.Background()
    client, err := secretmanager.NewClient(ctx)
    if err != nil {...}
    defer client.Close()

    req := &secretmanagerpb.AccessSecretVersionRequest{
        Name: "projects/<project_id>/secrets/SECRET/versions/latest",
    }

    result, err := client.AccessSecretVersion(ctx, req)
    if err != nil {...}

    return string(result.Payload.Data), nuil
}

secret, err := accessSecret()

using Google.Cloud.SecretManager.V1;
using System;
using System.Threading.Tasks;


public static async Task<string> AccessSecretAsync(string projectId, string secretId, string versionId)
{
    var client = await SecretManagerServiceClient.CreateAsync();
    var secretVersionName = SecretVersionName.FromProjectSecretVersion("<project_id>", "SECRET", "latest");
    var result = await client.AccessSecretVersionAsync(secretVersionName);
    return result.Payload.Data.ToStringUtf8();
}

var secret = await AccessSecretAsync();

See SecretManagerServiceClient.

Using a Third-Party Secret Vault

HashiCorp Vault

HashiCorp Vault is a centralized secrets management system that provides secure storage of sensitive information, such as password, API keys, access tokens or cryptographic keys, encrypted in transit and at rest. It permits dynamic generation for temporary, on-demand credentials, and advanced features like automated key rotation and leasing/renewal of secrets, plus some built-in support for secret revocation.

In what follows, we assume a mount point of "secret" and a vault path of "SECRET", and the secret is stored under key "value".

See node-vault, unofficial NPM package for Vault.

const options = {
  apiVersion: "v1",
  endpoint: process.env.VAULT_URL,
  token: process.env.VAULT_TOKEN
};
const vault = require("node-vault")(options);

async function getSecret() {
    const result = await vault.read("secret/SECRET");
    return result.data.data.value;
}

var secret = getSecret();

The following uses hvac, a Python client for Vault and other secret managers.

import hvac
import os

client = hvac.Client(url = os.environ['VAULT_URL'], token = os.environ['VAULT_TOKEN'])

client.secrets.kv.v2.configure(
    max_versions = 20,
    mount_point = 'secret',
)

# Assuming you've already authenticated
res = client.secrets.kv.v2.read_secret_version(path='SECRET')
secret = res.data.data.value

The example uses the (unofficial) Vault Java Driver.

import com.bettercloud.vault.Vault;
import com.bettercloud.vault.VaultConfig;
import com.bettercloud.vault.VaultException;

var config = new VaultConfig()
  .address(System.getenv("VAULT_URL"))
  .token(System.getenv("VAULT_TOKEN"))
  .build();

Vault vault = new Vault(config);
String secret = vault.logical().read("secret/SECRET").getData().get("value");

The following example uses the Vault API official Go library.

import (
    "os"
    vault "github.com/hashicorp/vault/api"
)

config := vault.DefaultConfig()
config.Address = os.Getenv("VAULT_URL")

client, err := vault.NewClient(config)
if err != nil {...}
client.SetToken( os.Getenv("VAULT_TOKEN") )

// default mount point secret. Secret data with string keyed by "value"
resp, err := client.Logical().Read("secret/SECRET")
if err != nil {...}
if resp == nil {...}

secret, ok := resp.Data["value"].(string)

The following example uses VaultSharp, .NET library for Vault.

using System;
using VaultSharp;
using VaultSharp.V1.AuthMethods.Token;
using VaultSharp.V1.Commons;

// recommendation: get url and token from environment
string vaultUrl = Environment.GetEnvironmentVariable("VAULT_URL");
string token = Environment.GetEnvironmentVariable("VAULT_TOKEN");
var settings = new VaultClientSettings(vaultUrl, new TokenAuthMethodInfo(token));
var client = new VaultClient(settings);
Secret<SecretData> res = await client.V1.Secrets.KeyValue.V2.ReadSecretAsync(path: "SECRET", mountPoint: "secret").Result;
string secret = res.Data.Data["value"];

CyberArk Conjur

CyberArk Conjur is an open-source security tool for managing secrets and credentials in modern IT environments.

The following show how to fetch a secret from CyberArk Conjur for popular programming languages. The environment variables CONJUR_APPLIANCE_URL, CONJUR_ACCOUNT, CONJUR_USERNAME and CONJUR_APIKEY contain the configuration needed to authenticate for fetching the secret.

Go to Client Libraries or https://docs.cyberark.com/conjur-open-source/Latest/en/Content/Developer/lp_REST_API.htm?tocpath=Developer%7CREST%C2%A0APIs for further detail.

The following code uses the official conjur-api-java library.

import com.cyberark.conjur.api.Conjur;

String conjurUrl = System.getenv("CONJUR_APPLIANCE_URL");
String conjurAccount = System.getenv("CONJUR_ACCOUNT");
String conjurUsername = System.getenv("CONJUR_USERNAME");
String conjurApikey = System.getenv("CONJUR_APIKEY");

var credentials = new Credentials(conjurUsername, conjurApikey);
var conjur = new Conjur(credentials);
String secret = conjur.variables().retrieveSecret("SECRET");

The following code uses the official conjur-api-java library.

const axios = require('axios');

cont conf = {
  url: process.env.CONJUR_APPLIANCE_URL,
  account: process.env.CONJUR_ACCOUNT,
  username: process.env.CONJUR_USERNAME,
  apikey: process.env.CONJUR_APIKEY
};

async function getConjurSecret() {
    // Authenticate and get session token
    const authResponse = await axios.post(
      `${conf.url}/authn/${conf.account}/${conf.username}/authenticate`,
      conf.apiKey,
      { headers: { 'Content-Type': 'text/plain' } }
    );
    const sessionToken = Buffer.from(authResponse.data).toString('base64');

    // Retrieve secret
    const secretResponse = await axios.get(
      `${conjurUrl}/secrets/${conf.account}/variable/SECRET`,
      { headers: { Authorization: `Token token="${sessionToken}"` } }
    );

    return secretResponse.data;
}

var secret = await getConjurSecret();

In Python, there is no official SDK but the REST api could be invoked directly using the requests library:

import requests
import base64
import os

conjur_url = os.environ['CONJUR_APPLIANCE_URL']
conjur_account = os.environ['CONJUR_ACCOUNT']
conjur_username = os.environ['CONJUR_USERNAME']
conjur_apikey = os.environ['CONJUR_APIKEY']

def get_conjur_secret():
    # Authenticate and get session token
    auth_url = f"{conjur_url}/authn/{conjur_account}/{conjur_username}/authenticate"
    auth_response = requests.post(auth_url, data = conjur_apikey)
    auth_response.raise_for_status()
    session_token = base64.b64encode(auth_response.content).decode('utf-8')

    # Retrieve secret
    secret_url = f"{conjur_url}/secrets/{conjur_account}/variable/SECRET"
    headers = {"Authorization": f'Token token="{session_token}"'}
    secret_response = requests.get(secret_url, headers=headers)
    secret_response.raise_for_status()

    return secret_response.text

secret = get_conjur_secret()

The following example retrieves a secret using the community-supported conjurapi Go module.

import (
    "os"
    "github.com/cyberark/conjur-api-go/conjurapi"
    "github.com/cyberark/conjur-api-go/conjurapi/authn"
)

// Assumes CONJUR_APPLIANCE_URL and CONJUR_ACCOUNT passed as environment variables
config, err := conjurapi.LoadConfig()
if err != nil {...}

conjur, err := conjurapi.NewClientFromKey(config,
    authn.LoginPair{
        Login:  os.Getenv("CONJUR_USERNAME"),
        APIKey: os.Getenv("CONJUR_APIKEY"),
    },
)
if err != nil {...}

// Retrieve a secret into []byte.
secretValue, err := conjur.RetrieveSecret(variableIdentifier)
if err != nil {...}
secret = string(secretValue)

// Retrieve a secret into io.ReadCloser, then read into []byte.
// Alternatively, you can transfer the secret directly into secure memory, vault, keychain, etc.
secretResponse, err := conjur.RetrieveSecretReader("SECRET")
if err != nil {...}

secret, err = conjurapi.ReadResponseBody(secretResponse)

This example uses the official Conjur API for .NET.

using System;
using Conjur;

string conjurUrl = Environment.GetEnvironmentVariable("CONJUR_APPLIANCE_URL");
string conjurAccount = Environment.GetEnvironmentVariable("CONJUR_ACCOUNT");
string conjurUsername = Environment.GetEnvironmentVariable("CONJUR_USERNAME");
string conjurApikey = Environment.GetEnvironmentVariable("CONJUR_APIKEY");

Client client = new Client("https://myorg.com", account);
string token = client.Login(conjurUsername, conjurApikey);
Variable variable = new Variable(client, "SECRET");
var secret = variable.GetValue();

PreviousSecret Leaks Handling NextSecret Leaks Handling CheatSheet

Last updated 1 month ago