How to Prevent Hard-Coded Secrets

PreviousSecret Leaks Handling NextSecret Leaks Handling CheatSheet

Last updated 7 months ago

How to Prevent Hard-Coded Secrets

Instead of writing a secret directly into the source code, you should define an alternative mechanism for obtaining the secret at the places where it is used, such as using environment variables or local files not under version control, or relaying to an external secret vault (aka secret manager). These are the most common options:

Environment Variables or Configuration files

Taking secrets from environment variables or configuration files works for any programming language and operating system.

Environment variables are not hard-coded, but they should be given the value somewhere. Application code and scripts may read the environment variable, but environment variables must be set before the application or script runs.

With local files, you may need to enforce that the exclude patterns in .gitignore or .dockerignore configurations are properly excluding the secret-holding files

With environment variables, it is more difficult to accidentally leak secrets, but be aware that the software may write environment variables for debugging to standard output/error, potentially disclosing the value in publicly available logs.

A popular way to setup environment variables is to load them from an .env file, but remember: that file should never be under version control.

A template.env file containing only the variable names with empty/blank values, and comments explaining which is the secret and how it is used, is a good technique for documenting the secrets needed and for inventory of the secrets used.

If that innocuous template.env file is under version control and .env in .gitignore, users can copy template.env to .env, and then edit the git-ignored .env with the real secrets.

The following are examples for how to get a secret from .env file for each ecosystem. Replace the name of the environment variable SECRET_VAR and <path> to the .env file accordingly.

Code examples below are illustrative and simplified, with no error handling and configuration.

var dotenv = require('dotenv');
dotenv.config({ path: '<path>/.env' });

const secret = process.env.SECRET_VAR;

// alternative: load map instead of setting env vars
const secrets = {}
dotenv.config({ path: '<path>/.env', processEnv: secrets });
const secret = secrets.SECRET_VAR;

// See io.github.cdimascio:dotenv-java in Maven Central
Dotenv dotenv = Dotenv.configure()
  .directory("<path>").filename("env")
  .load();
String secret = dotenv.get("SECRET_VAR");
//

import (
  "os"
  "github.com/joho/godotenv"
)

func my_func() {
  err := godotenv.Load("<path>/.env")
  if err != nil { ... }
  secret := os.Getenv("SECRET_VAR")

  // alternative: load map instead of setting env vars
  reader := getRemoteFile("<path>/.env")
  secrets, err := godotenv.Parse(reader)
  secret := secrets["SECRET_VAR"]
}

using dotenv.net;
using System;

DotEnv.Load(options: new DotEnvOptions(envFilePaths: new[] {"<path>/.env"}));
string secret = Environment.GetEnvironmentVariable("SECRET_VAR");

// alternative: load map instead of setting env vars
var secrets = DotEnv.Read(options: new DotEnvOptions(envFilePaths: new[] {"<path>/.env"}));
string secret = secrets["SECRET_VAR"];

Using SCM Secrets

Collaboration platforms (aka Source Code Management Systems, SCM) and CI/CD tools often provide Secret Management, so CI/CD pipelines may get the secret securely.

GitHub

Secrets in GitHub are variables set in an organization, repository, or repository environment, available to use in GitHub Actions workflows.

For secrets stored at the organization-level, access policies control which repositories can use organization secrets. Organization-level secrets let secrets be shared between multiple repositories, which reduces the need for creating duplicate secrets. Updating an organization secret in one location also ensures that the change takes effect in all repository workflows that use that secret.

For secrets stored at the environment level, you can enable required reviewers to control access to the secrets. A workflow job cannot access environment secrets until approval is granted by required approvers.

Once a secret is registered, it can be referenced in a CI/CD workflow using a {{ secret.SECRET }} expression. But if possible, do not pass the secret value to the command to be executed. The command should read the environment variable instead. In the following example, a secret named API_KEY is passed to the workflow step in the environment variable API_KEY, but its value is then hard-coded in the command line, so it will be visible in the process table:

steps:
  - name: Hello world action
    shell: bash
    env: # pass the secret as environment variable
      API_KEY: ${{ secrets.API_KEY }}

    run: |
      # Not recommended! ps will show the clear-text secret
      my-command --key="$API_KEY" ...
      # my-command must read environment variable API_KEY

GitLab

GitLab provides support for external secret management providers:

job_using_vault:
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://vault.example.com # use your own
  secrets:
    SECRET:
      # translates to secret `ops/data/production/db`, field `password`
      vault: production/db/password@ops
      file: false
      token: $VAULT_ID_TOKEN

This stores the value of the secret fetched from the vault into the SECRET variable.

Using Cloud Secret Management Services

AWS Secrets Manager

 import {
  GetSecretValueCommand,
  SecretsManagerClient,
} from "@aws-sdk/client-secrets-manager";

export const getSecretValue = async (secretName: string) => {
  const client = new SecretsManagerClient();
  const response = await client.send(
    new GetSecretValueCommand({
      SecretId: secretName,
    }),
  );
  console.log(response);

  if (response.SecretString) {
    return response.SecretString;
  } else if (response.SecretBinary) {
    return response.SecretBinary;
  }
};

let secret = await get_secret("SECRET")

import boto3

client = boto3.client('secretsmanager')
secret = client.get_secret_value('SECRET')['SecretString']

import com.amazonaws.secretsmanager.caching.SecretCache;

SecretCache cache  = new SecretCache();
String secret  = cache.getSecretString("SECRET");

import (
	"context"
	"encoding/json"
	"fmt"
	"log"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/secretsmanager"
)

cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion(your_region))
if err != nil { ... }
client := secretsmanager.NewFromConfig(cfg)
input := &secretsmanager.GetSecretValueInput{ SecretId: aws.String(secretName), }
result, err := svc.GetSecretValue(input)
if err != nil { ... }
secret := *result.SecretString

using Amazon.SecretsManager.Extensions.Caching;

var client = new AmazonSecretsManagerClient();
var response = await GetSecretAsync(client, "SECRET");
if (response is not null && response.SecretString is not null)
{
    string secret = response.SecretString;
}
...

private static async Task<GetSecretValueResponse>
  GetSecretAsync( IAmazonSecretsManager client, string secretName )
{
    var request = new GetSecretValueRequest()
    {
        SecretId = secretName,
        VersionStage = "AWSCURRENT", // the default
    };

    return await client.GetSecretValueAsync(request);
}

Azure Key Vault

const { SecretClient } = require("@azure/keyvault-secrets");
const { DefaultAzureCredential } = require("@azure/identity");

const keyVaultName = "your-key-vault-name";
const kvUri = `https://${keyVaultName}.vault.azure.net`;
const credential = new DefaultAzureCredential();
const client = new SecretClient(kvUri, credential);

const secret = await client.getSecret("SECRET");

from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient

key_vault_name = "your-key-vault-name"
kv_uri = f"https://{key_vault_name}.vault.azure.net"

credential = DefaultAzureCredential()
client = SecretClient(vault_url=kv_uri, credential=credential)

secret = client.get_secret('SECRET').value

The following shows how to fetch a secret from Key Vault using the com.azure:azure-security-keyvault-secrets library.

String keyVaultName = "your-key-vault-name";
String kvUri = "https://" + keyVaultName + ".vault.azure.net";
SecretClient client = new SecretClientBuilder()
    .vaultUrl(kvUri)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildClient();
String secret = client.getSecret("SECRET").getValue();

import (
    "context"
    "fmt"
    "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
    "github.com/Azure/azure-sdk-for-go/sdk/keyvault/azsecrets"
)

keyVaultName := "your-key-vault-name"
kvUri := fmt.Sprintf("https://%s.vault.azure.net", keyVaultName)

cred, err := azidentity.NewDefaultAzureCredential(nil)
if err != nil {...}

client, err := azsecrets.NewClient(kvUri, cred, nil)
if err != nil {...}

secretResp, err := client.GetSecret(context.Background(), secretName, nil)
if err != nil {...}
secret = *secretResp.Value

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

string keyVaultName = "your-key-vault-name";
string kvUri = $"https://{keyVaultName}.vault.azure.net";
var client = new SecretClient(new Uri(kvUri), new DefaultAzureCredential());
var secret = client.GetSecret("SECRET").Value;

Google Cloud Secret Manager

const { SecretManagerServiceClient } = require('@google-cloud/secret-manager').v1;
const client = new SecretManagerServiceClient();

async function accessSecretVersion() {
  const [version] = await client.accessSecretVersion({
    name: 'projects/<project_id>/secrets/SECRET/versions/latest',
  });

  return version.payload.data.toString('utf8');
}

const secret = await accessSecretVersion();

from google.cloud import secretmanager

def access_secret():
    client = secretmanager.SecretManagerServiceClient()
    name = "projects/<project_id>/secrets/SECRET/versions/latest"

    response = client.access_secret_version(request={"name": name})
    return response.payload.data.decode("UTF-8")

# Usage
secret = access_secret()

import com.google.cloud.secretmanager.v1.SecretManagerServiceClient;
import com.google.cloud.secretmanager.v1.AccessSecretVersionRequest;
import com.google.cloud.secretmanager.v1.AccessSecretVersionResponse;

public static String accessSecret() {
    String secretName = "projects/<project_id>/secrets/SECRET/versions/latest";

    try (var client = SecretManagerServiceClient.create()) {
        var request = AccessSecretVersionRequest.newBuilder()
            .setName(secretName)
            .build();
        var response = client.accessSecretVersion(request);
        return response.getPayload().getData().toStringUtf8();

    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

String secret = accessSecret();

import (
    "context"
    "fmt"
    "log"

    secretmanager "cloud.google.com/go/secretmanager/apiv1"
    secretmanagerpb "google.golang.org/genproto/googleapis/cloud/secretmanager/v1"
)

func accessSecret() (string, error) {
    ctx := context.Background()
    client, err := secretmanager.NewClient(ctx)
    if err != nil {...}
    defer client.Close()

    req := &secretmanagerpb.AccessSecretVersionRequest{
        Name: "projects/<project_id>/secrets/SECRET/versions/latest",
    }

    result, err := client.AccessSecretVersion(ctx, req)
    if err != nil {...}

    return string(result.Payload.Data), nuil
}

secret, err := accessSecret()

using Google.Cloud.SecretManager.V1;
using System;
using System.Threading.Tasks;


public static async Task<string> AccessSecretAsync(string projectId, string secretId, string versionId)
{
    var client = await SecretManagerServiceClient.CreateAsync();
    var secretVersionName = SecretVersionName.FromProjectSecretVersion("<project_id>", "SECRET", "latest");
    var result = await client.AccessSecretVersionAsync(secretVersionName);
    return result.Payload.Data.ToStringUtf8();
}

var secret = await AccessSecretAsync();

Using a Third-Party Secret Vault

HashiCorp Vault

In what follows, we assume a mount point of "secret" and a vault path of "SECRET", and the secret is stored under key "value".

const options = {
  apiVersion: "v1",
  endpoint: process.env.VAULT_URL,
  token: process.env.VAULT_TOKEN
};
const vault = require("node-vault")(options);

async function getSecret() {
    const result = await vault.read("secret/SECRET");
    return result.data.data.value;
}

var secret = getSecret();

import hvac
import os

client = hvac.Client(url = os.environ['VAULT_URL'], token = os.environ['VAULT_TOKEN'])

client.secrets.kv.v2.configure(
    max_versions = 20,
    mount_point = 'secret',
)

# Assuming you've already authenticated
res = client.secrets.kv.v2.read_secret_version(path='SECRET')
secret = res.data.data.value

import com.bettercloud.vault.Vault;
import com.bettercloud.vault.VaultConfig;
import com.bettercloud.vault.VaultException;

var config = new VaultConfig()
  .address(System.getenv("VAULT_URL"))
  .token(System.getenv("VAULT_TOKEN"))
  .build();

Vault vault = new Vault(config);
String secret = vault.logical().read("secret/SECRET").getData().get("value");

import (
    "os"
    vault "github.com/hashicorp/vault/api"
)

config := vault.DefaultConfig()
config.Address = os.Getenv("VAULT_URL")

client, err := vault.NewClient(config)
if err != nil {...}
client.SetToken( os.Getenv("VAULT_TOKEN") )

// default mount point secret. Secret data with string keyed by "value"
resp, err := client.Logical().Read("secret/SECRET")
if err != nil {...}
if resp == nil {...}

secret, ok := resp.Data["value"].(string)

using System;
using VaultSharp;
using VaultSharp.V1.AuthMethods.Token;
using VaultSharp.V1.Commons;

// recommendation: get url and token from environment
string vaultUrl = Environment.GetEnvironmentVariable("VAULT_URL");
string token = Environment.GetEnvironmentVariable("VAULT_TOKEN");
var settings = new VaultClientSettings(vaultUrl, new TokenAuthMethodInfo(token));
var client = new VaultClient(settings);
Secret<SecretData> res = await client.V1.Secrets.KeyValue.V2.ReadSecretAsync(path: "SECRET", mountPoint: "secret").Result;
string secret = res.Data.Data["value"];

CyberArk Conjur

The following show how to fetch a secret from CyberArk Conjur for popular programming languages. The environment variables CONJUR_APPLIANCE_URL, CONJUR_ACCOUNT, CONJUR_USERNAME and CONJUR_APIKEY contain the configuration needed to authenticate for fetching the secret.

import com.cyberark.conjur.api.Conjur;

String conjurUrl = System.getenv("CONJUR_APPLIANCE_URL");
String conjurAccount = System.getenv("CONJUR_ACCOUNT");
String conjurUsername = System.getenv("CONJUR_USERNAME");
String conjurApikey = System.getenv("CONJUR_APIKEY");

var credentials = new Credentials(conjurUsername, conjurApikey);
var conjur = new Conjur(credentials);
String secret = conjur.variables().retrieveSecret("SECRET");

const axios = require('axios');

cont conf = {
  url: process.env.CONJUR_APPLIANCE_URL,
  account: process.env.CONJUR_ACCOUNT,
  username: process.env.CONJUR_USERNAME,
  apikey: process.env.CONJUR_APIKEY
};

async function getConjurSecret() {
    // Authenticate and get session token
    const authResponse = await axios.post(
      `${conf.url}/authn/${conf.account}/${conf.username}/authenticate`,
      conf.apiKey,
      { headers: { 'Content-Type': 'text/plain' } }
    );
    const sessionToken = Buffer.from(authResponse.data).toString('base64');

    // Retrieve secret
    const secretResponse = await axios.get(
      `${conjurUrl}/secrets/${conf.account}/variable/SECRET`,
      { headers: { Authorization: `Token token="${sessionToken}"` } }
    );

    return secretResponse.data;
}

var secret = await getConjurSecret();

In Python, there is no official SDK but the REST api could be invoked directly using the requests library:

import requests
import base64
import os

conjur_url = os.environ['CONJUR_APPLIANCE_URL']
conjur_account = os.environ['CONJUR_ACCOUNT']
conjur_username = os.environ['CONJUR_USERNAME']
conjur_apikey = os.environ['CONJUR_APIKEY']

def get_conjur_secret():
    # Authenticate and get session token
    auth_url = f"{conjur_url}/authn/{conjur_account}/{conjur_username}/authenticate"
    auth_response = requests.post(auth_url, data = conjur_apikey)
    auth_response.raise_for_status()
    session_token = base64.b64encode(auth_response.content).decode('utf-8')

    # Retrieve secret
    secret_url = f"{conjur_url}/secrets/{conjur_account}/variable/SECRET"
    headers = {"Authorization": f'Token token="{session_token}"'}
    secret_response = requests.get(secret_url, headers=headers)
    secret_response.raise_for_status()

    return secret_response.text

secret = get_conjur_secret()

import (
    "os"
    "github.com/cyberark/conjur-api-go/conjurapi"
    "github.com/cyberark/conjur-api-go/conjurapi/authn"
)

// Assumes CONJUR_APPLIANCE_URL and CONJUR_ACCOUNT passed as environment variables
config, err := conjurapi.LoadConfig()
if err != nil {...}

conjur, err := conjurapi.NewClientFromKey(config,
    authn.LoginPair{
        Login:  os.Getenv("CONJUR_USERNAME"),
        APIKey: os.Getenv("CONJUR_APIKEY"),
    },
)
if err != nil {...}

// Retrieve a secret into []byte.
secretValue, err := conjur.RetrieveSecret(variableIdentifier)
if err != nil {...}
secret = string(secretValue)

// Retrieve a secret into io.ReadCloser, then read into []byte.
// Alternatively, you can transfer the secret directly into secure memory, vault, keychain, etc.
secretResponse, err := conjur.RetrieveSecretReader("SECRET")
if err != nil {...}

secret, err = conjurapi.ReadResponseBody(secretResponse)

using System;
using Conjur;

string conjurUrl = Environment.GetEnvironmentVariable("CONJUR_APPLIANCE_URL");
string conjurAccount = Environment.GetEnvironmentVariable("CONJUR_ACCOUNT");
string conjurUsername = Environment.GetEnvironmentVariable("CONJUR_USERNAME");
string conjurApikey = Environment.GetEnvironmentVariable("CONJUR_APIKEY");

Client client = new Client("https://myorg.com", account);
string token = client.Login(conjurUsername, conjurApikey);
Variable variable = new Variable(client, "SECRET");
var secret = variable.GetValue();

PreviousSecret Leaks Handling NextSecret Leaks Handling CheatSheet

Last updated 7 months ago