Key Management: The Hardest Part of Cryptography
| , 22 minutes reading.
1. Why Should You Care?
Youโve implemented AES-256-GCM encryption. Your data is protected by one of the strongest ciphers available. But whereโs the key?
# Your secure encryption
key = b"my-super-secret-key-12345678901" # ๐ฅ Hardcoded!
ciphertext = encrypt(key, data)That hardcoded key just made your encryption theater. Anyone with access to your code has access to all your encrypted data.
Key management is where most cryptographic systems fail. Letโs fix that.
2. Key Lifecycle
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ KEY LIFECYCLE โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ โ
โ Generation โ Distribution โ Storage โ Usage โ Rotation โ Destruction
โ โ โ โ โ โ โ
โ โผ โผ โผ โผ โผ โผ
โ Secure Secure Secure Minimize Regular Secure
โ random channels location exposure schedule deletion
โ โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโEach stage has its own risks and requirements.
3. Key Generation
The Right Way
import os
import secrets
# CORRECT: Use cryptographically secure randomness
key_256 = secrets.token_bytes(32) # 256 bits
key_128 = secrets.token_bytes(16) # 128 bits
# Or using os.urandom (equivalent)
key = os.urandom(32)
# For specific algorithms, use their key generation
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
key = AESGCM.generate_key(bit_length=256)
from cryptography.hazmat.primitives.asymmetric import ed25519
private_key = ed25519.Ed25519PrivateKey.generate()The Wrong Ways
import random
import hashlib
# WRONG: Using random module (not cryptographically secure)
key = bytes([random.randint(0, 255) for _ in range(32)])
# WRONG: Deriving from predictable sources
key = hashlib.sha256(b"password").digest()
# WRONG: Using timestamp
key = hashlib.sha256(str(time.time()).encode()).digest()
# WRONG: Using username or other predictable data
key = hashlib.sha256(username.encode()).digest()Key Derivation from Passwords
from cryptography.hazmat.primitives.kdf.scrypt import Scrypt
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.primitives import hashes
import os
def derive_key_from_password(password: str, salt: bytes = None) -> tuple[bytes, bytes]:
"""Derive an encryption key from a password"""
if salt is None:
salt = os.urandom(16)
# scrypt is memory-hard (preferred)
kdf = Scrypt(
salt=salt,
length=32,
n=2**17,
r=8,
p=1
)
key = kdf.derive(password.encode())
return key, salt
# Usage
password = "user's passphrase"
key, salt = derive_key_from_password(password)
# Store salt alongside encrypted data
# Never store the password or derived key4. Key Storage
Storage Options (From Worst to Best)
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Option โ Security โ Use Case โ
โโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโค
โ Hardcoded in source โ โโโ โ Never โ
โ Config file โ โโ โ Development only โ
โ Environment variableโ โ โ Simple deployments โ
โ Encrypted file โ โ โ When HSM not available โ
โ Secrets manager โ โ โ Cloud deployments โ
โ HSM/KMS โ โโ โ Production, compliance โ
โ Hardware key โ โโโ โ Highest security โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโEnvironment Variables
import os
# Minimal improvement over hardcoding
def get_encryption_key():
key_hex = os.environ.get('ENCRYPTION_KEY')
if not key_hex:
raise ValueError("ENCRYPTION_KEY environment variable not set")
return bytes.fromhex(key_hex)
# Set in environment (NOT in code!)
# export ENCRYPTION_KEY=$(python -c "import secrets; print(secrets.token_hex(32))")Encrypted Key File
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives.kdf.scrypt import Scrypt
import os
import json
class KeyFile:
"""Store encryption keys in an encrypted file"""
def __init__(self, filepath: str, master_password: str):
self.filepath = filepath
self.master_key = self._derive_master_key(master_password)
def _derive_master_key(self, password: str) -> bytes:
# In production, store salt separately or in file header
salt = b"static-salt-for-demo" # Use random salt in production!
kdf = Scrypt(salt=salt, length=32, n=2**17, r=8, p=1)
return kdf.derive(password.encode())
def store_key(self, key_name: str, key_value: bytes):
"""Store a key in the encrypted file"""
keys = self._load_all()
keys[key_name] = key_value.hex()
self._save_all(keys)
def get_key(self, key_name: str) -> bytes:
"""Retrieve a key from the encrypted file"""
keys = self._load_all()
if key_name not in keys:
raise KeyError(f"Key '{key_name}' not found")
return bytes.fromhex(keys[key_name])
def _load_all(self) -> dict:
if not os.path.exists(self.filepath):
return {}
with open(self.filepath, 'rb') as f:
encrypted = f.read()
fernet = Fernet(self._to_fernet_key(self.master_key))
decrypted = fernet.decrypt(encrypted)
return json.loads(decrypted)
def _save_all(self, keys: dict):
data = json.dumps(keys).encode()
fernet = Fernet(self._to_fernet_key(self.master_key))
encrypted = fernet.encrypt(data)
with open(self.filepath, 'wb') as f:
f.write(encrypted)
def _to_fernet_key(self, key: bytes) -> bytes:
import base64
return base64.urlsafe_b64encode(key)
# Usage
keyfile = KeyFile("/secure/keys.enc", os.environ['KEY_FILE_PASSWORD'])
keyfile.store_key("database_key", os.urandom(32))
db_key = keyfile.get_key("database_key")Cloud KMS Integration
# AWS KMS
import boto3
class AWSKeyManager:
def __init__(self, key_id: str):
self.kms = boto3.client('kms')
self.key_id = key_id
def encrypt(self, plaintext: bytes) -> bytes:
response = self.kms.encrypt(
KeyId=self.key_id,
Plaintext=plaintext
)
return response['CiphertextBlob']
def decrypt(self, ciphertext: bytes) -> bytes:
response = self.kms.decrypt(
KeyId=self.key_id,
CiphertextBlob=ciphertext
)
return response['Plaintext']
def generate_data_key(self) -> tuple[bytes, bytes]:
"""Generate a data key encrypted by KMS"""
response = self.kms.generate_data_key(
KeyId=self.key_id,
KeySpec='AES_256'
)
# Return plaintext key for immediate use
# Store encrypted key for later retrieval
return response['Plaintext'], response['CiphertextBlob']
# Usage
km = AWSKeyManager('alias/my-master-key')
plaintext_key, encrypted_key = km.generate_data_key()
# Use plaintext_key for encryption
# Store encrypted_key with the ciphertext
# Discard plaintext_key from memory# Google Cloud KMS
from google.cloud import kms
class GCPKeyManager:
def __init__(self, project_id: str, location: str, keyring: str, key_name: str):
self.client = kms.KeyManagementServiceClient()
self.key_path = self.client.crypto_key_path(
project_id, location, keyring, key_name
)
def encrypt(self, plaintext: bytes) -> bytes:
response = self.client.encrypt(
request={'name': self.key_path, 'plaintext': plaintext}
)
return response.ciphertext
def decrypt(self, ciphertext: bytes) -> bytes:
response = self.client.decrypt(
request={'name': self.key_path, 'ciphertext': ciphertext}
)
return response.plaintext5. Key Hierarchy
Envelope Encryption
Master Key (in HSM/KMS)
โ
โโโ Decrypt โ Data Encryption Key 1 (encrypted)
โ โโโ Encrypts โ Data Set A
โ
โโโ Decrypt โ Data Encryption Key 2 (encrypted)
โ โโโ Encrypts โ Data Set B
โ
โโโ Decrypt โ Data Encryption Key 3 (encrypted)
โโโ Encrypts โ Data Set C
Benefits:
- Master key never leaves HSM
- Data keys can be rotated independently
- Data keys stored alongside encrypted data (encrypted form)
- Compromised data key only affects one datasetImplementation
import os
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
class EnvelopeEncryption:
"""Envelope encryption with KMS-protected master key"""
def __init__(self, kms_client):
self.kms = kms_client
def encrypt(self, plaintext: bytes) -> dict:
"""Encrypt data using envelope encryption"""
# Generate a new data key for this encryption
dek_plaintext, dek_encrypted = self.kms.generate_data_key()
# Encrypt data with DEK
nonce = os.urandom(12)
aesgcm = AESGCM(dek_plaintext)
ciphertext = aesgcm.encrypt(nonce, plaintext, None)
# Clear DEK from memory (Python limitation, but best effort)
del dek_plaintext
return {
'encrypted_dek': dek_encrypted,
'nonce': nonce,
'ciphertext': ciphertext
}
def decrypt(self, encrypted_data: dict) -> bytes:
"""Decrypt data using envelope encryption"""
# Decrypt the DEK using KMS
dek = self.kms.decrypt(encrypted_data['encrypted_dek'])
# Decrypt data with DEK
aesgcm = AESGCM(dek)
plaintext = aesgcm.decrypt(
encrypted_data['nonce'],
encrypted_data['ciphertext'],
None
)
# Clear DEK from memory
del dek
return plaintext6. Key Rotation
Why Rotate Keys?
Reasons for key rotation:
1. Limit exposure from potential compromise
2. Compliance requirements (PCI-DSS, etc.)
3. Personnel changes (people with key access leave)
4. Algorithm updates (SHA-1 โ SHA-256)
5. Key material exhaustion (theoretical for good ciphers)
Rotation strategies:
- Time-based: Rotate every N days/months
- Event-based: Rotate on security events
- Usage-based: Rotate after N encryptionsRotation Implementation
from datetime import datetime, timedelta
import json
class RotatingKeyManager:
"""Manage key rotation with versioning"""
def __init__(self, storage, kms):
self.storage = storage
self.kms = kms
def get_current_key(self, key_name: str) -> tuple[bytes, int]:
"""Get the current active key and its version"""
metadata = self._get_metadata(key_name)
current_version = metadata['current_version']
encrypted_key = metadata['versions'][str(current_version)]['key']
key = self.kms.decrypt(encrypted_key)
return key, current_version
def get_key_by_version(self, key_name: str, version: int) -> bytes:
"""Get a specific key version for decryption"""
metadata = self._get_metadata(key_name)
version_data = metadata['versions'].get(str(version))
if not version_data:
raise KeyError(f"Key version {version} not found")
return self.kms.decrypt(version_data['key'])
def rotate_key(self, key_name: str) -> int:
"""Create a new key version and set as current"""
metadata = self._get_metadata(key_name)
# Generate new key
new_key = os.urandom(32)
encrypted_key = self.kms.encrypt(new_key)
# Create new version
new_version = metadata['current_version'] + 1
metadata['versions'][str(new_version)] = {
'key': encrypted_key,
'created_at': datetime.now().isoformat(),
'status': 'active'
}
# Update current version
metadata['current_version'] = new_version
metadata['versions'][str(new_version - 1)]['status'] = 'decrypt-only'
self._save_metadata(key_name, metadata)
return new_version
def encrypt_with_rotation(self, key_name: str, plaintext: bytes) -> dict:
"""Encrypt with current key, including version for later decryption"""
key, version = self.get_current_key(key_name)
nonce = os.urandom(12)
aesgcm = AESGCM(key)
ciphertext = aesgcm.encrypt(nonce, plaintext, None)
return {
'version': version,
'nonce': nonce.hex(),
'ciphertext': ciphertext.hex()
}
def decrypt_with_rotation(self, key_name: str, encrypted_data: dict) -> bytes:
"""Decrypt using the correct key version"""
version = encrypted_data['version']
key = self.get_key_by_version(key_name, version)
aesgcm = AESGCM(key)
return aesgcm.decrypt(
bytes.fromhex(encrypted_data['nonce']),
bytes.fromhex(encrypted_data['ciphertext']),
None
)
def _get_metadata(self, key_name: str) -> dict:
return self.storage.get(f"key_metadata:{key_name}") or {
'current_version': 0,
'versions': {}
}
def _save_metadata(self, key_name: str, metadata: dict):
self.storage.set(f"key_metadata:{key_name}", metadata)Re-encryption After Rotation
def re_encrypt_all_data(key_manager, data_store, key_name: str):
"""Re-encrypt all data with the new current key"""
current_key, current_version = key_manager.get_current_key(key_name)
for record_id in data_store.list_all():
record = data_store.get(record_id)
# Skip if already using current key version
if record['key_version'] == current_version:
continue
# Decrypt with old key
old_key = key_manager.get_key_by_version(key_name, record['key_version'])
plaintext = decrypt(old_key, record['ciphertext'], record['nonce'])
# Re-encrypt with new key
new_ciphertext, new_nonce = encrypt(current_key, plaintext)
# Update record
data_store.update(record_id, {
'ciphertext': new_ciphertext,
'nonce': new_nonce,
'key_version': current_version
})7. Key Destruction
Secure Key Deletion
import ctypes
import os
def secure_zero(data: bytearray):
"""Securely zero out memory (best effort in Python)"""
# Get the address of the bytearray buffer
addr = id(data) + 32 # Python object header offset
size = len(data)
# Overwrite with zeros
ctypes.memset(addr, 0, size)
def secure_delete_key(key: bytes) -> None:
"""Best-effort secure deletion of key material"""
# Convert to mutable bytearray
key_array = bytearray(key)
# Overwrite multiple times
for _ in range(3):
secure_zero(key_array)
for i in range(len(key_array)):
key_array[i] = os.urandom(1)[0]
secure_zero(key_array)
# Clear references
del key_array
# Note: Python's memory management makes true secure deletion difficult
# For critical applications, use a language with better memory control
# or keep keys in HSM where they never leave secure hardwareKey Destruction Policy
from datetime import datetime, timedelta
class KeyDestructionPolicy:
"""Manage key lifecycle and destruction"""
def __init__(self, key_manager, archive_storage):
self.key_manager = key_manager
self.archive = archive_storage
def schedule_destruction(self, key_name: str, version: int,
days_until_destruction: int = 90):
"""Schedule a key version for destruction"""
destruction_date = datetime.now() + timedelta(days=days_until_destruction)
self.key_manager.update_version_status(
key_name, version,
status='pending-destruction',
destruction_date=destruction_date.isoformat()
)
def execute_pending_destructions(self):
"""Destroy keys that have passed their destruction date"""
pending = self.key_manager.get_pending_destructions()
for key_info in pending:
if datetime.now() >= datetime.fromisoformat(key_info['destruction_date']):
# Archive metadata (not the key itself!)
self.archive.store({
'key_name': key_info['key_name'],
'version': key_info['version'],
'destroyed_at': datetime.now().isoformat(),
'created_at': key_info['created_at']
})
# Destroy the key
self.key_manager.destroy_key_version(
key_info['key_name'],
key_info['version']
)8. Key Security Best Practices
Access Control
from enum import Enum
from functools import wraps
class KeyPermission(Enum):
ENCRYPT = 'encrypt'
DECRYPT = 'decrypt'
ROTATE = 'rotate'
DESTROY = 'destroy'
ADMIN = 'admin'
class KeyAccessControl:
"""Control who can do what with keys"""
def __init__(self):
self.permissions = {} # key_name -> {user_id -> set(permissions)}
def grant(self, key_name: str, user_id: str, permission: KeyPermission):
if key_name not in self.permissions:
self.permissions[key_name] = {}
if user_id not in self.permissions[key_name]:
self.permissions[key_name][user_id] = set()
self.permissions[key_name][user_id].add(permission)
def check(self, key_name: str, user_id: str, permission: KeyPermission) -> bool:
user_perms = self.permissions.get(key_name, {}).get(user_id, set())
return permission in user_perms or KeyPermission.ADMIN in user_perms
def require(self, key_name: str, permission: KeyPermission):
"""Decorator to require permission for a function"""
def decorator(func):
@wraps(func)
def wrapper(self, user_id: str, *args, **kwargs):
if not self.acl.check(key_name, user_id, permission):
raise PermissionError(
f"User {user_id} lacks {permission.value} permission for {key_name}"
)
return func(self, user_id, *args, **kwargs)
return wrapper
return decoratorAudit Logging
from datetime import datetime
import json
class KeyAuditLog:
"""Log all key operations for audit trail"""
def __init__(self, storage):
self.storage = storage
def log(self, event_type: str, key_name: str, user_id: str,
details: dict = None, success: bool = True):
entry = {
'timestamp': datetime.now().isoformat(),
'event_type': event_type,
'key_name': key_name,
'user_id': user_id,
'success': success,
'details': details or {}
}
self.storage.append('key_audit_log', entry)
def log_key_access(self, key_name: str, user_id: str, operation: str):
self.log('key_access', key_name, user_id, {'operation': operation})
def log_key_rotation(self, key_name: str, user_id: str,
old_version: int, new_version: int):
self.log('key_rotation', key_name, user_id, {
'old_version': old_version,
'new_version': new_version
})
def log_key_destruction(self, key_name: str, user_id: str, version: int):
self.log('key_destruction', key_name, user_id, {'version': version})9. Common Mistakes
Mistake 1: Key in Source Control
# WRONG: Key in code
SECRET_KEY = "aGVsbG8gd29ybGQK"
# WRONG: Key in config file that gets committed
# config.yaml:
# encryption_key: "aGVsbG8gd29ybGQK"
# RIGHT: Key from environment or secrets manager
SECRET_KEY = os.environ.get('SECRET_KEY')
# Check your .gitignore!
# .env files should never be committedMistake 2: Same Key for Everything
# WRONG: One key for all purposes
MASTER_KEY = os.environ['KEY']
encrypt_data(MASTER_KEY, data)
sign_token(MASTER_KEY, token)
encrypt_session(MASTER_KEY, session)
# RIGHT: Derive separate keys for each purpose
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
from cryptography.hazmat.primitives import hashes
def derive_purpose_key(master_key: bytes, purpose: str) -> bytes:
hkdf = HKDF(
algorithm=hashes.SHA256(),
length=32,
salt=None,
info=purpose.encode()
)
return hkdf.derive(master_key)
encryption_key = derive_purpose_key(master_key, "data-encryption")
signing_key = derive_purpose_key(master_key, "token-signing")
session_key = derive_purpose_key(master_key, "session-encryption")Mistake 3: Not Rotating After Compromise
# When a breach is detected:
def incident_response(key_manager, key_name: str):
# 1. Immediately rotate to new key
new_version = key_manager.rotate_key(key_name)
# 2. Mark old version as compromised (not just decrypt-only)
key_manager.mark_compromised(key_name, new_version - 1)
# 3. Begin re-encryption of all data (prioritize sensitive data)
schedule_reencryption(key_name, priority='critical')
# 4. Audit who had access to compromised key
generate_access_report(key_name, new_version - 1)
# 5. Notify security team
alert_security_team(key_name, 'key_compromise')10. Summary
Three things to remember:
Keys need a complete lifecycle. Generation, storage, distribution, rotation, and destructionโeach phase requires careful security considerations. Use HSM/KMS when possible.
Use envelope encryption. Protect data keys with a master key in KMS. This limits exposure, enables key rotation, and keeps master keys in hardware.
Rotate keys regularly and rotate immediately after any suspected compromise. Include key version with encrypted data so you can decrypt with old keys while encrypting with new ones.
11. Whatโs Next
Weโve covered the cryptographic building blocks and key management. Now itโs time to put everything together.
In the final article: Building Secure Systemsโapplying everything weโve learned to design end-to-end encrypted systems, secure APIs, and defense in depth.