AES Modes of Operation: Security vs Disaster Is Just One Option

1. Why Should You Care?

Here’s a penguin image encrypted with AES:

Original                ECB Encrypted          CBC Encrypted
┌──────────────┐       ┌──────────────┐       ┌──────────────┐
│   🐧🐧🐧     │       │   🐧🐧🐧     │       │ ▓▒░█▓▒░█▓▒  │
│   🐧🐧🐧     │  →    │   🐧🐧🐧     │       │ ░█▓▒░█▓▒░█  │
│   🐧🐧🐧     │       │   🐧🐧🐧     │       │ █▓▒░█▓▒░█▓  │
└──────────────┘       └──────────────┘       └──────────────┘
                       Penguin still visible!  Completely random

After ECB mode encryption, you can still see it’s a penguin. Why? Because identical plaintext blocks produce identical ciphertext blocks. Same-color regions in the image have identical ciphertext, leaking the outline.

The AES algorithm is exactly the same, but the mode choice determines security.

2. Definition

A mode of operation defines how to use a block cipher to encrypt data longer than one block.

AES can only encrypt 16-byte blocks. If your data is longer (almost always), you need a way to:

1. Split data into blocks
2. Decide how blocks relate to each other
3. Handle the last incomplete block (padding)

Different modes solve these problems differently, with different security properties.

3. ECB: Never Use This

How It Works

Plaintext: P₁ P₂ P₃ P₄
            │  │  │  │
            ▼  ▼  ▼  ▼
          ┌──┐┌──┐┌──┐┌──┐
Key ───►  │E ││E ││E ││E │
          └──┘└──┘└──┘└──┘
            │  │  │  │
            ▼  ▼  ▼  ▼
Ciphertext: C₁ C₂ C₃ C₄

Each block encrypted independently

Why It’s Insecure

If P₁ = P₃, then C₁ = C₃

Attackers can:
- Detect repeated patterns
- Rearrange blocks
- Replace blocks without decrypting

The Famous ECB Penguin

This example is so famous that “ECB Penguin” became a cryptography meme. Any image with repeated patterns leaks structural information when ECB-encrypted.

When ECB Is Acceptable

Almost never. Only exceptions:

• Encrypting a single block (16 bytes) of random data
• Key wrapping algorithms (specially designed)

Even in these cases, better options exist.

4. CBC: Classic but Needs Care

How It Works

Plaintext: P₁ P₂ P₃ P₄
            │  │  │  │
IV ───────►⊕  │  │  │
            │  │  │  │
            ▼  │  │  │
          ┌──┐│  │  │
Key ───►  │E │▼  │  │
          └──┘│  │  │
            │ ⊕  │  │
            │ │  │  │
            ▼ ▼  │  │
C₁ ────────┬──┐│  │
           │E │▼  │
           └──┘│  │
             │ ⊕  │
             │ │  │
             ▼ ▼  │
C₂ ──────────┬──┐ │
             │E │ ▼
             └──┘ │
               │  ⊕
               │  │
               ▼  ▼
C₃ ────────────┬──┐
               │E │
               └──┘
                 │
                 ▼
                 C₄

Each block's encryption depends on previous ciphertext block

The Critical Role of IV

Same plaintext + same key:
  IV = "random1234567890" → Ciphertext A
  IV = "different7654321" → Ciphertext B (completely different!)

IV ensures same plaintext doesn't produce same ciphertext

CBC Advantages

• Same plaintext produces different ciphertext (if IV differs)
• One-bit error in ciphertext affects only two plaintext blocks
• Well-studied, widely used

CBC Problems

1. IV must be random and unpredictable

# Wrong
iv = b"0" * 16  # Fixed IV
iv = str(counter).zfill(16).encode()  # Predictable IV

# Correct
iv = os.urandom(16)  # Random IV

2. Padding Oracle Attacks

If the server leaks "whether padding is correct" during decryption:
Attackers can recover plaintext byte-by-byte
This is how POODLE and Lucky 13 attacks work

3. No Integrity Protection

Attackers can modify ciphertext
Decryption produces garbage, but you might not know
Must add HMAC separately to verify integrity

4. Cannot Parallelize Encryption

Each block depends on previous block
Encryption must be sequential
(Decryption can be parallel)

5. CTR: Turning Block Cipher into Stream Cipher

How It Works

Nonce || Counter: N|0  N|1  N|2  N|3
                   │    │    │    │
                   ▼    ▼    ▼    ▼
                 ┌──┐ ┌──┐ ┌──┐ ┌──┐
Key ──────────►  │E │ │E │ │E │ │E │
                 └──┘ └──┘ └──┘ └──┘
                   │    │    │    │
Keystream:         K₁   K₂   K₃   K₄
                   │    │    │    │
Plaintext:         P₁ ⊕ P₂ ⊕ P₃ ⊕ P₄
                   │    │    │    │
                   ▼    ▼    ▼    ▼
Ciphertext:        C₁   C₂   C₃   C₄

CTR Advantages

• Can parallelize encryption and decryption
• No padding needed
• Encryption and decryption use same operation
• Random access (compute block N without previous blocks)

CTR Problem

Nonce must NEVER be reused!

If same key + nonce encrypt two messages:
C₁ = P₁ ⊕ K
C₂ = P₂ ⊕ K

C₁ ⊕ C₂ = P₁ ⊕ P₂

Attacker gets XOR of two plaintexts
If one plaintext is known, the other is recovered

6. GCM: The Modern Default

What Is GCM

GCM (Galois/Counter Mode) is an authenticated encryption mode that provides:

• Confidentiality (encryption)
• Integrity (authentication)
• Additional Authenticated Data (AAD)

┌───────────────────────────────────────────────────────────┐
│ AES-GCM = AES-CTR Encryption + GHASH Authentication       │
└───────────────────────────────────────────────────────────┘

How It Works

              ┌─────────────────────────────────────┐
              │          AES-CTR Encryption         │
              │   Nonce → Counter → AES → Keystream │
              │              ⊕                      │
              │           Plaintext                 │
              │              ↓                      │
              │           Ciphertext                │
              └─────────────────────────────────────┘
                              │
                              ▼
              ┌─────────────────────────────────────┐
              │         GHASH Authentication        │
              │   AAD + Ciphertext + Lengths        │
              │              ↓                      │
              │      Authentication Tag             │
              └─────────────────────────────────────┘

Why GCM Is the Modern Choice

1. Built-in Integrity Check

Automatically verifies authentication tag during decryption
If data was tampered, decryption fails
No separate HMAC needed

2. Additional Authenticated Data (AAD)

Can authenticate data that doesn't need encryption (like headers)
AAD is not encrypted but included in tag computation
Tampering AAD causes authentication failure

3. High Performance

CTR mode can be parallel
GHASH can be hardware-accelerated
Modern CPUs have AES-NI and PCLMULQDQ instructions

GCM Considerations

1. Nonce Must Be Unique

Like CTR, nonce reuse is catastrophic
Common approaches:
- Counter (if you can guarantee no repeats)
- Random 96 bits (collision probability extremely low but not zero)

2. Tag Length

Recommended: 128 bits (16 bytes)
Acceptable: 96 bits for some applications
Shorter tag = weaker authentication

3. Data Volume Limits

Single key + nonce combination:
Maximum 2³⁹ - 256 bits (~64GB) per encryption
Beyond this limit, change key or nonce

7. Mode Comparison Table

8. Practical Code Examples

AES-GCM (Recommended)

from cryptography.hazmat.primitives.ciphers.aead import AESGCM
import os

def encrypt_aes_gcm(plaintext: bytes, key: bytes, aad: bytes = b"") -> tuple:
    """
    Encrypt using AES-GCM
    Returns (nonce, ciphertext_with_tag)
    """
    aesgcm = AESGCM(key)
    nonce = os.urandom(12)  # 96-bit nonce
    ciphertext = aesgcm.encrypt(nonce, plaintext, aad)
    return nonce, ciphertext

def decrypt_aes_gcm(nonce: bytes, ciphertext: bytes, key: bytes, aad: bytes = b"") -> bytes:
    """
    Decrypt using AES-GCM
    Raises exception if authentication fails
    """
    aesgcm = AESGCM(key)
    return aesgcm.decrypt(nonce, ciphertext, aad)

# Usage example
key = AESGCM.generate_key(bit_length=256)
message = b"Secret message"
header = b"public header"  # AAD

nonce, ciphertext = encrypt_aes_gcm(message, key, header)
plaintext = decrypt_aes_gcm(nonce, ciphertext, key, header)

print(f"Original:  {message}")
print(f"Decrypted: {plaintext}")

AES-CBC + HMAC (Legacy but Still Acceptable)

from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.primitives import hashes, hmac, padding
import os

def encrypt_aes_cbc_hmac(plaintext: bytes, enc_key: bytes, mac_key: bytes) -> tuple:
    """
    AES-CBC encryption + HMAC authentication
    Encrypt-then-MAC approach
    """
    # Padding
    padder = padding.PKCS7(128).padder()
    padded = padder.update(plaintext) + padder.finalize()

    # Encrypt
    iv = os.urandom(16)
    cipher = Cipher(algorithms.AES(enc_key), modes.CBC(iv))
    encryptor = cipher.encryptor()
    ciphertext = encryptor.update(padded) + encryptor.finalize()

    # Compute MAC (including IV)
    h = hmac.HMAC(mac_key, hashes.SHA256())
    h.update(iv + ciphertext)
    tag = h.finalize()

    return iv, ciphertext, tag

def decrypt_aes_cbc_hmac(iv: bytes, ciphertext: bytes, tag: bytes,
                         enc_key: bytes, mac_key: bytes) -> bytes:
    """
    Verify HMAC then decrypt
    """
    # Verify MAC first
    h = hmac.HMAC(mac_key, hashes.SHA256())
    h.update(iv + ciphertext)
    h.verify(tag)  # Raises exception if verification fails

    # Decrypt
    cipher = Cipher(algorithms.AES(enc_key), modes.CBC(iv))
    decryptor = cipher.decryptor()
    padded = decryptor.update(ciphertext) + decryptor.finalize()

    # Remove padding
    unpadder = padding.PKCS7(128).unpadder()
    plaintext = unpadder.update(padded) + unpadder.finalize()

    return plaintext

9. Common Mistakes

Mistake 1: Using ECB Mode

# Wrong
cipher = Cipher(algorithms.AES(key), modes.ECB())

# Correct
cipher = Cipher(algorithms.AES(key), modes.GCM(nonce))

Mistake 2: Reusing IV/Nonce

# Wrong
nonce = b"fixed_nonce_123"  # Fixed nonce
for message in messages:
    encrypt(message, key, nonce)

# Correct
for message in messages:
    nonce = os.urandom(12)  # Generate new each time
    encrypt(message, key, nonce)

Mistake 3: CBC Without Authentication

# Wrong
ciphertext = aes_cbc_encrypt(plaintext, key, iv)
# No MAC, attacker can tamper

# Correct
ciphertext = aes_cbc_encrypt(plaintext, key, iv)
tag = hmac(mac_key, iv + ciphertext)
# Verify tag before decryption

Mistake 4: MAC-then-Encrypt vs Encrypt-then-MAC

# Wrong (MAC-then-Encrypt)
tag = hmac(plaintext)
ciphertext = encrypt(plaintext + tag)
# Cannot verify integrity before decryption

# Correct (Encrypt-then-MAC)
ciphertext = encrypt(plaintext)
tag = hmac(ciphertext)
# Can verify integrity before decryption

10. Summary

Three things to remember:

1. Never use ECB. It leaks plaintext patterns. If you see ECB in code, it’s a bug.

2. Prefer GCM. It provides encryption and authentication, and is the modern default. If you must use CBC, always add HMAC (Encrypt-then-MAC).

3. IV/Nonce must be unique. CBC’s IV needs to be unpredictable. CTR and GCM’s nonce just needs to be unique, but reuse leads to total compromise.

11. What’s Next

We’ve understood AES modes of operation. But how is symmetric encryption used in real systems?

In the next article, we’ll explore: Symmetric encryption in real systems—the symmetric encryption phase in HTTPS, file encryption, and database encryption best practices.