Core Architecture & Compliance Mapping

Signing Compliance Chains with HMAC in Python for Tamper-Evident Logs

Chain each record to the previous with an HMAC so any edit or reorder breaks verification.

This page solves one specific production failure: a compliance record is quietly edited or two entries are swapped after the fact, and nothing in the log itself reveals that the sequence is no longer the one that was written.

A waste-operations audit log is only worth as much as its integrity. If any row can be altered without trace, the log cannot substantiate a disposal event, a duty-status change, or a route decision to a regulator. The fix is to bind each record to the one before it with a keyed hash: chain_mac = HMAC(key, prev_mac + record_hash). Because every link folds in the previous MAC, editing a record in the middle invalidates that link and every link after it — tampering is not merely detectable, it is unmissable on a single verification pass. This is a foundational control in Security & Access Boundaries within the Core Architecture & Compliance Mapping framework.

HMAC chain: each link folds in the previous MAC, so one edit breaks every link after it Three compliance records are each canonicalized to sorted-JSON and hashed to a record_hash. Each chain_mac is HMAC of the key over prev_mac concatenated with record_hash, the first link using a fixed genesis MAC. Each chain_mac becomes the next link's prev_mac. Editing a middle record changes its hash and every downstream MAC, which a full verify pass detects. genesis fixed MAC seed record #1 canonical JSON → hash HMAC(k, prev+hash) record #2 edited after write hash changes record #3 MAC no longer recomputes mac0 mac1 mac2 verify pass recomputes every link from genesis and compares first mismatch at #2 → chain broken from here forward

Environment & Data Prerequisites

The signer needs nothing beyond the standard library — hmac, hashlib, json, os, and logging on Python 3.10+. The signing key is read from the environment, never hard-coded. pytest is the only third-party dependency, used to prove that tampering is caught:

python -m pip install pytest==8.2.0
export WASTE_COMPLIANCE_HMAC_KEY="$(python -c 'import secrets; print(secrets.token_hex(32))')"

Each record entering the chain is a flat compliance event. The signer folds these fields into a canonical form; the surrounding chain metadata it adds is described in the output section:

Field Unit / type Rule
record_id str unique per event, monotonic within a chain
event_type str e.g. disposal_logged, duty_status_change
body object the event payload, canonicalized before hashing
ts ISO 8601 string tz-aware UTC event time

Two rules matter before any code. The record must be serialized canonically — sorted keys, no insignificant whitespace — because HMAC is byte-exact and two logically-identical dicts with different key order would otherwise produce different MACs and appear tampered. And the key lives only in the environment or a secret manager: a key committed to source can be used by anyone to forge a valid chain, which defeats the entire control.

Step-by-Step Implementation

Step 1 — Canonical serialization and record hashing

The first job is to turn a record into the exact same bytes every time. json.dumps with sort_keys=True and tight separators is deterministic across processes and Python versions, and hashing that with SHA-256 gives a stable record_hash.

import hmac
import hashlib
import json
import os
import logging
from datetime import datetime, timezone
from dataclasses import dataclass


def canonical_bytes(record: dict) -> bytes:
    """Deterministic byte representation: sorted keys, no insignificant whitespace."""
    return json.dumps(record, sort_keys=True, separators=(",", ":")).encode("utf-8")


def record_hash(record: dict) -> str:
    return hashlib.sha256(canonical_bytes(record)).hexdigest()

Step 2 — The keyed chain MAC

The chain MAC binds the current record hash to the previous MAC under the secret key. Using hmac.new with SHA-256 keys the digest so it cannot be forged without the secret, and folding prev_mac in is what links the records into a chain rather than a set of independent signatures. The genesis MAC is a fixed, well-known seed used only for the first link.

GENESIS_MAC = "0" * 64  # fixed seed for the first link; there is no prior record


def _key() -> bytes:
    key = os.environ.get("WASTE_COMPLIANCE_HMAC_KEY")
    if not key:
        raise RuntimeError("WASTE_COMPLIANCE_HMAC_KEY is not set")
    return key.encode("utf-8")


def chain_mac(prev_mac: str, rec_hash: str) -> str:
    msg = (prev_mac + rec_hash).encode("utf-8")
    return hmac.new(_key(), msg, hashlib.sha256).hexdigest()

Step 3 — Append a signed entry

Appending computes the record hash, folds in the previous link’s MAC, and stores all three values with the record. The previous MAC comes from the last entry already in the chain, or the genesis seed for an empty chain.

@dataclass(frozen=True)
class ChainEntry:
    record: dict
    record_hash: str
    prev_mac: str
    chain_mac: str


def append_entry(chain: list[ChainEntry], record: dict) -> ChainEntry:
    prev_mac = chain[-1].chain_mac if chain else GENESIS_MAC
    rh = record_hash(record)
    entry = ChainEntry(
        record=record,
        record_hash=rh,
        prev_mac=prev_mac,
        chain_mac=chain_mac(prev_mac, rh),
    )
    chain.append(entry)
    return entry

Step 4 — Verify the whole chain

Verification recomputes every link from the genesis seed forward and compares against the stored MACs using a constant-time compare. It returns the index of the first broken link, or None if the chain is intact — so a reviewer learns not just that tampering occurred but exactly where.

class JSONFormatter(logging.Formatter):
    def format(self, record: logging.LogRecord) -> str:
        return json.dumps({
            "logged_at": datetime.now(timezone.utc).isoformat(),
            "level": record.levelname,
            "event": record.getMessage(),
            "chain": getattr(record, "chain", None),
        })


audit_logger = logging.getLogger("waste_ops.hmac_chain")
if not audit_logger.handlers:
    _h = logging.StreamHandler()
    _h.setFormatter(JSONFormatter())
    audit_logger.addHandler(_h)
audit_logger.setLevel(logging.INFO)


def verify_chain(chain: list[ChainEntry]) -> int | None:
    prev_mac = GENESIS_MAC
    for i, entry in enumerate(chain):
        rh = record_hash(entry.record)            # recompute from the record itself
        expected = chain_mac(prev_mac, rh)
        if not hmac.compare_digest(expected, entry.chain_mac):
            audit_logger.warning("chain_broken", extra={"chain": {"broken_at": i}})
            return i
        prev_mac = entry.chain_mac
    return None


if __name__ == "__main__":
    chain: list[ChainEntry] = []
    for rec in [
        {"record_id": "C1", "event_type": "disposal_logged", "body": {"tons": 4.2}, "ts": "2026-07-16T08:00:00+00:00"},
        {"record_id": "C2", "event_type": "duty_status_change", "body": {"status": "on_duty"}, "ts": "2026-07-16T08:05:00+00:00"},
        {"record_id": "C3", "event_type": "disposal_logged", "body": {"tons": 3.9}, "ts": "2026-07-16T08:40:00+00:00"},
    ]:
        e = append_entry(chain, rec)
        audit_logger.info("entry_appended", extra={"chain": {"record_id": rec["record_id"], "chain_mac": e.chain_mac}})
    print("intact:", verify_chain(chain) is None)

Compliance Output

Each appended entry stores the three chain values with its record. A single signed entry serializes as:

{"record_id": "C2", "record_hash": "9f2c...e41a", "prev_mac": "3ab8...77d0", "chain_mac": "c15e...9b42"}

Each field earns its place in the audit trail:

  • record_hash — the SHA-256 over the canonical bytes of the record. It pins the exact content signed; any change to the record body, even a reordered key that canonicalization would normalize, is captured here.
  • prev_mac — the previous link’s chain MAC, the ingredient that turns independent signatures into a chain. Its presence is what makes reordering detectable: swap two entries and each one’s prev_mac no longer matches its neighbour.
  • chain_mac — the keyed HMAC over prev_mac + record_hash. Because it requires the secret key, no one without key custody can forge or repair a link, which is what makes the log tamper-evident rather than merely tamper-detecting. Key custody is therefore the whole control: the key belongs in a secret manager with restricted access, rotated on a schedule, and the RBAC setup for waste ops dashboards reference governs who may hold it.

A chain built this way is what the generating tamper-evident compliance reports layer relies on to assert that a rendered report reflects the records exactly as originally written, and it satisfies the retained-record integrity expectations that compliance reporting and automation carries end to end.

Verification

The decisive test edits a record in the middle of a sealed chain and confirms verification points to that exact link.

import pytest


def _fresh_chain() -> list[ChainEntry]:
    chain: list[ChainEntry] = []
    for i in range(4):
        append_entry(chain, {"record_id": f"C{i}", "event_type": "disposal_logged",
                             "body": {"tons": 1.0 + i}, "ts": f"2026-07-16T08:0{i}:00+00:00"})
    return chain


def test_intact_chain_verifies():
    assert verify_chain(_fresh_chain()) is None


def test_edited_middle_record_breaks_chain_at_that_link():
    chain = _fresh_chain()
    # tamper: someone changes the tonnage on entry index 1 after the fact
    object.__setattr__(chain[1], "record", {**chain[1].record, "body": {"tons": 99.0}})
    assert verify_chain(chain) == 1


def test_reordering_two_records_is_detected():
    chain = _fresh_chain()
    chain[1], chain[2] = chain[2], chain[1]   # swap without recomputing MACs
    assert verify_chain(chain) is not None

Common Errors

A chain that was intact yesterday fails verification today with no edits. Root cause: the record was re-serialized with a non-canonical dump — a different key order or added whitespace — so record_hash no longer matches. Fix: hash only through canonical_bytes (Step 1) with sort_keys=True and tight separators, and never feed the signer a pretty-printed or differently-ordered copy of the record.

RuntimeError: WASTE_COMPLIANCE_HMAC_KEY is not set — the key is missing from the environment. Root cause: the secret was hard-coded in a config file that was then removed, or the process was started without the env var exported. Fix: load the key from a secret manager or environment as _key() does; do not fall back to a literal default, because a default key lets anyone forge a valid chain.

AttributeError: 'NoneType' object has no attribute 'chain_mac' on the first append. Root cause: reaching for chain[-1] on an empty chain instead of seeding with the genesis MAC. Fix: use chain[-1].chain_mac if chain else GENESIS_MAC (Step 3) so the first link folds in the fixed genesis seed and every later link folds in its predecessor.

FAQ

Why HMAC instead of a plain SHA-256 chain?

A plain hash chain is tamper-evident only if the attacker cannot recompute the hashes — but SHA-256 is public, so anyone who edits a record can recompute every downstream hash and reseal the chain, leaving no trace. HMAC folds in a secret key, so resealing requires key custody. Without the key an edit cannot be repaired, which is what upgrades the log from detectable-if-you-kept-a-copy to genuinely tamper-evident.

What does the genesis MAC protect against?

The genesis MAC anchors the chain so the first record cannot be silently dropped or replaced. Because link one folds the fixed genesis seed into its MAC, removing the original first entry and promoting the second changes what the new first link should hash to, and verification fails. Without a defined genesis, the head of the chain would be unanchored and a truncation from the front could go unnoticed.

Up: Security & Access Boundaries