How Crypto Custody Fails: Keys, Wallets, Recovery, and Controls
A product-neutral guide to key generation, signing policy, multisignature and threshold custody, backups, recovery drills, monitoring, and incident response.
Digital-asset custody is the ability to authorize valid state changes while preventing unauthorized ones and retaining a dependable recovery path. It is not achieved merely by keeping a private key “offline.” A secure device can still sign the wrong transaction, a quorum can be socially engineered, a backup can be unreadable, or an operator can discover during an incident that nobody knows which policy controls which address.
The asset ledger may be decentralized, but custody is an operational security system. Its failure modes span cryptography, software, hardware, people, vendors, networks, governance, and business continuity.
Begin with the authority model
A private key authorizes whatever the relevant protocol assigns to it. That may include transferring assets, changing contract logic, rotating other keys, adding an owner, changing a threshold, or approving a bridge or staking operation. Treating every key as equivalent obscures the highest-impact paths.
For each account or contract, document:
- the assets, roles, and administrative functions it controls;
- the exact signing or validation rule;
- every key, share, device, contract, and service involved;
- who can request, review, approve, assemble, submit, and reconcile an operation;
- transaction, velocity, destination, time, and network constraints;
- emergency pause, rotation, recovery, and retirement procedures;
- dependencies on vendors, identity systems, communication channels, and staff availability.
NIST’s general key-management guidance treats generation, distribution, storage, use, backup, recovery, compromise, revocation, destruction, inventory, and audit as a lifecycle. A custody design should do the same. A diagram of where keys are stored is not a lifecycle plan.
Key generation is a ceremony, not a button
The security of every later signature depends on key generation and randomness. Define the approved software and hardware versions, entropy sources, participants, environment, network state, evidence captured, and method for verifying resulting public identifiers. Prevent one operator from silently substituting a key or address.
Ceremony evidence should be useful without exposing secret material. Record device identifiers, firmware and application versions, configuration, public keys or addresses, derivation policy, test results, participant approvals, and tamper-evident hashes of relevant artifacts. Use a second trusted path to verify destinations and public keys before funding or assigning authority.
Do not take photographs, screen recordings, clipboard logs, cloud notes, or support transcripts that contain seeds or private material. A process that generates a strong secret and then places it in ordinary collaboration software has failed.
Deterministic wallets change the backup problem
BIP 32 describes hierarchical deterministic wallets that derive a tree of key pairs from a root. This can reduce the need to back up every newly generated key and can let an online system derive public receiving addresses without holding spending authority. It also concentrates risk: compromise or loss of the relevant root material and derivation information can affect many addresses.
BIP 39 defines a mnemonic representation used to generate a deterministic seed. A mnemonic is not a password chosen by a person, and the specification explicitly warns against using user-created sentences as wallet seeds. It is high-value key material in a human-readable encoding.
A usable recovery record may need more than the words:
- the exact wallet and derivation standards and paths;
- network and account information;
- whether an additional passphrase is required;
- scripts, descriptors, or contract configuration;
- quorum and signer ordering where applicable;
- software needed to reconstruct and verify control;
- a process for distinguishing an authentic backup from a planted one.
Never assume that importing the same words into an arbitrary wallet will reproduce the intended accounts and policies. Test restoration with the actual supported implementation and a controlled, non-production account before depending on it.
Backups must resist both theft and loss
Availability and confidentiality pull in opposite directions. One easily accessible copy may be stolen; one perfectly concealed copy may be destroyed or forgotten. Design backup placement against fire, flood, confiscation, insider access, coercion, supplier failure, and simultaneous regional events.
Useful controls include geographic separation, tamper evidence, split knowledge where appropriate, dual control for access, access logs, sealed recovery instructions, and periodic inspection. Encrypting a backup creates another key-management problem; document how that encryption key is protected and recovered.
A backup test is not “the package exists.” A proper exercise reconstructs the intended public keys or account, confirms the policy and derivation paths, verifies balances or test assets, signs a harmless test operation, and records the people, time, tools, and failures involved. The drill should not expose production secrets or weaken their later protection.
Multisignature and threshold systems solve different problems
Protocol-level multisignature requires multiple signatures or approvals that the ledger or smart contract can validate. It makes the authorization rule visible and can distribute control, but it may reveal signer structure and increase transaction size or contract complexity.
Threshold cryptography distributes a secret or cryptographic operation among parties so that a threshold can sign without reconstructing the whole key in one place. NIST’s report on threshold schemes for cryptographic primitives describes how distributing an operation can preserve security goals even when some components are compromised.
Threshold or MPC does not automatically provide good governance. Ask:
- Were shares generated jointly or did one system see the complete key first?
- Can a vendor update coordinator code or substitute participants?
- Which parties control authentication, policy, and transaction data?
- Does one cloud, identity provider, network, or administrator disable the whole quorum?
- Are implementations interoperable, reviewable, and recoverable without the current vendor?
- How are malicious participants, aborted sessions, nonce handling, and share refresh addressed?
- Can the organization prove which human decision authorized the resulting signature?
A cryptographic threshold can still sit behind one compromised web session. Separate the signing quorum from the request, policy, and user-identity planes.
Policy must bind to what is signed
Approval interfaces should present the complete, canonical meaning of an operation. Human reviewers cannot reliably validate raw hexadecimal strings, truncated addresses, untrusted token symbols, or a friendly label supplied by the requester.
Decode and independently verify:
- network and chain identifier;
- operation type and contract method;
- destination addresses and their provenance;
- asset identifiers, amount, fee, and decimal interpretation;
- nonce, validity window, and replay domain;
- allowance, delegation, upgrade, ownership, and policy changes;
- internal transfers and downstream calls produced by contract execution.
Use allowlists carefully. An allowlisted contract can be upgradeable, compromised, or called with dangerous parameters. Transaction simulation is useful evidence, not an oracle: it depends on chain state, simulation infrastructure, decoding, and assumptions about what will execute later.
Require stronger review for new destinations, policy changes, large or unusual values, contract upgrades, recovery actions, and break-glass operations. Rate and velocity limits reduce blast radius only if attackers cannot change the limits through the same compromised path.
Smart accounts move risk into code and governance
Some platforms allow programmable account validation. ERC-4337, for example, supports smart-contract account logic that can implement multisignature arrangements, custom validation, and recovery. This can improve user and organizational controls, but it adds contract code, upgrade mechanisms, entry-point dependencies, simulation rules, and new denial-of-service or authorization surfaces.
Audit the deployed bytecode and proxy or upgrade path, not only a wallet application’s marketing description. Determine who can upgrade the account, replace guardians, change a threshold, disable a module, or recover access. A recovery feature is itself an authority path and often becomes the easiest route around strong day-to-day signing.
Separate duties end to end
Four eyes on one compromised laptop is not meaningful separation. Put independent control around the full operation:
- A requester creates a business instruction in an authenticated system.
- A policy service checks limits, destinations, roles, and change context.
- Reviewers receive independently decoded transaction details.
- Signers verify the approved digest or canonical operation through trusted displays or separate paths.
- A broadcaster submits the signed result without being able to alter it.
- Reconciliation confirms the expected on-chain result and raises exceptions.
Do not let the same person create a destination, approve it, change policy, and execute the transfer. Protect administrative actions at least as strongly as asset transfers. Inventory service accounts, API keys, device enrollment, recovery channels, and vendor support procedures that can influence signing.
Monitoring needs protocol context
Monitor both attempted and completed actions. Useful events include failed approvals, quorum changes, new devices, share refresh, recovery attempts, policy overrides, firmware updates, unusual signing-session failures, new destinations, abnormal fees, contract upgrades, and transactions that differ from their business instruction.
Reconcile public-ledger events against internal request identifiers and approvals. Alerting only after value leaves an address is too late for prevention, but it remains essential for containment and evidence. Preserve signed requests, policy decisions, decoded transaction views, device attestations where available, submission responses, chain identifiers, and finality state without logging secret key material.
Rehearse compromise as well as loss
Loss asks how to restore authority. Compromise asks how to stop using authority that an attacker may possess. The second problem is harder because the attacker may race the response.
Prepare playbooks for suspected seed exposure, stolen signer devices, malicious insiders, compromised policy systems, vendor outages, fraudulent destination changes, vulnerable wallet software, contract bugs, and chain disruption. Define who can pause workflows, contact counterparties, rotate keys, move assets, revoke device access, preserve evidence, and communicate externally.
Practice with bounded test assets. Measure detection time, decision time, availability of required people, ability to reconstruct policy, and time to establish a clean signing environment. A recovery that depends on the same identity provider, vendor console, or communication channel as the compromised system is not independent.
A defensible custody acceptance test
Before assigning meaningful authority, require evidence that:
- key generation and public-key verification follow a recorded ceremony;
- no single operator, device, credential, location, vendor, or cloud is an unintended critical point;
- transaction intent is independently decoded and bound to the signed payload;
- governance changes and recovery receive stronger, not weaker, controls;
- backups reproduce the expected accounts and policies in a tested environment;
- monitoring correlates requests, approvals, signatures, submissions, and ledger results;
- vendor failure and software migration have documented exit paths;
- compromise response can rotate or move authority without relying on the suspected system;
- secrets never enter logs, tickets, screenshots, analytics, or ordinary backups.
The central question is not whether a key is hot or cold. It is whether every path that can create, use, change, recover, or destroy signing authority has an explicit owner, independent controls, testable evidence, and a survivable failure mode.