Facilitator notes · Technology, 2012 · About 40 minutes
The Knight Capital incident, 2012
On 1 August 2012, new trading software went live at Knight Capital Americas, then a major market maker in US shares. Old code came back to life on one of its eight servers, and in about 45 minutes the firm made more than 4 million executions in 154 stocks and lost over $460 million. In Before the Bell the group takes the seats of a developer, a technician, an operations analyst, a senior trader and a technology lead. It is written for technology teams, and no trading knowledge is needed.
For the facilitator. In the session this event stays masked, as Before the Bell, until the reveal at the end, so the group meets it as the people involved did, with only what they knew at the time. Keep this page for yourself before the session.
Run Before the BellAll scenarios
What happened
Knight Capital Americas, then a major market maker in US shares. The router was called SMARS; the old function was Power Peg, and the emails had said “Power Peg disabled”.
212 customer orders triggered more than 4 million executions, over 397 million shares, in about 45 minutes.
Within days the firm needed about $400 million in emergency investment from a group of other firms. It was taken over the following year.
What the group does
Over this session we’ll sit in several seats: a developer, a technician, an operations analyst, a senior trader and a technology lead.
No trading knowledge is needed. The firm routes customers’ orders to stock exchanges through its own software, spread across several servers.
- An ordinary day. Nothing goes wrong. We see how routine technical work usually gets done.
- The incident day. Launch day for a new exchange programme. Information arrives in the order it arrived for the people involved.
- The reveal and debrief. What really happened, why it made sense, and what it means for us.
It needs a screen everyone can see and about 40 minutes. A facilitator is optional, and nobody needs to sign in or know the field.
What went right
The debrief looks at what people did well as well as what went wrong: most days, the same adaptations are why things go right.
- The firm got its new code into production in stages, ahead of a fixed launch date.
- The system itself flagged the problem: 97 emails before the open named the function that was about to misbehave.
- Senior staff noticed the unusual positions as soon as the market opened.
- The technology team stopped a failure they had no procedure for, in a live market, in about 45 minutes.
- Other firms stepped in with emergency investment within days, and the firm kept operating.
What the debrief opens up
Its sections:
- Old code, recycled switches
- Signals nobody was designed to read
- When the obvious fix makes it worse
Some of the questions it asks the group:
- On the ordinary days, what made manual deployments and ignored emails safe? What took that away?
- What dormant code, old settings or forgotten processes are still live in our systems?
- Which of our automated messages would we ignore? Who decided they don’t matter?
- Who here is allowed to stop a misbehaving system? Would they know they were allowed?
Ways to debrief it
Each real incident can be debriefed three ways, chosen on its briefing screen: the standard debrief, a quick 20 to 30 minute debrief, or a learning team. To run a learning team on something from your own work, use the learning team guide.
Further reading
- Stopping the line. Toyota’s Andon cord: anyone can stop the line, the first thing that happens is that they are thanked, and false alarms count as useful drills. The tool was easy to copy; the conditions that made it usable were not. The Andon Cord, Tom Geraghty.
- Tight coupling. Perrow’s normal accidents: in complex, tightly coupled systems a small failure spreads fast. Incentivise simplicity over complexity. Normal Accidents, Tom Geraghty.
- Production pressure. The launch date was fixed before the code was ready. A fixed date doesn’t cause an incident on its own; it decides which checks feel affordable. The Vasa is the same story four centuries earlier: changing requirements, a deadline, and a failed stability test before the launch went ahead. The Vasa, Tom Geraghty.
- Change introduces new forms of failure. One of Richard Cook’s eighteen points on how complex systems fail, alongside the failures that sit latent in every system, waiting for the right conditions. Dr Richard Cook: How Complex Systems Fail, Tom Geraghty.
- Work as imagined, work as done. With no written procedure, the deployment as imagined lived in people’s heads, and each smooth one done from memory made the next feel safer. Working without a second check was a sensible adaptation that had worked for years; the gap between how deployment was imagined and how it was done only showed on the day one server was missed. Work as Imagined vs Work as Done, Tom Geraghty.
- Human error is a starting point. If one missed step by one person can bring a system down, the system is the problem. Human error should be the start of an investigation, not the end. Types of Human Error, Tom Geraghty.
- Counterfactuals. “They should have” explains little about why people did what they did. The article points to John Allspaw’s piece on counterfactuals and this accident. Counterfactuals, Tom Geraghty.
Key papers
- Normal accidents. Why complex, tightly coupled systems produce failures that no amount of training or procedure removes. Perrow (1984), Normal Accidents: Living with High-Risk Technologies, on the Psych Safety field guide.
- How complex systems fail. Richard Cook’s eighteen points, including that change introduces new forms of failure and that hindsight biases how we judge people afterwards. Cook (1998), How Complex Systems Fail, on the Psych Safety field guide.
- Behind human error. Error as the starting point of an investigation rather than its conclusion, and how knowing the outcome changes how we judge the same actions. Woods et al. (1994), Behind Human Error: Cognitive Systems, Computers, and Hindsight, on the Psych Safety field guide.
- Adaptive capacity in incident response. A field study of a technology organisation bringing in more people as an incident grew, and how teams scale their response to events bigger than anyone planned for. Cook & Long (2021), Building and Revising Adaptive Capacity Sharing for Technical Incident Response, on the Psych Safety field guide.