Database migration latency

Incident Report for Alloy

Resolved

At 10:41AM today, a database migration resulted in unexpected latency under load. Alloy’s incident response team was notified immediately by automated alerting, and was able to quickly diagnose the problem. The root cause was resolved within a couple of minutes, but it caused a backlog of requests that resulted in latency, timeouts, and, in some cases, failed requests until 10:54AM.

Posted Dec 19, 2022 - 10:41 EST