Database lock resulting in API outage
Incident Report for Alloy
Resolved
Between the hours of 22:02 and 22:12 ET (10 minutes), a scheduled database migration that had been tested successfully in lower environments acquired an unintended long lock on a database table, resulting in a backup of queries and increasingly high latencies. The migration was automatically detected and killed, freeing up database performance and returning operations to normal. The postmortem process is underway and multiple new checks for this type of failure are already in process.
Posted Jan 16, 2024 - 22:12 EST
Investigating
Our API integration tests have encountered an increase in errors. We are currently investigating. Stay tuned for updates.
Posted Jan 16, 2024 - 22:00 EST
This incident affected: Production API.
← Current Status Powered by Atlassian Statuspage