AWS Database Storage Cluster Issue

Incident Report for VisualVault

Postmortem

Root Cause:
The SQL cluster temporarily lost access to its backend storage due to a network protocol failure on the storage virtual machine.

Resolution:
Engineers restored connectivity by restarting key networking protocols on the storage VM. All services were fully operational by 1:11 PM.

Duration:
Approximately 40 minutes.

Preventive Measures:
We are adding proactive monitoring and validation checks for storage networking services to prevent recurrence.

Posted Nov 12, 2025 - 14:25 MST

Resolved

This incident has been resolved.

Posted Nov 12, 2025 - 13:16 MST

Investigating

We are currently investigating an issue with database storage within AWS

Posted Nov 12, 2025 - 12:46 MST

This incident affected: VisualVault - North America 3 (NA3) (VisualVault Web UI, REST API, Document Viewing), VisualVault - South America (VisualVault Web UI - SA1), VisualVault - North America (VisualVault - EF, VisualVault - ME), and VisualVault - Public Sector (Arizona EWFS Self Exclusion, Arizona OPG, Arizona Self Exclusion, California CDPH, City Of Lincoln, Florida DCF, Florida OLS, PCHS).