Microsoft Azure: Distribution failure causes power outage at Microsoft data center

A Microsoft data center experienced a two-hour outage as its electrical distribution system failed when the company attempted to swap from utility power to its backup generators. Microsoft noted in a newly published report that a power issue impacted some customers in a single Availability Zone within the West Europe region in the Netherlands, between 07:31 and 09:15 UTC on 20 October 2023.

The company noted that it has noticed some instability from the utility power grid which led it to transfer the load to backup generators. However, an issue with the generator startup led to an outage for some racks. Microsoft did not give details about the cause of the distribution failure but noted that the company was able to swap back from generators to utility powers after stabilizing the grid. 

“In total, five Storage scale units were impacted by this incident. Following power restoration, four recovered completely by 09:10 UTC, while the fifth required hardware diagnostics and part replacements on approximately 5 percent of its storage nodes,” Microsoft noted in its report. “As a result, it took longer to restore availability for the last <1 percent of storage accounts, with downstream impact to customers and services reliant on this final Storage scale unit. By 14:30 UTC, all but a few storage accounts had their availability restored, and by 17:10 UTC, full restoration was complete.”