← → or space to move between slides · f full screen · g all slides
Bi-weekly service review
Prepared for Tata Communications by Zixi
20h 59m of combined downtime across 23 outages, affecting 5 customers (4h 46m elapsed). 4 unplanned incidents, the most serious a Sev 2 (partial outage). 5 maintenance windows, 1 of them emergency.
This period at a glance
All times UTC
Overview
Combined downtime
20h 59m
across 23 outages · 4h 46m elapsed
Longest single outage
4h 1m
Mercury Radio Arts (Glenn Beck)
GLENNBECK-BX-WEST-01
4
Unplanned incidents
0
Still open
5
Customers affected
5
Maintenance windows
1 emergency
Overview
Service impact
All times UTC.
Each row is one broadcaster, and each bar shows how long an outage lasted. Bars that line up across rows come from the same event. Showing the 8 broadcasters with the most downtime, of 9 affected; the full timeline is on the dashboard.
Service impact
Combined downtime by customer, region and host server.
Incidents
Incident 1 of 4
19h 52m
Combined downtime
4h 6m
Elapsed time
6 Jul 04:14
Started (UTC)
8
Broadcasters affected
Customers affected: CS Test · Mercury Radio Arts (Glenn Beck) · Multimedios · TCN The Country Network · Zixi ContentXchange - CXE
Tata's planned full-node reboot of the LA Cloudlyte environment overran, taking five customer broadcasters down for close to four hours from 04:14 UTC on 6 July. The NJ region then flapped repeatedly the following afternoon.
Root cause: Ceph RBD remap remediation required a reboot of every node in the LA Cloudlyte environment; NJ flapping was reported by Tata separately and not attributed.
Resolution: All LA broadcasters recovered by 08:16 UTC. NJ flaps self-cleared within a minute each.
Incident 2 of 4
41m
Combined downtime
14m
Elapsed time
10 Jul 04:15
Started (UTC)
3
Broadcasters affected
Customers affected: CS Test · Mercury Radio Arts (Glenn Beck) · Multimedios
Three NJ broadcasters were acknowledged down from 04:15 UTC on 10 July. The tracker was resolved the same morning but its outage table was never closed out, so no end time or duration was ever recorded.
Resolution: Done
Incident 3 of 4
0m
Combined downtime
0m
Elapsed time
30 Jun 17:47
Started (UTC)
—
Broadcasters affected
Resolution: Done
Incident 4 of 4
0m
Combined downtime
0m
Elapsed time
30 Jun 03:48
Started (UTC)
—
Broadcasters affected
Maintenance
5 windows in the period, 1 of them emergency.
Action items
3 open of 3 tracked.
Before the next review
Thank you. Review period 30 Jun – 14 Jul 2026 · all times UTC