Thermal Failure Modes

derate · hotspot · trip · runaway

Thermal failures follow repeatable patterns and announce themselves in telemetry. If throttling is routine, the system is already past its thermal limit.

How thermal failures escalate, in five steps: drift, where approach temperature, pump power and branch delta-T creep up; derate, where chargers, GPUs and inverters roll back power; hotspot, where one connector, rack or battery module overheats; trip, where a chiller, pump or HVAC unit drops out; and outage or fire, with examples of the CME Group trading halt in November 2025 and the Moss Landing BESS fire in January 2025. Each step is visible in telemetry before the next one arrives. HOW THERMAL FAILURES ESCALATE 1 Driftapproach, pump power and delta-T creep up 2 Deratechargers, GPUs and inverters roll back power 3 Hotspotone connector, rack or battery module overheats 4 Tripa chiller, pump or HVAC unit drops out 5 Outage or fireCME Group halt, Nov 2025; Moss Landing BESS, Jan 2025 Each step is visible in telemetry before the next one arrives.
Most common mode
Derating
Best KPI
Derate frequency
Redundancy floor
N+1
Riskiest moment
Mode transitions
Hub
Supply Chain

Finding

Most serious thermal failures begin as slow drift that was logged but not acted on, and end when a single cooling component fails during peak load

In November 2025 a cooling failure at CyrusOne's data center in Aurora, Illinois halted CME Group's futures markets for about 10 hours. In January 2025 the Moss Landing battery building in California, a Vistra facility using LG Energy Solution NMC cells, burned for days after thermal runaway. Both fit the pattern: a cooling or containment path with too little margin, then one failure that the remaining equipment could not cover.

The catalog below is ordered by how often each mode appears. Derating and hotspots are daily events at dense sites; trips and runaway are rare but expensive. The common defenses are design for the degraded state - one unit out, a fouled exchanger, reduced water, at peak ambient - instrumentation by zone or branch rather than plant totals, and hysteresis with minimum run times at mode transitions, where control loops most often oscillate.

Battery sites have moved the most: since Moss Landing, the industry has shifted from large buildings of NMC racks to outdoor LFP containers with liquid cooling, gas detection and spacing tested to UL 9540A and NFPA 855.

Catalog

Failure mode catalog

Thermal failure modes
ModeSymptomRoot causesEarly warningMitigation
DeratingPower rollback; throughput dropsToo little rejection capacity; hot ambient; fouled exchangerRising delta-T and approach; more fan and pump dutyHeadroom; hybrid rejection; cleaning plan; buffering
HotspotsSingle-component overtemperatureConcentrated heat flux; manifold imbalance; connector resistanceThermal imaging; branch delta-T divergenceCooling at the source; rebalance flow; liquid-cooled cables
BESS thermal runawayCell alarms; shutdown; fireNon-uniform cooling; HVAC failure; cell defectsWidening cell temperature spread; off-gas sensorsZoning; detection; propagation barriers; LFP; spacing
Control instabilityTemperature hunting; mode-switch faultsPoor tuning; sensor latency; setpoint conflictsRapid valve and fan cycling; overshootCommissioning; hysteresis; time-synced telemetry
Water chemistry driftScaling, corrosion, capacity lossUnderdesigned treatment; poor monitoringConductivity drift; rising pressure dropInstrumented treatment; preventive maintenance
Fouling and cloggingLower flow; higher pump powerDirty strainers; biofouling; scalingPressure drop and pump power riseFiltration; differential-pressure alarms; cleaning
Leak or contaminationCoolant loss; downtimeMaterial incompatibility; poor joints; vibrationMakeup water rise; pressure decayLeak detection; isolation zones; pressure testing
Chiller or compressor tripSudden capacity lossCondenser fouling; low flow; refrigerant faultsHigh condenser pressure; short cyclingN+1 chillers; staged controls; power conditioning
Fan or pump single pointZone temperature excursionNo redundancy or sparesVibration, bearing temperature, currentN+1; quick-swap spares; predictive maintenance
Heat sink mismatchWorks in test, fails on hot daysDesigned to average ambient; ignored wet-bulbDerates correlate with weather extremesDesign to extremes and degraded states

By site

Failures by deployment

Thermal failures by deployment
DeploymentMost common failuresImpactHigh-leverage fixes
AI data centersHotspots, throttling, coolant distributionIdle GPUs; SLA riskDirect-to-chip; telemetry; stable transitions
Fleet DCFC and MCS depotsCharger rollback, connector overheatingMissed dispatch windowsLiquid-cooled cables; load scheduling
BESS sitesUneven cooling, HVAC failureLost capacity; safety deratesZoning; redundancy; conservative envelopes
Semiconductor fabsChilled-water excursionsYield loss; tool downtimeRedundant plants; predictive maintenance
GigafactoriesDry-room excursions, process heat overloadThroughput caps; quality driftZoned plants; buffering

Telemetry

Minimum telemetry

Minimum thermal telemetry
SignalWhat to logAutomation
Derate eventsTime, ambient, load, zone, durationAlert when weekly trend rises
Delta-T per zoneSupply and return by branch; flowFlag divergence beyond threshold
Approach temperatureWet-bulb or dry-bulb vs reject temperatureTrigger mode change or pre-cooling
Pump and fan powerkW, duty, starts and stopsFlag abnormal duty for inspection
Water chemistryConductivity, pH, biocide, blowdownHard alarms on drift, soft on trend
Mode transitionsEconomizer-to-chiller switches, stagingEnforce hysteresis and minimum run time

Sources

Sources and scope

Scope
Thermal Failure Modes
Sources
CME Group and CyrusOne statements (November 2025); Monterey County and Vistra (January 2025); UL 9540A; NFPA 855
Updated
October 2026