That smell again … A PSU about to die. Possibly n0002's (?).

Indeed died two hours later. Replace PSU, restart job.

Restart the jobs while waiting for the rain (if not winter) to pass …

Change the A/C setup : make the direction of air stream constant (and pointing to the lower part of the cluster) → significant difference in average core temperatures (to the better). Wait for possible icing problems → still looking stable after three days …