We’re troubleshooting an odd issue that appears to have gotten worse over the weekend. 6 servers housing 36 OSDs are all behaving very slowly. We’re looking for a common thread.
This is impacting performance in Canada this morning.
UPDATE 4:28AM: We have enough spare capacity to turn down these laggy servers while we work on them. To do this requires us to move data around. This process has started.
UPDATE 5:50AM: This is connected to the CHIA farming, we’ll post updates in that thread above starting now. We’ve already started moving data from the impacted machines and it’s actually safer to let the process continue while we also go after the Chia farms.