[08:16:31] <_dcaro>	 !log admin After enabling auto pg scaling on ceph eqiad cluster, osd.44 (cloudcephosd1005) got stuck, trying to restart 
[08:16:34] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[08:16:40] <_dcaro>	 !log admin After enabling auto pg scaling on ceph eqiad cluster, osd.44 (cloudcephosd1005) got stuck, trying to restart the osd service
[08:16:42] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[08:19:01] <_dcaro>	 !log admin Restarting serivce osd.44 resulted on osd.44 being unable to start due to some config inconsistency (can not reset class to hdd)
[08:19:02] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[08:45:17] <_dcaro>	 !log admin Tried resetting the class for osd.44 to ssd, no luck, the cluster is in noout/norebalance to avoid data shuffling (opened root@cloudcephosd1005:/var/lib/ceph/osd/ceph-44# ceph osd crush set-device-class ssd osd.44)
[08:45:19] <_dcaro>	 set osd(s) 44 to class 'ssd'
[08:45:20] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[08:45:28] <_dcaro>	 damn, bad paste xd
[08:45:42] <_dcaro>	 !log admin Tried resetting the class for osd.44 to ssd, no luck, the cluster is in noout/norebalance to avoid data shuffling (opened T268722)
[08:45:44] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[08:45:45] <stashbot>	 T268722: Ceph eqiad cluster: osd.44 failing to start - https://phabricator.wikimedia.org/T268722
[08:54:29] <_dcaro>	 !log admin Unsetting noup/nodown to allow re-shuffling of the pgs that osd.44 had, will try to rebuild it (T268722)
[08:54:32] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[08:54:32] <stashbot>	 T268722: Ceph eqiad cluster: osd.44 failing to start - https://phabricator.wikimedia.org/T268722
[09:31:19] <_dcaro>	 !log admin The OSD seems to be up and running actually, though there's that misleading log, will leave it see if the cluster comes fully healthy (T268722)
[09:31:23] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[09:31:23] <stashbot>	 T268722: Ceph eqiad cluster: osd.44 failing to start - https://phabricator.wikimedia.org/T268722
[19:03:33] <legoktm>	 !log codesearch fixed permissions on /srv/codesearch/.git/objects, forcing puppet run which is triggering restarts
[19:03:36] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Codesearch/SAL
[19:35:29] <bstorm>	 !log admin repairing ceph pg `instructing pg 6.91 on osd.117 to repair`
[19:35:31] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL