[08:46:34] Anyone else having problems accessing https://en.wikipedia.beta.wmflabs.org/w/api.php? [08:46:50] Connection timesout [08:56:21] !log admin icinga downtime cloudvirt[1028-1030].eqiad.wmnet for 1h for reboots [08:56:23] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [08:57:02] dwalden: I see some weird css-less page, but no connection timeout [08:57:29] Hmm. Well it is working for me now. [09:00:43] !log admin rebooting cloudvirt1028 for kernel updates [09:00:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:11:29] !log admin reboot of cloudvirt1028 went fine [09:11:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:12:57] !log admin rebooting cloudvirt1029 for kernel updates [09:12:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:17:29] !log tools due to the reboot of cloudvirt1029, several sgeexec nodes (8) are offline, also sgewebgrid-lighttpd (8) and tools-worker (3) and the main toolforge proxy (tools-proxy-03) [09:17:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:17:50] arturo: I assume tools.wmflabs.org down is known? [09:18:17] if you're a channel op and this is assumed to last for some time, could you update the channel topic? [09:18:21] yes, unfortunately. I'm doing a reboot of the server. Will take a couple of minutes [09:18:32] okay [09:18:48] thanks [09:25:35] Urbanecm: I would say things are back to normal? [09:28:27] !log admin rebooting cloudvirt1030 for kernel updates [09:28:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:28:43] !log admin reboot of cloudvirt1029 went fine [09:28:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:29:14] !log tools toolforge is recovered from the reboot of cloudvirt1029 [09:29:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:40:14] !log admin reboot of cloudvirt1030 went fine [09:40:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:17:57] !log codesearch ladsgroup@codesearch4:~$ sudo service hound-services restart (T235596) [10:18:00] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Codesearch/SAL [10:18:01] T235596: Services codesearch is down - https://phabricator.wikimedia.org/T235596 [10:39:30] !log tools.wikibase-databridge-storybook reran update script manually, because the static storybook was missing/broken. Cause unclear. Some errors during script execution, but storybook is available again 🤷 [10:39:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibase-databridge-storybook/SAL [10:41:37] arturo: yes! [10:41:51] cool! [14:01:00] Technical Advice IRC meeting starting in 60 minutes in channel #wikimedia-tech, hosts: - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [14:50:58] Technical Advice IRC meeting starting in 10 minutes in channel #wikimedia-tech, hosts: - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [16:21:00] !log tools Deployed toollabs-webservice 0.46 to buster-tools and stretch-tools (T218461) [16:21:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [16:21:04] T218461: `webservice --backend=kubernetes python restart` starts a php5.6 webservice - https://phabricator.wikimedia.org/T218461 [21:59:37] !log admin resync wiki replica tool and user accounts T235697 [21:59:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [21:59:41] T235697: labsdb1011 missing tools user accounts - https://phabricator.wikimedia.org/T235697 [22:25:16] !log tools.lexeme-forms deployed b480b6d07e (Czech translations + adjectives with more forms) [22:25:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL