[03:24:46] 17 netsplits in 40 minutes earlier today - wtf [11:03:46] !log tools.openstack-browser restarted webservice by hand for nothing, given we don't have here yet the webservice version I was expecting [11:03:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.openstack-browser/SAL [14:55:58] In about five minutes we're going to restart the openstack database -- there may be some brief service hiccups [15:02:37] 👍 [15:03:42] that's it, restart is done and things should all be back up [17:47:51] !log tools.admin-beta Update to c153daf "Fix main page of tools.wmflabs.org to work correctly on mobile devices" (T194943) [17:47:54] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.admin-beta/SAL [17:47:54] T194943: Make Toolforge main page mobile-friendly - https://phabricator.wikimedia.org/T194943 [17:48:55] !log tools.admin Update to c153daf "Fix main page of tools.wmflabs.org to work correctly on mobile devices" (T194943) [17:49:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.admin/SAL [21:18:57] !log toolsbeta upgraded toollabs-webservice package for stretch toolsbeta to 0.62 T244293 T244289 T234617 T156626 [21:19:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [21:19:03] T244293: Add a function to webservice called "migrate" that will push a tool from the old cluster on Kubernetes to the new one - https://phabricator.wikimedia.org/T244293 [21:19:03] T244289: Improve limit range management in webservice for Kubernetes - https://phabricator.wikimedia.org/T244289 [21:19:04] T234617: Toolforge. introduce new domain toolforge.org - https://phabricator.wikimedia.org/T234617 [21:19:04] T156626: k8s webservice restart failure with `ValueError: get() more than one object; use filter` - https://phabricator.wikimedia.org/T156626 [21:25:10] !log tools upgraded toollabs-webservice package for tools to 0.62 T244293 T244289 T234617 T156626 [21:25:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [21:25:16] T244293: Add a function to webservice called "migrate" that will push a tool from the old cluster on Kubernetes to the new one - https://phabricator.wikimedia.org/T244293 [21:25:16] T244289: Improve limit range management in webservice for Kubernetes - https://phabricator.wikimedia.org/T244289 [21:25:17] T234617: Toolforge. introduce new domain toolforge.org - https://phabricator.wikimedia.org/T234617 [21:25:17] T156626: k8s webservice restart failure with `ValueError: get() more than one object; use filter` - https://phabricator.wikimedia.org/T156626 [21:47:13] !log tools Depooling tools-sgewebgrid-lighttpd-0928 (T244791) [21:47:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [21:47:17] T244791: Scale up 2020 Kubernetes cluster for final migration of legacy cluster workloads - https://phabricator.wikimedia.org/T244791 [22:01:16] !log tools Manually starting webservices for tools that were running on tools-sgewebgrid-lighttpd-0928 (T244791) [22:01:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [22:01:19] T244791: Scale up 2020 Kubernetes cluster for final migration of legacy cluster workloads - https://phabricator.wikimedia.org/T244791 [22:51:08] !log tools all docker images now use webservice 0.62 [22:51:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [23:39:53] !log tools updated tools-manifest to 0.21 on aptly for stretch [23:39:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [23:54:32] !log tools.coord Shutdown webservice and removed symling from $HOME/public_html to /home/myst/public_html; contained TOU violating adminer software (phpmyadmin clone) [23:54:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.coord/SAL