[09:04:17] !log cloudinfra manually testing patch https://gerrit.wikimedia.org/r/c/operations/puppet/+/655019 to the puppetmaster to test (T271509) [09:04:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Cloudinfra/SAL [09:04:21] T271509: [wmcs][haproxy][puppet] Puppet failing on clouddb-wikireplicas-proxy - https://phabricator.wikimedia.org/T271509 [09:41:04] !log admin Taking osd.48 from eqiad ceph cluster out to do performance tests (T271417) [09:41:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:41:07] T271417: [ceph] test how disabling the disk cache affects the io throughtput - https://phabricator.wikimedia.org/T271417 [09:59:55] !log admin Started performance tests on sdc (od.48) for eqiad ceph cluster (T271417) [09:59:59] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:59:59] T271417: [ceph] test how disabling the disk cache affects the io throughtput - https://phabricator.wikimedia.org/T271417 [10:40:02] !log admin Finished tests, brining osd online (od.48) for eqiad ceph cluster (T271417) [10:40:05] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:40:05] T271417: [ceph] test how disabling the disk cache affects the io throughtput - https://phabricator.wikimedia.org/T271417 [10:53:26] !log admin root@cloudcontrol2001-dev:~# openstack subnet create --network wan-transport-codfw --gateway 208.80.153.185 --ip-version 4 --network wan-transport-codfw --no-dhcp --subnet-range 208.80.153.184/29 cloud-instances-transport1-b-codfw (T271517) [10:53:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:53:30] T271517: codfw1dev: drop cloudgw setup briefly for testing openstack stein - https://phabricator.wikimedia.org/T271517 [10:55:13] !log admin aborrero@labtestvirt2003:~ $ sudo ifdown eno2.2120 (T271517) [10:55:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:55:22] !log admin aborrero@labtestvirt2003:~ $ sudo ifdown eno2.2107 (T271517) [10:55:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:01:14] !log admin enabling neutron hacks in codfw1dev (cloudnet2002-dev, cloudnet2003-dev) (T271517) [11:01:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:01:22] T271517: codfw1dev: drop cloudgw setup briefly for testing openstack stein - https://phabricator.wikimedia.org/T271517 [11:02:47] !log admin root@cloudcontrol2001-dev:~# openstack router set --enable-snat cloudinstances2b-gw --external-gateway wan-transport-codfw (T271517) [11:02:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:06:10] !log admin root@cloudcontrol2001-dev:~# openstack router set --external-gateway wan-transport-codfw --fixed-ip subnet=cloud-instances-transport1-b-codfw,ip-address=208.80.153.190 cloudinstances2b-gw (T271517) [11:06:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:22:34] !log admin connecting cloudnet2002-dev cloudnet2003-dev back to vlan 2120 (T271517) [11:22:37] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:22:39] T271517: codfw1dev: drop cloudgw setup briefly for testing openstack stein - https://phabricator.wikimedia.org/T271517 [11:25:05] !log admin rebooting both cloudnet2002-dev/cloudnet2003-dev to make sure interfaces are set up correctl (T271517) [11:25:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:58:19] !log tools.pagepile-visual-filter deployed 4feee0dd9c (finish toolforge.org migration) [16:58:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.pagepile-visual-filter/SAL [23:35:56] !log wikistats deleting web proxy wikistats-old [23:35:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikistats/SAL [23:47:47] !log wikistats - shutting down and removing instance wikistats-dancing-goat - backup stored on -wild-tiger [23:47:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikistats/SAL [23:49:48] heh, when you click delete instance in Horizon and the next t creen shows no more instances for a moment and you think you deleted them all by accident instead of just one..wah. but reload fixes it :)