[06:16:29] <wikibugs>	 10serviceops, 10Operations, 10Patch-For-Review, 10User-Elukey: Reimage one memcached shard to Buster - https://phabricator.wikimedia.org/T252391 (10elukey) Note after checking slab distribution on the gutter pool. The last slab sizes seem to not follow the prediction made by the script:  ` STAT 49:chunk_si...
[09:27:55] <wikibugs>	 10serviceops, 10Operations, 10Kubernetes, 10Patch-For-Review: Migrate to helm v3 - https://phabricator.wikimedia.org/T251305 (10MoritzMuehlenhoff) Since we don't use helm3 in production yet, just adding it to the task. There's a new vulnerability specific to 3.x, so let's upgrade to 3.2.4 before we roll th...
[12:45:51] <wikibugs>	 10serviceops, 10Operations, 10ops-codfw, 10Patch-For-Review: (Need by: TBD) rack/setup/install kubernetes20[07-14].codfw.wmnet and kubestage200[1-2].codfw.wmnet. - https://phabricator.wikimedia.org/T252185 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by akosiaris on cumin2001.codfw.wmnet for...
[13:08:26] <wikibugs>	 10serviceops, 10Operations, 10ops-codfw, 10Patch-For-Review: (Need by: TBD) rack/setup/install kubernetes20[07-14].codfw.wmnet and kubestage200[1-2].codfw.wmnet. - https://phabricator.wikimedia.org/T252185 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['kubernetes2013.codfw.wmnet', 'kubernetes2...
[13:18:33] <wikibugs>	 10serviceops, 10CX-cxserver, 10Release-Engineering-Team (Pipeline): Migrate apertium to the deployment pipeline - https://phabricator.wikimedia.org/T255672 (10akosiaris)
[13:18:43] <wikibugs>	 10serviceops, 10CX-cxserver, 10Release-Engineering-Team (Pipeline): Migrate apertium to the deployment pipeline - https://phabricator.wikimedia.org/T255672 (10akosiaris) p:05Triage→03Medium
[13:19:24] <wikibugs>	 10serviceops, 10CX-cxserver, 10Release-Engineering-Team (Pipeline): Migrate apertium to the deployment pipeline - https://phabricator.wikimedia.org/T255672 (10akosiaris)
[13:39:53] <wikibugs>	 10serviceops: Replace rdb200[34] with rdb200[78] - https://phabricator.wikimedia.org/T255250 (10akosiaris)
[14:27:52] <wikibugs>	 10serviceops, 10Operations, 10Patch-For-Review: move all 86 new codfw appservers into production (mw2[291-2377].codfw.wmnet) - https://phabricator.wikimedia.org/T247021 (10ops-monitoring-bot) Icinga downtime for 1:00:00 set by dzahn@cumin1001 on 5 host(s) and their services with reason: new_install ` mw[2335...
[14:29:26] <wikibugs>	 10serviceops: Replace rdb200[34] with rdb200[78] - https://phabricator.wikimedia.org/T255250 (10akosiaris)
[14:52:50] <wikibugs>	 10serviceops, 10Operations, 10Patch-For-Review: move all 86 new codfw appservers into production (mw2[291-2377].codfw.wmnet) - https://phabricator.wikimedia.org/T247021 (10ops-monitoring-bot) Icinga downtime for 1:00:00 set by dzahn@cumin1001 on 5 host(s) and their services with reason: new_install ` mw[2335...
[15:24:25] <wikibugs>	 10serviceops, 10Operations, 10ops-codfw: (Need by: TBD) rack/setup/install 86 new codfw mw systems - https://phabricator.wikimedia.org/T241852 (10Dzahn) >>! In T241852#6211870, @Papaul wrote: > @Dzahn the 5 servers in C3 are ready for services   They are now in production. (details in T247021)
[15:25:02] <wikibugs>	 10serviceops, 10Operations: move all 86 new codfw appservers into production (mw2[291-2377].codfw.wmnet) - https://phabricator.wikimedia.org/T247021 (10Dzahn) mw2335 through mw2339 in rack C3 have also been taken into production now.  This should complete the ticket. All hosts from mw2291 through mw2376 are po...
[15:30:09] <wikibugs>	 10serviceops, 10Operations, 10ops-codfw: (Need by: TBD) rack/setup/install 86 new codfw mw systems - https://phabricator.wikimedia.org/T241852 (10Dzahn) @Papaul This ticket talks about mw2377 but it seems to me we never had a host mw2377 (only up to mw2376). Can you confirm that?
[15:32:51] <wikibugs>	 10serviceops, 10Operations, 10ops-codfw: (Need by: TBD) rack/setup/install 86 new codfw mw systems - https://phabricator.wikimedia.org/T241852 (10Dzahn) 76 servers are pooled as appservers. 10 have been used for kubernetes. Adds up to 86.
[15:33:12] <wikibugs>	 10serviceops, 10Operations: move all 86 new codfw appservers into production (mw2[291-2377].codfw.wmnet) - https://phabricator.wikimedia.org/T247021 (10Dzahn) 05Open→03Resolved 76 servers are pooled as appservers. 10 have been used for kubernetes. Adds up to 86.
[15:33:15] <wikibugs>	 10serviceops, 10Operations, 10ops-codfw: (Need by: TBD) rack/setup/install 86 new codfw mw systems - https://phabricator.wikimedia.org/T241852 (10Dzahn)
[17:36:29] <wikibugs>	 10serviceops, 10Operations, 10ops-codfw: (Need by: TBD) rack/setup/install 86 new codfw mw systems - https://phabricator.wikimedia.org/T241852 (10Papaul) @Dzahn confirm we have up to mw2376
[18:57:54] <wikibugs>	 10serviceops, 10Operations, 10Release-Engineering-Team, 10Developer Productivity, and 2 others: All debug hosts give (likely spurious) message: PHP Fatal error:  The UdpSocket to 127.0.0.1:10514 has been closed (from Monolog/SyslogUdp) - https://phabricator.wikimedia.org/T214734 (10Krinkle) Debug-only, not...