[04:36:05] 10serviceops, 10Operations, 10Core Platform Team (Security, stability, performance and scalability (TEC1)), 10Core Platform Team Kanban (Doing), and 3 others: Enabling api-request eventgate to group1 caused minor service disruptions - https://phabricator.wikimedia.org/T218255 (10Mh-3110) a:03Ottomata [04:38:41] 10serviceops, 10Operations, 10Core Platform Team (Security, stability, performance and scalability (TEC1)), 10Core Platform Team Kanban (Doing), and 3 others: Enabling api-request eventgate to group1 caused minor service disruptions - https://phabricator.wikimedia.org/T218255 (10Mh-3110) Hi @Ottomata, assi... [07:16:58] hey DST-suffering people of which I am one, are we going to move the meeting today or have it in the evening? [07:17:42] joe, mark, akosiaris, mutante, jijiki [07:19:22] fine by me [07:19:53] q: would you like chocolate or strawberry? a: fine by me [07:19:59] which is fine? :-P [07:20:37] moving it ofc. I thought the default is to have it in the evening, so the question was more like "do we move it?" to me [07:21:41] move it [07:26:09] <_joe_> I prefer to keep it in the afternoon if possible [07:26:23] <_joe_> so I have 2 hours of meetings in a row [07:26:28] <_joe_> well, 3 [07:26:35] <_joe_> and I don't disrupt the rest of my day [07:26:41] <_joe_> anyways, bbiab [07:27:54] I am alright with afternoon as well [07:43:39] Moved it [08:05:42] not here right now but saw it and will be there in an hour [08:05:56] acked invite, bbiab [08:10:50] so we are doing 12? [08:11:08] I am confused:/ [08:22:43] I just gave up and looked at the clendar [08:22:53] it seems to say 9 am utc [08:31:02] duh, yes sorry, 12 local time [08:46:09] yes [09:00:13] <_joe_> I'll be a minute late, my chrome logged me out and I have to reopen keepassx [10:46:25] 10serviceops, 10Operations, 10docker-pkg: Getting registry metadata from a public client fails on our registry - https://phabricator.wikimedia.org/T220085 (10Joe) [10:46:33] 10serviceops, 10Operations, 10docker-pkg: Getting registry metadata from a public client fails on our registry - https://phabricator.wikimedia.org/T220085 (10Joe) p:05Triage→03Normal [10:47:17] 10serviceops, 10Operations, 10docker-pkg: Getting registry metadata from a public client fails on our registry - https://phabricator.wikimedia.org/T220085 (10Joe) [10:52:15] <_joe_> this is such a fun bug... [11:06:12] <_joe_> oh ok got it [11:17:19] 10serviceops, 10Operations, 10docker-pkg: Getting registry metadata from a public client fails on our registry - https://phabricator.wikimedia.org/T220085 (10Joe) I think I found the culprit: when requesting directly to the registry, I do see the correct manifest is returned *and* a content-length header is... [11:17:25] 10serviceops, 10Operations, 10Traffic, 10docker-pkg: Getting registry metadata from a public client fails on our registry - https://phabricator.wikimedia.org/T220085 (10Joe) [11:22:05] 10serviceops, 10Operations, 10Traffic, 10docker-pkg: Getting registry metadata from a public client fails on our registry - https://phabricator.wikimedia.org/T220085 (10Joe) p:05Normal→03Low To be clear, here "docker client" means "the docker daemon running on some computer", as any docker client libra... [11:26:17] 10serviceops, 10Domains, 10Operations, 10Traffic, 10Patch-For-Review: contact Wikivoyage e. V. and figure out status of wikivoyage-old.org / fix or park broken domain - https://phabricator.wikimedia.org/T219867 (10Dzahn) contacted Fussi at https://de.wikivoyage.org/wiki/Benutzer_Diskussion:DerFussi#Statu... [12:45:13] 10serviceops, 10Operations, 10Thumbor, 10ops-eqiad, 10User-jijiki: thumbor1004 memory errors - https://phabricator.wikimedia.org/T215411 (10Dzahn) fresh Icinga alert for this since about 23 hours [12:49:02] 10serviceops, 10Operations, 10Thumbor, 10ops-eqiad, 10User-jijiki: thumbor1004 memory errors - https://phabricator.wikimedia.org/T215411 (10jijiki) @robH do we have an update? [15:15:15] 10serviceops, 10Domains, 10Operations, 10Traffic, 10Patch-For-Review: contact Wikivoyage e. V. and figure out status of wikivoyage-old.org / fix or park broken domain - https://phabricator.wikimedia.org/T219867 (10RolandUnger) wikivoyage-old.org was used only at the transfer process of the Wikivoyage dom... [15:29:04] 10serviceops, 10Domains, 10Operations, 10Traffic, 10Patch-For-Review: contact Wikivoyage e. V. and figure out status of wikivoyage-old.org / fix or park broken domain - https://phabricator.wikimedia.org/T219867 (10Dzahn) Thanks for the confirmation @RolandUnger . Appreciate it and will remove it. [15:29:24] 10serviceops, 10Domains, 10Operations, 10Traffic, 10Patch-For-Review: contact Wikivoyage e. V. and figure out status of wikivoyage-old.org / fix or park broken domain - https://phabricator.wikimedia.org/T219867 (10Dzahn) [15:32:23] 10serviceops, 10Domains, 10Operations, 10Traffic, 10Patch-For-Review: contact Wikivoyage e. V. and figure out status of wikivoyage-old.org / fix or park broken domain - https://phabricator.wikimedia.org/T219867 (10Dzahn) @RolandUnger @DerFussi Just to make sure, this also includes the email aliases we on... [15:43:03] 10serviceops, 10Domains, 10Operations, 10Traffic, 10Patch-For-Review: contact Wikivoyage e. V. and figure out status of wikivoyage-old.org / fix or park broken domain - https://phabricator.wikimedia.org/T219867 (10RolandUnger) Yes, this should also include email aliases. The wikivoyage-ev.org addresses a... [15:45:50] 10serviceops, 10Domains, 10Operations, 10Traffic, 10Patch-For-Review: contact Wikivoyage e. V. and figure out status of wikivoyage-old.org / fix or park broken domain - https://phabricator.wikimedia.org/T219867 (10Dzahn) Ack, removing wikivoyage-old.org for email and web, not touching wikivoyage.org in a... [16:18:50] 10serviceops, 10Operations, 10Core Platform Team (Session Management Service (CDP2)), 10Core Platform Team Kanban (Doing), and 4 others: Session storage Cassandra cluster configuration - https://phabricator.wikimedia.org/T215883 (10fgiunchedi) [20:42:35] 10serviceops, 10Citoid, 10Release Pipeline, 10Patch-For-Review, and 2 others: Migrate citoid to kubernetes - https://phabricator.wikimedia.org/T213194 (10Jdforrester-WMF) Is this now completed? [20:43:22] 10serviceops, 10CX-cxserver, 10Release Pipeline, 10Patch-For-Review, and 3 others: Migrate cxserver to kubernetes - https://phabricator.wikimedia.org/T213195 (10Jdforrester-WMF) Is this now completed? [20:47:46] 10serviceops, 10CX-cxserver, 10Release Pipeline, 10Patch-For-Review, and 3 others: Migrate cxserver to kubernetes - https://phabricator.wikimedia.org/T213195 (10jijiki) Traffic is being served from k8s, we will mark this as resolved after we merge a couple of cleanup patches (like the last ones in T213194)... [20:49:31] 10serviceops, 10Release Pipeline, 10Release-Engineering-Team, 10Core Platform Team Backlog (Watching / External), 10Services (watching): TEC3:O3:O3.1:Q3 Goal - Move cxserver, citoid, changeprop, eventgate (new service) and ORES (partially) through the production ... - https://phabricator.wikimedia.org/T212801 [20:49:35] 10serviceops, 10Citoid, 10Release Pipeline, 10Patch-For-Review, and 2 others: Migrate citoid to kubernetes - https://phabricator.wikimedia.org/T213194 (10jijiki) 05Open→03Resolved a:03jijiki @Jdforrester-WMF @akosiaris I am marking this as resolved, please reopen if you feel we are missing something.