[07:28:38] hi! I am wondering if there is a public log of deployments of WMF sites? [07:29:51] for instance, to find out when a particular commit was pushed to production [07:31:44] I am trying to understand which commit introduced mixed case in HTTP responses ("Set-Cookie" and "set-cookie"), which breaks login in Wikidata-Toolkit (T249680) [07:31:45] T249680: Wikidata password login via MediaWiki API fails - https://phabricator.wikimedia.org/T249680 [07:51:25] pintoch: easiest way afaik is to search the Gerrit patch title on mediawiki.org's internal search which mediawiki version (1.XX-wmf.XX) [07:51:45] the wmf version's page tells you a tuesday of the week when it was deployed [07:53:24] then you can take a look at https://wikitech.wikimedia.org/wiki/Deployments/Train; the version is deployed to "group 0" wikis on that week's tuesday, to "group 1" on wednesday and to "group 2" on thursday [07:58:20] tassu: thank you very much, that is really helpful [07:59:37] so the change was indeed deployed this morning, now I just need to inspect the commits between this deployment and the previous one [08:00:13] pintoch: mw.org pages have a list of changes in that train if that's helpful [08:00:28] (or perhaps git bisect could do) [08:02:45] yes it's very convenient, thanks :) [08:10:16] pintoch: https://tools.wmflabs.org/versions/ for which versions are deployed where. https://wikitech.wikimedia.org/wiki/Deployments for additional SWAT deployments (see table). git logs for anything in code repositories. [08:10:34] ...or for MW Core the mw.org pages, indeed [08:12:51] yes, thanks, now I am just faced with my total cluelessness with MediaWiki's internals, so I might give git bisect a try, it could be fun [08:20:01] pintoch: https://phabricator.wikimedia.org/T249526 sounds related to me, see very last comment [08:21:46] andre__: very relevant indeed, thank you! [08:21:51] yw [13:40:29] pintoch: I don't think you'll find a specific MW commit as the cause, as anomie notes. in WMF production there's a few different kinds of proxies in the middle of the user request flow. we recently switched from using nginx as the TLS terminator for our API servers to using Envoy for that purpose, on April 6th https://tools.wmflabs.org/sal/log/AXFONKX90fjmsHBaqKCr [17:08:43] pintoch: I saw several SREs talking in another channel about the [sS]et-[cC]ookie bug. Ultimately this is a client bug, but hopefully something that we can find a bandaid for. It is a client bug because RFC 2116 (HTTP/1.1) says "Field names are case-insensitive." for headers -- https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 [17:10:21] The RFC for http/2 actually makes lowercase canonical which folks seem to think is the trigger here with the introduction of the envoy proxies in our internal network. [17:11:39] "However, header field names MUST be converted to lowercase prior to their encoding in HTTP/2." -- https://tools.ietf.org/html/rfc7540#section-8.1.2 [17:31:58] bd808: yeah, see also https://phabricator.wikimedia.org/T249680#6039918 :) [17:33:38] Someday I will actually make the "MediaWiki is edge cases all the way down" shirt/sticker :) [20:48:52] bd808: if you make that t-shirt I will get one [20:50:38] apergos: deal. :) Logo might be something like a jenga tower made of razor blades [20:50:56] {{worksforme}} [20:52:06] or a stack of semi-transparent turtles? Not sure what kind of iconography works best for this kind of "edge" [20:52:38] maybe a GCI task if I don't move on it sooner :) [20:57:31] I do like the turtles but I don't know how they can be turned into edges unles [20:57:31] s [20:57:44] tey are somehow stacked on edge :-D [20:59:32] maybe a house of cards made of razor blades? [21:00:01] I don't know how you make a jenga tower of razor blades, they are thin, that is the point [21:11:06] TimStarling: stop putting your reality in my fantasy! :p [21:11:30] but a house of cards + razor blades is a nice visual too [22:12:01] Hm, if everything is edges then there is no inside? So we're possibly talking of something that is not compact or connected. [22:12:10] Maybe a https://en.wikipedia.org/wiki/Totally_disconnected_space or https://en.wikipedia.org/wiki/Extremally_disconnected_space [22:12:52] I guess this might be printed on a t-shirt https://en.wikipedia.org/wiki/File:Cantors_cube.jpg [22:57:43] Nemo_bis: you tricked me into reading maths things. Now my head hurts because I'm trying to remember bits from my undergrad work so many, many years ago.