[02:51:29] !admin [02:52:54] wikidatauser: need some help? [02:53:35] https://www.wikidata.org/w/index.php?title=Q3163141&action=history vandalism Protected plis [02:54:27] proteger* [02:55:00] proteger por favor [02:56:22] hay admin aqui? [02:56:51] hay TAB como en wiki-en o wiki-es? [02:57:13] I'll post it on the admin noticeboard for you [02:57:29] thanks =) [02:58:18] https://www.wikidata.org/wiki/Wikidata:Administrators%27_noticeboard#Vandalism_on_.7B.7BQ.7CQ3163141.7D.7D [02:59:06] ok [02:59:09] =) [02:59:24] glad I could be of some help [03:00:40] Bye [09:58:24] Lydia_WMDE: aude: Did you manage to resolve the cookie issue stuff? [10:00:04] hoo: not solved but lego said he'd look into it [10:00:19] Ok [10:00:28] Is he on now? [10:00:38] The solution is to unset the old cookies via varnish, we did that before [10:00:48] ok [10:00:50] let me get the ticket [10:00:52] one sec [10:00:54] then you can comment [10:01:04] Nice :) [10:01:06] hoo: https://phabricator.wikimedia.org/T109038 [10:02:43] Lydia_WMDE: Commented [10:02:49] thanks! [10:05:36] hoo: thanks [10:05:42] how's camp? [10:06:18] Awesome! :) [10:06:22] You should have joined [10:06:23] :D [10:06:29] * aude is moving this weekend [10:06:41] out of the hostel :) [10:06:49] hehe, nice :) [10:38:04] aude: no idea why this one is failing.... https://gerrit.wikimedia.org/r/#/c/231509/ [10:38:12] but I guess thats okay as we dont use it yet.... [10:54:32] addshore: hmm... [10:54:38] indeed [11:31:30] aude: seen comment in https://phabricator.wikimedia.org/T107319 ? [11:46:09] Lydia_WMDE: out of curiosity: Was there actually communication between https://phabricator.wikimedia.org/T100722#1322620 and closing in https://phabricator.wikimedia.org/T100722#1538951 ? [11:54:48] Anyone know the offending cookie that needs killing? ;) [11:58:17] hoo: ^ [11:58:50] Not offhand, no [11:59:01] It's just the current cookies with a different path [11:59:23] but in the end, I don't know [11:59:37] I think this notebook is newer than that, so doesn't have the old cookies [12:35:54] Reedy: I have the old cookies and can reproduce. I think any cookie for the domain wikidata.org instead of something more specific like www.wikidata.org [12:36:58] oh *sigh* we totall forgot m.wikidata.org [12:37:36] andre__: not really. the feature was killed though now for all i can tell [12:41:37] sigh [12:42:13] Lydia_WMDE: if you expected some communication (on what exactly?) I'm happy to be the bad guy pointing that out. If wanted. [12:42:29] andre__: aprechiated ;-) [12:43:09] hey [12:43:27] https://phabricator.wikimedia.org/T109038#1539407 [12:43:51] ^ I think the problem is the centralauth_Token cookie getting set for "wikidata.org", or still existing there from the past, anyways [12:44:14] after deleting it manually, I haven't managed to trigger it to re-appear by any combination of login/logout on wikidata and other wikis [12:44:47] (a cookie for wikidata.org will get sent to *.wikidata.org too) [12:45:32] but since https://wikidata.org is just a redirect and probably no normal activity other than old links/bookmarks hit it, there's no simple universal fix at the varnish level. [12:45:51] if we delete centralauth_Token without stricitly filtering on the Host header, we'll repeatedly kill legit logins [12:46:12] if we filter on Host: wikidata.org for the cookie-kill it would DTRT, but then not all visitors will ever make a request to https://wikidata.org/ [12:46:46] (I think....) [12:47:08] it's early in the morning to be sure of anything about how browsers match up an update/delete of a cookie based on the source domain and the domain specified :/ [12:56:28] it's possible that, from a request to "www.wikidata.org", we can send a cookie delete for "wikidata.org", and it will delete only the wikidata.org version of it and not the www version of it, maybe [12:56:43] they overlap for that request in some way, I imagine it's subject to all kinds of browser differences in this case :/ [13:01:50] bblack: thanks for looking into it! from our side hoo and jzerebecki can probably say most about it [13:02:09] they're both at CCCamp though and i don't know how much/when they're around [13:02:11] bblack: what is the life time for the central auth token? [13:02:20] ah! :D [13:02:23] :) [13:02:54] good question [13:03:29] I think 30d, but there was recent unrelated chatter about raising it to a year [13:03:33] I don't think that was merged, yet [13:04:18] bblack: can we fix central auth to try both central auth token cookie values instead of failing? [13:04:46] I don't see any good solution that is able to delete the cookie [13:04:59] assuming getting 2 values is why it is failing [13:05:18] I'm not even sure it's getting two values. it may be that the browser picks one of the two to send [13:05:36] I guess we can test that, though [13:05:44] https://gerrit.wikimedia.org/r/#/c/230954/ is the 1y expiry thing, not merged yet [13:10:19] testing that now... [13:10:28] oh you're right I think [13:10:33] https://github.com/jshttp/cookie/issues/18 mentions it [13:10:46] you supposedly get two values in the cookie string with the same key [13:11:03] so yes, perhaps a CentralAuth patch to try both if two values exist and the first fails [13:11:25] yea my browser sends two values [13:11:53] also had the same problem years ago with another php application [13:12:11] although I'm imagining CA uses some standard php or mw library code to get cookie values. it may not be trivial to get raw access to the string? [13:12:51] though we fixed it by adding a prefix to all cookie names [13:14:31] I never looked at mediawiki session handling... [13:15:13] ah but! the two-values thing should be abnormal and only exist for this broken case [13:15:31] so perhaps I could just hack up something in varnish that deletes the CA cookie only if it sees a double-value [13:15:40] oh yes. [13:15:47] whether that deletes only the wikidata.org variant or deletes both, either way worst case they have to log back in once [13:16:26] bblack: it will only work if it deletes the wikidata.org one. otherwise the next try we again get both. [13:17:02] I'm assuming if I delete the value with domain=wikidata.org, it will either delete just that one, or kill both [13:17:13] so we could do one redirect back to wikidata.org [13:17:24] only if there are two values [13:17:25] but yes, I guess depending on horrible browser behaviors, it could just delete www.wikidata.org one, but I wouldn't think so [13:17:38] hmmm [13:18:02] yeah that would be more certain to correct it [13:18:05] complex, though [13:18:19] let me see how bad it looks if I do the redirect->delete thing [13:18:36] (we can send the delete on the response that MW's sending to redirect them back to www.wikidata.org too) [13:19:02] yes [13:19:28] we can test if the delete without redirect works, to make sure it is not needless complexity. [13:23:49] hi, I need someone with admin/editinteface rights in enwiki [13:35:45] jzerebecki: https://gerrit.wikimedia.org/r/#/c/231556/ ? [13:36:37] oh I have VCL syntax errors there, fixing [13:37:49] PS2 looks better [13:39:09] I think in the best case this fixes it (with the user maybe having to reload/relogin after they hit the fix), worst case it deletes the one for www.wikidata.org but not for wikidata.org, in which case the broken users just stay broken, but doesn't affect unbroken users [13:58:18] I'm salting the fix attempt around the cluster now, will take a few mins to get it deployed [13:59:35] jzerebecki: re: redirect, anything like that starts getting much uglier in terms of varnish complexity [14:00:14] bblack: yea I also forgot that this is only the centralauth token, not the session cookie [14:00:21] ah right [14:00:48] once the salt finishes, since you still have the double-cookie, we can have you hit a page and see what happens to your cookie store [14:00:54] (at least for your browser variant heh) [14:01:45] ok it's live everywhere now [14:03:32] bblack: I think it deleted the www.wikidata.org cookie and then automatically logged me in [14:03:42] did it delete both of them? [14:03:49] when I vistied https://www.wikidata.org [14:03:58] no it didn't, only one [14:04:14] heh, so it's the worst possible behavior? it deleted www.wikidata.org but not wikidata.org? [14:04:36] ok let me look at fixing this up to use a one-shot redirect [14:05:44] bblack: no it actually deleted .wikidata.org so it is exactly fine [14:06:02] that was me being confused [14:06:10] oh nice [14:06:20] so session stayed up, and logout->login worked after? [14:08:01] bblack: before your fix: I manually logged out, tried to login, didn't work. after your fix I accessed https://www.wikidata.org it got the cookie delete and was a redirect to the main page. the main page got delivered to me as logged in. [14:08:26] retrying [14:09:10] login&logout still works as expected. [14:09:22] so, fix is probably good enough? [14:09:44] bblack: yes we can take it out after 31 days. [14:09:58] right, I'll add the commentary on that I should've put in the first patch, + a bug ref [15:42:21] JeroenDeDauw: https://github.com/wmde/WikibaseDataModelServices/pull/42 [16:37:53] Hmm, anybody knows who in WMDE is playing with phab09.wmflabs.org by creating lots of test tickets by (ab)using my user account over there? [16:38:05] (I know this isn't Wikidata stuff but if there's a better WMDE channel just tell me) [16:42:47] andre__: addshore got a lot of mails from the test instance [16:42:55] he wasn't playing wiht it though and was wondering also [16:43:27] yeh, I got somewhere between 300 and 400 emails today :/ [16:43:33] that went straight to my inbox :/ [17:17:40] addshore: somehow you are popular then ;) [17:28:44] benestar: Didn't you saw my pings? :( [17:29:07] sjoerddebruin: nope, sry [17:29:24] Your references script is not working on certain items. [17:44:47] Oi, algum brasileiro aí? [21:11:33] PROBLEM - wikidata.org dispatch lag is higher than 300s on wikidata is CRITICAL: HTTP CRITICAL: HTTP/1.1 200 OK - pattern not found - 1431 bytes in 0.151 second response time [21:12:25] Hm [21:13:53] on it [21:29:21] RECOVERY - wikidata.org dispatch lag is higher than 300s on wikidata is OK: HTTP OK: HTTP/1.1 200 OK - 1411 bytes in 0.113 second response time [23:56:15] PROBLEM - wikidata.org dispatch lag is higher than 300s on wikidata is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Service Unavailable - pattern not found - 50727 bytes in 0.082 second response time