[11:08:05] Is anyone else seeing an "Unhandled Exception ("AphrontMalformedRequestException")" error when trying to log in to Phabricator? [11:08:59] I actually could not edit an existing task I created. So I decided to log out. Then it turns out that I can not login back! [11:12:47] Nevermind. My adblocker was deleting all tokens from my browser. [11:13:00] I disabled it and it seems to be working fine [15:16:24] Wikipedia got successive slower, and now nowiki has stopped responding [15:16:36] yup nothing wrks [15:16:52] SRE knows and is working on it, stay tuned [15:16:57] Completely dead… [15:17:30] Loaded [15:18:23] … a single page… [15:18:33] it's still able to be read (and edited) if you're logged out [15:18:35] ironically [15:21:56] it shoul recover in a few minutes, fix being pushed around. [15:26:01] Any idea what's going on with Wikimedia sites? Why are they down? [15:26:11] I can't use Commons. [15:26:55] known issue, being worked on [15:27:00] should hopefully be back soon [15:30:26] fix being sent around now, we should see recovery shortly [15:30:30] if not already tbh [15:31:02] soke too soon, still some esams issues [15:31:07] being looked at [15:31:19] Is T242226 a side effect of this? [15:31:20] T242226: VE over en.wiki gives a 502 error - https://phabricator.wikimedia.org/T242226 [15:31:58] we'll know in a few minutes [15:32:57] users in EMEA are likely particularly affected because of knock-on issues in the esams caching cluster [15:34:13] EMEA = Europe, Middle-East, Asia? [15:34:20] (was going to look it up on Wikipedia, but, well ^^) [15:35:54] Lucas_WMDE: s/asia/africa/ [15:36:24] ok thanks [15:37:41] I guess Asia is closer to eqsin [15:38:24] asia is split, but yeah most of it goes to eqsin [15:42:42] Anyone an idea what is happening? German Wikipedia was almost unusable in the last hour. [15:42:46] https://grafana.wikimedia.org/d/000000479/frontend-traffic?orgId=1&refresh=1m [15:42:55] CountCount: ops folks are working on it [15:43:51] @tassu Thanks [16:00:54] did i make it? [16:01:39] you're on irc, if that's what you mean [16:04:34] KF2019-LF: Hi. Make what? [16:04:46] meeting [16:04:53] There is no meeting. AFAIK. [16:05:11] You may want to provide more context though. :) [16:05:16] k [16:20:59] I think he might be taking about the Wikimedia Tech Advice meetings which have been cancelled [16:22:41] oh yeah, that would have been the same time [16:25:07] ah yeah sadly it no longer exists [16:25:13] I guess some other group could step up to run it [16:25:29] but it would have to be planned and announced