[00:06:21] I am off, but will read the log later if there any comments to my problem. [00:27:36] uwe, looks to me like you tried uploading a file where the last char of the filename was a " ? [03:12:23] https://meta.wikimedia.org/wiki/Special:RecentChanges [03:12:31] I get "502 Bad Gateway" error [03:14:25] Is anyone else getting 502 errors or slow loading times? [03:14:35] i am [03:14:42] on en.wikinews [03:14:43] dammit [03:14:46] I got it on meta [03:14:49] nothing loading or taking too damn long [03:15:16] looks like bits is having issues [03:15:24] Hazard-SJ: can you confirm it too? [03:15:26] TimStarling: You around by any chance? [03:15:33] Seems to be loading nowYes [03:15:41] *yes [03:15:49] Only the title seems to be loading [03:16:14] Sometimes it loads, sometimes it's slow, sometimes it's a "502" error:/ [03:16:55] :/ [03:16:57] OK, came up now :/ [03:17:35] DragonFire1024: works now? [03:17:58] mhm...seems to be good now [03:18:11] same here [03:18:22] * p858snake|l wonders if by off chance if RobH is around [03:18:32] The problem was possibly also experienced on Wikidata [03:18:33] wtf, watchmouse didn't even notice? [03:18:44] * ori-l notices that he can hear p858snake|l think [03:18:52] jeremyb: watchmouse = ? [03:18:59] usor: status.wikimedia.org [03:19:10] ori-l: i'm just a figment of your imagination [03:20:12] and we are down again [03:20:15] "502 Bad Gateway" [03:20:17] :( [03:20:48] p858snake|l: :P [03:21:08] jeremyb: It's showing disruptions now [03:21:15] good [03:21:50] It's good that there are disruptions? :? [03:22:12] well that is a debatable view [03:22:36] "The connection was reset" now [03:23:26] hi Soapy [03:23:29] hi [03:23:34] some of us are having problems getting to WP [03:23:35] jeremyb: The times displayed are a few mins behind [03:23:39] Soapy: same here [03:23:49] apparently it isn't down for everyone, since there's still notices in the bot channels [03:23:50] Hazard-SJ: sometimes it responds very fast... [03:23:56] 502 errors and connections reset and slow pages :S [03:23:57] Maybe someone would like to update the topic? [03:24:03] seems like it's only eqiad per nagios [03:24:06] but not certain [03:24:16] jeremyb: Quite strange [03:24:34] Soapy: bits has two zones/clusters [03:24:50] europe and everywhere else in the galaxy [03:24:56] ok [03:25:39] meganhernandez: hello... are you experiencing "technical difficulties"? [03:25:50] * Hazard-SJ didn't kntice the topic was freely editable [03:25:50] I was able to get one page to load with no CSS [03:26:18] is there a problem with the site right now? [03:26:21] yes [03:26:24] !!! [03:26:24] Stop using so many exclamation marks ! [03:26:30] meganhernandez: yes [03:26:31] shut up wm-bot [03:26:48] do you know how long it's been going on? [03:26:55] meganhernandez: 13-14 mins [03:27:07] meganhernandez: at least 17 mins for me [03:27:18] wm-bot do you like question marks too ???? [03:27:28] meganhernandez: i mailed to summon some people to fix it ~5 mins ago but you're welcome to sound an alarm too if you like [03:27:33] so ?? is ok but !! isn't :S [03:27:38] meganhernandez: no one's online yet (to fix it) [03:29:14] ??? [03:29:17] :) [03:29:20] robla? Ryan_Lane around? [03:29:27] meganhernandez: already tried both of htem [03:29:28] them* [03:29:30] a multi million dollar non profit website [03:29:36] and no one around to fix it? [03:29:42] jeremyb: Do we know what exactly needs fixing? [03:29:46] Hazard-SJ: no [03:30:02] DragonFire1024: Care to fix it for us then? ; [03:30:07] * );) [03:30:10] * ;) [03:30:21] sure Hazard-SJ where's the hammer? [03:30:24] :P [03:30:38] Let me try to load Commons :P [03:30:39] Still no loading, can't we get someone to fix this? :/ [03:30:53] usor: hold your horses. we're well aware that it's broken [03:31:11] a few pages are loading [03:31:24] mutante: TimStarling: ping? [03:31:25] Soapy: But very slowly, right? [03:31:43] it even seems to be getting better slightly [03:31:45] yeah, slowly [03:31:50] Soapy: https://en.wikipedia.org/wiki/m:Main_Page <- nothing on meta is loading for me, but enwiki works [03:31:57] even the Javaxcript clock loaded this time [03:32:06] which is usually the last thing to appear on a page for me [03:32:09] usor: I got a timeout from enwiki [03:32:42] And commons, so no hammer for DragonFire1024 [03:32:55] Hammer time? [03:32:58] :( [03:33:28] (not sure if right channel) does anyone else have serious super lag right now? everyewhere on en.wiki? [03:33:31] this all happened right when i started to edit...note: I have had a very bad day and nothing has gone right...blame me [03:33:35] gwickwire: yes [03:33:37] gwickwire: /topic [03:33:42] yes gwickwire [03:33:55] Okay :) Just making sure it ain't my old crap computer. [03:34:05] nope [03:34:06] ;) [03:34:13] * ori-l is making phonecalls [03:34:54] ori-l: Call the people with access [03:35:00] Hazard-SJ: orly?? [03:35:39] Hazard-SJ: I thought ori-l would be calling the janitor tbh [03:36:20] p858snake|l: No, the hardware for a hammer for DragonFire1024 to use :P [03:36:43] :D [03:37:12] yeah its down again [03:38:31] A DDOS? [03:38:32] idk how much this helps, but I get text just not any graphics, etc. [03:38:35] meganhernandez: obviously don't turn on any extra regions/campaigns/etc. for now [03:39:17] Jasper_Deng: what? [03:39:26] what's up? [03:39:29] is this possibly a Ddos attack? [03:39:38] Jasper_Deng: we don't discuss DDOS [03:40:06] robla: LVS flapping for lots of services. people have complained extra about bits but i think everything's been effected [03:40:25] seems like it's coming back [03:40:40] working fine if slowly [03:40:44] meganhernandez: we said that 20 mins ago and look where we are now [03:41:26] yeah, it *seems* to be working [03:41:47] cool [03:41:58] it's probably going to take someone in ops to fix it, but I can poke around a bit [03:42:28] (maybe find something more specific to look at...that's about as helpful as I'm going to be) [03:42:47] pfft who needs css and js anyway :p [03:42:57] plain text is where its at [03:47:24] when was the last time ya'll had a problem? [03:48:19] about 5 minutes ago [03:48:23] working now, thanks [03:48:36] well main page is working [03:49:04] uncached pages appear to be loading slightly slower than usual, but that might be power of suggestion [03:49:11] right [03:51:41] Could be just me, but I'm sensing a slight delay in the page load time... [03:52:44] jeremyb: something's still wrong [03:52:51] http://en.wikipedia.org/w/index.php?title=Wikipedia:Administrators%27_noticeboard/Incidents&curid=5137507&diff=525963729&oldid=525962530 seems to have no CSS [03:53:05] nvmd [03:53:05] worksforme [03:57:44] meganhernandez: in case you weren't paying attention we're mostly back to normal. but that's just empirical observations and non-scientific surveys; nothing official [03:58:09] yeah seems to be working to e [03:58:10] me [03:58:16] good [03:59:38] jeremyb: The status page still shows some problems [03:59:59] least nothing's *down* [04:00:01] Hazard-SJ: at least they're all yellow (or green) and not red [04:00:17] idk so much about how that page works anyway [04:00:34] must be a rolling average or max/min or something. but of what time window? [04:00:38] 1 min? 10 mins? [04:00:59] looks like a proprietary technology [04:01:01] i'm slightly more interested in what you're all experiencing personally [04:01:03] it is [04:01:10] it's a CNAME [04:01:19] The time at the top (Dec 2, 2012 03:59 UTC) is the last update of the status.wikimedia page [04:01:21] to a hosted service. SaaS [04:01:43] gwickwire: still have the same questions [04:02:07] Haha okay. Just a question, does Helpmebot run on toolserver? [04:02:50] hey [04:02:54] iz back [04:03:10] If so, toolserver may have taken a hit, as it just quit wiki-en-help and wiki-en-helpers and came back a few seconds (10-15) later. [04:04:13] well it is the same network... [04:04:34] although eqiad was hit worse than esams in ganglia/nagios [04:04:36] who knows [04:07:23] But HMB went like 2 mins after the issues with WP were supposedly resolved. Rolling issue maybe? [04:08:36] on an unrelated note, how is the deployment of FlaggedRev on enwiki going? [04:09:36] Blech. Like 2 pages have PC1 and the rest are tests. No pending edits made today at all (as far as i saw from sitting here F5ing the Special:PendingChanges page) [04:09:58] No way to tell if its working or not because no edits need reviewing yet. [04:13:40] think it's *working* [04:14:13] i think the people to talk about that feature are either not online now or came on only for the outage and left already [04:15:48] Mobile's not having problems for me anymore. [04:16:18] someone else said mobile was working for them too. i neglected to check myself [04:16:42] Oh. Really? It was completely down for like a few mins for me. [04:16:55] Just now when I tried is it working, albeit a bit slowly. (yes im on wifi) [04:17:10] 02 03:50:07 < ^demon> Mobile seems to be fine too. [04:17:16] 27 mins ago [04:17:30] Oh. :P Welp. IDK. Could be me. [04:17:37] so about 10 mins after we had mostly recovered (at least per nagios) [04:17:46] Hey, status.wikimedia is showing almost all green now! That's good, right? [04:18:09] or it means that the status check is broken too [04:18:09] ;) [04:18:25] yeah, we don't care so much about brewster [04:18:33] (ubuntu mirror lives there i think) [04:18:38] Haha. If it's broken, then thats really bad. [04:18:50] Status is always up... As far as ive seen. [04:19:08] well it's a CNAME as i said [04:20:18] im not techy. [04:20:20] idk what that is. [04:20:21] :( [04:29:36] gwickwire: i know of this on line encylopedia :P [04:29:46] I'm not sure I follow. [04:30:46] Well you're in the tech channel for a online encyclopaedia and you say you don't know what something is, it was a joke [04:31:29] Oh lol. [04:31:31] Yeah. Haha. [04:31:38] I'd look it up. But I'm lazy [11:42:51] I try to upload a jpg file with pywikipediabot to commons and the api and get this error: {u'servedby': u'srv296', u'error': {u'info': u'This file did not pass file verification', u'code': u'verification-error', u'details': [u'filetype-mime-mismatch', u'jpg"', u'image/jpeg']}} Looks for me like the api has a bug, because the mimetype for jpg is image/jpeg. If I upload by hand, it works fine. I asked in #pywikipediabot, but for me looks [11:42:51] like a bug in the upload-api. [12:53:23] Is there a way to print debug output on a Jenkins-induced parser test run? I know it suppresses most output from the tests [13:22:28] https://en.wikipedia.org/w/index.php?title=Special:Search&go=Go&search=Media:Example.ogg [13:22:53] it's only me getting Internal error? [13:36:49] wizardist: that is server side :-D [13:37:08] hashar: ? [13:37:08] wizardist: you can open a bug report referencing the message [13:37:30] wizardist: the hexadecimal between [] will let us find the full stacktrace [13:37:53] I'll leave it just here: 66bf8c48 [13:38:28] wizardist: thenit will be ignored :-] [13:39:11] not my fault, I'm busy with something more interesting :( [13:40:15] wizardist: I looked at it :-] [13:40:25] wizardist: you can't search for Media:Foobar.jpg :-] [13:40:31] use File:Foobar.jpg instead [13:40:52] I know, but anyway it should be catched and redirected to File: NS or whatever [13:40:59] showing an exception sucks [13:41:05] :) [14:04:03] hashar: Would you be a good person to ask about the Jenkins->parsertests setup? [14:04:22] Jarry1250: yup I am the one in charge of Jenkins :-) [14:06:03] hashar: Okay, so I'm trying to debug a parser tests fail that can't be reproduced locally, but I'm struggling because ech-style output naturally gets suppressed. Is there a mechanism for output? A debug method that doesn't get suppressed? [14:06:09] *echo-style [14:07:37] so it fails on Jenkins but not on your comp ? [14:07:48] most of the time that is related to SQLite (which is used on Jenkins) [14:07:56] what is the change number Jarry1250 ? [14:08:49] hashar: https://gerrit.wikimedia.org/r/#/c/25838/, I think you've seen it before, but note that in the final patch set, what I've done is comment out the file_put_contents function -- and the error is the same. [14:09:09] Therefore I *think* file_put_contents is failing, or it simply never gets called [14:10:52] ah that test, so yeah it was failing for me [14:11:02] let me try it again [14:11:46] hashar: Ah, thanks, I was trying to sort it out myself like a good dev but as you can see that attempt failed :) You'll want to grab one of the older changesets [14:12:08] s/changesets/patchsets/ [14:12:53] (The same one as you commented on before, perhaps) [14:13:17] Jarry1250: that was patchset 8 which does fail on my comp [14:16:24] Jarry1250: maybe it is missing the Foobar.svg file ? :-) [14:17:31] hashar: Foobar.svg is dynamically written using file_put_contents [14:17:57] hashar: The other test files are already present but copied using copy() [14:18:01] (they work) [14:18:06] (this one doesn't) [14:35:23] hshar: Any thoughts? Does it create Foobar.svg on your install correctly? [14:37:01] no idea :( [14:37:11] seems latest patchset uncomment it [14:37:25] you might have to register the file in the database too [14:37:43] I have no idea how those tests work to be honest [14:40:28] hashar: Well, to explain, they copy() (the two existing tests) or file_put_contents() (the new test) material into a test images folder in one method, then register it in another -- except you can't register it unless it exists in the upload directory. [14:40:53] The two old images are registered successfully -- but the new image isn't. [14:41:19] Which makes me suspect that it is never created. But I don't know why copy() would work when file_put_contents() doesn't -- either you have write perms or you don't [14:42:07] So I wanted to test the return value of file_put_contents() directly, but I'm not sure how to output that. [14:44:37] Jarry1250: sorry i am out [14:44:41] daughter woke up [14:44:52] feel free to post on wikitech-l to get some assistance :/ [14:44:53] Hello. Is somebody with access to db1017's mysql here? If so: May he/she check if the masterlog-file db1017-bin.000906 is still there? [14:44:55] can't really help today [14:58:22] haashar: No problem :) [16:03:59] since people were all asleep yesterday... "how is the deployment of FlaggedRev on enwiki going?" [18:31:12] quick question: how quickly are configuration change requests usually handled? (yes, i know it's weekend, and that the answer is "not right now") [18:32:09] MatmaRex: depends on the issue/severity and need for review [18:32:56] trivial config change. https://bugzilla.wikimedia.org/show_bug.cgi?id=42579 [18:35:16] MatmaRex: you need to find someone to +1 it and then push it live [18:35:53] why do *I* need to *find* someone? :/ [18:36:02] isn't it somebody's job to handle these? [18:36:12] given that it is a weekend, you should be able to poke the right people tomorrow [18:36:28] MatmaRex: there is someone who does it [18:36:36] i understand it's weekend, just asking in general [18:37:10] but if you want it expedited you need to poke the right people otherwise it can take a week+ [18:37:21] heh [18:37:30] alright, thanks [22:41:29] Just in case anyone cares, I've been having some slight issues with Vector on Commons, don't know why or what it means. [22:44:56] can you describe them? [22:46:56] I think its the objects (maybe)? [22:47:04] They just aren't showing up first load. [22:47:14] After like two reloads they'll show up, most of them at least. [22:47:22] Like the prettyness of the page. [23:04:23] zzz =_= [23:26:00] gn8 folks [23:32:35] andre__: s/mood/moot/ ;) [23:33:28] * andre__ needs context [23:33:33] ah! [23:33:36] make it moot. [23:33:38] true. [23:35:03] hah [23:35:26] andre__: fosdem? [23:35:42] kinda sorta your neighborhood [23:37:47] definitely! [23:37:56] best beers EVAH [23:38:04] flights are booked [23:42:08] jeremyb: also thinking of going? [23:42:58] andre__: maybe some future time. i've been thinking it's one i want to do within the next 5 years. not happening this time [23:43:30] andre__: although i did have a couple days in .be [23:44:14] I think I haven't missed one since the first time I went there. Just too much fun and so many things to learn by going to talks of other projects [23:47:31] andre__: 150ish USD round trip for you? vs. 800ish for me [23:47:38] ;-( [23:47:44] meh. yeah :(