[00:48:11] New patchset: Stefan.petrea; "Adding new pageview reports mobile (in progress)" [analytics/wikistats] (master) - https://gerrit.wikimedia.org/r/41979 [08:48:53] New patchset: Hashar; "(bug 42628) lint whitespaces using git diff --check" [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/37803 [08:49:22] New review: Hashar; "PS7: made extensions to check whitespaces before PHP lint." [integration/jenkins-job-builder-config] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/37803 [09:06:56] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/39850 [09:09:48] New review: Hashar; "deployed on server" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/39850 [09:10:41] New review: Hashar; "Deployed live." [integration/jenkins-job-builder-config] (master); V: 2 C: 2; - https://gerrit.wikimedia.org/r/37803 [09:10:42] Change merged: Hashar; [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/37803 [09:24:43] New patchset: Hashar; "(bug 43579) EventLogging pep8 job" [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/42517 [09:26:42] hi hashar [09:27:29] New patchset: Hashar; "(bug 43579) run pep8 on EventLogging extension" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42518 [09:41:17] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42518 [09:43:34] Change merged: Hashar; [integration/jenkins-job-builder-config] (master) - https://gerrit.wikimedia.org/r/42517 [09:46:21] Nikerabbit: hello Niklas :-) [09:47:49] hashar: you're back refreshed? [09:47:59] New review: Hashar; "Hello Marius, could you seek out additional approvals from Wikimedia staffer? Will be happy to merge..." [integration/zuul-config] (master); V: 2 C: 1; - https://gerrit.wikimedia.org/r/39711 [09:48:47] Nikerabbit: kind of :-) The first week was all about being on the road, drinking and eating. Then I caught a flu or something and had to stay 2 days in bed :( [09:49:06] finally managed to rest this weekend. So I am almost refreshed this morning :) [09:50:36] hashar: aww too bad [09:51:21] hashar: just FYI I don't know if you are aware: TranslationNotifications and CLDR extension tests seem to have some problems and always fail [09:53:19] New review: Hashar; "Uploaded this version on commons https://commons.wikimedia.org/wiki/File:Wikimedia_CI_workflow.svg" [integration/doc] (master) - https://gerrit.wikimedia.org/r/39085 [09:54:02] Nikerabbit: have they ever been successful ? [09:54:29] hashar: dunno, maybe not [09:54:33] apparently not :( https://integration.mediawiki.org/ci/job/mwext-cldr-testextensions/ [09:56:14] for CLDR, the issue is that there are no PHPUnit tests to be found. So the testextension job is always failling [09:56:21] though it should not be voting :/ [09:56:48] Nikerabbit: do you have bugs for both issues? If not I will fill them [10:06:46] hashar: I don't remember seeing those in bugzilla [10:07:37] Nikerabbit: I found out the bug [10:07:37] New patchset: Hashar; "non voting extension tests were voting" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42522 [10:07:47] Nikerabbit: probably an issue with Zuul code base. The above change fix it [10:08:00] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42522 [10:11:20] New review: Thehelpfulone; "Whilst I'm not quite a WMF staffer, I'm highly trusted.. :-)" [integration/zuul-config] (master) C: 1; - https://gerrit.wikimedia.org/r/39711 [10:32:50] !g I1b00b00f5ddf34248f158303f176b3cc32bbcc3c [10:32:50] https://gerrit.wikimedia.org/r/#q,I1b00b00f5ddf34248f158303f176b3cc32bbcc3c,n,z [10:33:05] !g Id454bad62258c2f33c13f8fc43e45e69b0f5e8a7 [10:33:05] https://gerrit.wikimedia.org/r/#q,Id454bad62258c2f33c13f8fc43e45e69b0f5e8a7,n,z [10:44:18] Reedy: please stop killing the 'static-master' symbolic link for bits docroot -:) https://gerrit.wikimedia.org/r/42526 [10:44:35] Reedy: that breaks beta which use 'master' as a MediaWiki version [10:44:43] New review: Hoo man; "IMO it's a bit weird that I can merge code to run in production, manage central notices and edit eve..." [integration/zuul-config] (master) C: 0; - https://gerrit.wikimedia.org/r/39711 [10:53:40] hashar: PFFFT [10:53:45] ;) [10:53:47] Sorry [10:55:25] Reedy: tis ok :-] [10:56:13] hashar: besides, I need to go break beta with git deploy stuff [10:56:14] :D [10:56:48] Reedy: yeah I have been tasked the same thing apparently [10:57:07] There's a couple of conversations in the backscroll [10:57:18] Basically, we need to decide on a rough layout, then create lots of symlinks [10:57:21] Or osmething like that... [11:03:00] Reedy: at first: I have no idea what git deploy is :-] [11:03:08] so will have to find out a 30000 feet overview of it [11:03:16] hashar: deploying from git [11:03:18] would most probably read the source code which is on github somewhere [11:03:19] duh :D [11:03:25] and look at the recipes written by Roan for parsoid [11:03:42] oh [11:03:43] and salt [11:03:45] oh my god [11:05:02] Reedy: well I am going to get to lunch [11:05:09] will have a look at the list of stuff at https://bugzilla.wikimedia.org/showdependencytree.cgi?id=43338&hide_resolved=1 [11:05:15] heh [11:05:17] and find out how to deploy salt + git deploy from puppet [11:05:23] hopefully we have some puppet classes to do that [11:06:10] ah here is my bug https://bugzilla.wikimedia.org/show_bug.cgi?id=43339 [11:09:07] lunnnch time [12:24:27] back [12:42:13] hashar: I blocked your bug with my bug [12:42:32] Reedy: noticed that [12:42:39] heh [12:42:40] Reedy: thought I am not really sure what your bug is about :-] [12:42:51] still finding out how to get git-deploy from puppet [12:42:57] I can't find the perl script in our repo :( [12:43:04] Changing the layout from /home/wikipedia/common layout [12:43:32] moving mediawiki-config.git repo up to the same level as the code checkouts, not checking them out into it [12:43:40] ah [12:43:58] And then also tidying up /usr/local/apache and such [12:44:05] To make it all nice and consistent [12:44:20] do we have any design / architecture documentation about that ? [12:44:34] http://wikitech.wikimedia.org/view/Git-deploy is not really helpful :-D [12:44:59] Not really, as we've not designed it yet :D [12:45:05] ahh [12:45:25] something like /srv/deployment/mediawiki [12:45:26] probably [12:45:29] I forgot about the WMF project methodology : hack first, figure it out later :-D [12:45:40] Need to find the discussions from Tim in backscroll and put them on the bug [12:45:52] I don't have any backscroll myself :/ [12:46:12] Channel logged http://goo.gl/ckvIW [12:47:28] Damn it [12:48:16] Got the first convo, let me pull out hte bits not needed [12:49:00] I am updating the wikitech doc [12:49:59] hashar: sam@reedyboy.net [12:50:00] FAIL [12:50:08] hashar: https://bugzilla.wikimedia.org/show_bug.cgi?id=43340#c3 [12:50:34] wtf, some of them display in browser [12:50:35] some it downloads [12:51:26] Ok, so what channel was the other part of the discussion [12:52:46] that's probably the useful bit at least [12:56:55] Reedy: looking [12:57:56] "# Speculative fix: have the server copy the files using BitTorrent." [12:58:20] grr [12:58:26] Wasn't it just all git... Make them all pull, wait for that to work [12:58:38] I thought that the MediaWiki servers could all peer together using bittorent and happily update their copies [12:58:39] then do the checkout/whatever to get them all using the same code in a small amount of time [12:59:00] 700 servers all hitting the same 10GB link to download files is a bit dumb [12:59:08] ; WARNING : the issues below should be made bug reports in our Bugzilla}} [12:59:40] removing the curly braces [12:59:59] i believe there was a discussion about having a peer server in each rack or something [13:00:05] I'm slightly out of the loop [13:00:15] I don't buy the per rack based solution [13:00:28] there is no way we will ever manage to properly track which server is in which rack [13:00:41] but maybe i am underestimating ourselves :-] [13:00:42] lol [13:00:51] NFS for life. [13:00:56] I would go to have the 700 servers to grab the changes using BitTorrent [13:01:04] 1 copy of the data would be so much easier [13:01:14] then once all Bittorent clients have completed the tasks, send a salt run to actually switch to the new MW copies [13:01:34] New patchset: Siebrand; "Automatically merge UniversalLanguageSelector on +2/+2" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42540 [13:01:49] siebrand: :-] [13:02:04] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42540 [13:02:09] hashar: thanks. [13:02:26] hashar: Jenkins had a fit on https://gerrit.wikimedia.org/r/#/c/37395/ [13:02:53] New review: Hashar; "deployed :-)" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42540 [13:03:08] hashar: I'm sure they'd found a way to do it [13:03:14] You'd have to ask Ryan (tm) [13:03:35] Reedy: indeed [13:04:05] ahh if only we had done all of that directly on the 'beta' cluster [13:04:28] siebrand: I think I had zuul reloaded but I am not sure. It is probably still processing ongoing jobs. [13:05:02] I should try and start creating lots of symlinks today [13:05:19] siebrand: it was. UniversalLangueSelector should now have its change merged by Jenkins after CR+2 :-] Thanks for the patch! [13:05:44] Reedy: on my side I am figuring out the puppet stuff and will try to get classes for 'beta' [13:06:02] Reedy: why do we need symlinks farm anyway? [13:06:14] Reedy: can't we just update the apache conf to points to /srv/deployment/slot0 ? [13:07:15] I think it's keeping it incremental [13:07:25] rather than changing loads of things and then not knowing what actually broke [13:07:38] And also it might take a while to catch up and fix other scripts etc [13:08:39] sure :) [13:08:56] * hashar coffee + cigarette [13:09:01] and reading the puppet manifests [13:09:49] Also, it's not 700 [13:09:54] it's currently just under 200 ;) [13:10:21] Though, that's mostly just tampa? [13:11:12] 95 more in eqiad [13:12:19] hashar smoking is bad :P [13:12:37] * petan says [13:13:45] petan: yeah it is slowly killing me :/ [13:13:54] stop it? [13:14:32] http://en.wikipedia.org/wiki/Electronic_cigarette [13:14:44] It's usb powered! [13:14:50] :D [13:15:45] hashar you have mac air? [13:15:53] or just some... eh, mac? [13:16:02] I don't really know much about macs [13:16:30] I just know they are all white and girls like them [13:16:51] btw mac air has no usb for you :/ [13:16:54] the plastics ones are white [13:16:58] aha [13:17:04] there are some non-plastic? [13:17:07] they have to be heavy [13:17:07] nowadays most are in aluminum though and hence are gray [13:17:14] mm [13:18:01] I got the MacBookAir4,2 (13 inches screen from mid 2011) [13:18:12] waits 1.34kg [13:18:14] doesn't it prevent wi-fi from working properly? I mean aluminium blocks the signal far more than plastic case [13:18:17] which is light by my standard [13:18:36] na wifi works fine in all of my flat [13:18:45] well, at some point it's cute and it has terminal, so it's not so bad for me :D [13:18:48] better than my smartphone can handle [13:19:25] I like the microkernel as well :D [13:19:27] I highly recommend the 13 inches Air cause of the SSD and 1440 x 900 resolution. [13:19:39] wish there was a microkernel linux project [13:20:04] petan: such as GNU Hurd? http://en.wikipedia.org/wiki/GNU_Hurd [13:20:36] weeeee is it linux compatible? [13:21:25] (latest macbook pro 13inches from mid 2012 is 2.04kg (700g more), overpriced SSD and 1280x800 resolution :( [13:21:30] though it is probably cheaper [13:22:50] 13'' Air and basic 13''Pro are boths at 32 761 Kč [13:22:51] doh [13:23:31] 1300 euros doh [13:23:40] vs 1050€ on the french apple store [13:23:47] haha I was just looking at the price you found it for me :D [13:24:13] yes in CZ all stuff is far more expensive [13:24:14] err the Air 13'' is 1250€ [13:24:29] 1050€ is for the 11 inches air [13:24:30] it's cheaper to buy it from US e-shop :P [13:24:36] or in UK [13:24:45] unless the custom strike and make you pay the taxes :-D [13:25:14] government here sucks hard [13:26:10] It's useful having an academic proxy in the uk [13:26:20] get education pricing as you don't have to give proof of academic status [13:26:39] really? [13:26:41] Yup [13:26:50] Yay for having an internal linux cluster at my old uni ;D [13:26:58] haha [13:27:11] Also, buying it in the US can be cheaper still if you get it in states that there isn't sales tax... [13:29:45] you can still have your country customs to strike in and charge you import taxes + VAT :( [13:30:04] that's our case [13:32:06] hashar: send it to the office and pick it up from SF ;) [13:32:43] IIRC Krinkle bought one when he was in Portland for oscon [13:35:31] * Reedy symlinks hashar [13:36:23] yeah should do that next time I got to the states [13:36:36] I need a DSLR camera btw :-] [13:36:50] Amazon is quite good as they don't charge sales tax [13:37:16] hence me buying multiple SSDs, ram, cpu, graphics card... [13:38:35] Reedy: even in the US ? [13:38:44] I mean how can Amazon skip tax ? [13:38:47] Especially in the US [13:38:51] The same way they do everywhere else [13:39:08] Item(s) Subtotal: $1,115.96 [13:39:08] Shipping & Handling: $8.90 [13:39:08] ----- [13:39:08] Total before tax: $1,124.86 [13:39:08] Sales Tax: $0.00 [13:39:10] ----- [13:39:13] Total for This Shipment: $1,124.86 [13:39:14] ----- [13:39:19] ka-ching [13:41:12] :) [13:41:48] so bad gnu hurd doesn't support most of filesystems yet :/ [13:42:12] Reedy: that is crazy [13:42:18] The no tax? [13:42:24] Reedy: yup [13:42:25] Or my $1000+ Amazon US order? :P [13:42:27] btw hashar what FS is being used by apple [13:42:37] amazon charges sales tax only for states where it has offices [13:42:38] HFS? [13:42:45] Reedy: I know that US have different tax per states and hence most site show the prices without tax then apply the tax [13:42:57] can't charge residents of other states, because it's interstate taxes [13:43:01] hfs is kind of old, isn't it [13:43:06] Reedy: so when ordering for a delivery to the WMF office in SF, Amazon should apply the California tax I believe. [13:43:11] does it even support journaling [13:43:23] aude: so that is tax evasion ? :( [13:43:26] hashar: see what aude said [13:43:33] US are doomed huuh [13:44:16] It makes the savings quite a bit larger too [13:44:51] nah [13:56:29] New patchset: Siebrand; "Automatically merge translatewiki on +2/+2" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42546 [14:11:31] omg [14:11:33] lrwxrwxrwx 1 root root 30 May 18 2012 common -> /usr/local/apache/common-local [14:14:12] Reedy what's wrong? [14:14:17] it's a symlink... [14:14:44] target file permissions are used when you access it [14:15:34] ahaeharha git-deploy is a f*** package!! [14:15:46] I was looking for a perl script in our puppet repo :-] [14:16:11] Change merged: Hashar; [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42546 [14:16:49] New review: Hashar; "deployed." [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42546 [14:17:03] siebrand: I have deployed the auto submit for Translatewiki [14:17:19] hashar: cool. Thanks. No hurry there. [14:17:41] siebrand: well once the patch is written, it just take a couple minutes to deploy it. So I might as well do that sooner than later. [14:17:52] siebrand: one less trivial issue in my work backlog :-] [14:18:14] siebrand: + it get frustrating to not have such trivial changes merged on sight. [14:18:50] Reedy: any idea how salt work ? [14:19:00] Reedy: do we need a client on the app servers? [14:27:40] hashar: "A salt module lives on every salt minion and can be called from the salt master or from any peer which is allowed access." [14:27:45] So yes, the clients need *something* [14:28:00] petan: It was more that it wasn't actually in /h/w/c ;) [14:28:16] ah [14:28:30] Reedy: guess I will figure it out :-] [14:28:32] long live NFS [14:28:40] Reedy: I did write the puppet class for beta :D [14:28:52] hashar: Surely Ryan would've written puppet modules to do the client installs? [14:28:59] I hope so [14:29:14] damn you timeouts [14:29:52] oh [14:30:04] a salt client is named a 'minion' [14:31:19] yeah, looks to be... [14:32:16] fun [14:32:26] I like discovering new technology [14:32:39] tbh, if we were planning to move to this and ryan hadn't written the puppet configs, I'd be scared! [14:33:38] Reedy: so role::salt::minion is installed by default via the 'base' class [14:33:42] so we have it already :-] [14:33:43] \O/ [14:33:46] sounds sensible [14:33:47] wheee [14:35:27] err: /Stage[main]/Accounts::L10nupdate/Ssh_authorized_key[l10nupdate@fenari]: Could not evaluate: can't find user for 996 [14:35:30] grmlblblb [14:42:56] If I start moving files on bastion I'm going to break shit, aren't I? [14:44:02] Reedy: on the beta bastion ? [14:44:03] yup [14:44:08] the apaches configurations are not in puppet [14:44:17] and have their document root pointing somewhere [14:44:23] lol [14:44:33] I wonder how much point there is actually doing it here first... [14:44:52] maybe /usr/local/apache/common which symlink to /usr/local/apache/common-local which might smylink to /home/wikipedia/common which in turn link to /data/project/apache/conf or something [14:44:57] well that is certainly a HUGE mess [14:44:57] lol [14:45:10] I think we should get git deploy / salt deployed on the deployment-bastion host [14:45:29] make sure the salt minion/clients properly fetch the mediawiki content under /srv/deployment/something [14:45:34] then once THAT works [14:45:47] But they need somewhere to pull it from... [14:45:47] alter the beta Apaches configuration to point to the new dir. [14:46:13] apparently the minion pull using http on the deployment-bastion host [14:46:13] It might make more sense just doing it on fenari/whatever that host in eqiad is [14:46:26] and the role class install the files on that machine under /srv/deployment [14:46:36] oh I am talking about beta ) [14:46:37] not fenari [14:46:59] I think Tim was suggesting we create this base /srv [14:47:01] get the files in it [14:47:22] remove the old /usr/local/apache, and at the same time, symlink those back to /srv [14:47:46] yup [14:47:47] ` [14:47:54] sounds like an almost good idea :-] [14:48:02] But the question is whether we should do it on labs first.. [14:48:19] why not ? [14:48:30] if we screw something on labs, that is safer for production i guess [14:48:41] then it does not closely match production [14:48:44] lol [14:49:03] and we still have shell scripts on fenari which are not in puppet / wikimediamaintenance nor the deb package that hold scripts [14:49:09] lols [14:49:16] yeah that is a mess [14:49:48] anyway on beta I would like git-deploy to work / have the file pushed on apaches [14:49:52] but not serving content yet [14:50:05] once we are happy with git deploy we can switch the apaches [14:50:15] * Reedy starts copying files around [14:50:26] ie common-local to /srv/old [14:50:38] Reedy: what for ? ;)D [14:50:52] so they're in the new location [14:51:12] then /usr/local/apache/common-local can symlink back to it [14:51:35] This starts getting confusing if I overthink it [14:51:50] Reedy: not that / partition on beta apaches are very small [14:52:04] so /srv should symlink to /data/project/apaches_srv or something [14:52:13] lol [14:52:17] (and that should be in Puppet) ;-] [14:52:24] hmm [14:52:27] as well as moving files around, we need to start making the new structure [14:52:31] or we could use the /dev/vdb maybe [14:52:40] let me check [14:52:44] lol [14:53:01] each instances as a /dev/vda1 mounted on / [14:53:15] which is small (10G) [14:53:35] and a /dev/vdb1 (300G on our apaches) which is mounted on /mnt [14:54:38] so hmm [14:55:13] lol [14:55:34] I'm sure this just keeps getting more annoying [14:55:52] I would make /srv/ a symbolic link to /mnt/srv which is a local disk to the instance [14:56:40] /dev/vdb 43G 185M 40G 1% /mnt [14:57:41] lrwxrwxrwx 1 root root 8 Jan 7 14:57 srv -> /mnt/srv [14:58:00] except you need to have the symlink to be handled by puppet :-] [14:58:28] not sure in which class that should belong too though [14:58:55] heh [14:59:16] But making puppet do everything while we're still organising seems daft [15:07:16] hashar: so do the beta apaches serve directly from a remote FS mounted locally? [15:08:18] ie gluster i guess [15:08:37] 30349 root 20 0 413m 204m 2760 S 49 5.2 1381:48 glusterfs [15:09:08] so apaches have /usr/local/apache symlink to /data/project/apache [15:09:18] Reedy: btw don't trust df , sizes are all wrong [15:09:35] I wasn't using it for sizes [15:09:40] then /data/project/apache/common symlink to /data/project/apache/common-local [15:09:54] better safe than sorry [15:10:23] on the deployment-bastion host, /home/wikipedia/common is also a symbolic link to /data/project/apache/common-local [15:10:27] hmm [15:10:48] lol [15:11:09] I think we should get rid of /data/project entirely (since it is a shared filesystem) [15:11:16] and use /srv/ pointing to /mnt/srv [15:11:16] Next you're going to tell me that they're actually served from fenaris /home via sftp [15:11:22] figuring out which class could host that [15:11:29] Reedy: lol [15:15:28] ah role::applicationserver something [15:16:29] I'm sure I'm over thinking the whole thing [15:18:14] bah [15:18:21] the role::applicationserver class no more exist [15:18:25] refactored [15:18:43] role::applicationserver::appserver [15:20:03] fun fact [15:20:05] I should've started this copy in a screen session [15:20:11] I can't add a new puppet class under labsconsole :/ [15:20:27] Reedy: I don't think you need to copy anything [15:20:41] Reedy: as I understand it git deploy will handle it for us [15:21:42] [00:07:37] maybe we should try to avoid updating these 700 /usr/local/apache references more than once [15:21:42] [00:08:04] we could change scap to push out to a new location, say /srv/old [15:21:42] [00:08:21] then replace the old /usr/local/apache with a symlink farm [15:22:17] I'm confused. [15:23:34] Very confused. [15:26:06] I'm giving up till later on [15:31:39] I give up till later too [15:31:46] can't add the required class :/ [15:31:51] :( [15:32:09] https://labsconsole.wikimedia.org/wiki/Special:NovaPuppetGroup for deployment-prep does not let me add any class :D [15:32:48] Can you not get to https://labsconsole.wikimedia.org/w/index.php?title=Special:NovaPuppetGroup&action=create&project=deployment-prep ? [15:34:55] yeah that one create a puppet group [15:35:00] which group classes [15:35:07] I did create a new group named 'group' [15:35:14] does not let me edit the classes anyway [15:35:18] Ryan would know :) [15:59:53] !g Icf3a73387bc0a855ebec3805747685b1e8cc71e0 [15:59:53] https://gerrit.wikimedia.org/r/#q,Icf3a73387bc0a855ebec3805747685b1e8cc71e0,n,z [16:05:18] I am out [16:05:36] Reedy: will most probably be back later this evening to sync with Ryan [16:05:39] :) [16:06:07] * hashar waves [16:18:49] Nikerabbit, <-- why underscores? [16:24:16] MaxSem: that's what solr 4 excepts with the example configuration [16:25:00] still, with these underscores it looked like some magic var [16:25:14] maybe, content_version or something? [16:25:37] anyway, you or siebrand need to answer my email about all this stuff [16:26:27] MaxSem: It's on my list, but I may take about as much time as in the first roundtrip. [16:26:44] lol [16:37:52] New patchset: Yurik; "whitelist Yuri Astrakhan (yurik)" [integration/zuul-config] (master) - https://gerrit.wikimedia.org/r/42570 [16:51:40] hello [16:52:10] is there something or somebody i should poke about fixed bugs that really need a point-release? [16:52:34] right now i'm thinking about things describedin second paragraph of https://bugzilla.wikimedia.org/show_bug.cgi?id=42452#c41 [17:43:39] Reedy: if you're around, any idea what's happening with the new version of this file? try clicking the new file itself and the new thumb, get the old file and old thumb http://test2.wikipedia.org/wiki/File:0.8622626746865945.png [18:22:53] ori-l: hi :) so I think you get pep8 on the EventLogging extension :-) Not fully tested though [18:29:28]