[00:00:06] https://www.mediawiki.org/w/index.php?title=Special:Code/MediaWiki&path=%2Ftrunk%2Ftools%2Fmwmultiversion%2Fmultiversion [00:00:11] not really, you just check in svn and have a look at what the folders head is [00:00:53] oh, huh [00:01:00] i assumed that was a private svn repo [00:01:35] mostly the only thing in the private repo is the config settings afaik [00:01:40] everything else is almost public [02:00:12] !log LocalisationUpdate failed (php-1.20wmf1) at Thu May 3 02:00:12 UTC 2012 [02:00:13] !log LocalisationUpdate failed (php-1.20wmf2) at Thu May 3 02:00:13 UTC 2012 [02:00:19] Logged the message, Master [02:00:21] Logged the message, Master [02:04:52] !log aaron synchronized multiversion/activeMWVersions 'deployed r115116' [02:04:57] Logged the message, Master [02:14:50] !log tstarling synchronizing Wikimedia installation... : [02:14:53] Logged the message, Master [02:21:48] !log tstarling synchronizing Wikimedia installation... : [02:21:51] Logged the message, Master [02:42:49] there's a problem with the wikimedia recent changes feed, which makes the CVNBots report edits on the beta wiki as if they are edits on normal wikis [02:42:57] could someone solve this? [02:43:48] not right now [02:43:50] file a bug [02:44:24] ok [02:46:24] sync done. [02:47:02] to rephrase, edits made on wmflabs.org are reporting through production-wiki feeds on irc.wikimedia.org [02:47:05] TimStarling: i'll look into it [02:47:24] !log tstarling synchronizing Wikimedia installation... : [02:47:27] Logged the message, Master [02:48:57] sync done. [03:06:21] Mathonius: ping [03:06:27] :) [03:06:50] i need a sample wiki to check [03:06:57] which project? [03:07:38] for example, http://simple.wikipedia.beta.wmflabs.org/?diff=3264060&oldid=3264059 was reported in #cvn-simplewikis [03:08:17] is that channel supposed to be every revision or is it filtered? [03:08:40] filtered [03:08:50] it was reported as: 13User14 [[simple:User:PiRSquared]] 13Large removal14 [[simple:Project:Requests for deletion]] 04(-13010)14 10Diff:12 http://simple.wikipedia.beta.wmflabs.org/?diff=3264060&oldid=326405914 "[[Help:Revert a page|Reverted]] edits by [[Special:Contributions/10.4.0.17|10.4.0.17]] ([[User talk:10.4.0.17|talk]]) to last version by Chenzw" [03:19:24] Hi there. Are any admins available right now? I need some wiki-help [03:25:46] SmartGuy: as in on wiki admins? [03:25:49] or what kind? [03:26:43] Not NI'm not sure, actually. My first time here. [03:27:15] I am logged into the English wikipedia on two computers currently but have lost my password [03:27:36] I don't have an email set up on my wikipedia account [03:27:57] and someone suggested I come here and ask if an admin can add my email to my Wikipedia account so that I can reset the password [03:28:18] you can create a new user account [03:28:48] Yes, I could create a new account, but I have a long editing history that I don't want to lose. [03:31:25] how did you manage to lose your password? anyway, doesn't really matter. I don't think it will get reset unless you're known (IRL) to the right people who can verify it's yours. if it's an established account with history instead of just 2 edits from today then all the more reason not to do so [03:32:26] Had it in a password database. File is corrupt and so is my backup copy of it. [03:33:09] ahhh. it's been a long time since you typed it i guess. you're screwed. no point in trying. start over [03:33:30] I used to do that sort of thing but the workload is far too high [03:33:30] maybe if SmartGuy was a head of state or something... [03:33:34] (alternatively, you could remember it!) [03:33:49] yeah I figured that might be the case [03:34:08] LOL no chance of me remember it - it is something like k9*(8kdjfw*((JJJR [03:34:21] ah well. Thanks anyway. [03:34:36] always good to have an email set [03:37:39] you can always have someone rename your old account [03:37:57] it can be "SmartGuy (old)" or something [03:38:23] then you can reregister SmartGuy and keep using that name [03:38:41] Actually if that is possible, that would probably be okay [03:38:55] I'll probably just do that, thanks [03:39:03] https://en.wikipedia.org/wiki/Wikipedia:Changing_username [03:39:47] TimStarling: i was thinking that but then i wasn't sure about the GFDL implications (some wikis don't allow changes with past edits) [03:40:17] but i guess if it's substantially the same name as the original that's maybe different? then why can't you usurp any arbitrary name? [03:40:27] we don't use the GFDL [03:40:39] errr... right that too [03:40:45] so i guess it's historical [03:41:20] SmartGuy: if you do rename you'll need to rename individually for each wiki on the cluster where you had an account already and wish to continue using the original username [03:41:39] AFAIK [03:41:39] I only have an account on the English wiki [03:41:50] you never unified? [03:42:06] Nope. Never needed to. [03:42:27] so cool, I will follow this suggestion to get the current account renamed, since it is apparently hard to add my email address. [03:42:34] thanks for the suggestions guys [03:43:08] no, it's simple. we're just refusing to do it [03:43:47] there would need to be some kind of verification of your identity which you're not able to provide [03:43:51] even so, no worries [03:43:57] yes i understand [03:46:01] I did set up User Committed Identity, if that makes any difference [03:46:08] if not then I' [03:46:20] then I'll just ask to have the accout name changed like Tim suggested [04:00:27] SmartGuy: you can file a shell bug [04:00:38] !log tstarling synchronizing Wikimedia installation... : [04:00:40] maybe someone else will feel more charitable [04:00:42] Logged the message, Master [04:01:46] will do tks [04:01:51] thanks all good night [04:01:52] SmartGuy: username? [04:02:27] my Wikipedia name? SmartGuy, same as I am using here [04:04:40] sync done. [04:07:15] SmartGuy: the fact that you only established the committed identity ~4 days ago doesn't instill a lot of confidence [04:07:26] amazing coincidence... [04:07:46] yes I understand - did it on the advice of someone at the help desk [04:08:06] in fact how could we know that you didn't just do it *since* losing teh password? [04:08:19] itiit's alright - I'll just get the name changed [04:08:29] you can't know [04:08:44] i lost the password about a week ago [04:08:55] you know I wrote about how stupid the committed identity thing is on the wikipedia talk page [04:08:55] in which case it's kinda pointless. it's as good as someone here giving you a secret token to put on wiki and then you bringing the diff link with that token in it [04:09:10] but my comments were immediately contradicted and then wiped off the page by an early archive [04:09:16] what do I know? [04:09:47] Yes, I didn't understand how it was supposed to help, but I did it anyway since someone at the help desk suggested it. [04:09:52] TimStarling: [[Template talk:User committed identity]]? [04:09:58] i don't see it [04:10:32] it's archived [04:10:51] oh [04:11:20] see, someone archived it right after I said how stupid it was [04:12:02] anyway it's better than nothing, probably good enough to allow adding the email [04:12:28] especially if there is some record of that email address being used by SmartGuy [04:12:38] TimStarling: even if it was done so recently? [04:12:56] I doubt there is any record of the email that I provided in the string, at least not on Wikipedia [04:13:20] your committed identity is a hash of your email address? [04:13:57] It is a sentence that states a number of things about me, including my email. [04:14:23] I don't think it's a problem that it was done recently, as long as the edit was by the right user [04:14:46] if you can edit as a user, you can change the user's password, so it's not an escalation to do a password reset [04:15:05] TimStarling: seen http://www.asheesh.org/note/debian/short-key-ids-are-bad-news.html ? [04:15:10] do you know asheesh? [04:15:40] no [04:15:42] TimStarling: don't you need the current password to do a reset? [04:16:12] no [04:16:27] you need the password to change the password, but that is pointless as a security measure [04:16:32] huh. well you do on the beta (labs) wiki i just changed mine on [04:16:41] since you don't need the password to change the email address or to issue a password reset to a new email address [04:17:02] resetting is different to changing [04:17:03] so... SmartGuy set yourself an email address? [04:17:22] in prefs. and then do a passwd reset [04:17:47] It won't let me set an email address without the password [04:18:09] yeah, actually it asked me too on second though [04:18:12] (again on beta) [04:18:25] funny [04:18:28] at the least I am learning a lot about password security! [04:18:34] maybe that was changed recently [04:18:44] i certainly remember it being discussed before [04:20:14] i wonder what SimpleAntiSpam is (udp logging) [04:21:57] SimpleAntiSpam is a text box hidden with CSS [04:22:12] the label says "do not write anything in here" [04:22:21] if you write something in there, the edit is rejected as spam [04:22:43] hah [04:24:04] TimStarling: Does it work, out of curiosity? [04:24:51] yes [04:25:01] Cool :) [04:26:11] guys thanks for the help/education - I think I'll just ask to have my username changed and then re-create the SmartGuy id, like Tim suggested [04:26:17] good night to all and good luck [04:26:21] nacht [04:26:28] SmartGuy: Good luck! [04:27:20] oh yes - getting all my passwords reset has been a pain. Oh well. that will teach me not to ruin my password database [04:27:28] goodnight thanks [04:27:58] or just keep backups. and maybe a decrypted printout in a safe [04:28:03] lots of strategies [04:28:47] * matthewrbowker likes MD5. Makes a database dump easy, no worrying about plaintext passwords XD [04:38:45] matthewrbowker: huh? [04:38:51] matthewrbowker: md5 is quite weak [04:39:10] you need a slow hash for passwords not just one way [04:40:21] jeremyb: True, but my application doesn't require secure passwords necessarily. I'm opting to just put MD5 in as a precaution, as I'm dumping it soon to move it over to a new host. [04:47:19] matthewrbowker: is it a public website, where anyone is able to create an account? [04:47:42] TimStarling: No, it's an administrative backend where only select users can have an account. [04:48:16] right, so password hashing is probably not your biggest security problem [04:48:24] do you have CSRF protection? [04:49:21] TimStarling: fixed the UDP logging issue from Mathonius. logged in labs, idk if it should be logged here too? [04:49:44] may as well [04:52:14] !log fixed complaints of beta simplewiki appearing in #cvn-simplewikis on freenode on the labs side. [https://labsconsole.wikimedia.org/w/index.php?title=Nova_Resource:Deployment-prep/SAL&diff=3390&oldid=3366 details] [04:52:16] Logged the message, Master [04:52:39] not the first time that's happened. maybe it should be filtered at the leslie level [04:52:42] TimStarling: I'm not sure.. I store user authentication with a user ID using PHP's $_SESSION[''] global. The script checks if that's set... I've opted not to use cookies or anything client-side as they can be a mess. [04:52:45] or in nova [04:53:15] matthewrbowker: most admin web interfaces do not protect against CSRF [04:54:01] I even found one that led to a remote root compromise against a standard ubuntu desktop [04:54:25] That's not good at all... [04:54:27] you should read up about it [04:54:37] I will, thanks for the tip. [04:54:44] checking the Referer header is a simple way to fix it [04:55:27] at the expense of blocking people who are behind certain kinds of anonymising proxies, but they don't matter do they? [04:57:46] No, not really. [04:58:53] Thanks for the tip. [05:00:15] or just use a framework that can do stuff like that for you [05:03:22] jeremyb: There's a thought... [08:02:15] Job queues are stuck again. [08:02:16] http://ganglia.wikimedia.org/latest/graph_all_periods.php?c=Miscellaneous%20pmtpa&h=spence.wikimedia.org&v=3335&m=enwiki%20JobQueue%20length&r=hour&z=small&jr=&js=&st=1335395131&z=large [08:02:27] Can someone please start the queue runners (or whatever needs doing) [08:03:10] mark: ^^ [08:15:30] this seems worth a log [08:16:08] !log siebrand: job queue stuck, on en.wiki jumped from o to 37k in the last ~36h [08:16:12] Logged the message, Master [08:19:07] so a first update on the jobrunners status: the shell script that loops through them all is running, and it's looping properly [08:19:19] I see that all jobs have started something new within the last minute [08:19:48] php: PHP Warning: include_once(/home/wikipedia/common/wmf-config/CommonSettings.php): failed to open stream: No such file or directory in /usr/local/apache/common-local/php-1.20wmf2/LocalSettings.php on line 4 [08:19:49] nice [08:20:22] I'll try running a loop by hand and see what it does [08:23:19] I don't see any odd output. it returns virtually immediately. [08:23:32] I'm going to look at the nextJobDB.php script now. [08:24:39] nothing useful there. Too bad, I was hoping I could tell it to be verbose and get an idea of what it thinks it is doing [08:29:55] it loks like nextjob is performing ok, after looking at an strace [08:30:26] hmm [08:30:30] maaaaybe not [08:31:18] I see [08:31:35] almost all of the scripts that actually process jobs are stuck on testwiki [08:37:35] I'm going to push out a new copy of LocalSettings.php which should address the warning message [08:42:33] !log ariel synchronized php-1.20wmf2/LocalSettings.php 'job runners don't have /home mounted' [08:42:36] Logged the message, Master [08:46:03] the job runner processes look a lot more healthy; I see them running on a variety of wikis now [08:59:08] it will take a while for the runners to catch up enough on the backlog across all wikis to start to make a dent, but checking en wiki we seem to not be piling on new jobs any more. looks like LocalSettings was indeed the problem [09:06:48] apergos: great stuff, thank you [09:06:55] you're welcome [09:18:04] apergos: why is there no alerts if jobqueue gets big? [09:18:40] ordinarily what would an ops person do? [09:19:51] there's generally not a good means of monitoring php errors afaik [09:20:06] roan had some sort of tail of the log going but beyond that there's not much [09:20:39] well even alert on growing # of jobs at some rate - like noticed on a graph [09:22:17] Nikerabbit: reedy does have a naigos alert for it [09:22:36] triggers at 33K iirc [09:23:04] or possibly "did have" [10:22:54] apergos: job queue is shrinking now. [10:23:25] it's been going up and down for the last [10:23:28] well since the change went out [14:55:22] I guess we need some help in fighting spam: they're triggering the abusefilter so fast to make the filter auto-disabled. the only way we have is to split one filter into two, prefix one of them with "(timestamp % 2 == 0) &" and another "(timestamp % 2 == 1) &" [14:56:08] of course I can use a bot or some user script to duplicate a filter into twenty to make it never overload in theory but it looks too hacky [15:00:14] liangent: ouch [15:08:29] what do you suggest or should I go on the hacky way? [15:10:13] file a bug to request a configuration change [15:12:47] Beau_: then how to determine which value should be used [15:15:00] liangent: maybe this can be turned off completely? was that mechanism useful? [15:15:28] if you have a botnet you can always make more requests to hit the filter... [15:50:35] Reedy: when you marked the wikimania2013 wiki bug as "blocked by" 16976 - that's just a way to keep it under tracking right? [15:51:14] no, the other way round [15:51:24] 16976 is blocked by 36477 [15:51:29] but yes, it's more for tracking purposes [15:57:21] what's the process for creating wikis out of interest Reedy? I imagine it's a tad more complicated with SUL... [15:57:32] Not really [15:57:44] Make sure the dns entries and apache stuff exist [15:57:46] run a script [15:57:47] sync to site [15:58:18] <^demon> Add to db.php if it's not defaulting to s3 :) [16:06:13] !log reedy synchronized wmf-config/CommonSettings.php 'Bug 31406 - Set $wgUseMathJax = true on Wikimedia wikis' [16:06:15] Logged the message, Master [16:08:17] !log reedy synchronized wmf-config/CommonSettings.php 'Bug 36460 - Enable chunked uploads as opt-in user preference' [16:08:20] Logged the message, Master [16:16:06] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36359 - Add namespace 102 to $wgContentNamespaces on ptwiki Bug 36360 - Add namespace 102 to $wgNamespacesToBeSearchedDefault on ptwiki' [16:16:08] Logged the message, Master [16:49:55] is scap still broken? [17:07:46] hi RoanKattouw [17:07:56] Nikerabbit: I think it might have been fixed [17:07:59] But not tried it [17:08:09] Hi Nike [17:11:33] does the RT ticket mean that the wiki's been created you're just waiting for the DNS entry to be put in by someone or is it usually done before making the wiki? [17:11:49] No [17:11:51] DNS first [17:11:59] otherwise it can't be accessed/tested at all [17:12:02] so little point creating it [17:12:18] yeah fair enough ;) [17:20:51] robla: we should move all wikis to the new backend config [17:26:06] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36320 - Set $wgShowUpdatedMarker back to true on ptwiki' [17:26:09] Logged the message, Master [17:26:10] robla: it's been on commons, mw.org for some time [17:26:37] AaronSchulz: this is the logging thing? [17:26:55] well, basically [18:15:57] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36480 - Create namespace Comments: in Greek Wikinews' [18:15:59] Logged the message, Master [18:23:05] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36386 - cswikiquote user group changes' [18:23:08] Logged the message, Master [18:24:18] Reedy, what about some more enotif tweaks :) [18:31:29] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36344 - Remove file upload facility on Gujarati wikipedia' [18:31:33] Logged the message, Master [18:33:36] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36344 - Remove file upload facility on Gujarati wikipedia' [18:33:38] Logged the message, Master [18:34:26] good news [18:34:48] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36344 - Remove file upload facility on Gujarati wikipedia' [18:34:50] Logged the message, Master [18:38:26] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36171 - Imports from Wikibooks' [18:38:28] Logged the message, Master [18:41:11] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36183 - Fix namespace alias on Hindi Wikipedia' [18:41:14] Logged the message, Master [18:46:27] !log reedy synchronized wmf-config/InitialiseSettings.php 'Bug 36092 - Activation of flood flag on vec.wikipedia.org' [18:46:30] Logged the message, Master [18:55:33] Nemo_bis: for shits and giggles? [18:55:36] What this time? :p [18:56:22] Enable some more of the smaller big wikis? [19:06:13] Reedy, change the default for new users [19:06:25] or enable the markers on all wikis [19:06:39] I don't think all yet [19:07:21] It's on on commons... [19:07:43] Leave en, de, fr, ja and ru wiki off for the moment? [19:07:53] That's then +5 [19:09:21] !log reedy synchronized wmf-config/InitialiseSettings.php 'Enable show update markers for some more of the larger wikis' [19:09:24] Logged the message, Master [19:12:14] Reedy, does it really matter? aren't the other wikis on different DBs? [19:12:22] * Nemo_bis didn't really check [19:12:30] it's an improvement anyway, thanks [19:12:34] yeah, that's 3 different clusters [19:13:08] We can do $wgDefaultUserOptions['enotifwatchlistpages'] = 1; also [19:15:17] !log reedy synchronized wmf-config/CommonSettings.php '$wgDefaultUserOptions[enotifwatchlistpages] = 1' [19:15:20] Logged the message, Master [19:19:09] uh, nice [19:19:37] Reedy, there's also https://bugzilla.wikimedia.org/show_bug.cgi?id=36316 if you feel like it :D [19:19:43] just to complete last change [19:20:40] !log reedy synchronized wmf-config/CommonSettings.php 'Bug 36316 - Set Add pages I edit to my watchlist to true by default for new users' [19:20:43] Logged the message, Master [19:20:52] great :) [19:26:04] AaronSchulz: ping [19:27:59] hm Reedy, option was set to true for me on it.wiki [19:28:12] I'm getting notifications for watchlisted pages [19:28:16] Indeed [19:28:20] I think I mentioned this before [19:28:30] if the user has the default, they don't have it saved in the database [19:28:46] so changing the default changes it for everyone who's not using not the default [19:29:04] yep but the docs suggested it was otherwise in this case [19:29:17] It's been this way since werdnas rewrite AFAIK [19:29:22] I just started getting spammed with a huge number of watchlist notifications, without turning it on. [19:29:22] hmm [19:29:25] I had to go turn it off. [19:29:46] it needs to be reverted then [19:29:49] yeah [19:30:14] binasher: pong [19:30:15] I have 31K pages on my watchlist. I never intended to get individual notifications when any of those pages were edited, which turns out to be more often than once a minute. [19:30:26] CydeWeys, it's being fixed [19:30:35] hmm 4 min ping latency :) [19:30:37] <^demon> E-mail notifs should be opt-in...always. [19:30:54] Nemo_bis: Cool. [19:31:00] ^demon, they've never been for talk [19:31:20] !log reedy synchronized wmf-config/CommonSettings.php 'Revert $wgDefaultUserOptions[enotifwatchlistpages] = 1' [19:31:23] Logged the message, Master [19:31:41] AaronSchulz: mediawiki udp profiling is broken because parser::bracesubstitution is sending a separate stat with the name of every title again ("1.20wmf1:-:Parser::braceSubstitution-title-Eranthis_hyemalis_aka_winter_aconite_2005_5th_april_in_tradgardsforeningen_gothenburg_sweden.jpgeS") [19:31:54] that was taken out of 1.19 [19:32:11] site hack? [19:32:14] we probably lost it somewhere [19:32:14] it was a good test for mchenry, let's see how the load was :) [19:32:17] i think domas did it [19:32:20] but not sure [19:32:35] i'm not even sure how to do a site hack post-git to do a temporary fix [19:32:51] same way essentially [19:33:01] but it would be nice if it made it into master and 1.20wmf2 as well as wm1 [19:33:11] commit to the branch, review, git pull on fenari, sync-file [19:33:31] Commit it to trunk for starters then! :) [19:34:26] https://www.mediawiki.org/wiki/Special:Code/MediaWiki/98341 ? [19:34:40] that i can do! i'm actually not sure how to merge from master to branches though [19:34:42] yeah, that's it [19:35:01] checkout branch, cherry pick, git review [19:35:29] binasher: it can be live hacked for now and I can make a patch in master [19:35:29] <^demon> See, this is why live hacks are evil. Evil evil evil. [19:35:42] hehe [19:35:43] it should only profile when using ProfilerSimpleText or something [19:35:46] looks all so easy [19:35:55] just five more steps [19:36:00] ;) [19:36:14] Nemo_bis: I don't think we have a way to do that.... [19:36:28] Reedy, to do what? [19:36:42] mess with preferences [19:36:57] AaronSchulz: conditional on profiler type sounds good, any chance you could make that change? :) [19:37:06] binasher: I'll hack it now and do the change next [19:37:07] Reedy, it's been done several times [19:37:21] it's not trivial though [19:37:23] binasher: are both versions sending it? [19:37:35] Indeed [19:37:38] For usability stuff [19:38:07] AaronSchulz: yeah, i see a few from wmf2 as well [19:40:21] Reedy, I'm updating the docs: in what version was the rewrite of preferences? [19:40:41] !log aaron synchronized php-1.20wmf1/includes/parser/Parser.php 'live-hack out template profiling...again.' [19:40:44] Logged the message, Master [19:42:09] !log aaron synchronized php-1.20wmf2/includes/parser/Parser.php 'live-hack out template profiling...again.' [19:42:11] Logged the message, Master [19:46:25] ^demon: the Profiler class hierarchy is pretty suck [19:46:50] probably b/c or something, meh [19:47:21] binasher: ahh, good old singleton inheritence [19:47:24] :p [19:47:40] <^demon> Don't blame me, I didn't write it. I just made the config suck marginally less :p [20:12:54] Reedy: I have more than 10k pages on my main watchlist [20:13:08] And? :p [20:13:10] I'm wondering how many spam will produce mediawiki [20:13:40] suppose it depends how often the pages get changed [20:15:01] [a lot] [20:15:14] you'll likely get a lot of emails then [20:15:16] if you have it enabled [20:15:33] afaik you enabled it by default >:-D [20:15:55] an user is reporting me she gets notifs even if she removed the checkbox [20:17:46] I then reverted it [20:18:23] good boy :p [20:30:57] Vito_away, she had checked and then unchecked the option? [20:37:00] !log reedy synchronized php-1.20wmf1/extensions/Collection/Collection.templates.php [20:37:03] Logged the message, Master [20:37:47] !log reedy synchronized php-1.20wmf2/extensions/Collection/Collection.templates.php [20:37:50] Logged the message, Master [20:39:52] Nemo_bis: it happened while Reedy hadn't yet removed the option [20:41:16] hi [20:41:27] I have a problem with email notifications [20:41:37] Vito_away, yes but as far as I understand if one had changed to non-default and then to default again then they shouldn't be affected [20:41:48] poux2, are you sure? did it happen after 19.31 UTC? [20:42:02] Nemo_bis: yes, it is happening right now [20:42:16] poux2, define "right now" [20:42:42] Nemo_bis: Von: MediaWiki Mail [20:42:43] An: Poupou l'quourouce [20:42:44] Gesendet: 21:18 Donnerstag, 3.Mai 2012 [20:42:46] Betreff: [Wikipedia] Die Seite „1359“ wurde von DidiWeidmann geändert [20:42:49] for example [20:42:57] that's before 19.31 *UTC* [20:43:00] it's been fixed [20:43:03] is it? [20:43:06] yes [20:43:09] ok [20:43:14] hope it is true... [20:43:18] thanks anyway!! [21:02:36] !log about to move ES writes to cluster23 [21:02:38] Logged the message, Master [21:05:39] !log asher synchronized wmf-config/db.php 'setting wgDefaultExternalStore to cluster23' [21:05:42] Logged the message, Master [21:07:46] LBFactory_Multi::newExternalLB: Unknown cluster "cluster23" [21:07:53] Ah, same [21:07:59] With a very neat backtrace [21:08:24]