[00:00:36] hmm, well basically i'm trying to figure out whats changing the /srv/mediawiki permissions on me. puppet seems like the most likely candidate, but its turning into a real pain to debug :P [00:07:15] ebernhardson: If it is Puppet, it should log whenever it changes anything, either in /var/log/puppet* or interactively with "sudo puppetd -tv". So I would change the permissions of /srv/mediawiki, then run "sudo puppetd -tv" and see if there's a line about that. [00:07:35] ebernhardson: I thought role::mediawiki-install::labs left /srv/mediawiki alone. New files get created with the user's umask which defaults to 022 ; I don't know if it was different on pmtpa instances [00:23:07] spagewmf: There's no reason it should be (different), it's the same version of puppet running from the same manifest. [00:31:50] Coren: thanks. We're having more trouble updating /srv/mediawiki on our new eqiad labs instance and aren't sure why. I just did g+s on all subdirectories, I think it helped. [00:32:34] Well, g+s will just propagate the group down (which may be what you want); but if you have something resetting permission that won't help in the long run. [00:32:39] !log deployment-prep Converted deployment-graphite to use local puppet & salt masters [00:32:42] Logged the message, Master [01:31:07] Hi, I finished the migration and restarted the webservice, but nothing is showing on http://tools.wmflabs.org/robin/ - does anyone have any idea what the problem is? [01:48:13] SPQRobin: That gives for me "No webservice"?! [01:52:32] (03PS1) 10Tim Landscheidt: Fix Lintian errors in misctools man pages [labs/toollabs] - 10https://gerrit.wikimedia.org/r/122628 [01:54:35] scfc_de: yeah, that's what I mean with "nothing" :P [02:01:02] SPQRobin: did you run webservice start? [02:01:18] Betacommand: I did [02:01:39] when I do it again, it says "webservice already running", so... [02:01:49] and trying to stop it doesn't work [02:02:42] SPQRobin: you are doing this as the tool correct? [02:02:51] SPQRobin: Your job is in "E" state; "error: can't open output file "/data/project/robin/error.log": Perm" [02:03:03] scfc_de: ah [02:03:06] Betacommand: yes [02:03:27] Which I don't understand ... [02:03:36] heh [02:03:55] scfc_de: what are the permissions for that file? [02:05:04] SPQRobin: Ah, you started webservice as robin. You need to start it as tools.robin ("become robin" => "webservice start"). [02:05:13] Users don't have webservices. [02:06:05] scfc_de: well, the tool is named identical to my user (both robin; I know, not really a good idea) [02:06:42] so I did do "become robin" & "webservice start" [02:07:23] but I did also play around with file permissions a while ago.. maybe there's the problem [02:08:04] SPQRobin: not sure you can have a tool with the same name as a user [02:08:25] well, it did work before the migration [02:09:04] maybe I should just start again under a new tool name [02:09:12] SPQRobin: you where probably exploiting a bug that got fixed [02:09:41] No, that should work. [02:11:15] SPQRobin: Look at "qstat -j 157058". It says "owner: robin", so there is a *very* high probability that you didn't "become robin" first :-). Have you tried again? [02:12:25] I did do "become robin"; my commands are under "tools.robin@tools-login" [02:12:51] Ah, I see what happened: You tried to "webservice start" as robin (which launches the job as "lighttpd-robin" as well), and then as tools.robin, but the latter saw that a job "lighttpd-robin" was already running, so it didn't start a new one. [02:13:12] As robin, you need to "qdel -j 157058", then "become robin" and "webservice start". [02:13:51] scfc_de: do you know how to block access to a tools webspace based off user agents? [02:14:44] With lighttpd no; but there should be some docs for that. [02:15:11] Ive see a TweetmemeBot spidering my tools [02:15:26] http://redmine.lighttpd.net/projects/1/wiki/Docs_Configuration, "match on useragent" looks alright. [02:15:45] scfc_de: that worked; thanks a lot! [02:16:21] In pmtpa, we blocked TweetmemeBot globally in tools-webproxy, but I haven't caught up how that is set up in eqiad. [04:37:48] Good morning all [04:37:55] * Beetstra looks at Coren .. any news? [06:53:18] Betacommand: why? the bot is smart enough, just try !fingerprint [07:45:05] does anyone have any idea who maintains the http://ganglia.wmflabs.org/ interface? It is dead :D [08:05:00] hashar: " [08:05:00] I'm stumped, but matanya has offered to work on this. andrew (talk) 15:56, 17 March 2014 (UTC) [08:05:05] quote from Andrew [08:05:15] ah nice [08:05:17] thanks :-] [08:05:31] it is a bit of a pity since wikimedia got a contractor to handle that [08:05:34] https://wikitech.wikimedia.org/wiki/Nova_Resource:Ganglia [08:05:37] says petan based on the logs [08:05:43] would have expected the project / instance to have been movable easily [08:05:55] contractor for just ganglia? [08:06:03] yeah I think so [08:06:03] petan says what? [08:06:35] petan: the questions was who was working on ganglia project in the past [08:06:43] and i just saw your name on project SAL [08:06:45] I was one of many :P [08:07:04] I basically just fixed the OOM issues that resulted in random crashes [08:07:10] blames not enough logging by the others [08:07:24] gotcha, petan [08:07:38] hashar: i didnt realize we have a contractor [08:07:48] that was a while ago [08:08:53] hashar: poke me later about this, didn't even have time to look at [08:09:24] matanya: was just wondering whether someone knew ganglia is dead :-D [08:09:33] i have [08:15:41] hashar: can you recommend who will know about https://gerrit.wikimedia.org/r/#/c/74591/11/manifests/misc/maintenance.pp ? [08:18:01] matanya: what are the stats? (usage of apache vs. mwdeploy) [08:18:26] let me grep [08:19:19] 12 apache 7 mwdeploy mutante [08:20:02] i'd just make the patch with apache as user, add comment to it linking to the revert [08:20:10] and poke people for gerrit reviews [08:20:25] would have used apache because the other "purge" script does it [08:21:06] i did that [08:21:14] my concerns are the logging issue [08:21:51] i have no clue where is the logging happenning, i guess from mw-deployment-vars.sh some where [08:21:59] but not familiar with it [08:23:08] what do you mean, isn't it just changing the path to logfile? [08:24:07] no, can;t [08:24:16] it comes from templates/misc/mw-deployment-vars.erb [08:24:36] but i dont know if other things rely on that [08:24:55] try Reedy [08:24:59] it seems not, but who knows [08:25:13] i think i'll push, and wait for reviews [08:25:20] rely on the logfile that doesn't exist yet? sorry i dont get the whole context yet [08:25:26] but yea, do that [08:31:40] !log deployment-prep MediaWiki config paths tweaks for Math {{bug|63331}} and Captchas {{bug|63342}} [08:31:43] Logged the message, Master [08:41:37] Is icinga (at http://icinga.wmflabs.org/ ) also (longer) dead or just moved to another url? [08:49:47] se4598: try nagios.wmflabs.org :) [08:51:45] matanya: wasn't nagios the old system where nagios company asked why nagios.wikimedia.org redirects to icinga? ;P [08:51:54] but there also: Connect to 208.80.153.249 on port 80 ... failed Error 110: Connection timed out [08:51:58] that is surprising indeed [08:52:10] my point is all monitoring seems to be dead [08:52:22] remembers how much work it was for Leslie and others to switch prod from Nagios to Icinga [08:52:35] and what se4598 said [08:52:37] about the naming [08:54:22] AFAIR was icinga.wmflabs.org already a long time before migration to eqiad a 502 Bad Gateway [08:54:35] yes mutante i was abit involved in that effort [08:55:01] se4598: yes [08:55:36] you could ping Damianz [08:55:55] Damianz: can we ?:) [09:09:24] do we have or need a bug for icinga and ganglia? [09:12:30] se4598: i'd say need.. [09:13:01] labs migration people just said to please make 'em [09:17:17] se4598: if in doubt, fill a bug :-] [09:17:24] at worth someone will close it [09:17:40] will do, currently busy fighting vandalism :) [09:20:58] as long as it's not vandalism on bug reports:) [09:21:04] thanks [09:29:45] we are not able to access our instance wikidata-builder3 since yesterday.. [09:30:03] https://www.irccloud.com/pastebin/zaBSjG2U [09:30:35] should we consider it lost? [09:30:38] Tobi_WMDE: a bug in bugzilla will do, or poke andrewbogott_afk or Coren [09:39:02] Coren: do you have an idea? [09:45:59] se4598: What are you fighting vandalism with? [09:47:26] most definitely not the wiki software itself, as logical as that could be [09:47:43] gry: wiki software suck in fighting vandals [09:47:50] that can be fixed [09:47:55] I hope he uses huggle as he is one of its developers [09:47:59] gry: so fix it. [09:48:09] I would, if I knew the language; I don't [09:48:21] petan: I am ;) [09:48:29] I know the language, which makes me think it can't be fixed [09:48:29] gry: JavaScript? [09:48:36] no, wikis aren't written in js [09:48:44] Oh, you mean the core? [09:49:01] yes [09:49:13] How would that work? [09:49:23] huggle is a browser desinged to fight vandalism, mediawiki is software that is executed on server and then rendered in a browser, it by design is not possible to be ever that powerful as huggle [09:49:52] it's like you wanted to create a java interpretor written in java that would be faster than its c version [09:49:55] it never will be [09:50:50] a930913: in some way that any contributor can do his stuff without installing software in browser or computer [09:50:50] Huggle doesn't max CPU, so speed isn't an issue. [09:51:12] no, I don't really have an issue with these tools performance [09:51:17] gry: What do userscripts count as? [09:51:33] gry: yes that could be possible, but again it would be slow and ineffective, there is already twinkle for this afaik [09:51:33] as something you can put into core and maintain properly [09:52:23] I'm not sure how putting something into JS and loading it when he enters a vandal-fight mode could slow things [09:52:33] I just can't imagine someone would be spending time to create some js based tool that would be heavily multithreaded connected to multiple external resources and doing all the stuff what huggle does [09:52:43] possibly [09:53:21] maybe they want to run it on node.js? [09:53:25] it would slow things because software written in JS is slower than C++, that's why all these 3d games are written in C++ and not in JS [09:53:49] that's why linux kernel is written in c and not JS :P [09:54:09] it would be funny idea to create self-interpreted byte compiled JS kernel though [09:54:21] hm. they have parsoid which does many things at once, what is it written in? [09:54:32] JS and it's horribly slow [09:54:42] I installed parsoid to one of my servers, it just suck [09:55:01] it consumes tons of resources to do a little work [09:55:48] I think that the approach of WYSIWIG editor was much better than VisualEditor [09:56:31] that thing was really fast, stable and didn't require any 3rd software to be installed on server, like parsoid [10:06:19] petan: V8 is fast enough to run a Huggle. The main lag would be in IO anyway. [10:06:49] Remember, until recently huggle was in C# ;) [10:06:53] maybe but I see a little point in that [10:07:07] a930913: hg2 (which I use) is in Visual Basic [10:07:28] I see that it sounds cool that "user doesn't need to install anything, but open a webpage" but on other hand, typing one command in terminal in order to install huggle isn't that hard either [10:07:39] you need to install browser anyway :P [10:08:11] petan: A big barrier for for using huggle is the need to download, install and hand credentials over to software. [10:08:48] petan: Using wikipedia within wikipedia makes a lot more sense. [10:09:32] ok, but all that JS, wouldn't you need to download that too hm? the traffic of webpage based SW would be higher though, so in the end, you would end up downloading much bigger data [10:09:48] downloading and installing huggle is really very easy, its debian package has few kb only [10:10:04] petan: There's downloading a file and "downloading" a webpage. [10:10:09] regarding credentials: yes that suck, but unless wmf fix that I can't do anything [10:10:16] The webpage is transparent to the user. [10:10:32] is it? :P [10:11:04] more and more powerful browsers will be, more potential they will have to destroy target computers with some JS based viruses [10:11:21] I doubt that everyone is reading and understands the source code of JS they load in their browsers [10:11:24] petan: When you click a link on a webpage and are brought to another webpage, almost everyone doesn't even realise that the download has occured. [10:11:36] maybe but it did [10:12:20] petan: If anything, the source of the JS is transparent, where the compiled code given for huggle is not. [10:12:36] I wouldn't be that sure, this is only case of windows [10:12:52] ubuntu packages are built by launchpad which is service provided by canonical [10:12:53] petan: A majority OS. [10:12:59] they are just that transparent [10:13:40] you can view source code of any binary that was built there [10:13:57] Anyway, until recently huggle was windows only, with a hack for other OS's. [10:14:19] petan: My point was the source code has little effect on the end user. [10:14:22] yes, that has changed though [10:14:42] define "little effect" [10:15:01] petan: Most users don't check source. [10:15:26] yes, that makes it irrelevant whether they run byte compiled code (faster) or interpreted code [10:15:46] they don't care anyway [10:15:54] because as you said, they don't check the source [10:16:07] petan: That was in response to 11:11 < petan> more and more powerful browsers will be, more potential they will have to destroy target computers with some JS based viruses [10:16:27] As for speed, as I said, it's generally fast enough. [10:16:51] really? on your PC maybe [10:17:02] I have 1.1ghz 2gb ram netbook [10:17:07] that thing can barely run firefox [10:17:12] the latest version I mean [10:17:15] petan: My PC is a benchmark for slow ;p [10:17:24] it's slow mastodon eating lot of resources [10:17:28] Well yeah, FF is a beast. [10:17:31] not all people on world have latest computesr [10:17:33] * computers [10:18:10] I can't even run facebook on that PC, but huggle, hell that run so fast and well [10:18:23] facebook is web based, huggle is natively running [10:18:39] I could run 10 huggles simultaneously on that box [10:18:51] but that box can't handle 1 facebook window [10:19:03] that is my proof that webbased stuff is slow and resource expensive [10:19:55] one day people will have boxes with 20 cores and 100gb of ram, and they will be happily running the same software as they run now, just it would be web based [10:19:59] petan: You seem to be confusing memory leaks with programs. [10:20:20] what, are you saying there is memory leak in firefox? [10:20:49] **** yes. I switched to chrome because of that. [10:20:57] It's just bloat. [10:21:01] I have chrome too [10:21:10] it uses just as much ram as firefox [10:21:20] literally, I see no difference between these 2 [10:21:36] just firefox is IMHO more community based, while chrome is pretty much all about google [10:21:39] FF has so much more overhead. [10:21:48] chrome too [10:22:02] that's all the JS bloat wrapped all around these browsers [10:22:02] That does not logic.. [10:25:41] petan: The most computationally expensive thing in much of the browser stuff is the rendering of the page. [11:00:45] petan: i built huggle 3 on my machine, works good, and looks nice, thanks [11:00:55] any plans for supporting more wikis? [11:02:54] you are welcome :P [11:03:02] yes, huggle 3 supports any wiki, even non wmf [11:03:16] one guy already installed it and had it working on some private wiki [11:05:05] a930913: indeed, fortunatelly huggle uses api, which don't need to be rendered, and pages it display are mostly pre-parsed diffs only [11:16:22] petan: Rendering in both applications should be similar. [11:17:03] a930913: I don't think so, huggle only needs to render few lines of html, while web browser based application would need to render whole its interface that would be only html + JS etc [11:18:03] the interface itself changes a lot [11:18:28] just have a look how often the queue is changed, the lists with information about edit etc [11:20:32] petan: how do i choose a non-listed project ? [11:21:06] matanya: wmf or non-wmf? [11:21:11] wmf [11:21:41] well, most easy is probably to open a bugzilla ticket requesting to enable huggle on that project [11:21:57] the project needs to have huggle configuration page, warning templates and all that [11:22:19] then it just needs to be added to Huggle [11:22:27] petan: Fair point, but the queue still needs to be rendered in any language. [11:22:28] * https://meta.wikimedia.org/wiki/Meta:Huggle/List [11:22:58] a930913: rendering it in native (on linux GTK) library is much, much faster, then rendering using html [11:23:16] html needs to be parsed, binary code doesn't [11:24:14] petan: Also if done right, a DocumentFragments helps change only deltas. [11:24:55] maybe, but I am not going to do that, but you have my permission to rewrite huggle to JS :P [11:25:05] good luck :) [11:26:08] petan: As I said last time, I already have translated the core, just needs a bit more bugfix :) [11:26:18] Do I get to call it huggle though? :3 [11:26:43] that's up to you [11:27:57] So I won't get shouted at for infringing "trade" mark? :p [11:31:07] thanks petan [11:38:22] done for today: hg2 still works like a charm http://imgur.com/W3hlP2J petan, sorry for not using hg3 ;) [11:51:29] mutante, matanya: created bugs for icinga and ganglia: bugs 63361 and 63362 [11:51:55] great,thx [11:51:55] thanks, coren and andrewbogott_afk will probably look at it [12:48:48] after 'finish-migration render-tests': [12:48:51] tools.render-tests@tools-login:~$ mysql [12:48:51] ERROR 1045 (28000): Access denied for user 'rendertests'@'10.68.16.7' (using password: YES) [12:48:54] what am i missing? [12:49:03] have the credentials changed? [12:56:58] JohannesK_WMDE: was the tool created while you was logged in with the shell. If yes, restart your connection [12:57:22] ups, misread [12:57:28] se4598: no, it's been migrated... [13:04:44] JohannesK_WMDE: seems you are missing the conf file. try $ sql local [13:05:13] !mysql [13:05:24] !mysql is https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Shared_Resources/MySQL_queries [13:05:25] Key was added [13:06:01] tools.render-tests@tools-login:~$ mysql local [13:06:01] ERROR 1045 (28000): Access denied for user 'rendertests'@'10.68.16.7' (using password: YES) [13:06:18] .my.cnf exists if that's what you mean hedonil... [13:06:21] JohannesK_WMDE: just sql not mysql [13:06:37] tools.render-tests@tools-login:~$ sql local [13:06:37] ERROR 1045 (28000): Access denied for user 'rendertests'@'10.68.16.7' (using password: YES) [13:06:44] sql is a wrapper script, mysql is the binary [13:06:47] !log deployment-prep Applying role::beta::natfix on deployment-upload.eqiad.wmflabs . Might let it access images from commons.wikimedia.beta.wmflabs.org ( ex: http://upload.beta.wmflabs.org/wikipedia/commons/thumb/4/43/Feed-icon.svg/16px-Feed-icon.svg.png yields: Error retrieving thumbnail from scaling server: couldn't connect to host commons.wikimedia.beta.wmflabs.org ) [13:06:50] Logged the message, Master [13:07:02] hedonil: i know [13:07:21] JohannesK_WMDE: can you open replica.my.cnf and read the content? [13:07:49] JohannesK_WMDE: not mysql, but "sql local" [13:07:52] hedonil: yep... [13:07:57] se4598: yes, see above [13:08:51] try $ mysql --defaults-file=~/replica.my.cnf -h tools-db [13:09:38] sorry my irc connection apparently dropped and i doesn't saw the messages (reading now from bot-log) [13:12:27] hedonil: that works... apparently .my.cnf is bad/broken/whatever [13:12:55] JohannesK_WMDE: toss it to the bin ;) [13:13:00] tools.render-tests@tools-login:~$ cat .my.cnf [13:13:00] [client] [13:13:00] host=tools-db [13:13:00] user=rendertests [13:13:00] database=rendertests [13:13:01] password=[...] [13:13:32] hmmmm... [13:13:34] That is a strange password - you sure it is correct ;-) [13:14:13] Beetstra: sure! that's the one i always use for everything! ;) [13:14:43] :-D [13:15:32] seriously though... i didn't realize the database names and user names changed that much after the move. so we have to use the sXXXX__blah DB names for everything? [13:16:14] JohannesK_WMDE: yep. http://tools.wmflabs.org/tools-info/?dblist=tools-db [13:16:20] JohannesK_WMDE - changed that much: some even disappeared!! [13:16:32] Beetstra: lol [13:16:52] great [13:17:14] Beetstra: coren made your db going to stealth mode ;) [13:17:30] hedonil .. Iḿ not exactly laughing yet - may have to spend time to again regenerate the db since I .. forgot to dump the structure of it after the last build/redesign :-( [13:18:21] Beetstra: Sorry, my timezone doesn't really allow me to respond to you around 1am my time most of the time. :-) I'm going to restore your DB from a pickled instance in an hour or two. [13:18:40] OK, I hope to see it tomorrow then. [13:18:44] Thanks Coren [13:18:45] we have to move again soon. this is all very exciting and fun [13:18:54] JohannesK_WMDE: Move again? [13:19:14] YES, please Coren, schedule another move. Split all tools into their own instances or something like that [13:19:40] Please announce it today (and hope that people don't notice it is the 32nd of March) [13:19:44] Coren: yeah, TS to labs, then the equiad move, then ... move somewhere else just for the fun of it [13:20:02] and yes, today would be the right day to announce it Coren [13:20:11] Beetstra: I think Coren is going to recover your missing asset [13:20:39] hedonil, Coren just mentioned that, indeed [13:20:40] Given how much pain everyone has had to go through already, I don't think it'd be the right time for playing April fool. Srsly. :-) [13:20:47] Beetstra: Everything will be ok in the end. If it's not ok, it's not the end :-D [13:21:19] hedonil - I have spam fighters from all over the world complaining, asking when I bring my tools back online [13:21:29] did not hear any spammers complain, though .. strange [13:21:54] Beetstra: they know... [13:22:11] Coren: Can you send round a panicked, back-up-your-work-quickly email to labs-l? :p [13:24:06] OK, see you tomorrow [13:24:52] a930913: I'm all for good fun in general but I really don't think it'd be a good move to annoy people about Labs now. :-) [13:27:04] Though backing up is always a good idea, and with no Labs snapshots at the moment, it's mandatory :-). [13:29:19] scfc_de: Is there an easy command to snapshot your own stuff? [13:30:27] a930913: rm -rf ? [13:30:27] !log deployment-prep upgrading labs Elasticsearch to 1.1.0 [13:30:30] Logged the message, Master [13:31:20] !log deployment-prep deployment-upload is rejecting connection on port 80. Applying role::beta::uploadservice from {{gerrit|122786}} [13:31:23] Logged the message, Master [13:31:45] hedonil: Thanks, running now. [13:31:50] a930913: ;) [13:31:56] hedonil: ;) [13:32:48] !log deployment-prep Thumbs access more or less fixed [13:32:51] Logged the message, Master [13:33:03] On the more positive side of things, I note that Tool Labs has been rock solid in its new home. The pain wasn't for nothing. :-) [13:33:38] Coren: With your professional experience, at what point should we expect the creaking to start? :p [13:35:30] Coren: congrats! [13:36:00] a930913: "ssh -e none scfc-test@tools-login.wmflabs.org 'cd .. && tar jc $USER' > scfc-test.tar.bz2" [13:36:27] Coren: but you have provide a kind of 'legacy corner' with gluster [13:37:19] Coren: to remember - and just in case have a cuplrit that can be blamed randomly [13:37:21] lol [13:38:31] scfc_de: That's just files, not cron or db, etc., right? [13:44:57] a930913: Yes, just the home directory. That suits me, as I always generate/back up crontabs or DBs from/to disk. [13:47:08] a930913: I'm hoping "never"; the move to eqiad allowed us to do "right" some of the things that were hacked up in pmtpa and there is no plan to stop continual maintenance. [13:47:42] Also, I *finally* get to start working of feature requests again. [14:07:37] hashar: deployment-elastic04 won't even let me in! [14:08:13] :-( [14:08:31] manybubbles: same for me [14:08:49] have you attempted rebooting it ? [14:09:00] let me give it a shot [14:09:05] we fucked up the puppet master yesterday might have applied some wrong configuration to it [14:09:06] :( [14:09:55] manybubbles: if all fails you get to grab ops to fix it up [14:13:17] hashar, ganglia was running on a Lucid instance. There were only a couple of Lucid boxes still running in labs and they mostly didn't survive the move due to not supporting the NFS client that we require. [14:13:55] andrewbogott: ah nice [14:14:24] andrewbogott: was it using any puppet classes ? I will be more than happy to attempt to resurrect the service on a Precise instance [14:15:44] hashar: please do! I started work on the 'aggregator' instance, it should have the necessary classes applied. [14:16:17] I just made you a project admin :) [14:16:35] What actually happens in these office meetings for labs? [14:18:52] andrewbogott: thanks [14:19:15] manybubbles: seems elasticsearch04 is reachable again. [14:19:25] yeah! [14:19:26] I'm on it [14:19:40] manybubbles: note that we have our puppetmaster (deployment-salt) with patches cherry picked in /var/lib/git/operations/puppet [14:19:54] I figured it was something like that [14:20:06] can those be on a branch or something? [14:20:09] a930913: You're talking about the toolserver migration meeting? If you're already migrated then it probably won't be interesting :) If you have issues that are preventing you from moving away from toolserver then the office hour is a good opportunity to make sure your blocker is addressed. [14:21:46] andrewbogott: You know, I completely missed the toolserver part :p I just say migration and thought it was about the labs migration XD [14:21:53] s/say/saw/ [14:22:44] hm... why Update_time = NULL o_O [14:22:50] i am suprised [14:24:11] is there a way to enabe? [14:24:55] Steinsplitter: I guess nobody has a clue what you are talking about. [14:27:16] SELECT UPDATE_TIME FROM information_schema.tables WHERE TABLE_SCHEMA = 'userdb' [14:27:21] output is NULL... [14:27:31] strange... normally schold be a timestamp [14:29:35] andrewbogott: first step for ganglia: get it installed on hosts in eqiad and point to the new IP address :-D https://gerrit.wikimedia.org/r/122790 [14:30:19] scfc_de: hast du ne idee warum das auf den labsdbs abgeschaltet ist? [14:32:19] !log deployment-prep completed upgrade to Elasticsearch 1.1.0 and fixed deployment-elastic04. [14:32:21] Logged the message, Master [14:32:50] Steinsplitter: Nein. [14:33:26] springle: Is "SELECT UPDATE_TIME FROM information_schema.tables WHERE TABLE_SCHEMA = 'userdb'" being NULL a conscious decision? [14:34:04] !log ganglia aggregator puppet.conf had some HTML for certname entries. Fixed all occurrences to use i-000002a4.eqiad.wmflabs [14:34:06] Logged the message, Master [14:34:30] Steinsplitter: Welcher Server? [14:35:12] scfc_de: der für die userdatenbanken [14:35:22] s51291 O_O [14:35:53] hashar, it has to be done by ip? [14:35:58] andrewbogott: no idea [14:36:06] andrewbogott: merely reusing the same logic that was used previously [14:36:10] yeah, ok. [14:36:14] Best to just remove the pmtpa entry [14:36:16] one bug at a time :-] [14:36:40] Steinsplitter: did that ever work on labs? [14:36:46] Steinsplitter: http://comments.gmane.org/gmane.comp.db.mysql.general/112102 [14:36:54] Steinsplitter: tools-db oder der auf den replicas? [14:37:06] ... Beginning with MySQL 5.7.2 [14:37:10] Coren, any way to check server-status via the terminal? xtools is stuck again, and I want to check out why, but the server-status page isn't loading as well. [14:37:24] scfc_de: tools-db [14:37:30] hashar: does 'one bug at a time' refer to pulling out pmtpa code as well? [14:37:54] na just the ip / hostname [14:37:59] I have no idea how ganglia use that information [14:39:54] !cyberpowerresponse [14:39:54] and I say what I need, get no response and realize I just wasted the effort of typing what I need. [14:40:12] Steinsplitter: Hat hedonils Link Deine Frage beantwortet? [14:40:23] This is exactly why I ping before I talk. petan [14:40:52] Cyberpower678: I hope the effort of typing two sentences hasn't hurt you too much. [14:41:28] scfc_de, I lost about 2 minutes of my life. :p [14:41:33]