[02:27:02] * Hazard-SJ says hello and asks if either coren or andrewbogott is around [02:27:14] I'm kinda around. What can I do to you? [02:27:52] * Hazard-SJ tells Coren that he would like to be added to Tools [02:28:12] What's your wikitech username? [02:28:37] Hazard-SJ [02:29:19] {{done}} [02:29:23] !toolsdoc [02:29:32] !toolsdocs [02:29:32] http://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/Help [02:29:37] Useful pointers ^^ [02:34:32] Thanks :) [03:03:05] Coren: By chance, are you still around? [03:03:13] For a bit [03:04:06] I created a project, and running become gives a strange problem [03:04:20] hazard-sj@tools-login:~$ become hazard-bot [03:04:22] sudo: unable to change directory to /data/project/local-hazard-bot/: No such file or directory [03:04:23] sudo: unable to execute /bin/bash: No such file or directory [03:05:23] Coren: But /data/project/hazard-bot exists and no other directory uses "local-toolname" in /data/project [03:05:30] * Coren goes check. [03:05:41] * Hazard-SJ smiles and waits [03:06:33] o_O [03:07:00] * Hazard-SJ wonders what Coren found [03:08:11] Did you already put stuff in the tool's directory you need to keep? [03:08:45] No [03:09:37] Coren: I actually checked on the folders after I found that problem [03:11:04] ... I have _no_ idea why your service account was created with a broken home. I just deleted it and created it anew and this time it got set right. [03:11:17] (You'll have to log off then back on) [03:12:42] Coren: Works now, thanks [03:12:55] * Coren is immensely annoyed. [03:13:09] Not that it worked; that it was broken in the first place. [03:19:50] Coren: How so? :S [03:21:09] I disaprove of bugs. :-) [08:39:04] Warning: There is 1 user waiting for shell: Byrial (waiting 0 minutes) [08:47:03] o/ [08:52:29] Warning: There is 1 user waiting for shell: Byrial (waiting 13 minutes) [09:05:59] Warning: There is 1 user waiting for shell: Byrial (waiting 27 minutes) [09:16:23] Coren is there a page where you need to insert that pypi thing? [09:16:37] Coren: I mean, like did you insert it to some to-do? [09:16:44] Coren: because it needs to be flagged as done [09:17:07] @searchrx to.*do [09:17:07] No results were found, remember, the bot is searching through content of keys and their names [12:43:10] petan: http://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/TODO [12:47:06] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/TODO was modified, changed by Petrb link https://www.mediawiki.org/w/index.php?diff=686962 edit summary: [+14] [12:47:31] petan: Good job on all that footwork BTW. [12:47:51] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/TODO was modified, changed by Petrb link https://www.mediawiki.org/w/index.php?diff=686964 edit summary: [-67] [12:48:04] simple english? :P [12:48:09] what is "footwork" [12:48:20] http://en.wikipedia.org/wiki/Footwork [12:49:40] USian Idiom. I means complicated work to do something needed the "hard" way. [12:49:52] Like research, etc. [12:51:29] ah, this wasn't really complicated, I just used google :P [12:52:00] we should make a list of what needs to be converted, also what should I do with converted packages? do we install them? [12:52:18] I copied them to .system so far [12:53:46] Right now oursql and a recent requests was all that was needed; I'll have to talk to Ryan about the packages -- either we'll put them in a repo for non-ubuntu stuff or I'll write a puppet class to install from a Labs repo on NFS. [12:54:58] anyway - line $ - 1 on to-do can't we use what I have on bots? [12:55:39] there is a script which allow users create custom db's [12:55:39] "creating databases"? [12:55:42] yes [12:55:51] it's a mysql proc [12:56:08] I even sent a mail about it once [12:56:13] Yeah, probably something like this, but I have to wait for the read DB first. Probably sometime this week. [12:56:20] real* DB [12:56:24] users can do system.create_db("my_cool_db"); [12:56:32] k [12:56:44] so real db's are coming? [12:56:57] !log wikidata-dev wikidata-dev-9 /srv/devrepo/w/Wikibase "git submodule init && update " [12:56:58] :o [12:56:58] Logged the message, Master [12:57:15] Yeah, I'm hard at work on the column selection ATM. [12:57:39] I meant the user db's [12:57:41] the simple part [12:58:03] It's still contingent on having replication up -- same hardware. :-) [12:58:33] ok but why once hw is up it needs to wait for setup of sw? I thought you already have the hw? [12:58:51] it shouldn't have dependency on production copy [12:59:09] We do, but the DB is Asher's responsibility. He's the mysqlmeister. :-) [12:59:28] mhm [12:59:37] Patience, grasshopper. [12:59:41] :-) [13:03:43] <^demon> Abraham_WMDE: Fun tip to save yourself some typing, `git submodule update --init` [13:05:03] Will DB queries on labs be faster than toolserver? [13:08:07] ^demon: thx [13:08:50] ^demon: had some typos for the admin log anyway - to many tabs open ;) [13:10:04] greenrosetta: Likely they will; the cluster hardware is quite a bit beefier. The biggest advantage is likely to be the reduced replag though. [14:11:20] petan: [16:07:34] addshore? [16:07:46] i typed it all in #huggle ;p [16:13:33] addshore I think in order to do this on labs we need to somehow integrate it to mediawiki [16:13:48] addshore so that it would have some wikimedia use [16:15:20] !log bots addshore: installing nano on -login [16:15:23] Logged the message, Master [16:15:37] * petan puts all nano users to prison [16:15:52] * addshore puts petan in prison [16:16:10] * petan edits the prison using vim and escapes [16:16:42] i do use vim, but for little things nano is so much quicker :P [16:16:50] addshore imagine the irc interface right inside wikipedia :D [16:16:53] and generally i dont do big things ina termina; [16:17:05] so that people could use ircloud features in mediawiki itself [16:17:06] :P [16:17:20] that would improve collaboration a bit [16:17:40] it would indeed [16:17:41] I can see a potential for many cabals in that XD [16:18:37] addshore you think you could implement that python code of quakechat to mediawiki? [16:18:44] quake chat is what freenode uses in webchat [16:18:53] gimmie 1 sec [16:18:53] it's stolen from quakenet [16:19:05] just trying to updat my repos, im so behind :P [16:19:11] k [16:19:17] strangly they are my own repos that I am behind with >.< [16:23:24] can the bugzilla project on labs get an public IP allotted to it, please? [16:31:57] Coren: ? [16:35:16] andrewbogott: ? :) [16:35:30] Sure, hang on... [16:36:26] can i have a vanity ipv4 ip, like '208.CALL-BUGZILLA' [16:37:04] just kidding [16:37:58] Looking at the numbers on my keyboard, you'd need your slogan to consist strictly of punctuation marks. [16:38:12] Anyway -- one IP quota'd! [16:38:30] thanks [16:38:59] by the way, dunno if you saw, but the mysterious disappearing mongodb password was in the end due entirely to my stupidity [16:39:57] Yep, I saw. Although it was also broken by being only on Stafford. [16:44:16] * Coren returns from food run. [16:44:47] Looking at the numbers on my keyboard, you'd need your slogan to consist strictly of punctuation marks. [16:44:54] Isn't that appropriate for bugs? [16:45:02] @&*^#% piece of buggy software. [16:46:16] I don't think I could reproduce the exact sequence from memory though :( [16:50:48] ta-da! http://boogs.wmflabs.org/ [16:51:10] Coren: care to help me with a permissions problem? :/ [16:51:19] addshore: Sure. What be up? [17:00:31] addshore here :D [17:00:46] @notify binasher [17:00:46] I will notify you, when I see binasher around here [17:01:06] ok I need to go now [19:23:48] Warning: There is 1 user waiting for shell: Zoska (waiting 0 minutes) [19:25:01] petan: ill have a look into it ;p [19:25:13] in what [19:37:19] oh i thought u had gone :P [19:37:22] Warning: There is 1 user waiting for shell: Zoska (waiting 13 minutes) [19:37:47] http://www.mediawiki.org/wiki/Extension:WebChat << ugly uses mibbit [19:38:14] http://www.mediawiki.org/wiki/Extension:Chat << ugly install so no doubt its ugly too [19:38:19] ugly u say [19:38:23] and nothing else in recent years :P [19:38:27] mhm [19:38:36] well, problem is that I suck in javascript and all that [19:38:50] so best I can do is a backend (bouncer / php code) [19:39:17] frontend must be done by some ajax-master [19:39:22] i suck in js too ;p [19:39:54] another question is if this would be actually popular / had success [19:40:07] I don't want to work on that alone and it sounds like a lot of work [19:40:18] http://www.mediawiki.org/wiki/Extension:CgiIrcLogin [19:40:41] that looks familier [19:41:03] I know about this all, none of these extensions are actually working well, nor are simple enough for regular people [19:41:13] also 2 extensions for pjirc (open source) http://www.mediawiki.org/wiki/Extension:PJIRC http://www.mediawiki.org/wiki/Extension:IRC_Chat [19:41:46] what I would like is a chat that looks mediawiki native - like the interface is media wikish [19:41:58] i reacon the cig one is a good place to start, I played around with ye olde cgi client before and itts rather simple [19:42:01] not something that just iframe some 3rd framework [19:42:47] it needs to be very simple as well, so that people who have no clue how irc works are able to use it [19:42:56] if we want to use this on wikimedia sites eventually [19:43:14] irc is already being used a lot, this would open it to more people [19:45:34] petan: you know about the Freenode web interface, right? [19:45:40] yup [19:45:48] I even have a source code of that [19:47:07] sumanah we were discussing creation of mediawiki extension that would enable irc-cloud like features inside of mediawiki, but that's a rather huge project... [19:47:16] yeah, pretty huge [19:49:20] and ew. irc [19:49:26] it's a terrible protocol [19:49:57] well, it is... but it's being used most for this kind of chat [19:50:16] also it's quite simple to implement it [19:50:40] but I agree there could be lot of improvements in it, hope someone create irc 2.0 :D [19:51:00] Warning: There is 1 user waiting for shell: Zoska (waiting 27 minutes) [19:51:35] petan: what would you add? :P [19:51:49] video chat? :D [19:51:54] :/ [19:52:07] I would probably switch whole thing to json or something easy to parse [19:52:33] set up some strict & proper standards so that each network wouldn't have own interpretation of it [19:52:54] petan: well, or we could use jabber inside of the projects [19:53:03] jabber is multi user? [19:53:18] need to find what makes it better than irc.. [19:53:22] yes, it's possible to have multi-user chat [19:53:46] we could also send notification streams there [19:55:33] hmm jaber seems like a high-level chat to me... authorization is must, probably harder to implement, thus not many clients / servers, but indeed offers some interesting features [19:55:51] I am wondering if they can be disabled [19:56:20] like, being able to join a chat and read the history could be problem somewhere [19:56:35] but in most places it's advantage [19:57:32] I am wondering how irc bots like nagios bot would work in jabber [19:57:42] it's probably not so lighweight I guess [19:59:29] interesting, I just made echo break on my local instance xD [20:00:38] I think that irc 2.0 could be something between jabber and irc, it would offer these structured data streams, but it would be kept as much lightweight as possible [20:04:26] Warning: There is 1 user waiting for shell: Zoska (waiting 40 minutes) [20:08:33] petan: FYI: the tool labs classes are now in puppet; new package installations should go through there and no longer manually with ongrid. [20:08:53] module/toollabs in operations/puppet [20:08:59] OK [20:09:47] You normally want to add packages to exec_environ, though dev tools go in dev_environ [20:10:07] Coren can you link me to gerrit? [20:10:10] I can't find it [20:10:42] https://gerrit.wikimedia.org/r/#/admin/projects/operations/puppet [20:13:27] db upgrades worked fine for eqiad. I wonder if it's still working :D [20:15:03] !toolspack is git clone ssh://pingu@gerrit.wikimedia.org:29418/operations/puppet modules/toollabs [20:15:04] Key was added [20:15:46] Ryan_Lane: Receiving objects: 100% (56798/56798), 69.53 MiB | 475 KiB/s, done. [20:15:51] 70MB? :D [20:16:07] poor those with mobile connection [20:16:07] eh? [20:16:14] that repository is becoming huge [20:16:24] can't we shrink it somehow? [20:16:25] that's actually not so large [20:16:34] it's just a text... [20:16:39] 70mb for a text is lot [20:16:49] there's some binaries in the history [20:16:54] oh [20:16:55] and it's the history that makes it large [20:17:06] binaries in git o.O [20:17:10] even text can be large when there's a ton of commits [20:17:18] did someone accidentaly commit a movie there? :P [20:17:34] I had originally put gerrit in there, stupidly [20:17:52] I remember days when svn checkout was about getting 600kb of text for a large projects... [20:18:02] Warning: There is 1 user waiting for shell: Zoska (waiting 54 minutes) [20:18:51] dude our svn checkout was massive [20:19:02] well, it wasn't 70mb for sure :D [20:19:08] it was *way* larger [20:19:10] GBs [20:19:11] anyway Coren where are packages for -login [20:19:43] uh, maybe the whole repository, but surely not mediawiki [20:19:48] yes. the whole repo [20:19:57] well, that contained a lot of stuff... [20:20:02] MW is gigantic too [20:20:09] yes it is [20:20:10] petan: Login is a toollabs::bastion, so it gets both toollabs::exec_environ and toollabs::run_environ [20:20:22] dev_environ* [20:20:29] ah ok [20:20:43] there is no mono-complete [20:20:47] :P [20:21:42] I though I had all of http://www.mediawiki.org/wiki/Wikimedia_Labs/Tool_Labs/Notepad but I may have missed. [20:22:36] maybe I just can't find it.. [20:23:35] petan: line 37 of modules/toollabs/manifests/exec_environ.pp [20:26:14] oh [20:26:36] I thought you didn't want that to be on exec nodes [20:27:44] https://gerrit.wikimedia.org/r/#/c/62913/1/modules/toollabs/manifests/dev_environ.pp [20:28:07] Coren ^ I submitted a patch, doesn't break stuff, but having mono-complete on both places is not a best solution [20:28:50] petan: Actually, having in in both places /does/ break stuff. [20:28:59] howcome [20:29:20] puppet is teh evils is howcome. [20:29:22] I thought that apt-get package twice doesn't do anything wrong [20:29:39] mhm [20:29:41] ok [20:29:43] petan: It doesn't, but puppet doesn't want it in the manifest in two spots. [20:29:54] Coren did you already installed mono-complete on exec nodes? [20:30:12] petan: Yes. [20:30:16] ok [20:30:32] petan: Some support libs in there needed by some code; I hated having to install the whole mess but necessary for some bots. [20:31:11] People still use mc? In 2013?! [20:31:33] Warning: There are 2 users waiting for shell, displaying last 2: Zoska (waiting 67 minutes) Munaf (waiting 2 minutes) [20:31:48] Coren on toolserver yes [20:32:03] they won't be happy not to have it [20:32:34] basically having anything on toolserver what wouldn't be on labs would make labs worse [20:32:49] except for things like "having bugs" :D [20:33:00] or solaris :P [20:35:08] Coren should I insert ALL packages we are using on -dev there or only packages we REALLY need to have [20:35:19] like pypi-install [20:35:25] that is what we use to convert to .dev [20:35:33] but not really anything we need on -login [20:35:47] I suppose these 1 purpose things can be sorted out using apt [20:36:00] petan: What needs to go there is what needs to go on every bastion and bastion-like box. Stuff that's not needed for normal "business" needn't go there [20:36:21] so I won't put pypi in there [20:36:32] Right. One-offs are okay to not puppetize if (and only if) nothing that runs depends on it. [20:36:58] Coren ok, is it possible to enable this puppet class on some box? [20:37:16] Coren I would like to apply it on all bots boxes so that we can finally start using it as a staging area [20:37:16] "some box"? [20:37:33] like puppet changes could be tested on bots if they don't break stuff [20:37:43] I don't like breaking things [20:38:01] Ah, then you have to switch to NFS first; the root class puts things in /data/project/.system [20:38:08] aha [20:38:22] well, it's possible to create .system on gluster or not? [20:38:34] switch to nfs is something I need to schedule [20:38:44] It is, but it means that puppet runs now become dependent on gluster working. :-) [20:39:07] hmm, if gluster doesn't work, nothing will work there, so let's give it a try [20:39:21] most of bots that run there are depending on gluster as well [20:41:57] ::bastion should be 100% safe to use, but I'd avoid ::execnode for now, I'm about to put stuff in there that would break your cluster unless everything else was in place. [20:42:21] Look at manifests/role/labs.pp to see how I did it. [20:42:57] Onbviously, don't include role::labsnfs::client yet. :-) [20:47:23] Coren yes I tried execnode on 1 box I can break, it seems to reconfigure mounts [20:47:34] it changes auto.master [20:47:46] /etc/auto.space etc [20:47:57] hm I will see [20:48:07] Oh, nonono! Don't include the role::labs::tools classes! Make your own role::labs::bots or you'll break *everything* [20:48:13] break ALL THE THINGS! [20:49:29] Copypasta class role::labs::tools, make role::labs::bots (and change the inherits), remove 'include role::labsnfs::client' and change the $grid_master [20:50:03] You probably want to tweak the description => in your classes, too. [20:50:17] aha [20:50:23] Coren break things where? [20:50:28] Coren in tools or bots? [20:50:32] petan: In bots. :-) [20:50:45] ok this was a testing box anyway, nothing running there [20:51:02] Well, next time you reboot it it'll be on the NFS server now. :-) [20:51:58] okay but if I copy paste all stuff - how are we going to set up the staging process? like where the changes would go first? directly to tools? or first to bots and if it works, then to tools? or only big changes are going to go through that process? [20:52:37] I think we should rather set up some realm variable [20:53:13] like $realm == tools || $realm == bots [20:54:29] Alternately, you can copy the toollabs module itself, make one of experimental stuff. [20:56:20] yes but that would mean keeping 2 different files synced [20:56:37] that is problematic... [20:57:08] btw the $realm should contain production or staging [20:57:19] the bots project would use the staging [20:57:38] Mind you, stuff you add to the *role* only applies to your project; so you can to experimental stuff there before moving it to the toollabs module. [20:57:56] if you were to change something big, you would wrap it if ( $realm == "staging" ) ... and so the change goes to staging area only [21:00:18] petan: (did you see my message earlier) why is the head of huggle full of errors ? :P [21:00:54] addshore what errors [21:01:10] bbl [21:01:12] :P [21:01:12] http://grab.by/mlRs (i guess you didnt see the message) ;p [21:06:44] BTW, re Puppet: On Toolserver, I would like to include in Postfix's /etc/aliases a dynamically generated list (every hour perhaps) of "aliases". What's the best way to let that coexist with Puppet's mailalias? Just add/update a section "# == STARTS HERE".."# == ENDS HERE" and assume Puppet's mailalias will only append at the end? [21:10:43] scfc_de: That'd work, but it's generally considered to be Bad Practice™ to manually fiddle with a file you expect to be puppet-managed. For that specific case, I suppose it'd be mostly harmless, but you'll confuse things a lot if you end up having the same alias in both "parts" [21:14:52] Coren: Yep, those ones we would have to filter out ... Problem is, Postfix has no "include" option (or I haven't found it), and Puppet's templates are evaluated on the master side, if I understand it correctly. [21:16:14] None that I know of, at least. Are you at liberty of tweaking the puppet config, though? [21:18:53] What do you mean? *Everything* has to be approved by someone else, of course :-), but Puppet owners and roots are the same. [21:19:54] You might be able to get away with a different scheme, then. Make a new alias database, point postfix at it, and use a puppet Exec to take the /etc/aliases, add your entries, and push to the "real" aliases db. [21:23:08] well, instance creation and such work in eqiad with folsom [21:24:40] Ryan_Lane: That's random. What is this a response to? :-) [21:26:21] Coren: And Puppet's mailalias will amend the original /etc/aliases? So I would not only call "make-toolserver-aliases" per cron every hour or so, but also on mailalias-notify? That sounds like a good idea ... Thanks. [21:27:03] scfc_de: np [21:27:11] I'm testing the upgrade in eqiad :) [21:37:25] I'm testing to see if Sweet and Sour chicken with rice will fill my stomach ;p [21:44:24] petan: You'll really want to test your changes on a self-hosted puppet master before submitting them. gdb is already included by base.pp. :-) [22:36:36] Ryan_Lane: Your image still has booboos. [22:37:25] ganglia-monitor doesn't fully install and breaks every apt-get invokation. [22:37:30] invocation* [22:46:13] purging ganglia-monitor then running puppet again reinstalls it and it works that time. [23:18:39] andrewbogott_afk: ping [23:29:19] @notify andrewbogott [23:29:19] I will notify you, when I see andrewbogott around here