[03:47:13] PROBLEM Free ram is now: WARNING on incubator-bots incubator-bots output: Warning: 19% free memory [06:35:32] !log deployment-prep update ffmpeg on deployment-transcoding (new security release from ppa) [06:35:33] Logged the message, Master [06:37:00] anyone around who could svn up /usr/local/apache/common/live/extensions/TimedMediaHandler [07:55:41] j^: done [09:08:00] petan|wk: i still see r110005 but would need r110104 or later [09:08:22] ah, sec [09:33:53] j^: done [09:34:49] g-day [09:34:53] hey [10:11:12] !log deployment-prep add-apt-repository ppa:j/timedmediahandler and update ffmpeg on deployment-web to support frame extraction from WebM videos [10:11:13] Logged the message, Master [10:14:35] petan|wk: need an extra vote for global editinterface? [10:14:37] :P [10:15:32] oh, that is your choice if you want :) [10:15:56] looks like >90% support [10:16:14] I see one weak oppose, though the voter didn't say it [10:17:48] sigh, huggle is for windows [10:19:08] Hydriz: it's not really true [10:19:18] version 3 has native support for linux [10:19:31] right, not actually native [10:19:36] but still it's supported there [10:19:56] I see [11:44:21] !magic is wikimedia labs is the place where magic is to happen that's why we have a unicorn as symbol [11:44:22] Key was added! [11:44:42] :o [11:56:26] !magic delete [11:56:26] wikimedia labs is the place where magic is to happen that's why we have a unicorn as symbol [11:56:33] @del magic [11:56:42] @delete magic [11:56:47] !magic is No magic, Labs is there because of hard work of many people, no surprise we have a beast of burden as a symbol [11:56:47] Key exist! [11:56:56] bleh [12:33:16] !log bots Moving also XLinkBot to bots-3. LiWa3 slows down reverting on XLinkBot too much. [12:33:17] Logged the message, Master [12:43:14] petan|wk: can you svn up /usr/local/apache/common/live/extensions/TimedMediaHandler once more? [12:47:38] !log deployment updated /usr/local/apache/common/live/extensions/TimedMediaHandler to r110117 per j^request [12:47:39] deployment is not a valid project. [12:47:52] !log deployment-prep updated /usr/local/apache/common/live/extensions/TimedMediaHandler to r110117 per j^request [12:47:52] Logged the message, Master [13:05:15] !log deployment-prep touch /etc/wikimedia-image-scaler on deployment-transcoding; transcoding needs more wgMaxShellMemory too [13:05:16] Logged the message, Master [13:25:34] MaxSem: !magic del [13:26:08] MaxSem: I know what lab is and why it exist, I quoted Ryan [13:26:12] labs [13:30:44] !log deployment-prep install upstart script /etc/init/timedmediahandler.conf on deployment-transcoding and start service [13:30:45] Logged the message, Master [13:40:13] Anyone here remembers his/her old school days? [13:50:57] Hydriz: yes [13:51:14] do you have schemes like Casual Fridays? [13:51:22] casual fridays? [13:51:26] what you mean [13:51:31] Where you don't wear the school uniform and wear some casual shirt [13:51:37] or do you even wear uniform [13:51:49] we don't have to wear uniform in our coutry [13:51:53] I never did [13:51:57] okay, nevermind [13:52:16] I have to wear uniforms to school [13:52:22] aha [13:52:37] I don't really think it helps people to be better student :) [13:52:53] but maybe it's usefull at some point [13:53:11] heh [13:53:17] I mean the differences between people are not so significant [13:53:22] I am not really talking about the uniform part [13:53:31] I dunno [13:53:43] but we are supposed to wear uniforms to school [13:53:52] when all people wear same clothes, they don't look so different to each other [13:53:56] its quite confortable actually [13:54:03] yeah, true [14:17:13] PROBLEM Free ram is now: CRITICAL on incubator-bots incubator-bots output: Critical: 5% free memory [14:17:59] OMG [14:18:02] 5% [14:18:06] shyt [14:20:19] !log incubator Killed all bots on incubator-bots. Critical free memory available. [14:20:20] Logged the message, Master [14:22:13] RECOVERY Free ram is now: OK on incubator-bots incubator-bots output: OK: 88% free memory [14:22:28] 5% -> 88% [14:22:33] huge difference [14:32:47] !log incubator Created new instance incubator-bots2 with m1.large configuration to offset the load on incubator-bots [14:32:48] Logged the message, Master [14:35:42] !log incubator Enabled one of the three bots on incubator-bots, the remaining two on incubator-bots2 [14:35:43] Logged the message, Master [14:38:38] Hydriz: what kind of bots it is [14:38:52] some incubator specific ones [14:38:57] I know what you are thinking [14:38:57] there is a bots project dedicated for them [14:39:18] I know [14:39:22] ok [14:39:23] and I was considering it [14:39:33] but its okay [14:39:35] :) [14:39:58] petal|wk: Have you considered creating bots server instances with larger resources? [14:40:27] Especially like some bots/scripts that actually eat up lots of memory [15:16:40] hi sumanah! [15:16:45] hi Hydriz [15:16:50] g-day [15:16:57] Happy Friday to you [15:17:01] haha [15:17:14] Monday - Wednesday was holiday for me [15:18:00] petan|wk: Do you know if we are actually allowed to upgrade our instances from Ubuntu 10.04 -> 11.10 or whatever? [15:19:19] Hydriz: yes [15:19:27] we are? [15:19:33] Hydriz: I have no problem with creating larger instances if it's needed [15:19:37] .... [15:19:46] yeah, about that [15:19:58] regarding new version of ubuntu I think we maily support LTS only [15:20:05] and it's unlikely to change [15:20:18] I mean, we upgrade our instances ourselves [15:20:32] cos sudo does give quite a bit of rights [15:20:33] yes I think it's possible but it will probably break puppet [15:20:41] yeah [15:20:44] so you should not do that [15:20:50] heh [15:20:53] I am not that evil [15:21:04] I mean, if it was necessary for something Ryan could create a repository for that release [15:21:14] Lets put that as a rule on Help:Rules [15:21:23] (page does not exist) [15:21:30] I don't want to make Rules without prior discussion with others [15:21:32] especially Ryan [15:21:41] yeah, so it isn't there yet [15:21:47] + I don't mean it's disallowed I mean it's not recommended [15:22:00] I can create a page with recommendations, I don't like idea of page with rules now [15:22:02] heh upgrading your instance [15:22:11] yeah, I am just linking it first [15:22:14] for the future [15:22:18] hm... [15:22:39] maybe Help:Tips could be [15:22:56] like information how to name instances, how to create them, how to allocate space [15:23:16] if you read the mail I sent today you probably noticed that I found there are unused allocated resources [15:23:19] Just Do It [15:23:29] ah yes [15:23:30] that one [15:23:34] indeed [15:23:46] this is wasting of money at some point [15:23:50] so you referred to RAM? [15:23:59] because resources do cost money [15:24:06] space shouldn't be the issue here, since its all shared [15:24:16] but RAM... maybe [15:24:17] no it's issue [15:24:25] storage is bigger issue than ram [15:24:33] and it's not shared [15:24:42] every instance has allocated own space [15:24:54] m1 instances eat 30gb of space each [15:25:08] while users do not use more than 5 gb in most cases [15:25:19] 30? [15:25:22] yes [15:25:25] isn't it like 20? [15:25:27] no [15:25:32] ah [15:25:33] there are two vd's [15:25:34] I see [15:25:38] root + /mnt [15:25:42] yes [15:26:01] ./mnt is absolutely unused [15:26:04] in most of them [15:26:20] But what Labs does is making RAM in proportion to space [15:26:44] if we can resolve this issue, introduce more options (and smaller intervals) [15:26:50] um? [15:26:52] what [15:27:07] like, instead of RAM being 512MB, then next is 1024MB [15:27:17] making it like some 700+ or what [15:27:24] ram is fully allocated to instance if you have an instance with 2gb, it allocate 2gb of physical ram [15:27:54] okok, lets not talk about ram first [15:28:01] lets say space issue [15:28:34] if users just need 5GB to store their stuff, then they should be able to choose 5GB, and not 0 or the next available 20 [15:28:42] indeed [15:28:49] decrease the interval, thats what I am going at [15:28:58] that's what I wrote in mail [15:29:09] (sorry, I was reading the title only haha) [15:29:50] that's ok [15:30:11] getting >200 emails per day, you won't feel inclined to read the contents [15:30:27] actually it's usefull sometimes to read it [15:30:36] yeah [15:30:40] those catchy ones [15:30:41] :) [15:30:43] I read [15:30:51] like viagra [15:30:52] spam [15:30:57] anyway [15:31:06] :) [15:31:11] we probably need to let labs settle down first [15:31:29] what you mean [15:31:42] then, when Ryan introduces stuff like changing the size of storage/RAM, we can fix this issue (somewhat) [15:33:24] no I don't mean it's possible to change size of storage etc [15:33:37] I meant to create a new templates for instance [15:33:59] yeah, something like that [15:34:15] but still it requires labs to change from beta -> stable [15:34:17] preferably each project should have allocated lvm in shared gluster and project members could allocate space as they need [15:34:37] it doesn't [15:35:13] this is a part of it :) [15:35:24] making it stable means make this happen [15:35:31] or set it up [15:36:39] I would rather say making labs stable require this to happen [15:36:55] ;) [15:37:15] okay, lost in the jungle [15:37:26] I mean there are many proposals made [15:37:30] we are still making new and new [15:37:42] part of making labs stable is to finish the proposed stuff [15:37:53] we are not waiting for labs to become stable so that we can start with that [15:38:06] we are making labs stable by actually working on it [15:38:51] okay...? [15:39:01] so I think we should create new templates immediately [15:39:03] I lost my compass :P [15:39:04] not to wait for anything else [15:40:07] yeah, but Ryan is still busy setting other stuff up [15:40:26] not really [15:40:28] that's just now [15:40:34] he works on labs as much as he can [15:40:41] usually he is available often [15:40:51] but now he's on a conference or something [15:40:51] * Hydriz recalls yesterday's debate [15:41:00] yes he's not in office now [15:42:14] my point was that if more volunteers were allowed to do stuff only ops can do now, he could be working on other important things and let the simple requests for other volunteers [15:42:28] like creating projects [15:43:09] now we have to bother him with every simple request while I believe there are many of us who actually now have lot of experiences to do this ourselve [15:44:44] * Hydriz gives petan|wk a chilled drink [15:44:45] btw, petan how does the depops group work? [15:45:24] I svn upped an extension before [15:45:44] and not being there, ended up impersonating hexmode :P [15:46:11] ok, see you guys tomorrow! [15:46:21] bye Hydriz [15:46:26] Hydriz: why chilled drink? :) [15:46:28] bye [15:47:10] Platonides: just fyi, Reedy is driving now so I can focus on CR. Maybe he can fix it so you won't need to impersonate [15:47:24] ok, I can fix it too [15:47:46] theoretically, I think I could have added myself to /etc/groups [15:47:52] indeed [15:47:56] that's a way to fix that ;) [15:48:06] preferably on dbdump [15:48:12] although I don't know if it's generated from ldap [15:48:14] because that's the instance you should use for that [15:48:22] no it's not generated from ldap [15:48:24] I was in dbdump [15:48:27] ok [15:48:28] "adduser user group" is better [15:48:37] but I don't think it works [15:48:38] with ldap [15:48:51] it should [15:48:55] hm [15:48:58] we could check it [15:49:07] <^demon> hexmode: Not for ldap users, use modify-ldap-users? [15:49:33] petan|wk: ^^ there ya go [15:49:36] ah [15:49:41] I don't think I am allowed to use it [15:50:41] ldap is restricted to ops, that's why we use /etc/groups [15:50:56] I mean, I can't change ldap groups, etc [15:51:04] neither add ldap users [15:52:40] Platonides: feel free to insert you in there [15:52:45] yourself [16:07:29] btw need any help with irc? [16:07:38] Platonides, I mean the rc [16:10:27] I don't know if someone actually worked on that yet [16:10:37] hexmode: that's another thing we might need Reedy to help with [16:11:14] petan|wk: you can ask him ;) [16:11:31] ok, once he's around I will [16:12:23] also I don't know if Platonides didn't already start working on that, so we should probably discuss that with him too [16:46:21] petan, I started a patched ircd daemon [16:46:41] but it didn't seem to work properly [16:46:51] ok [16:46:57] let me know if you need any help [16:47:06] I am a staffer on one irc network so I know how it work [16:51:08] Platonides: I will probably create a nat on squid to redirect 6667 to feed [22:05:59] Ryan_Lane: got a minute? [22:06:05] sure [22:06:37] IIRC yesterday you told me to ask you to rename me today [22:06:52] well, I said we should talk about it today, anyway [22:07:16] what's your current name, and what do you want to switch to? [22:07:35] Bob, Dave [22:07:37] :P [22:07:59] Ryan_Lane: vasilievvv -> vvv [22:18:54] !account-questions | vvv [22:18:54] vvv: I need the following info from you: 1. Your preferred wiki user name. This will also be your git username, so if you'd prefer this to be your real name, then provide your real name. 2. Your SVN account name, or your preferred shell account name, if you do not have SVN access. 3. Your preferred email address. [22:19:06] ignore #2 [22:19:17] make sure you pick what you want permanently, too [22:19:37] our policy in labs is "no renames - suck it up" :) [22:19:43] Aha [22:20:11] #3 is vasilvv@gmail.com [22:20:26] And #1 is my git commit name, right? [22:20:33] Then it is Victor Vasiliev [22:20:34] well, it's your git username [22:20:35] yes [22:20:43] it's also your labsconsole wiki name [22:20:54] So they are the same [22:20:56] All right [22:21:56] Ryan_Lane: can you create an account for Jerith [22:22:35] !account-questions | jerith [22:22:35] jerith: I need the following info from you: 1. Your preferred wiki user name. This will also be your git username, so if you'd prefer this to be your real name, then provide your real name. 2. Your SVN account name, or your preferred shell account name, if you do not have SVN access. 3. Your preferred email address. [22:22:50] ^^ the bot is the reason I want people to ask in here [22:23:06] Yay bots! [22:23:32] 1. jerith 2. jerith [22:24:22] 3. jeremy@praekelt.com for this, I think. [22:24:33] you'll be able to change your email address later, if you'd like [22:24:49] !initial-login | vvv [22:24:49] vvv: https://labsconsole.wikimedia.org/wiki/Access#Initial_log_in [22:25:56] My instinct is usually to use my personal email rather than my work email for things like this, but this is really a work project. :-) [22:25:56] !initial-login | jerith [22:25:56] jerith: https://labsconsole.wikimedia.org/wiki/Access#Initial_log_in [22:26:12] * Ryan_Lane nods [22:26:22] well, at any point, change it over [22:27:12] 01/27/2012 - 22:27:11 - Creating a home directory for vvv at /export/home/bastion/vvv [22:27:14] *yawn* [22:27:37] Anyone have an opinion on node.js outside of it being a pain to work with? [22:28:12] 01/27/2012 - 22:28:11 - Updating keys for vvv [22:29:12] 01/27/2012 - 22:29:11 - Updating keys for vvv [22:29:15] Damianz: I don't trust the hype, but I haven't actually used it for anything. [22:29:28] I think we are using it for parser work [22:31:12] 01/27/2012 - 22:31:12 - Creating a home directory for jerith at /export/home/bastion/jerith [22:31:13] Huh [22:31:27] Now I feel the difference between SSH to NL server and to US server [22:32:12] 01/27/2012 - 22:32:12 - Updating keys for jerith [22:32:14] I kinda like and kinda hate callbacks vs threads... really weird way to program deamons. [22:33:51] Ryan_Lane: what username/password do I use for gerrit? [22:34:05] everything web related is your wiki username [22:34:10] Same as you've just used for the other thing. [22:34:13] everything shell related is your shell name [22:35:19] Out of curiosity, how many people have different wiki and shell usernames? [22:35:38] a lot [22:35:54] look at the ops puppet [22:35:55] :D [22:36:01] *puppet git repo [22:36:35] or ldaplist -l passwd on an instance [22:37:09] cn is the wiki user name [22:37:16] uid is shell name [22:38:12] Cool. [22:38:21] * jerith has infrastructue envy now. [22:39:34] well, you can always help build it ;) [22:40:22] Ryan_Lane: The reason I have infrastructure envy is because I haven't had time to build this stuff at my current employer yet. [22:40:48] (I did about half the work setting up something similar at my last company.) [22:41:18] all of this is open sourc [22:41:23] including the puppet manifests [22:41:32] Oh great [22:41:35] I have logged in [22:41:45] Now I have to figure out what does that thing do [22:42:13] Still trying to wrap my head around puppet, it's on my todo list before I can roll out openstack as I'm not doing it without packagemanagement. Least I have a load of hardware on order for it :D [22:43:23] Ryan_Lane: why am I seeing "No Nova credentials found for your account." [22:43:50] Logout and back in probably [22:44:06] Ryan_Lane: It's mostly a matter of setting aside the time to do it. We hired an awesome devops guy (who starts in March) so it stops being bumped down the priority list. [22:44:56] jerith: can you try to login to labs? [22:44:58] well, if you make it openstack based, we can help each other out ;) [22:45:05] 01/27/2012 - 22:45:05 - Creating a home directory for jerith at /export/home/mobile-sms/jerith [22:45:12] preilly: because there is a bug in mediawiki I haven't tracked down [22:45:20] Ryan_Lane: okay [22:45:20] preilly: what did you do to get that error? [22:45:25] something killed your session [22:45:34] Ryan_Lane: went to labs console [22:45:46] Tbf most of mediawiki is bugs :P [22:46:02] preilly: heh. yeah. I know that part ;) [22:46:05] 01/27/2012 - 22:46:04 - Updating keys for jerith [22:46:18] preilly: I'm logged in. [22:46:20] I don't see any errors in the debug log [22:46:39] preilly: log out and log back in and it'll start working again [22:46:53] this is a bug with an interaction between ldapauth plugin and mediawiki [22:47:02] and I trace a bunch of these down every mediawiki release [22:47:11] jerith: can you login to vumi-gw1? [22:47:48] preilly: I can. \o/ [22:48:01] ah. I see an erorr [22:49:06] jerith: do you think you could install vumi on this box? [22:49:22] jerith: also, are you going to be at global mobile congress [22:49:35] man, these bugs are infuriating to track down [22:50:00] preilly: I can be, if you can convince my boss I should be. :-) [22:50:23] (Probably not, unless there's a reason for me to go.) [22:50:33] But Gustav will be there. [22:50:57] preilly: You should come visit us in Cape Town, though. [22:52:04] jerith: yeah! [22:52:15] jerith: well, I wanted to demo vumi at the conference [22:52:33] We usually install to /var/praekelt/vumi/ but that's flexible. [22:52:59] Where do you guys usually put stuff like this? [22:53:20] jerith: that is fine by me [22:54:27] We theoretically have some puppet setup thingies, but I don't think they're current. [22:55:08] On the subject of puppet I'm going to go play with my nice shiny new install server now I finished sticking the new vlan tags out onto all the switches. [22:55:46] So I'll do the cowboy install-from-github-by-hand. [22:59:24] Password logins are disabled on labs machines, right? [22:59:32] Yeah [22:59:38] Just sudo takes password [23:00:01] Cool. So I don't need to worry about a strong password for my vumi user. [23:00:22] If it's a bot user I generally just lock the password anyway [23:08:58] jerith: good passwords are good practice ! [23:10:08] LeslieCarr: This password needs to be disabled, because nobody should be logging in as this user except via sudo. [23:10:46] And my sysadmin skills are very rusty, so I'll worry about that a bit later. [23:15:49] Locked. Yay manpages. :-) [23:16:08] lol [23:16:11] :) [23:16:26] life -l jerith [23:19:06] preilly: screen seems to be unhappy. Did you do something to it? [23:19:21] jerith: give me a second [23:20:27] * jerith only has another half hour or so before he has to sleep, but that should be plenty long enough to get stuff installed. [23:25:59] jerith: try it again [23:27:44] Done. [23:28:24] jerith: did you CTRL-A :multiuser on and :acladd preilly? [23:28:31] I did. [23:29:05] what is it called? [23:29:12] "jerith" [23:29:34] jerith@vumi-gw1:~$ screen -ls [23:29:34] There is a screen on: 15008.jerith (01/27/2012 11:26:08 PM) (Multi, attached) [23:29:37] 1 Socket in /var/run/screen/S-jerith. [23:32:17] Ooh! vim's all green. :-D [23:37:05] Yeah :( [23:37:09] Silly ubuntu [23:38:08] * jerith remembers that he actually has to install some packages. [23:49:55] hi Ryan_Lane [23:50:03] did you check some of my emails :) [23:50:07] howdy [23:50:20] ah [23:50:24] did :) [23:50:59] Ryan_Lane: vd's are dynamic now? [23:52:12] btw tiny with 1.5gb storage is really hard to use :) [23:52:24] actually I think 512 ram is ok for most of services [23:52:25]