[02:01:01] ok, LevelUp office hour starting now in #wikimedia-dev [02:31:02] [bz] (ASSIGNED - created by: Arthur Richards, priority: Normal - normal) [Bug 40605] Supporting MobileFrontend on beta labs - https://bugzilla.wikimedia.org/show_bug.cgi?id=40605 [02:32:38] [bz] (NEW - created by: Michelle Grover, priority: Unprioritized - normal) [Bug 46620] Automatic mobile redirects for MobileFrontend on Betalabs - https://bugzilla.wikimedia.org/show_bug.cgi?id=46620 [02:34:41] [bz] (NEW - created by: Michelle Grover, priority: Unprioritized - normal) [Bug 46621] SSL Support on betalabs - https://bugzilla.wikimedia.org/show_bug.cgi?id=46621 [12:41:52] I tried to create some instances for the oauth project, and they all show "instance state" as "error". So then I tried to delete one of them, and it seems to be gone except that Special:NovaInstance still lists it and still says the state is "error". What did I do wrong? How can I fix it? [12:50:56] hashar - ^ Any idea? [12:51:06] sure I always have ideas [12:51:24] ah [12:51:38] anomie: can't really help there, something failed on nova side and I don't have access to the logs [12:51:47] nor can I delete the faulty instance :( [12:51:53] So who do I bug about that? [12:52:59] the regular roots [12:53:04] Coren Ryan_Lane or paravoid [12:53:19] Oy! [12:54:04] I'm actually a highly irregular root. [12:54:53] And I have no magic bits in Nova either so I can't help yet. :-( [12:55:45] Coren- :( Thanks anyway. I now wait for paravoid or Ryan_Lane [14:40:55] Hm. Long weekend for most christian societies. [14:51:24] aahh [14:51:30] I forgot about the 3 days weekend [15:07:37] anomie, still having trouble with creating instances? [15:08:28] andrewbogott- Probably. I haven't tried anything since my message this morning. [15:08:52] due to the weirdness of my irc bouncer I couldn't tell how hold those messages were :) Let me look at some log files. [15:23:42] Coren, looks like WMF does not officially observe this holiday. Some folks will probably take a vacation day though. [15:24:30] andrewbogott: Not that it'd matter to me since I'm a contractor. :-) I mostly meant it in the "volunteers will probably extra active during this period" sense. [15:24:41] yep, true [15:34:57] [bz] (ASSIGNED - created by: Jan Luca, priority: Normal - enhancement) [Bug 46460] Allow tools to create databases - https://bugzilla.wikimedia.org/show_bug.cgi?id=46460 [15:44:33] !log tools reboot (still unactivated) tools-shadow [15:44:36] Logged the message, Master [15:50:50] Ryan_Lane, did you do any kind of tinkering with openstack packages on brewster? apt is full of complaints on virt2 [15:57:57] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Needed Toolserver features was modified, changed by MPelletier (WMF) link https://www.mediawiki.org/w/index.php?diff=665547 edit summary: [+466] /* End-user-support */ updates [15:58:49] andrewbogott: we don't use custom packages [15:59:00] that's what I thought. [15:59:05] Change on 12mediawiki a page Wikimedia Labs/Tool Labs/Needed Toolserver features was modified, changed by MPelletier (WMF) link https://www.mediawiki.org/w/index.php?diff=665548 edit summary: [+284] /* Render */ update [15:59:08] something… interesting is happening on virt2 [15:59:13] what's it complaining about? [15:59:52] well… [15:59:54] ls /var/lib/nova/instances [15:59:54] ls: cannot access /var/lib/nova/instances: Transport endpoint is not connected [16:00:02] that's normal [16:00:06] really? [16:00:24] yes [16:00:36] fixed now [16:00:48] we had gluster on the old nodes [16:00:56] virt2 has nova-compute disabled [16:01:14] /var/lib/nova/instances was still in /etc/fstab [16:01:19] Puppet is saying things like this: http://dpaste.org/yVLtw/ [16:01:20] I just unmounted and deleted it [16:01:30] And apt-get -f install was failing due to that weird file [16:01:37] hm [16:01:44] whys it trying to install compute? [16:03:09] !ping [16:03:09] pong [16:03:20] Packaging on virt2 may be unrelated to the actual problem I'm trying to solve (which is that new instances won't build; that looks like some kind of amqp failure) [16:03:41] * petan is becoming GTK guru [16:04:02] I told it to explicitly install all of the packages [16:04:41] Is that apt-get -f install, or something else? [16:04:48] no [16:04:55] apt-get install nova-api nova-common nova-compute nova-compute-kvm nova-network python-nova [16:05:05] oh, ok. [16:05:38] I'm going to purge nova-compute-kvm and nova-compute [16:06:29] * andrewbogott stands back, covers ears [16:06:37] it's not needed on that instance [16:06:40] err [16:06:42] on that host [16:06:50] virt2 just does api and network [16:07:04] virt 5-11 do compute [16:07:21] eventually we'll do network node per compute node and will put the api back on the controller [16:07:28] and virt2 will be free [16:09:06] andrewbogott: seems that fixed it [16:09:15] I wonder how the packages got into some weird state [16:09:53] OK, lemme see if that fixed instance creation for some reason [16:11:26] it did [16:11:29] I just tested it :) [16:11:38] cool! [16:11:45] How did your talk go yesterday? Any new recruits? [16:12:15] it's today :) [16:12:20] we'll see [16:12:41] I have no idea how many people show up. it's always kind of hit or miss at universities [16:12:50] ah, that makes sense; I thought you were cutting it pretty close last night [16:13:37] anomie: You should be able to delete and create instances now; let me know if you run into more problems. [16:13:51] andrewbogott- ok, will try it in a few minutes [16:16:55] yeah. I still have to work on my presentation today, too [16:17:26] andrewbogott: it looks to me like someone tried to do a dist-upgrade and it failed [16:30:03] Ryan_Lane: Pretty sure I didn't… is there anything in bash history? [16:30:22] * andrewbogott looks [16:32:37] Yep, there it is. [16:43:31] It's not me! [16:43:57] may have been me [16:44:00] * Ryan_Lane shrugs [16:44:10] either way, it's fixed and we tracked down the cause [17:30:13] [bz] (NEW - created by: silke.meyer, priority: Normal - normal) [Bug 45483] Make instance creation failures more verbose - https://bugzilla.wikimedia.org/show_bug.cgi?id=45483 [17:53:46] Is there a puppet class to install mysql listening for connections from another instance? [19:08:34] [bz] (NEW - created by: Željko Filipin, priority: Unprioritized - normal) [Bug 46649] Link to login page broken at en.m.wikipedia.beta.wmflabs.org - https://bugzilla.wikimedia.org/show_bug.cgi?id=46649 [19:13:23] hi all [19:13:30] everything works? [19:13:54] btw Coren GTK isn't that bad, at least it's fast and eats low ram :) that's what I like more than fancy skin [19:19:13] <^demon> Ryan_Lane: Can you review a couple of gerrit things for me? [19:19:45] <^demon> s/couple/4/ [19:20:14] <^demon> Whoops....s/4/5/ :\ [19:44:20] ^demon: sure [19:44:34] petan: should something not be working? :) [19:47:48] Ryan_Lane I don't know that's why I ask :P [19:47:58] !ping [19:47:58] pong [19:48:03] at least this is working :D [19:48:14] wm-bot responding is a good sign... [19:48:29] petan: did something break? [19:49:35] not sure why the default thought wouldn't be "everything is ok" :D [20:15:02] Ryan_Lane I was just asking so that I know all people in bots project etc are not having troubles :P [20:15:07] :) [20:15:48] @seen sumanah [20:15:48] petan: Last time I saw sumanah they were quitting the network with reason: Quit: Leaving N/A at 3/28/2013 3:03:47 AM (17:12:01.0360890 ago) [20:16:36] is it more proper to use "they" or "he/she" [20:16:57] english has no universal gender-neutral version [20:26:54] Is there a puppet group to install mysql listening for connections from another instance? role::labs-mysql-server only listens on localhost, and most of this stuff I can't even find in puppet to see what it might do. [20:30:37] anomie: hm. that role should probably be modifiable with a variable [20:30:42] if it currently isn't [20:34:32] Ryan_Lane- Underneath the checkbox line for that role are two text fields labeled "db_cluster" and "writable". Neither of which I even see in role::labs-mysql-server, or generic::mysql::server. [20:35:01] we really need to clean up the configuration page [20:35:20] anomie: I meant the role class likely needs to be modified to allow it [20:38:15] Ryan_Lane- I'd help, if I knew anything about it. [20:50:02] [bz] (NEW - created by: Andrew Bogott, priority: Unprioritized - normal) [Bug 46656] Allow service users to be added and removed from sudo policies - https://bugzilla.wikimedia.org/show_bug.cgi?id=46656 [22:22:34] md_5|holiday [22:35:44] [bz] (REOPENED - created by: Antoine "hashar" Musso, priority: High - major) [Bug 41104] glusterfs log files are not rotated - https://bugzilla.wikimedia.org/show_bug.cgi?id=41104 [22:40:16] <^demon> Ryan_Lane: https://gerrit.wikimedia.org/r/#/c/53759/, https://gerrit.wikimedia.org/r/#/c/54989/, https://gerrit.wikimedia.org/r/#/c/55057/, https://gerrit.wikimedia.org/r/#/c/55259/, https://gerrit.wikimedia.org/r/#/c/56485/ [22:53:05] ^demon: you should really add me as a reviewer to these ;) [22:53:36] <^demon> Last I remembered, your dashboard was pretty full :p [22:53:43] <^demon> So I figure pinging you on IRC is easier :) [23:02:43] yeah, but both is likely best ) [23:02:44] ;) [23:02:56] just in case I happen to close those tabs [23:03:00] otherwise I need to find you [23:06:46] !log cvn Installing subversion from apt on cvn-app1 (for pywikipedia) [23:06:48] Logged the message, Master [23:11:46] Ryan_Lane: http://cl.ly/image/3y1P1F3P3z3X [23:11:56] "Agree and create account", uh ? [23:12:15] bleh [23:12:28] we use the same damn message for the button and the link [23:12:33] I forgot to switch it back [23:12:41] that's a localization bug [23:13:06] local wikitech edit [23:13:07] by you [23:13:16] yes [23:13:30] and I was trying to change the button on the account creation page [23:13:35] k [23:13:44] btw, may want to delete the page so it doesn't shadow the native [23:13:49] it uses the same messag [23:13:51] *message [23:13:53] I figured [23:13:53] that's a bug [23:14:05] How is that a bug [23:14:08] which is? [23:14:19] that is uses the same message for the link and the button on the creation page [23:14:25] they do two different things [23:14:38] one message brings you to the form, the other submits the form [23:14:46] two completely separate actions [23:14:58] It's set back to normal now [23:15:21] Oh, that, yes that is a bug. I saw that edit already. [23:15:54] then I said, you may want to delete the page so it doesn't shadow the english translation of mw [23:16:00] I did :) [23:16:06] just now [23:16:09] I had forgotten earlier [23:16:13] it hasn't changed in years, but you never know :) [23:16:15] k [23:16:16] got distracted when I went for it [23:16:17] thx [23:16:20] yw [23:21:01] <^demon> Ryan_Lane: Do you wanna just wait for tomorrow morning? We can knock this out first thing... [23:21:10] yep [23:21:16] I'm giving a talk in like 40 minutes [23:21:27] <^demon> Ok, that's cool. Just ping me in the morning when you're free. [23:40:46] !log cvn Installing mysql-server and mysql-client from apt on cvn-app1 (for cvnclerkbot.py) [23:40:48] Logged the message, Master