[00:06:00] is Adam Miller still employed by the foundation? [00:07:47] i don't know who that is (to be fair, i don't know a lot of employees) [00:08:42] Usability Initiative developer [00:08:48] http://www.mediawiki.org/wiki/User:Adammiller [00:08:55] doesn't look like he's ever edited [00:09:52] I don't see him at http://wikimediafoundation.org/wiki/Staff_and_contractors [00:10:01] hehe that's wher ei was checking [00:10:42] some staff aren't on there, but I don't think he's on any staff lists elsewhere [00:10:58] I think they _should_ appear there [00:11:16] there's also https://meta.wikimedia.org/wiki/Wikimedia_Foundation_contractors [00:12:11] maybe you should just send an email "Are you still employed by WMF?" to the wikimedia.org address and see if it bounces :P [00:12:19] oh i didn't know that existed [00:12:49] that page in meta was started because http://wikimediafoundation.org/wiki/Staff didn't list contractors [00:12:57] which are an important part of wmf employees [00:13:29] http://wikimediafoundation.org/wiki/Staff was then (years later) changed to http://wikimediafoundation.org/wiki/Staff_and_contractors [00:13:39] so I'm not sure there's a reason to keep it, though [00:21:45] No he isn't [00:23:14] No, Adam isn't with WMF any more [00:23:17] Hasn't been for a while [00:30:09] PROBLEM - Host ms-be5 is DOWN: PING CRITICAL - Packet loss = 100% [00:35:42] RECOVERY - Host ms-be5 is UP: PING OK - Packet loss = 0%, RTA = 1.09 ms [00:37:27] !log restarting exim4 on mchenry with split_spool_directory = true [00:37:32] Logged the message, Mistress of the network gear. [00:48:37] PROBLEM - Host ms-be5 is DOWN: PING CRITICAL - Packet loss = 100% [00:54:09] RECOVERY - Host ms-be5 is UP: PING OK - Packet loss = 0%, RTA = 0.21 ms [00:54:58] damn, i know why that's not reducing the backlog [00:55:01] it only matters for new messages [01:03:59] PROBLEM - Packetloss_Average on locke is CRITICAL: CRITICAL: packet_loss_average is 18.6763888889 (gt 8.0) [01:13:35] RECOVERY - Packetloss_Average on locke is OK: OK: packet_loss_average is 0.2422868 [01:41:56] PROBLEM - MySQL Slave Delay on db1025 is CRITICAL: CRIT replication delay 280 seconds [01:42:23] PROBLEM - MySQL Slave Delay on storage3 is CRITICAL: CRIT replication delay 244 seconds [01:48:59] PROBLEM - Misc_Db_Lag on storage3 is CRITICAL: CHECK MySQL REPLICATION - lag - CRITICAL - Seconds_Behind_Master : 639s [01:49:17] RECOVERY - MySQL Slave Delay on db1025 is OK: OK replication delay 1 seconds [01:51:50] RECOVERY - Misc_Db_Lag on storage3 is OK: CHECK MySQL REPLICATION - lag - OK - Seconds_Behind_Master : 4s [01:52:35] RECOVERY - MySQL Slave Delay on storage3 is OK: OK replication delay 18 seconds [02:12:59] PROBLEM - Puppet freshness on ocg3 is CRITICAL: Puppet has not run in the last 10 hours [04:36:05] New patchset: Faidon; "partman: fix boot for Ciscos" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12565 [04:36:38] New review: Faidon; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/12565 [04:36:38] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/12565 [04:37:41] Change merged: Faidon; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12565 [04:45:06] PROBLEM - Puppet freshness on ms-be5 is CRITICAL: Puppet has not run in the last 10 hours [04:46:09] PROBLEM - Puppet freshness on nfs2 is CRITICAL: Puppet has not run in the last 10 hours [04:49:09] PROBLEM - Puppet freshness on nfs1 is CRITICAL: Puppet has not run in the last 10 hours [05:57:32] up at 7:30 am? I mean, so was I... but that was after sleeping all night [06:02:40] aw [06:03:02] sleep is good. [06:22:04] New review: Dereckson; "I still need to add aliases." [operations/mediawiki-config] (master) C: -1; - https://gerrit.wikimedia.org/r/12556 [06:24:18] PROBLEM - Host lvs1001 is DOWN: PING CRITICAL - Packet loss = 100% [06:24:27] PROBLEM - BGP status on cr1-eqiad is CRITICAL: CRITICAL: host 208.80.154.196, sessions up: 9, down: 1, shutdown: 0BRPeering with AS64600 not established - BR [07:35:56] PROBLEM - Puppet freshness on db1047 is CRITICAL: Puppet has not run in the last 10 hours [07:44:02] PROBLEM - Puppet freshness on db42 is CRITICAL: Puppet has not run in the last 10 hours [07:44:13] $#$%$@!%$!$%@!$% PARTMAN [07:44:23] what happened with lvs1001? [07:44:28] anyone investigating? [07:45:17] eh? [07:45:31] crap it did not page me and I was busy typing (code) [07:45:53] that really can'tbe true or we would have a huuuuge numebr of complaints [07:47:09] we assume that the cr1 whine is the real issue I guess? [07:56:40] going to powercycle it I guess [07:57:16] seems like the cr1 whine is a symptom not a cause [07:59:01] !log powercycled lvs1001, not pingable, nothing good from mgmt console, etc. [07:59:08] Logged the message, Master [08:01:50] yes, cr1 is complaining that it lost a bgp session [08:01:52] with lvs1001 [08:02:16] host is up again [08:02:21] also, we didn't get any other alerts, so either 1001 was already secondary or it wa primary and cr1 fell back to its backup [08:02:28] wasn't worried much :) [08:02:47] RECOVERY - Host lvs1001 is UP: PING OK - Packet loss = 0%, RTA = 26.36 ms [08:03:09] yeah, I guess that there was a failover (after looking at the config) [08:03:23] RECOVERY - BGP status on cr1-eqiad is OK: OK: host 208.80.154.196, sessions up: 10, down: 0, shutdown: 0 [08:03:47] and there we go [08:10:29] :) [08:10:30] thanks [08:10:34] I'm so deep in partman shit right now [08:10:44] going through logs & source [08:12:14] source? sounds bad already [08:15:59] PROBLEM - Puppet freshness on maerlant is CRITICAL: Puppet has not run in the last 10 hours [08:25:42] New patchset: Hashar; "labs use the same wgCentralDBname on all wiki" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12566 [08:25:49] New review: jenkins-bot; "Build Successful " [operations/mediawiki-config] (master); V: 1 C: 0; - https://gerrit.wikimedia.org/r/12566 [08:26:31] New review: Hashar; "(no comment)" [operations/mediawiki-config] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/12566 [08:26:34] Change merged: Hashar; [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12566 [08:30:51] New review: Hashar; "Deployed on live site" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12566 [08:39:14] New patchset: Hashar; "Disable wgNoticeInfrastructure on 'beta' cluster" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12568 [08:39:20] New review: jenkins-bot; "Build Successful " [operations/mediawiki-config] (master); V: 1 C: 0; - https://gerrit.wikimedia.org/r/12568 [08:39:33] New review: Hashar; "(no comment)" [operations/mediawiki-config] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/12568 [08:39:35] Change merged: Hashar; [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12568 [08:40:34] New review: Hashar; "Deployed on live site." [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12568 [08:48:06] New patchset: Hashar; "Load transcode conf on -e /etc/wikimedia-transcoding" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12569 [08:48:12] New review: jenkins-bot; "Build Successful " [operations/mediawiki-config] (master); V: 1 C: 0; - https://gerrit.wikimedia.org/r/12569 [08:48:25] New review: Hashar; "(no comment)" [operations/mediawiki-config] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/12569 [08:48:31] Change merged: Hashar; [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12569 [08:49:20] New review: Hashar; "deployed on live site." [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12569 [08:53:03] New patchset: Faidon; "partman: more fixes for Cisco" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12570 [08:53:36] New review: Faidon; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/12570 [08:53:36] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/12570 [08:53:45] Change merged: Faidon; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12570 [09:15:16] mark: ping? [09:15:54] pong? [09:15:56] hi [09:15:58] i'm about to leave to the datacenter [09:16:27] so, I've spent like an hour or two thinking that I had a third bug with the d-i for virt[678] [09:16:35] because it didn't ping after installing [09:16:50] I just learned that we're dropping that traffic from prod [09:17:05] I'm fine with that, but could we make it to reply icmp prohibited instead? [09:17:21] silently dropping traffic is unintuitive [09:17:49] PROBLEM - Puppet freshness on db29 is CRITICAL: Puppet has not run in the last 10 hours [09:18:57] we don't drop silently [09:19:07] unless someone changed it [09:19:12] I set it up as admin prohibited [09:19:55] i'll look at it later [09:19:58] and also arrange your access ;) [09:20:06] :-) [09:20:08] leaving now or it'll get very late [09:20:09] bye [09:20:10] ! [09:20:18] bye & thanks [09:30:43] New patchset: Faidon; "partman: do not warn for not having a swap" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12572 [09:31:05] Ryan_Lane: okay, gerrit question for you [09:31:15] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/12572 [09:31:56] ? [09:32:14] commit 9e18e152096641ba79c5fba26d587a74941f1e0d ebcc2ddc112be4e83b1e81f341e745cc45d0a175 728f1f1dc47d1524dbb933a07b08e76f2148a5b8 [09:32:17] commit 728f1f1dc47d1524dbb933a07b08e76f2148a5b8 ebd1c8e20750f455498ef0c179d08a6596b8261b [09:32:21] commit ebcc2ddc112be4e83b1e81f341e745cc45d0a175 2e3d4d2e0ce79e0a185aebc7e30d986434dfb1c9 ebd1c8e20750f455498ef0c179d08a6596b8261b [09:32:24] why the hell did it do a merge? [09:32:30] New review: Faidon; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/12572 [09:32:33] Change merged: Faidon; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12572 [09:32:48] I commited 728f1f1d right on top ebd1c8e [09:32:58] submitted to gerrit which did a merge of those two [09:33:03] that's just crazy. [09:34:17] hmmmm [09:34:25] it didn't do it with the latest commit [09:34:45] the only difference was that last time I did a +2, waited for gerrit2 to v: +1 and hit "submit" [09:34:53] so review & submit is different from review + submit? [09:36:49] so you had the old item as +2/+1 when you review/submitted the new commit? [09:38:14] New review: Jens Ohlig; "I find the functional style readable and elegant, but it really makes things a lot harder to debug f..." [operations/puppet] (production) C: 1; - https://gerrit.wikimedia.org/r/8344 [09:40:21] I don't understaand the question [09:41:26] that means I don't understand your question :-D [09:42:48] you had two commits that you pushed for review, and then what exactly? [09:49:00] they were separate commits that were merged independently [09:49:12] the git log on the first is a mess [09:49:20] anyway, got to run to catch the bank open [09:49:23] ok [09:49:26] see ya later [10:31:56] !log installing security upgrades on formey (gerrit) [10:32:01] Logged the message, Master [10:41:48] !log installing security upgrades on fenari [10:41:53] Logged the message, Master [10:42:48] !log fenari upgrade - this included replace wikimedia-lvs-realserver 0.04 (using .../wikimedia-lvs-realserver_0.08 [10:42:53] Logged the message, Master [10:46:34] !log installing security upgrades and kernel on bast1001 (still needs reboot, but dont break user sessions) [10:46:39] Logged the message, Master [11:33:30] New patchset: Hashar; "detect cluster with /etc/wikimedia-realm" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12583 [11:33:36] New review: jenkins-bot; "Build Successful " [operations/mediawiki-config] (master); V: 1 C: 0; - https://gerrit.wikimedia.org/r/12583 [11:44:33] !log installing upgrades and kernel on pdf1, can reboot? (also needs puppetizing and precise reinstall) [11:44:38] Logged the message, Master [12:04:57] New patchset: Hashar; "(bug 37700) update stewardwiki logo & favicon" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/11943 [12:05:06] New review: jenkins-bot; "Build Successful " [operations/mediawiki-config] (master); V: 1 C: 0; - https://gerrit.wikimedia.org/r/11943 [12:05:41] New review: Hashar; "Updated commit message and rebased." [operations/mediawiki-config] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/11943 [12:06:02] New review: Hashar; "(no comment)" [operations/mediawiki-config] (master); V: 1 C: 2; - https://gerrit.wikimedia.org/r/11943 [12:06:05] Change merged: Hashar; [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/11943 [12:07:02] New review: Hashar; "deployed on live site" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/11943 [12:14:03] PROBLEM - Puppet freshness on ocg3 is CRITICAL: Puppet has not run in the last 10 hours [12:23:47] New patchset: Hashar; "enhance account throttling" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12185 [12:23:53] New review: jenkins-bot; "Build Failed " [operations/mediawiki-config] (master); V: -1 C: 0; - https://gerrit.wikimedia.org/r/12185 [12:24:25] New review: Hashar; "Patchset 2 fix issues mentioned in the inline diff of patchset 1." [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12185 [12:46:14] mark: so, we just tested the ircd package I made [12:46:17] it works [12:47:48] on a side note, we're officially out of public IPs in labs [12:58:10] New patchset: Hashar; "enhance account throttling" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12185 [12:58:16] New review: jenkins-bot; "Build Failed " [operations/mediawiki-config] (master); V: -1 C: 0; - https://gerrit.wikimedia.org/r/12185 [13:01:30] New review: Hashar; "Patchset3 just rewrite most of the original patch and code :-D" [operations/mediawiki-config] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/12185 [13:04:02] New patchset: Hashar; "enhance account throttling" [operations/mediawiki-config] (master) - https://gerrit.wikimedia.org/r/12185 [13:04:08] New review: jenkins-bot; "Build Successful " [operations/mediawiki-config] (master); V: 1 C: 0; - https://gerrit.wikimedia.org/r/12185 [13:04:51] New review: Hashar; "Patchset 4 is a rebase to latest master. We had a change to raise throttle on enwiki : https://gerr..." [operations/mediawiki-config] (master); V: 0 C: 0; - https://gerrit.wikimedia.org/r/12185 [13:55:52] New patchset: Ryan Lane; "Ensure the apparmor profile is added before mysql is reconfigured." [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12590 [13:56:23] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/12590 [14:02:06] New review: Ryan Lane; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/12590 [14:02:08] Change merged: Ryan Lane; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12590 [14:27:53] hey guys, nrpe question da room [14:28:08] if a new check file is created in /etc/nagios/nrpe.d [14:28:17] does nagios-nrpe-server need to be reloaded in order to see it? [14:28:21] (I assume yes) [14:28:26] I ask, because currently puppet does not do this [14:28:29] should it? [14:29:31] ottomata: yes it needs [14:29:43] reload :o [14:29:51] ok, i will make it so in puppet, subscribing the service [14:30:00] that would be cool [14:30:08] because right now puppet has troubles with this, on labs [14:30:19] sometimes I need to reload nrpe by hand [14:30:34] New review: Ryan Lane; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/12387 [14:30:37] Change merged: Ryan Lane; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12387 [14:31:41] oh i take it back! [14:31:44] it is supposed to already do this [14:31:44] notify => Service["nagios-nrpe-server"] [14:31:45] hmm [14:32:59] is there a way I can ask nrpe what a check is currently returning? [14:34:52] ottomata: there's a 'returns' resource (proper term?) which you can act on [14:35:04] hmm [14:35:06] i mean manually [14:35:10] oh ha [14:35:12] something like [14:35:27] nagios-check —name check_udp2log_log_age-lucene [14:35:32] and have nrpe run it [14:35:43] rather than me run the command manually [14:35:45] that I don't know [14:35:47] ok [14:35:49] well hmm [14:35:50] so [14:35:58] yesterday notpeter added a new udp2log instance [14:36:05] and it has a check to make sure the log files aren't old [14:36:05] http://nagios.wikimedia.org/nagios/cgi-bin/extinfo.cgi?type=2&host=oxygen&service=udp2log+log+age+for+lucene [14:36:15] trying to understand why it says this check is not defined [14:36:30] it exists in the /etc/nagios/nrpe.d directory [14:36:43] otto@oxygen:/etc/nagios/nrpe.d$ cat check_udp2log_log_age-lucene.cfg [14:36:43] command[check_udp2log_log_age-lucene]=/usr/lib/nagios/plugins/check_udp2log_log_age lucene [14:36:56] and running that command manually works [14:37:36] New patchset: Ryan Lane; "Allow the labs mysql role to be more configurable" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12592 [14:38:10] New review: gerrit2; "Change did not pass lint check. You will need to send an amended patchset for this (see: https://lab..." [operations/puppet] (production); V: -1 - https://gerrit.wikimedia.org/r/12592 [14:38:38] hey Ryan_Lane, why not pass the $mysql_datadir as a class parameter [14:38:49] instead of resolving the variable in global scope? [14:39:19] because I can't call parameterized classes from labs [14:39:23] ohhhhhh [14:39:25] righto [14:39:26] cool [14:41:10] can I seriously not do: if !$::mysql_datadir { [14:41:10] ? [14:41:19] but if !$mysql_datadir { is allowed? [14:42:13] New patchset: Ryan Lane; "Allow the labs mysql role to be more configurable" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12592 [14:42:20] if that's true, I'm going to be really annoyed [14:42:40] spoken like a true puppet user [14:42:44] New review: gerrit2; "Change did not pass lint check. You will need to send an amended patchset for this (see: https://lab..." [operations/puppet] (production); V: -1 - https://gerrit.wikimedia.org/r/12592 [14:42:48] maybe not [14:43:10] oh [14:43:11] I'm dumb [14:43:37] New patchset: Ryan Lane; "Allow the labs mysql role to be more configurable" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12592 [14:44:10] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/12592 [14:44:33] New review: Ryan Lane; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/12592 [14:44:36] Change merged: Ryan Lane; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/12592 [14:45:50] PROBLEM - Puppet freshness on ms-be5 is CRITICAL: Puppet has not run in the last 10 hours [14:46:44] PROBLEM - Puppet freshness on nfs2 is CRITICAL: Puppet has not run in the last 10 hours [14:47:12] Ryan_Lane, not that it really matters, but it might look cleaner to use conditional assignment instead of an if/else block for those [14:47:40] datadir => $::mysql_datadir ? { [14:47:40] false => "/mnt/mysql", [14:47:40] default => $::mysql_datadir [14:47:40] } [14:47:51] that works? [14:47:55] yeah [14:48:02] ah. cool [14:48:03] you can even do that in the class inclusion, instead of setting a temp variable [14:48:07] was going to do that priginally [14:48:51] something like that [14:48:52] https://gist.github.com/2973201 [14:48:52] <^demon> drdee: Ping. [14:48:54] but yeah, doesn't really mater [14:49:04] ^demon: pong [14:49:24] <^demon> Hey :) I was trying to setup webstatscollector yesterday, but I couldn't find the existing code in SVN. [14:49:27] Jeff_Green: ping! [14:49:28] hehe [14:49:35] ^demon: hold on [14:49:44] PROBLEM - Puppet freshness on nfs1 is CRITICAL: Puppet has not run in the last 10 hours [14:50:03] http://svn.mediawiki.org/viewvc/mediawiki/trunk/webstatscollector/ [14:50:19] so I was giving access to evan rosen, a new global dev to some db machines [14:50:23] <^demon> Ah, I was looking in trunk/tools. Thanks :) [14:50:31] notpeter just informed me that the default wikidev group is not available on all machines [14:50:44] and he suggested I ask you (Jeff_Green) what to do about it [14:52:03] ottomata: hm. if this is being included via the node config, it's global, right? [14:52:21] because my change didn't work when I ran it. heh [14:52:36] oh. hell, it's not set [14:52:39] the var? [14:52:45] I didn't set the var [14:52:54] aye [14:53:00] and actually, i don't know much about the $:: syntax [14:53:01] oh. something's broken [14:53:04] i've only used that here [14:53:08] i think if you define it like this in your node [14:53:12] $mysql_datadir = '...' [14:53:14] project puppet groups are broken [14:53:19] then it is local? [14:53:38] maybe $:: only qualifies things that are declared outside of a class? [14:53:39] (not sure) [14:53:42] sorry I was afk for a minute there [14:53:46] np [14:54:08] well, it doesn't matter, because it's something broken in labs anyway [14:54:11] hehe, aye [14:54:35] does this make any sense (nowadays or ever): puppet: systemuser ... groups => [ 'project-foo' ] } ? [14:54:42] ooohhhh [14:54:45] ottomata: I don't know the history of the wikidev group, my impression is that it's a legacy and a throwawy in terms of security [14:55:00] it seems setting 0 doesn't work? [14:55:04] aye, i don't really care much I think, but admins.pp sets it as the default group for new users [14:55:16] I wonder how my logic is fucked up for that one [14:55:25] and we do use wikidev on stat servers, to allow for all of us to access data we are working on in /a (and other) places [14:55:58] is it missing from one of the "class admins::..." blocks? [14:56:30] i see it all there [14:56:41] but this is a new users, and maybe not in one of the admins groups? [14:56:49] also, this is on one of the db servers [14:56:50] umm [14:56:52] which server? [14:57:04] db42 and db1047 [14:57:16] not sure which puppet is complaining on, notpeter knows [14:57:24] garg puppet is blinding me [14:57:30] hehe [14:57:33] ottomata: once you have time: there is a new issue in RT-3180 (cant find libcairo), and 3119 (shell for Evan) in RT-3119. fyi and no worries [14:57:54] ottomata: save me some pain--how is the user being added to db1047 for example? [14:58:01] hm, i Know about libcairo, was waiting til someone cared to fix it. that is a problem with precise [14:58:04] can you use true/false in the config, rather than 1/0? [14:58:05] # RT 3119 [14:58:05] if $hostname == "db1047" { [14:58:05] include accounts::erosen [14:58:05] } [14:58:14] hmmm [14:58:14] o_0 [14:58:20] ottomata: but that did not work on db1047 and db42. Could not find dependency Group[500] for User[erosen] [14:58:21] no sure Ryan_Lane, would have to try it with mysql [14:58:25] not sure if my.cnf likes that [14:58:35] could make the my.cnf.erb file smarter though [14:58:48] holy cow there have been a ton of changes since my last git pull [14:58:51] <%= innodb_file_per_table ? 1 : 0 %> [14:59:09] file_per_table is my doing [14:59:10] mutante, right, I am talking to Jeff_Green about that right now [14:59:20] Jeff_Green: because I did a merge from test to production [14:59:23] Jeff_Green: see above, dependency group and RT-3119 ottomata: just saw backlog :) [14:59:29] :) [15:00:32] i'm the wrong person to ask about this because I hate our entire user creation scheme so much I wrote a new one [15:00:37] which is sitting on the shelf [15:01:22] where did /var/log/daemon.log go btw? moved in precise? [15:01:28] i guess my suggestion would be to create a new flavor of admin:: class in admins.pp, and apply that to db1047 instead of doing individual accounts [15:02:13] that seems to be sort of the standard, and imo that's the logical place to make sure the group is added as well--sticks with the standard as awful as it is [15:02:21] is this a case for virutal resources? [15:02:23] http://docs.puppetlabs.com/guides/virtual_resources.html [15:02:50] or um, can we just put wikidev group everywhere? :) [15:02:54] New review: MaxSem; "(no comment)" [operations/mediawiki-config] (master); V: 0 C: -1; - https://gerrit.wikimedia.org/r/12185 [15:02:57] those seem more about preventing conflicts [15:03:19] my issue is that we sprinkle user creation all over the place and it makes it very hard to control what's going on for a particular host [15:03:41] wikidev is a good example--you want to use it as a point of control for access to directory [15:04:02] and you know what users you're adding to that group within the classes you deal with [15:04:37] but suddenly you discover some other class somewhere got sucked onto the box and granted some other user that group [15:05:13] that's totally unacceptable in the FR context where I've been working--I have to know exactly who is going to have what access and why [15:05:23] (FR==?) [15:05:27] fundraising [15:05:28] aye [15:05:29] hm [15:05:39] ottomata: / Jeff: well, in class erosen there is gid => $gid, but its not set to 500 or something else, and you dont have group 500, so use another gid? shrug [15:06:02] even better [15:06:10] that's true [15:06:21] create a new group specific to your purpose, and hope nobody else touches it :-) [15:06:25] can't I just not set it? [15:06:29] and it will be the user's group? [15:06:30] erosen? [15:06:45] ah, but I do want him to be in wikidev on stat machines [15:06:53] ok so this is the other problem [15:07:02] well not *the* but *an* [15:07:03] ^demon: yeah not sure why webstatscollector is not in trunk/tools anyways [15:07:09] right [15:07:13] we use wikidev as the login group all over the place [15:07:18] yeah, that is wrong [15:07:26] default group should be individual, i think, no? [15:07:32] and then they shoudl be in this other 'wikidev' group [15:07:32] <^demon> drdee: It's all good--it'll be in analytics/webstatscollector in just a bit :) [15:07:34] if needed [15:07:34] I agree yeah [15:07:38] if I did that for erosen [15:07:38] exactly [15:07:41] puppet might not be angry [15:07:43] ^demon: Thx! [15:07:50] i think it might be ok with a non-existent secondary group [15:07:55] ottomata: you're going to find that that is forced in several places [15:08:12] grep "$gid = 500" in manifests [15:08:35] oh actually it looks a little better than last time I looked [15:08:37] heh [15:08:38] ottomata: one of the issues is that you can't add existing users to existing groups using the puppet linux provider afaik and you would have to use an Exec to add a user to additional groups after the creation [15:08:52] oh no, grep "$gid=500" [15:09:07] that's the one that made me cry [15:09:24] so you either accept wikidev as the default group or redo half the world [15:11:04] mutante, really? [15:11:05] das crazy [15:11:22] is this worth a bigger discussion? [15:11:38] (and then redo half the world?) [15:11:49] probably not, i mean, yes [15:12:00] it would be amazing to make our user management much smoother and better [15:12:02] i'd be happy to put my new fully parameterized approach out there again for bashing [15:12:02] so many ways to do that [15:12:05] buuuuuuuut [15:12:11] it would probably break tons of stuff [15:12:12] it allows totally granular control, but it's a total redo [15:12:15] yep [15:12:15] and people would be pretty unhappy [15:12:19] https://groups.google.com/forum/?fromgroups#!topic/puppet-users/gRjXoaukopE [15:12:22] me and Jeff would be happy! [15:12:33] that's why i said first a bigger discussion [15:12:43] because the problem (not treated) will only get worse [15:12:49] with additional hires in the coming year [15:13:27] hey Jeff_Green, what about this: [15:13:28] https://gist.github.com/2973365 [15:13:29] would that work? [15:13:41] "By default most Linux distributions will use the ‘groupadd’ provider, which doesn’t allow you to manage group members, so you’ll have to do it on the user resources instead." http://www.puppetcookbook.com/posts/add-a-unix-group.html [15:14:20] this is getting to complicated for IRC but . . . [15:14:31] ottomata, for adding yes, for limiting it gets tricky [15:14:50] mutante: i found puppet and linux to work fine together for full control of both flavors of group [15:15:08] it's just that our class is not well designed [15:15:12] limiting it? [15:15:17] so [15:15:33] puppet treats groups as inclusive|minimum [15:15:41] Jeff_Green: cool, i always ran into the problem when wanting to add a user to a group after both existed already. [15:16:03]