[01:14:33] RECOVERY - MySQL replication status on db1025 is OK: CHECK MySQL REPLICATION - lag - OK - Seconds_Behind_Master : 0s [01:16:00] 29 01:13:00 < DaBPunkt> was the nagios-password changed during the last two hours? [01:16:03] 29 01:13:18 < DaBPunkt> nagios-mysql-password [01:16:03] in #-tech [01:31:22] New patchset: Ryan Lane; "libvirt-bin's upstart script is rediculous, and doesn't let you change daemon options in the default file. So, let's update it's upstart to appease its stupidity." [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1751 [01:32:08] New review: Ryan Lane; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1751 [01:32:08] Change merged: Ryan Lane; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1751 [01:44:02] PROBLEM - MySQL replication status on db1025 is CRITICAL: CHECK MySQL REPLICATION - lag - CRITICAL - Seconds_Behind_Master : 1412s [01:46:43] New patchset: Ryan Lane; "Weirdness with -'s?" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1752 [01:46:57] New review: Ryan Lane; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1752 [01:46:58] Change merged: Ryan Lane; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1752 [01:50:57] New patchset: Ryan Lane; "Work around upstart_job's failure." [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1753 [01:51:38] New review: Ryan Lane; "(no comment)" [operations/puppet] (production); V: 0 C: 2; - https://gerrit.wikimedia.org/r/1753 [01:51:40] Change merged: Ryan Lane; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1753 [02:29:26] New patchset: Ryan Lane; "%s in this context is the short hostname, not the FQDN. This causes a certificate error." [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1754 [02:29:38] New review: Ryan Lane; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1754 [02:29:39] Change merged: Ryan Lane; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1754 [02:29:39] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1754 [03:44:07] RECOVERY - MySQL replication status on db1025 is OK: CHECK MySQL REPLICATION - lag - OK - Seconds_Behind_Master : 0s [04:07:57] PROBLEM - Puppet freshness on ms1002 is CRITICAL: Puppet has not run in the last 10 hours [04:33:02] !log restarting virt4 because upstart is an unreliable POS [04:33:12] Logged the message, Master [13:54:51] PROBLEM - Backend Squid HTTP on knsq11 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:03:01] PROBLEM - Host knsq11 is DOWN: PING CRITICAL - Packet loss = 100% [14:16:51] PROBLEM - Puppet freshness on ms1002 is CRITICAL: Puppet has not run in the last 10 hours [15:43:54] New patchset: Jgreen; "fix notification and puppetize offhost_backups-storage3, typo in dump_fundraisingdb" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1762 [15:44:07] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1762 [15:44:10] New review: Jgreen; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1762 [15:44:10] Change merged: Jgreen; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1762 [15:50:02] scum [15:50:19] so they are telling me that the web platform requires mac or windows, can't be done with linux [15:50:24] for the juniper courses. [15:50:37] yes [15:50:43] they aadvised me to run a vm of mac os or windows. since I don't actually have either of those [15:50:50] I can't very well run them in a vm >_< [15:51:00] so no courses for me :-( [15:51:14] this is why we hated their SSL VPN [15:51:22] they're linux-lame as a company [15:51:36] they were barely Mac non-lame [15:51:37] when I looked at the course requirements it said someting else [15:51:45] something about adobe connect etc and it offered a test page [15:51:56] I checked all that out and made sure before registering [15:52:02] but no. [15:53:01] I suppose they would expect me to buy a copy of one of these os's in order to run it in a vm [15:53:04] not going to happen [15:53:48] borrow a netbook or something? [15:53:51] you could launch an ec2 windows instance, and use it via vnc :D [15:54:01] *eyeroll* [15:54:21] knowing juniper they'll make you sslvpn and break your network stack [15:54:21] sure that won't have any latency.... [15:54:28] I am sure they will [15:57:49] apergos: depending on where you launch the instance, the latency shouldn't be terrible [15:58:37] it already sucks because the isp does some sort of proxying [15:58:49] remember how I have a kind of crappy connection? [15:59:02] it can't proxy vnc ;) [15:59:10] anyways it just pisses me off because I *did* due diligence before signing up [15:59:18] * Ryan_Lane nods [15:59:20] only to then be told "ha ha" [16:00:05] I sent em an email and said if there's no linux option they'll have to cancel my registration [16:00:07] we'll see [16:11:08] they are going to see if it runs on wine :-D [16:19:17] jesus h [16:19:27] electricity rates going up 9.2% jan 1 [16:19:38] it's already not cheap [16:30:59] strike! [16:31:59] don't worry [16:32:05] we're taking a break right now [16:32:12] the tax collectors are on strike today and tomorrow [16:32:18] then it's the weekend and the holiday [17:42:47] PROBLEM - Puppet freshness on es1002 is CRITICAL: Puppet has not run in the last 10 hours [17:51:17] New patchset: Jgreen; "moving misc::fundraising out of misc-servers and into its own file" [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1763 [17:51:31] New review: gerrit2; "Lint check passed." [operations/puppet] (production); V: 1 - https://gerrit.wikimedia.org/r/1763 [17:55:27] New review: Jgreen; "(no comment)" [operations/puppet] (production); V: 1 C: 2; - https://gerrit.wikimedia.org/r/1763 [17:55:27] Change merged: Jgreen; [operations/puppet] (production) - https://gerrit.wikimedia.org/r/1763