[00:38:01] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[00:38:32] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[00:38:51] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[00:48:52] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[00:49:52] <tsnag>	 /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 34151 MB (3% inode=99%):  
[01:38:01] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[01:38:42] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[01:38:52] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[01:49:52] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[02:13:52] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.334961/1.10, alarm hl:np_load_long=0.733398/1.55, alarm hl:mem_free=18294.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.334961/1.00, alarm hl:np_load_long=0.733398/1.50, alarm hl:mem_free=18294.000000M/300M, alarm hl:available=1/0  
[02:14:51] <tsnag>	 Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK  
[02:38:13] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[02:39:12] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[02:39:40] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[02:40:42] <Krinkle>	 Hm.. are there any known issues with following symlinks?
[02:40:56] <Krinkle>	 I'm getting reports that 2 of my tools are showing internal server errors as of today
[02:41:15] <Krinkle>	 itt's the 2 tools that are symlinked from $HOME/SVN/krinkle/trunk/toolname to $HOME/public_html/toolname
[02:41:29] <Krinkle>	 ln -s svnpath public-htmlhtml 
[02:44:20] <Krinkle>	 Hm.. looks like according to $ ls -n the symlink is pointing to $HOME/SVN/krinkle/trunk/toolname// (<!-- 2 slashes
[02:44:25] <Krinkle>	 re-created the link and working fine now
[02:45:12] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.627930/1.10, alarm hl:np_load_long=0.910156/1.55, alarm hl:mem_free=17929.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.627930/1.00, alarm hl:np_load_long=0.910156/1.50, alarm hl:mem_free=17929.000000M/300M, alarm hl:available=1/0  
[02:47:13] <tsnag>	 Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK  
[02:50:12] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[02:52:12] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.207031/1.10, alarm hl:np_load_long=1.015625/1.55, alarm hl:mem_free=17878.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.207031/1.00, alarm hl:np_load_long=1.015625/1.50, alarm hl:mem_free=17878.000000M/300M, alarm hl:available=1/0  
[03:38:13] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[03:39:41] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[03:40:12] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[03:51:11] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[04:02:19] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.266601/1.10, alarm hl:np_load_long=0.959961/1.55, alarm hl:mem_free=17570.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.266601/1.00, alarm hl:np_load_long=0.959961/1.50, alarm hl:mem_free=17570.000000M/300M, alarm hl:available=1/0  
[04:04:12] <tsnag>	 Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK  
[04:11:11] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.195312/1.10, alarm hl:np_load_long=1.007812/1.55, alarm hl:mem_free=17427.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.195312/1.00, alarm hl:np_load_long=1.007812/1.50, alarm hl:mem_free=17427.000000M/300M, alarm hl:available=1/0  
[04:39:12] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[04:39:51] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[04:40:12] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[04:41:04] <Joan>	 Krinkle: :-)
[04:41:35] <Krinkle>	 solaris seems to have a different way of handling symlinks 
[04:41:58] <Joan>	 "ls -l" is the canonical form, I think.
[04:42:30] <Joan>	 >
[04:42:31] <Joan>	      In legacy mode, the -f option does not turn on the -a option and the -g,
[04:42:34] <Joan>	      -n, and -o options do not turn on the -l option.
[04:42:36] <Joan>	 Fun fact.
[04:42:38] <Joan>	 >
[04:42:43] <Joan>	 Solaris is a special child.
[04:43:01] <Krinkle>	   -s, --symbolic              make symbolic links instead of hard links
[04:43:03] <Krinkle>	 that's all I need
[04:43:17] <Krinkle>	 to directorates it's mostly fine
[04:43:37] <Krinkle>	 but when linking to files solaris makes an invalid link when the target is not specified as an absolute path
[04:44:14] <Joan>	 I was talking about ls, not ln. -s is the for symlinks, yes. :-)
[04:44:15] <Krinkle>	 e.g krinkle@willow: $public_html/foo: ln -s ../../SVN/krinkle/trunk/foo/config.ini config.ini
[04:44:24] <Joan>	 Remove "the" there.
[04:44:36] <Krinkle>	 that fails to work
[04:44:45] <Krinkle>	 the link is there but something is broken about it
[04:45:06] <Krinkle>	 e.g krinkle@willow: $public_html/foo: ln -s $HOME/SVN/krinkle/trunk/foo/config.ini $HOME/public_html/foo/config.ini
[04:45:09] <Krinkle>	 then it does work
[04:45:21] <Krinkle>	 ArRrGgh!
[04:45:27] <Joan>	 Yeah, I wouldn't trust dot shit with symlinks.
[04:45:29] <Joan>	 Be explicit. :-)
[04:51:11] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[05:24:13] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.067383/1.00, alarm hl:np_load_long=0.803711/1.50, alarm hl:mem_free=18052.000000M/300M, alarm hl:available=1/0  
[05:29:12] <tsnag>	 Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK  
[05:39:11] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[05:40:01] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[05:40:11] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[05:51:11] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[05:54:37] <Joan>	 @replag
[05:54:37] <tsbot>	 Joan: s2-user: 1m 14s [+0.00 s/s]; s3-rr-a: 1m 36s [+0.00 s/s]; s3-user: 1m 36s [+0.00 s/s]; s7-rr-a: 10s [+0.00 s/s]; s7-user: 10s [+0.00 s/s]
[06:00:03] <reba>	 MZMcBride * [Toolserver-l] enwiki_p corrupt?
[06:17:03] <reba>	 MZMcBride * Re: [Toolserver-l] enwiki_p corrupt?
[06:20:12] <tsnag>	 Load avg. on willow is WARNING: WARNING - load average: 19.58, 14.40, 11.71  
[06:24:11] <tsnag>	 Load avg. on willow is OK: OK - load average: 10.64, 14.61, 12.65  
[06:33:12] <tsnag>	 Load avg. on willow is WARNING: WARNING - load average: 17.17, 15.55, 13.67  
[06:39:12] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[06:40:01] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[06:40:11] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[06:44:12] <tsnag>	 Load avg. on willow is OK: OK - load average: 10.32, 14.50, 14.88  
[06:48:12] <tsnag>	 Load avg. on willow is WARNING: WARNING - load average: 13.46, 16.03, 15.57  
[06:51:11] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[06:57:12] <tsnag>	 Load avg. on willow is OK: OK - load average: 9.27, 13.39, 14.71  
[07:00:03] <reba>	 Simon Kågedal Reimer * Re: [Toolserver-l] How to silence qsub/qcronsub?
[07:02:27] <wikirc_>	 [[Category:Edit counters]] ! 10https://wiki.toolserver.org/w/index.php?diff=6810&oldid=6771&rcid=8981 * 82.145.216.57 * (+29) ()
[07:13:41] <tsnag>	 Sun Grid Engine execd on wolfsbane is WARNING: short@wolfsbane exceedes load threshold: alarm hl:np_load_short=0.253906/1.10, alarm hl:np_load_long=0.281250/1.55, alarm hl:mem_free=244.000000M/300M, alarm hl:available=1/0: all.q@wolfsbane exceedes load threshold: alarm hl:np_load_short=0.253906/1.00, alarm hl:np_load_long=0.281250/1.50, alarm hl:mem_free=244.000000M/300M, alarm hl:available=1/0  
[07:24:41] <tsnag>	 Sun Grid Engine execd on wolfsbane is OK: short@wolfsbane OK: all.q@wolfsbane OK  
[07:27:41] <tsnag>	 Sun Grid Engine execd on wolfsbane is WARNING: short@wolfsbane exceedes load threshold: alarm hl:np_load_short=0.265625/1.10, alarm hl:np_load_long=0.257324/1.55, alarm hl:mem_free=236.000000M/300M, alarm hl:available=1/0: all.q@wolfsbane exceedes load threshold: alarm hl:np_load_short=0.265625/1.00, alarm hl:np_load_long=0.257324/1.50, alarm hl:mem_free=236.000000M/300M, alarm hl:available=1/0  
[07:39:22] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[07:40:01] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[07:40:21] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[07:51:21] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[08:33:03] <reba>	 Simon Kågedal Reimer * Re: [Toolserver-l] How to silence qsub/qcronsub?
[08:39:31] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[08:40:01] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[08:40:32] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[08:51:31] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[09:39:32] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[09:40:01] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[09:40:31] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[09:51:32] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[10:39:42] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[10:40:12] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[10:40:41] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[10:51:42] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[11:39:42] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[11:40:12] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[11:40:41] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[11:52:42] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[12:40:41] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[12:41:11] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[12:41:40] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[12:50:10] <tsnag>	 /sql on rosemary is CRITICAL: DISK CRITICAL - free space: /sql 33993 MB (3% inode=99%):  
[12:53:41] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[13:34:02] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.068359/1.00, alarm hl:np_load_long=0.630860/1.50, alarm hl:mem_free=17033.000000M/300M, alarm hl:available=1/0  
[13:35:05] <tsnag>	 Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK  
[13:41:11] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[13:41:22] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[13:41:22] <tsnag>	 s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1925.000000  
[13:42:12] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[13:47:23] <tsnag>	 s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1520.000000  
[13:54:12] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[14:15:02] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=2.121094/1.10, alarm hl:np_load_long=0.968750/1.55, alarm hl:mem_free=17304.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=2.121094/1.00, alarm hl:np_load_long=0.968750/1.50, alarm hl:mem_free=17304.000000M/300M, alarm hl:available=1/0  
[14:17:02] <tsnag>	 Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK  
[14:20:03] <tsnag>	 s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1943.000000  
[14:39:03] <tsnag>	 s4 replag on rosemary is OK: QUERY OK: SELECT ts_rc_age() returned 1744.000000  
[14:41:23] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[14:41:23] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[14:42:22] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[14:54:13] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[14:56:14] <wikirc_>	 [[GeoShape]] ! 10https://wiki.toolserver.org/w/index.php?diff=6811&oldid=6398&rcid=8982 * Guverneu * (-170) (/* Frontend Examples */ )
[15:14:03] <reba>	 Aaron Halfaker * Re: [Toolserver-l] enwiki_p corrupt?
[15:31:28] <wikirc_>	 [[List of projects that highlight OSM ways]] !N 10https://wiki.toolserver.org/w/index.php?oldid=6812&rcid=8983 * Guverneu * (+9244) (extract from [[GeoShape]], written by me)
[15:41:23] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[15:41:32] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[15:42:32] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[15:54:24] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[16:10:30] <wikirc_>	 [[Comparasion between osm2pgsql and osmosis for GeoShape]] !N 10https://wiki.toolserver.org/w/index.php?oldid=6813&rcid=8984 * Guverneu * (+7934) (excerpt from [[GeoShape]] by me)
[16:26:29] <wikirc_>	 [[GeoShape]] ! 10https://wiki.toolserver.org/w/index.php?diff=6814&oldid=6811&rcid=8985 * Guverneu * (-17530) (outsourced 2 parts)
[16:41:34] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[16:41:34] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[16:42:32] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[16:47:51] <Jeeves>	 who may I ask help about TUSC password not werking?
[16:49:32] <wikirc_>	 [[Job scheduling]] ! 10https://wiki.toolserver.org/w/index.php?diff=6815&oldid=6807&rcid=8986 * Nemobis * (+166) (/* obligatory resources */ although long jobs (more than ~200 hours currently, from anecdotical evidence) will probably end up in the longrun queue on willow, which has less resources)
[16:50:36] <Jeeves>	 nevermind, it works now
[16:54:33] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[17:07:45] <wikirc_>	 [[OpenStreetMap/Priorities 2012]] ! 10https://wiki.toolserver.org/w/index.php?diff=6816&oldid=6554&rcid=8987 * Guverneu * (+18) (/* highlighting OSM Objects in the map */ +user guverneu, still interested)
[17:35:11] <wikirc_>	 [[Comparasion between osm2pgsql and osmosis for GeoShape]] ! 10https://wiki.toolserver.org/w/index.php?diff=6817&oldid=6813&rcid=8988 * Guverneu * (+123) (expanded cons of osm2pg)
[17:41:32] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[17:42:33] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[17:42:33] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[17:55:33] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[18:03:02] <tsnag>	 Load avg. on willow is WARNING: WARNING - load average: 17.07, 17.22, 12.19  
[18:06:02] <tsnag>	 Load avg. on willow is OK: OK - load average: 10.51, 14.44, 12.00  
[18:24:31] <wikirc_>	 [[GeoShape]] ! 10https://wiki.toolserver.org/w/index.php?diff=6818&oldid=6814&rcid=8989 * Guverneu * (+6) (/* update procedures todo.. */ )
[18:41:33] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[18:42:34] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[18:42:42] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[18:43:03] <reba>	 DaB. * Re: [Toolserver-l] enwiki_p corrupt?
[18:55:32] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[19:34:01] <jeremyb>	 does skagedel do IRC? aka Simon Kågedal Reimer
[19:34:37] <jeremyb>	 skagedal*
[19:41:43] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[19:42:43] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[19:42:43] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[19:55:42] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[20:34:24] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.037109/1.00, alarm hl:np_load_long=0.816406/1.50, alarm hl:mem_free=15834.000000M/300M, alarm hl:available=1/0  
[20:35:22] <tsnag>	 Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK  
[20:41:43] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[20:42:51] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[20:42:52] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[20:55:43] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[21:05:22] <tsnag>	 Sun Grid Engine execd on ortelius is WARNING: short@ortelius exceedes load threshold: alarm hl:np_load_short=1.231445/1.10, alarm hl:np_load_long=0.804688/1.55, alarm hl:mem_free=16058.000000M/300M, alarm hl:available=1/0: all.q@ortelius exceedes load threshold: alarm hl:np_load_short=1.231445/1.00, alarm hl:np_load_long=0.804688/1.50, alarm hl:mem_free=16058.000000M/300M, alarm hl:available=1/0  
[21:06:23] <tsnag>	 Sun Grid Engine execd on ortelius is OK: short@ortelius OK: all.q@ortelius OK  
[21:30:18] <kaldari>	 I just ran a query on sql-s1-rr that seems to be taking longer than expected. Feel free to kill it if it's still running.
[21:31:26] <valhallasw>	 kaldari: you can do that yourself
[21:31:28] <valhallasw>	 show processlist;
[21:31:30] <valhallasw>	 then
[21:31:37] <valhallasw>	 kill <query id>;
[21:32:03] <kaldari>	 cool thanks
[21:32:26] <kaldari>	 it was just a count() query so it shouldn't have been bad, but it timed out in phpmyadmin
[21:35:22] <kaldari>	 hmm I don't seem to have command line access to sql-s1-rr
[21:35:34] <kaldari>	 unless there's someway to get to it I don't know about
[21:37:06] <kaldari>	 I'm beginning to think something else is wrong as phpmyadmin.toolserver.org seems to be completely inoperable now
[21:37:26] <valhallasw>	 kaldari: those are sql queries
[21:41:43] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[21:41:52] <DarkoNeko>	 nighty o/
[21:42:52] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[21:42:52] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[21:48:54] <kaldari>	 well I tried killing it several times, but it didn't work.
[21:49:09] <kaldari>	 the id of the process is 10590532
[21:50:45] <kaldari>	 I also don't understand why a simple count(*) query would be running for 19 minutes
[21:51:14] <kaldari>	 it doesn't even join any tables
[21:53:05] <kaldari>	 I could have counted them by hand by now
[21:53:11] <valhallasw>	 I doubt that
[21:53:14] <kaldari>	 :)
[21:53:23] <valhallasw>	 count(*) always is slow for large tables
[21:53:56] <valhallasw>	 in any case; what happens after you do 'kill 10590532;'?
[21:55:08] <kaldari>	 it just returns false
[21:55:13] <kaldari>	 as the result
[21:55:27] <valhallasw>	 err
[21:55:35] <valhallasw>	 you're running this /in/ mysql, right?
[21:55:48] <Akoopal>	 number sounds big for a pid?
[21:55:50] <kaldari>	 no, I can't get to the server to access mysql directly
[21:55:52] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[21:56:07] <valhallasw>	 kaldari: ...
[21:56:12] <valhallasw>	 either use phpmyadmin
[21:56:14] <valhallasw>	 or sql -h sql-s1-rr.toolserver.org
[21:56:32] <kaldari>	 php myadmin is hosed
[21:56:42] <kaldari>	 I'll try sql -h sql-s1-rr.toolserver.org
[21:57:20] <valhallasw>	 Akoopal: it's a query/thread id, not a pid
[21:57:32] <Akoopal>	 valhallasw: ahh
[21:59:34] <kaldari>	 ERROR 1094 (HY000): Unknown thread id: 10590532
[21:59:44] <kaldari>	 lemme check the list again....
[22:00:25] <kaldari>	 ok, all fixed now
[22:09:03] <reba>	 Platonides * Re: [Toolserver-l] How to silence qsub/qcronsub?
[22:35:54] <msgbot>	 3(work started) [UTRS-63] Test interface <10https://jira.toolserver.org/browse/UTRS-63>  (Hersfold)
[22:41:43] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[22:42:52] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[22:42:52] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[22:55:52] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output  
[23:41:43] <tsnag>	 SMF on damiana is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[23:41:52] <msgbot>	 3(created) [UTRS-66] Have UTRSBot report changes to the beta site; UTRS: UTRSBot; New Feature <10https://jira.toolserver.org/browse/UTRS-66>  (Hersfold)
[23:42:52] <tsnag>	 SMF on willow is CRITICAL: ERROR - maintenance:  svc:/network/puppetmasterd:default  
[23:42:52] <tsnag>	 SMF on turnera is CRITICAL: ERROR -  offline:  svc:/system/cluster/scsymon-srv:default  
[23:55:51] <tsnag>	 Sun Grid Engine execd on willow is WARNING: NRPE: Unable to read output