[00:00:37] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [00:09:58] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 615283.000000 [00:09:58] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 615284.000000 [00:26:59] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [00:29:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [00:30:08] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [00:31:58] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [00:34:28] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [00:42:17] bonne nuit~ [00:52:48] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:10:58] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 618944.000000 [01:10:58] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 618944.000000 [01:14:27] Environment IPMI on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [01:16:28] Environment IPMI on adenia is OK: ok: temperature ok fan ok voltage ok chassis ok [01:17:28] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [01:27:59] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [01:29:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [01:30:07] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [01:31:58] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [01:34:38] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [02:11:59] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 622604.000000 [02:11:59] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 622604.000000 [02:27:59] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [02:29:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [02:30:08] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [02:31:59] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [02:34:37] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [02:55:37] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.941894/1.75, alarm hl:np_load_avg=0.937012/2.0, alarm hl:mem_free=149.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.941894/1.9, alarm hl:np_load_long=0.940430/2.25, alarm hl:mem_free=149.000000M/200M, alarm hl:available=1/0 [02:58:19] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 140109 MB (14% inode=99%): [03:12:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 626263.000000 [03:12:57] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 626264.000000 [03:28:59] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [03:29:38] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [03:30:08] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [03:30:37] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [03:31:58] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [03:35:37] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [03:44:48] /sql on thyme is CRITICAL: DISK CRITICAL - free space: /sql 68706 MB (7% inode=99%): [03:56:37] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.047851/1.00, alarm hl:np_load_long=0.828125/1.50, alarm hl:mem_free=20595.000000M/350M, alarm hl:available=1/0 [03:57:37] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [04:12:58] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 629865.000000 [04:12:59] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 629865.000000 [04:18:17] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 41572 MB (10% inode=99%): [04:18:37] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.908691/1.75, alarm hl:np_load_avg=0.966309/2.0, alarm hl:mem_free=237.000000M/350M, alarm hl:available=1/0 [04:20:37] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [04:20:37] Sun Grid Engine execd on ortelius is WARNING: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.086914/1.00, alarm hl:np_load_long=0.799805/1.50, alarm hl:mem_free=20810.000000M/350M, alarm hl:available=1/0 [04:21:37] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [04:23:17] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 73294 MB (18% inode=99%): [04:28:59] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [04:29:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [04:30:08] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [04:31:59] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [04:35:38] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [05:12:37] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.437988/1.75, alarm hl:np_load_avg=1.185547/2.0, alarm hl:mem_free=225.000000M/350M, alarm hl:available=1/0 [05:12:58] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 633465.000000 [05:12:58] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 633464.000000 [05:15:37] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [05:28:59] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [05:30:07] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [05:30:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [05:31:58] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [05:36:37] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [06:06:37] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.422851/1.75, alarm hl:np_load_avg=1.272461/2.0, alarm hl:mem_free=235.000000M/350M, alarm hl:available=1/0 [06:12:58] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 637065.000000 [06:12:58] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 637065.000000 [06:13:37] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [06:27:37] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.452149/1.75, alarm hl:np_load_avg=1.456055/2.0, alarm hl:mem_free=177.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.452149/1.9, alarm hl:np_load_long=1.441895/2.25, alarm hl:mem_free=177.000000M/200M, alarm hl:available=1/0 [06:29:58] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [06:30:07] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [06:30:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [06:32:57] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [06:36:37] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [06:38:37] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=4.249023/1.10, alarm hl:np_load_long=1.197266/1.55, alarm hl:mem_free=21459.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=4.249023/1.00, alarm hl:np_load_long=1.197266/1.50, alarm hl:mem_free=21459.000000M/350M, alarm hl:available=1/0 [06:39:58] Load avg. on ortelius is WARNING: WARNING - load average: 17.25, 10.84, 5.91 [06:40:58] Load avg. on ortelius is OK: OK - load average: 11.02, 10.35, 6.05 [06:47:37] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [07:00:37] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [07:04:37] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.748535/1.75, alarm hl:np_load_avg=1.600098/2.0, alarm hl:mem_free=132.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=1.748535/1.9, alarm hl:np_load_long=1.522461/2.25, alarm hl:mem_free=132.000000M/200M, alarm hl:available=1/0 [07:10:37] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [07:12:59] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 640665.000000 [07:12:59] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 640665.000000 [07:29:58] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [07:30:08] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [07:30:37] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [07:32:58] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [07:36:37] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [07:57:47] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [07:58:53] 3(commented) [MAGNUS-311] Catscan broken on plwiki <10https://jira.toolserver.org/browse/MAGNUS-311> (bulwersator) [08:01:18] [[Special:Log/newusers]] create 10 * Frshven6 * (New user account) [08:01:56] [[User:Frshven6]] !N 10https://wiki.toolserver.org/w/index.php?oldid=6939&rcid=9141 * Frshven6 * (+63) (Created page with "[http://workplacehealthyvending.com fresh healthy vending scam]") [08:13:57] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 644326.000000 [08:13:57] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 644326.000000 [08:23:49] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.113770/1.75, alarm hl:np_load_avg=1.071289/2.0, alarm hl:mem_free=303.000000M/350M, alarm hl:available=1/0 [08:24:49] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [08:29:57] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [08:30:18] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [08:30:48] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [08:32:49] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.196289/1.75, alarm hl:np_load_avg=1.181641/2.0, alarm hl:mem_free=298.000000M/350M, alarm hl:available=1/0 [08:33:07] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [08:36:58] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [08:47:27] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [08:58:07] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [09:07:22] hey everybody ! is there anybody around who knows what's new with the MMP-interwikibot ? I see that the page https://wiki.toolserver.org/view/Interwiki_bot_MMP_planning hasn't been edited very much since its creation few weeks ago… [09:14:08] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 647934.000000 [09:14:08] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 647936.000000 [09:30:08] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [09:30:18] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [09:31:07] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [09:33:09] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [09:37:08] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [09:55:34] hey, ho [09:55:53] have you guys experienced problems with logging into nightshade? [09:56:06] I can log in to any given server, but nightshade just hangs [09:57:09] ~ ping nightshade.toolserver.org [09:57:09] PING nightshade.toolserver.org (91.198.174.201): 56 data bytes [09:57:09] Request timeout for icmp_seq 0 [09:57:25] Holek: nightshade is out of service since several weeks ago [09:57:31] oh, okay [09:57:50] so, where did my crontab go? [09:58:17] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [09:58:18] Well, if your crontab wasn't read from a file under your home dir, then probably it's lost [09:59:03] (or for sure it's lost, as DaB. said in the list a few days ago) [09:59:30] oh, then that's good ;) [10:00:22] That's the spirit :) [10:01:29] I had some weird task that would screw up half of my tools after yesterday's commits, but is the cronjobs are not run from nightshade, it's all good ;) [10:02:00] Ok :) [10:03:28] /sql on z-dat-s4-a is WARNING: DISK WARNING - free space: /sql 36829 MB (9% inode=99%): [10:08:28] /sql on z-dat-s4-a is CRITICAL: DISK CRITICAL - free space: /sql 22644 MB (5% inode=99%): [10:15:08] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 651600.000000 [10:15:08] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 651600.000000 [10:30:19] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [10:30:28] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [10:31:18] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [10:33:19] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [10:35:29] [[GeoShape]] ! 10https://wiki.toolserver.org/w/index.php?diff=6940&oldid=6818&rcid=9142 * Guverneu * (-3600) (update+close) [10:36:28] /sql on z-dat-s4-a is OK: DISK OK - free space: /sql 83090 MB (20% inode=99%): [10:37:18] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [10:47:47] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [10:48:28] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:48:29] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:48:29] SSH on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [10:48:47] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [10:48:57] MySQL on z-dat-s3-a is CRITICAL: (Service Check Timed Out) [10:49:07] MySQL on z-dat-s3-a is OK: Uptime: 2337527 Threads: 11 Questions: 2415785671 Slow queries: 131706 Opens: 15849661 Flush tables: 1 Open tables: 16384 Queries per second avg: 1033.479 [10:49:18] SSH on z-dat-s7-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [10:49:18] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [10:49:18] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [10:58:19] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [11:03:08] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.133789/1.10, alarm hl:np_load_long=0.643555/1.55, alarm hl:mem_free=20458.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.133789/1.00, alarm hl:np_load_long=0.643555/1.50, alarm hl:mem_free=20458.000000M/350M, alarm hl:available=1/0 [11:04:08] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [11:07:08] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.202149/1.10, alarm hl:np_load_long=0.778320/1.55, alarm hl:mem_free=21339.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.202149/1.00, alarm hl:np_load_long=0.778320/1.50, alarm hl:mem_free=21339.000000M/350M, alarm hl:available=1/0 [11:07:12] [[Toolserver Intuition]] 10https://wiki.toolserver.org/w/index.php?diff=6941&oldid=6636&rcid=9143 * Krinkle * (-1) (/* Register entire text-domain locally */ typo) [11:13:41] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=1.219238/1.75, alarm hl:np_load_avg=1.025391/2.0, alarm hl:mem_free=349.000000M/350M, alarm hl:available=1/0 [11:15:17] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 655208.000000 [11:15:27] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 655221.000000 [11:15:38] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [11:30:40] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [11:30:57] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [11:31:39] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [11:33:42] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [11:35:57] 3(updated) [MNT-1225] Growing replag on S1 due to a database migration at WMF <10https://jira.toolserver.org/browse/MNT-1225> (Anonymous) [11:37:38] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [11:54:52] 3(updated) [MNT-1225] Growing replag on S1 due to a database migration at WMF <10https://jira.toolserver.org/browse/MNT-1225> (Marlen Caemmerer) [11:58:39] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [12:01:03] hello all [12:06:25] morning DaBPunkt [12:12:55] 3(commented) [ACCAPP-481] I am active user in http://fa.wiktionary.org and i want this account to run bot on this wiki also updating it's statics. <10https://jira.toolserver.org/browse/ACCAPP-481> (DaB.) [12:15:18] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 658809.000000 [12:15:28] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 658821.000000 [12:16:53] 3(assigned) [ACCAPP-478] Set up anything needed to allow easy analysis of data held by Education Program extension <10https://jira.toolserver.org/browse/ACCAPP-478> (DaB.) [12:19:03] 3(moved) [TS-1336] Template checker <10https://jira.toolserver.org/browse/TS-1336> (DaB.) [12:20:53] 3(resolved) [TS-1336] Template checker <10https://jira.toolserver.org/browse/TS-1336> (DaB.) [12:20:57] 3(commented) [ACCAPP-468] Rcsprinter <10https://jira.toolserver.org/browse/ACCAPP-468> (DaB.) [12:20:59] 3(commented) [ACCAPP-478] Set up anything needed to allow easy analysis of data held by Education Program extension <10https://jira.toolserver.org/browse/ACCAPP-478> (Jeroen De Dauw) [12:28:53] 3(assigned) [ACCAPP-454] I want to share data from the Walters Museum with Wikimedia Commons <10https://jira.toolserver.org/browse/ACCAPP-454> (DaB.) [12:30:48] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [12:31:06] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [12:31:48] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [12:32:58] 3(moved) [TS-1337] Renew expired account josh <10https://jira.toolserver.org/browse/TS-1337> (DaB.) [12:33:03] 3(commented) [TS-1337] Renew expired account josh <10https://jira.toolserver.org/browse/TS-1337> (DaB.) [12:33:59] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [12:37:38] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [12:44:18] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.383789/1.10, alarm hl:np_load_long=0.715820/1.55, alarm hl:mem_free=19722.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.383789/1.00, alarm hl:np_load_long=0.715820/1.50, alarm hl:mem_free=19722.000000M/350M, alarm hl:available=1/0 [12:45:17] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [12:45:54] 3(created) [TS-1338] Install site-wide SSL certificates; Toolserver: Loginserver: General/Unknown; Bug <10https://jira.toolserver.org/browse/TS-1338> (Tim.Landscheidt) [12:52:49] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [12:57:27] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [12:57:53] 3(commented) [ACCAPP-481] I am active user in http://fa.wiktionary.org and i want this account to run bot on this wiki also updating it's statics. <10https://jira.toolserver.org/browse/ACCAPP-481> (Abdollah Abdollahi) [12:58:48] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [12:59:53] 3(commented) [ACCAPP-481] I am active user in http://fa.wiktionary.org and i want this account to run bot on this wiki also updating it's statics. <10https://jira.toolserver.org/browse/ACCAPP-481> (Abdollah Abdollahi) [13:01:53] 3(commented) [ACCAPP-439] Creating tools for automated template:book filling in articles <10https://jira.toolserver.org/browse/ACCAPP-439> (Engraver) [13:06:48] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.997070/1.75, alarm hl:np_load_avg=0.927734/2.0, alarm hl:mem_free=126.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.997070/1.9, alarm hl:np_load_long=0.875977/2.25, alarm hl:mem_free=126.000000M/200M, alarm hl:available=1/0 [13:10:47] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [13:13:48] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.908691/1.75, alarm hl:np_load_avg=0.912598/2.0, alarm hl:mem_free=143.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.908691/1.9, alarm hl:np_load_long=0.889648/2.25, alarm hl:mem_free=143.000000M/200M, alarm hl:available=1/0 [13:15:28] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 662417.000000 [13:15:37] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 662424.000000 [13:30:47] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [13:30:58] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [13:31:07] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [13:31:58] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [13:33:59] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [13:34:49] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.619141/1.75, alarm hl:np_load_avg=0.692383/2.0, alarm hl:mem_free=288.000000M/350M, alarm hl:available=1/0 [13:38:38] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [13:58:58] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [14:00:47] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [14:04:47] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.895019/1.75, alarm hl:np_load_avg=0.868652/2.0, alarm hl:mem_free=148.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.895019/1.9, alarm hl:np_load_long=0.785156/2.25, alarm hl:mem_free=148.000000M/200M, alarm hl:available=1/0 [14:05:47] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [14:15:38] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 666024.000000 [14:16:28] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 666078.000000 [14:31:17] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [14:32:08] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [14:32:09] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [14:34:09] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [14:39:38] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [14:58:39] /sql on rosemary is WARNING: DISK WARNING - free space: /sql 138633 MB (14% inode=99%): [14:59:11] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:05:37] @replag [15:05:38] matthewrbowker: s1-rr-a: 1w 17h 50m 33s [+1.00 s/s]; s1-user: 1w 17h 50m 33s [+1.00 s/s]; s3-rr-a: 13s [-0.00 s/s]; s3-user: 13s [-0.00 s/s] [15:16:38] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 669689.000000 [15:17:27] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 669738.000000 [15:31:17] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [15:32:08] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [15:32:09] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [15:34:13] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [15:39:39] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [15:42:54] 3(commented) [ACCAPP-468] Rcsprinter <10https://jira.toolserver.org/browse/ACCAPP-468> (David Moon) [15:43:28] Sun Grid Engine execd on ortelius is WARNING: short-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.528320/1.10, alarm hl:np_load_long=0.712890/1.55, alarm hl:mem_free=20998.000000M/300M, alarm hl:available=1/0: medium-sol@ortelius exceedes load threshold: alarm hl:np_load_short=1.528320/1.00, alarm hl:np_load_long=0.712890/1.50, alarm hl:mem_free=20998.000000M/350M, alarm hl:available=1/0 [15:45:28] Sun Grid Engine execd on ortelius is OK: short-sol@ortelius OK: medium-sol@ortelius OK [15:45:39] /sql on thyme is CRITICAL: DISK CRITICAL - free space: /sql 50970 MB (5% inode=99%): [16:00:09] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [16:00:38] SMTP on z-dat-s7-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:00:47] RAID on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:00:57] SSH on z-dat-s4-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:00:57] Environment IPMI on hyacinth is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [16:00:57] SSH on z-dat-s3-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:00:57] SSH on z-dat-s6-a is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:01:18] SSH on hyacinth is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:01:28] SMTP on z-dat-s7-a is OK: SMTP OK - 2.526 sec. response time [16:01:28] Environment IPMI on hyacinth is OK: ok: temperature ok fan ok voltage ok chassis ok [16:01:38] RAID on hyacinth is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [16:01:38] SSH on z-dat-s4-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [16:01:47] SSH on z-dat-s3-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [16:01:48] SSH on z-dat-s6-a is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [16:02:09] SSH on hyacinth is OK: SSH OK - OpenSSH_5.8p2-hpn13v11 (protocol 2.0) [16:10:00] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.701660/1.75, alarm hl:np_load_avg=0.731445/2.0, alarm hl:mem_free=298.000000M/350M, alarm hl:available=1/0 [16:16:40] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 673291.000000 [16:17:31] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 673340.000000 [16:20:01] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [16:23:49] fisheye.toolserver.org on web.amaranth is CRITICAL: CRITICAL - Socket timeout after 21 seconds [16:29:15] [[Special:Log/newusers]] create 10 * Velimir Ivanovic * (New user account) [16:31:41] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [16:32:10] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [16:32:11] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [16:34:31] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [16:40:11] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [16:41:41] fisheye.toolserver.org on web.amaranth is OK: HTTP OK: HTTP/1.1 200 OK - 272 bytes in 10.008 second response time [16:49:54] 3(updated) [ACCAPP-465] Request for new account <10https://jira.toolserver.org/browse/ACCAPP-465> (Luca Rabinelli) [17:00:12] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [17:11:04] 3(created) [TS-1339] Multi-maintainer project request: citegen; Toolserver: Accounts; Task <10https://jira.toolserver.org/browse/TS-1339> [17:17:14] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 676917.000000 [17:17:33] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 676943.000000 [17:31:43] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [17:32:14] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [17:32:14] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [17:34:36] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [17:40:14] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [18:00:14] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [18:00:54] 3(commented) [OSM-10] mapnik doesn't run from console <10https://jira.toolserver.org/browse/OSM-10> (Kai Krueger) [18:16:56] 3(closed) [OSM-6] mapnik tirex-backend occasionally failing <10https://jira.toolserver.org/browse/OSM-6> (Kai Krueger) [18:17:42] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 680547.000000 [18:18:13] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 680577.000000 [18:27:56] 3(created) [TS-1340] Cronie jobs not being run intermittently; Toolserver: General/Unknown; Bug <10https://jira.toolserver.org/browse/TS-1340> (Russell Blau) [18:29:53] 3(commented) [TS-1340] Cronie jobs not being run intermittently <10https://jira.toolserver.org/browse/TS-1340> (Russell Blau) [18:32:02] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [18:32:24] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [18:32:24] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [18:34:54] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [18:40:33] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [18:54:56] is there any kind of public http access log for each user domain? [19:00:33] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [19:03:33] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.840820/1.75, alarm hl:np_load_avg=0.789551/2.0, alarm hl:mem_free=165.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.840820/1.9, alarm hl:np_load_long=0.718262/2.25, alarm hl:mem_free=165.000000M/200M, alarm hl:available=1/0 [19:10:33] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [19:14:52] 3(updated) [ACCAPP-469] Help development with CVN (and commit access to p_cvn), possibly run query services <10https://jira.toolserver.org/browse/ACCAPP-469> (Daniel Salciccioli) [19:17:44] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 684151.000000 [19:19:11] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 684246.000000 [19:19:53] 3(commented) [ACCAPP-464] I want this account to run my bot on wikipedia. it will do pywiki codes <10https://jira.toolserver.org/browse/ACCAPP-464> (Abbas Salamat) [19:24:53] 3(commented) [ACCAPP-464] I want this account to run my bot on wikipedia. it will do pywiki codes <10https://jira.toolserver.org/browse/ACCAPP-464> (Abbas Salamat) [19:26:34] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.609375/1.75, alarm hl:np_load_avg=0.651856/2.0, alarm hl:mem_free=276.000000M/350M, alarm hl:available=1/0 [19:26:53] 3(commented) [ACCAPP-464] I want this account to run my bot on wikipedia. it will do pywiki codes <10https://jira.toolserver.org/browse/ACCAPP-464> (Abbas Salamat) [19:26:57] 3(updated) [ACCAPP-464] I want this account to run my bot on wikipedia. it will do pywiki codes <10https://jira.toolserver.org/browse/ACCAPP-464> (Abbas Salamat) [19:26:58] 3(updated) [ACCAPP-464] I want this account to run my bot on wikipedia. it will do pywiki codes <10https://jira.toolserver.org/browse/ACCAPP-464> (Abbas Salamat) [19:28:57] 3(commented) [TS-1340] Cronie jobs not being run intermittently <10https://jira.toolserver.org/browse/TS-1340> (DaB.) [19:31:43] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [19:32:03] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [19:32:42] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [19:32:42] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [19:34:53] DaBPunkt: Does rosemary have enough disk space? [19:35:03] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [19:35:47] Dispenser: 135GB left for the sql-partition at the moment. That should be enough for the TABLE-alter [19:38:12] or do you ask for another reason? [19:38:27] No that was it [19:38:36] How much does disk space does each server have? [19:38:50] Firebolt: that's differ from server to server [19:38:54] Max? [19:39:04] Firebolt: https://wiki.toolserver.org/view/Servers (might be out of date) [19:39:10] thank you [19:39:24] Firebolt: no db-server has more then 1TB [19:40:43] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [19:41:25] If rosemary and thyme are similarly spec'ed, Why did thyme run out of disk space [19:47:07] Dispenser: thyme has 44GB left at the moment. And mysql- is sometimes a little bit strange in questions of deletion [19:49:33] (for example: enwiki on thyme is 100GB bigger as enwiki on rosemary) [19:53:03] RAID on adenia is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:54:03] Something wrong with your numbers: rosemary has 146 GB total, 135 free, but the pagelinks/categorylinks/templatelinks/externallinks are 7.6 GB compressed leaving 3.4 GB for the 40 other tables [19:54:53] Dispenser: rosemary hast 16x146GB :-) [19:54:58] -t [19:54:58] oh [19:55:28] alone enwiki is 782GB big [19:55:52] 3(updated) [ACCAPP-464] I want this account to run my bot on wikipedia. it will do pywiki codes <10https://jira.toolserver.org/browse/ACCAPP-464> (Abbas Salamat) [19:56:25] * Dispenser resizes browser window [19:59:29] Dispenser: I doubt that that saves disc-space ;) [20:00:44] APT on yarrow is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [20:17:52] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 687759.000000 [20:19:22] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 687852.000000 [20:32:12] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [20:32:53] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [20:32:53] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [20:32:53] APT on yarrow is OK: APT OK: 0 packages available for upgrade (0 critical updates). [20:36:02] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [20:38:02] RAID on adenia is OK: OK - TOTAL: 2: FAILED: 0: DEGRADED: 0 [20:40:54] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [21:17:54] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 691363.000000 [21:19:22] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 691456.000000 [21:32:32] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [21:32:54] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [21:32:54] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [21:36:33] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [21:40:52] 3(commented) [TS-1340] Cronie jobs not being run intermittently <10https://jira.toolserver.org/browse/TS-1340> (Russell Blau) [21:41:03] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [21:43:41] anyone here who can tell me about how to set cron jobs to send output to email? [21:44:03] I've noticed that on occasion my cron job runs out of memory on the TS so I have to run it on my home machine [21:44:54] basically, if the process returns a value, I'd like it to email me [21:45:28] that's done automaticaly [21:45:31] I know the statements in bash to track "if this returns a value" - if [ $? -eq 0 ]; THEN ... FI [21:45:53] it's supposed to but I've only once ever received an email of the output [21:52:58] 3(commented) [TS-1340] Cronie jobs not being run intermittently <10https://jira.toolserver.org/browse/TS-1340> (DaB.) [21:53:08] Magog_the_Ogre: did you run a test which produced output for sure? And did you check your spam-folder? [21:55:33] oh it definitely produces output [21:56:15] I know because it's saving the output to out [21:56:28] *,out [21:56:32] *.out [21:56:35] (argh) [21:57:33] apparently that's a cronsub functionality [21:57:41] heh [21:57:48] I guess I've answered my own question [21:58:53] yes [22:01:39] I imagine cronsub takes all of the following commands if there is more than one on a line [22:01:55] how does one circumvent cronsub on the rest of the line? [22:05:20] pardon? [22:06:32] nvm I'll figure it out [22:10:13] [[Toolserver notice]] 10https://wiki.toolserver.org/w/index.php?diff=6942&oldid=6852&rcid=9145 * Dispenser * (+49) (Added AJAX note slightly better Python implementation) [22:14:15] [[Toolserver notice]] 10https://wiki.toolserver.org/w/index.php?diff=6943&oldid=6942&rcid=9146 * Dab * (+3) (Ages, and ages, and ages…) [22:16:55] [[Database status files]] N 10https://wiki.toolserver.org/w/index.php?oldid=6944&rcid=9147 * Dispenser * (+51) (Redirected page to [[Database access#Database status files]]) [22:18:55] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 695026.000000 [22:19:35] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 695062.000000 [22:20:04] Sun Grid Engine execd on willow is WARNING: medium-sol@willow exceedes load threshold: alarm hl:np_load_short=0.738281/1.75, alarm hl:np_load_avg=0.805176/2.0, alarm hl:mem_free=172.000000M/350M, alarm hl:available=1/0: longrun-sol@willow exceedes load threshold: alarm hl:np_load_short=0.738281/1.9, alarm hl:np_load_long=0.845703/2.25, alarm hl:mem_free=172.000000M/200M, alarm hl:available=1/0 [22:27:26] [[Template:Obsolete]] 10https://wiki.toolserver.org/w/index.php?diff=6945&oldid=4629&rcid=9148 * Dispenser * (+107) (Fixed line break; distinctive look to differentiate from text) [22:27:57] [[Toolserver notice]] 10https://wiki.toolserver.org/w/index.php?diff=6946&oldid=6943&rcid=9149 * Dispenser * (-661) (More information on what happened) [22:28:02] Sun Grid Engine execd on willow is OK: medium-sol@willow OK: longrun-sol@willow OK [22:32:42] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [22:33:23] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [22:33:23] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [22:36:45] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [22:39:41] [[Status.toolserver.org]] N 10https://wiki.toolserver.org/w/index.php?oldid=6947&rcid=9150 * Dab * (+598) (basic knowledge) [22:41:19] [[Status.toolserver.org]] M 10https://wiki.toolserver.org/w/index.php?diff=6948&oldid=6947&rcid=9151 * Dab * (+0) () [22:41:44] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [22:48:54] 3(updated) [ACCAPP-458] Run bots in ptwiki <10https://jira.toolserver.org/browse/ACCAPP-458> (Francisco Carvalho Venancio) [22:50:53] [[Template:Lowercase title]] N 10https://wiki.toolserver.org/w/index.php?oldid=6949&rcid=9152 * MZMcBride * (+105) (new template) [22:51:05] [[Template:Lowercase title]] 10https://wiki.toolserver.org/w/index.php?diff=6950&oldid=6949&rcid=9153 * MZMcBride * (+2) (counting is hard) [22:54:12] [[Status.toolserver.org]] 10https://wiki.toolserver.org/w/index.php?diff=6951&oldid=6948&rcid=9154 * MZMcBride * (+91) (updated page) [22:54:51] [[Domains]] 10https://wiki.toolserver.org/w/index.php?diff=6952&oldid=4460&rcid=9155 * MZMcBride * (+60) (/* status.toolserver.org */ +link) [22:59:56] [[Domains]] 10https://wiki.toolserver.org/w/index.php?diff=6953&oldid=6952&rcid=9156 * Dab * (+6) (/* confluence.toolserver.org */ ) [23:01:19] [[Code snippets]] 10https://wiki.toolserver.org/w/index.php?diff=6954&oldid=6922&rcid=9157 * MZMcBride * (+49) (/* shell */ I always forget this one...) [23:03:02] [[Snippets]] N 10https://wiki.toolserver.org/w/index.php?oldid=6955&rcid=9158 * MZMcBride * (+27) (redirect) [23:03:24] [[Special:Log/move]] move 10 * MZMcBride * (moved [[02Queries10]] to [[MySQL queries]]: this page is specific to MySQL) [23:04:53] [[MySQL queries]] 10https://wiki.toolserver.org/w/index.php?diff=6958&oldid=6956&rcid=9160 * MZMcBride * (+4) (minor fixes) [23:15:06] [[MySQL queries]] 10https://wiki.toolserver.org/w/index.php?diff=6959&oldid=6958&rcid=9161 * MZMcBride * (-84) (cleanup) [23:19:15] s1 replag on thyme is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 698639.000000 [23:19:17] nacht ts [23:19:43] s1 replag on rosemary is CRITICAL: QUERY CRITICAL: SELECT ts_rc_age() returned 698675.000000 [23:33:03] FMA on yarrow is CRITICAL: ERROR - unexpected output from snmpwalk [23:33:43] SMF on willow is CRITICAL: ERROR - maintenance: svc:/network/puppetmasterd:default [23:34:13] SMF on turnera is CRITICAL: ERROR - offline: svc:/system/cluster/scsymon-srv:default [23:34:53] @replag [23:34:54] Joan: s1-rr-a: 1w 1d 2h 19m 50s [+1.00 s/s]; s1-rr-a-c: 21m 2s [+0.01 s/s]; s1-user: 1w 1d 2h 19m 50s [+1.00 s/s]; s2-user: 28s [+0.00 s/s]; s2-user-c: 17m 51s [+0.01 s/s]; s3-rr-a: 1m 24s [+0.00 s/s]; s3-user: 1m 24s [+0.00 s/s]; s5-user-c: 17m 51s [+0.01 s/s] [23:37:42] RAID on daphne is CRITICAL: ERROR - TOTAL: 2: FAILED: 0: DEGRADED: 1 [23:42:44] SMF on damiana is CRITICAL: ERROR - maintenance: svc:/network/ldap/client:default [23:54:14] s4 replag on rosemary is WARNING: QUERY WARNING: SELECT ts_rc_age() returned 1885.000000