[00:04:13] aliasd on nightshade is CRITICAL: Connection refused [00:30:06] Hello wolfgang42 [00:30:15] Hi [00:30:36] I will /msg you, so we can talk without bothering the channel [00:30:42] OK [00:49:44] Free Memory on damiana is WARNING: WARNING - 6.2% (523124 kB) free! [00:53:53] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [00:55:43] Free Memory on damiana is OK: OK - 7.1% (594176 kB) free. [01:04:14] aliasd on nightshade is CRITICAL: Connection refused [01:53:54] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [02:04:14] aliasd on nightshade is CRITICAL: Connection refused [02:53:54] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [03:04:14] aliasd on nightshade is CRITICAL: Connection refused [03:53:54] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [04:04:14] aliasd on nightshade is CRITICAL: Connection refused [04:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [05:04:14] aliasd on nightshade is CRITICAL: Connection refused [05:23:40] The centralauth_p tables are gone. Is this related to the s7 problems? [05:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [06:04:13] aliasd on nightshade is CRITICAL: Connection refused [06:26:54] Free Memory on damiana is WARNING: WARNING - 6.3% (528868 kB) free! [06:27:54] Free Memory on damiana is CRITICAL: CRITICAL - 3.6% (305324 kB) free! [06:30:53] Free Memory on damiana is OK: OK - 11.5% (964788 kB) free. [06:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [07:04:14] aliasd on nightshade is CRITICAL: Connection refused [07:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [08:04:13] aliasd on nightshade is CRITICAL: Connection refused [08:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [09:04:14] aliasd on nightshade is CRITICAL: Connection refused [09:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [10:04:14] aliasd on nightshade is CRITICAL: Connection refused [10:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [11:02:13] aliasd on nightshade is OK: TCP OK - 0.005 second response time on port 984 [500 Not found.] [11:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [12:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [13:27:54] Free Memory on damiana is CRITICAL: CRITICAL - 3.5% (294660 kB) free! [13:29:54] Free Memory on damiana is WARNING: WARNING - 5.6% (467964 kB) free! [13:34:53] Free Memory on damiana is OK: OK - 7.4% (619808 kB) free. [13:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [14:32:53] Free Memory on damiana is WARNING: WARNING - 5.9% (494116 kB) free! [14:38:54] Free Memory on damiana is OK: OK - 7.1% (595764 kB) free. [14:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [15:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [16:24:54] Free Memory on damiana is WARNING: WARNING - 5.7% (476084 kB) free! [16:27:53] Free Memory on damiana is CRITICAL: CRITICAL - 3.6% (304096 kB) free! [16:30:54] Free Memory on damiana is WARNING: WARNING - 5.8% (489980 kB) free! [16:51:54] Free Memory on damiana is WARNING: WARNING - 6.5% (546448 kB) free! [16:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [17:01:54] Free Memory on damiana is OK: OK - 7.1% (598772 kB) free. [17:45:54] Free Memory on damiana is WARNING: WARNING - 5.9% (493712 kB) free! [17:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [17:55:53] Free Memory on damiana is CRITICAL: CRITICAL - 3.8% (321224 kB) free! [18:04:54] Free Memory on damiana is WARNING: WARNING - 5.1% (423656 kB) free! [18:10:54] Free Memory on damiana is OK: OK - 7.2% (602720 kB) free. [18:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [19:27:24] toolserver.org HTTP on wolfsbane is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:27:33] Environment IPMI on wolfsbane is CRITICAL: CHECK_NRPE: Socket timeout after 30 seconds. [19:37:14] Environment IPMI on wolfsbane is OK: ok: temperature ok fan ok voltage ok chassis ok [19:38:13] toolserver.org HTTP on wolfsbane is OK: HTTP OK: HTTP/1.1 200 OK - 239 bytes in 0.175 second response time [19:52:44] hello [19:52:55] i will move s6 tonight [19:53:00] i am going to prepare [19:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [19:59:25] ok i will now shut down s6 [20:06:54] MySQL on z-dat-s6-a is CRITICAL: Cant connect to MySQL server on z-dat-s6-a (146) [20:13:31] I'm having trouble getting scheduled jobs to run properly. [20:13:54] See http://etherpad.wikimedia.org/delinker for details. [20:14:57] cgcreate: can't create cgroup users/delinker/923807-undefined: Cgroup, operation not allowed [20:15:06] No idea what this means. [20:15:25] Things seem to have broken somehow end of November. [20:15:32] Any ideas? [20:15:55] siebrand: currently only know sge was broken at the end of november [20:16:19] nosy: And has it supposedly been fixed, or is it still broken? [20:16:31] nope supposed to be fixed [20:16:45] but it hung in a strange way [20:17:35] nosy: Who would be able to help me in getting this fixed again? [20:20:13] siebrand: oh...this problem is not related to sge is it? [20:20:30] honestly i dont exactly know what cgcreate is [20:20:36] nosy: I have no idea. I don't know where "cgcreate: can't create cgroup users/delinker/923807-undefined: Cgroup, operation not allowed [20:20:37] " comes from. [20:20:50] but google says https://wiki.archlinux.org/index.php/Cgroups [20:20:56] cgroups (aka control groups) is a Linux kernel feature to limit [20:21:00] linux... [20:21:04] solaris... [20:21:09] siebrand: I've seen something similar. how do you submit that job? [20:21:58] johang: I don't know… Bryan configured that, and the only thing I know is how to update commons delinker sources, and kill a stuck process (qdel). [20:22:12] Bryan is not active anymore, as far as I know. [20:22:26] you maintain delinker now? [20:23:14] well, I've done so for the past X years, on and off. [20:23:36] But it's not been able to start since end of November. [20:23:44] It's starting to be a serious problem by now. [20:23:49] i see. how is the regarding program called that is submitted? [20:24:11] I'm thinking of stepping down, because I don't have the time anymore to shove this shit away :( [20:24:27] start-cd, start-cdh? [20:24:47] I've seen that before, but I don't know how/where that's configured. [20:25:12] there is a cron job on clematis [20:25:19] looks like this: [20:25:21] * * * * * cronsub -l cd $HOME/bin/start-cd [20:25:21] * * * * * cronsub -l cdh $HOME/bin/start-cdh [20:25:24] climatic. Never heard of. [20:25:50] clematis? climatic? typo? :P [20:26:09] cronie -l -u delinker [20:26:24] Autocorrect :( [20:26:29] can it be the command that should be called? [20:26:35] very funny though [20:26:41] you should use qcronsub [20:26:55] When I logged in 16 mins ago, qstat -u delinked gave me two lines of output. [20:27:05] They're out of my scroll back now :( [20:27:41] They're there again, so I pasted them in the etherpad.s [20:28:19] the names of the jobs can be related to the scripts with some luck [20:32:59] nosy: Anyway, thanks for the help. I think the end of CommonsDelinker is near. [20:36:11] is it? [20:36:43] @replag [20:36:44] Danny_B|backup: s1-rr-a-wd: 10s [-0.00 s/s]; s1-user-wd: 10s [-0.00 s/s]; s2-user: 14s [-0.00 s/s]; s2-user-wd: 10s [-0.00 s/s]; s3-user: 34s [-0.00 s/s]; s4-user-wd: 20m 47s [+0.02 s/s]; s5-user-wd: 10s [-0.00 s/s]; s6-user: error [20:36:45] Danny_B|backup: s7-user-wd: 10s [-0.00 s/s] [20:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [20:54:54] Free Memory on damiana is WARNING: WARNING - 6.7% (560304 kB) free! [21:00:54] Free Memory on damiana is OK: OK - 7.2% (603324 kB) free. [21:06:54] MySQL on z-dat-s6-a is CRITICAL: Cant connect to MySQL server on z-dat-s6-a (146) [21:44:54] MySQL on z-dat-s6-a is OK: Uptime: 38 Threads: 5 Questions: 389 Slow queries: 4 Opens: 79 Flush tables: 1 Open tables: 72 Queries per second avg: 10.236 [21:54:04] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [21:55:32] s6 is back again [22:00:06] Marlen Caemmerer * Re: [Toolserver-l] Maintenance: Move of s6 [22:32:53] Free Memory on damiana is WARNING: WARNING - 5.6% (468492 kB) free! [22:38:54] Free Memory on damiana is OK: OK - 7.1% (590772 kB) free. [22:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146) [23:54:03] MySQL on z-dat-s4-a is CRITICAL: Cant connect to MySQL server on z-dat-s4-a (146)