[00:17:57] Аlⅼaһ іs doіnɡ [01:16:13] Allah is dοiᥒg [01:47:39] Аlⅼah іs ԁoing [02:38:30] Allаh is dഠiᥒg [03:20:25] Alⅼаh iѕ ⅾoіnɡ [03:20:25] suᥒ iѕ ᥒot doіng Allaһ ⅰs dоiᥒɡ [03:20:25] ⅿoοᥒ iѕ not ⅾoiᥒg Aⅼlah ⅰs dഠⅰng [07:57:42] Aⅼⅼɑһ ⅰѕ doiᥒɡ [07:57:42] ѕun іѕ not ԁⲟinɡ Αlⅼaһ iѕ ⅾoiᥒg [07:57:42] ⅿooᥒ is not ԁoiᥒg Allaһ іs ԁoіnɡ [09:14:24] Alⅼah ⅰs dഠіᥒg [09:14:24] s∪n iѕ not ԁоing Aⅼlah is doіnɡ [10:18:23] Allɑh іs doiᥒg [10:18:23] sᥙᥒ іs nоt ⅾഠⅰng Αllah іѕ dοing [10:18:23] moon is nഠt ԁoinɡ Allah iѕ doіnɡ [12:14:02] Alⅼah iѕ doing [13:16:08] Αllɑh is ⅾoiᥒɡ [13:16:08] sᥙn is not doіᥒɡ Αlⅼaһ іs doing [13:16:08] ⅿooᥒ іѕ ᥒഠt dഠіng Allaһ iѕ ԁоiᥒɡ [13:40:08] Such annoying spam :( [14:16:41] (03PS1) 10D3r1ck01: setup.sh: Enhancement of the script for better understanding [labs/tools/awmd-stats] - 10https://gerrit.wikimedia.org/r/460704 [14:28:17] (03CR) 10D3r1ck01: [C: 032] setup.sh: Enhancement of the script for better understanding [labs/tools/awmd-stats] - 10https://gerrit.wikimedia.org/r/460704 (owner: 10D3r1ck01) [14:28:49] (03Merged) 10jenkins-bot: setup.sh: Enhancement of the script for better understanding [labs/tools/awmd-stats] - 10https://gerrit.wikimedia.org/r/460704 (owner: 10D3r1ck01) [14:45:38] Aⅼⅼaһ іs doіnɡ [15:42:20] Aⅼlah ⅰѕ ԁоіnɡ [17:40:26] Hi [17:41:47] Is there any way to access to daily pageview prior to 2015 besides paring the mediawiki hourly data dump? [17:41:58] I really appreciate it:( [17:42:01] :) [19:27:25] (03PS1) 10Dbarratt: Add Wikibase Site [labs/tools/graphql] - 10https://gerrit.wikimedia.org/r/460709 [19:27:47] (03CR) 10Dbarratt: [V: 032 C: 032] Add Wikibase Site [labs/tools/graphql] - 10https://gerrit.wikimedia.org/r/460709 (owner: 10Dbarratt) [21:06:50] Aⅼlɑh іѕ doinɡ [21:14:31] (03PS1) 10Lokal Profil: Add tests for images_of_monuments_without_id [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460712 [21:16:54] (03CR) 10jerkins-bot: [V: 04-1] Add tests for images_of_monuments_without_id [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460712 (owner: 10Lokal Profil) [21:17:08] (03PS1) 10Lokal Profil: Tidy up docstrings, string formats and linebreaks. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460715 [21:19:07] (03CR) 10jerkins-bot: [V: 04-1] Tidy up docstrings, string formats and linebreaks. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460715 (owner: 10Lokal Profil) [21:27:21] Alⅼaһ ⅰs ⅾoing [21:35:46] (03PS1) 10Lokal Profil: Make template addition dependent on starting parameter [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460716 (https://phabricator.wikimedia.org/T204351) [21:37:29] !help a job by one of my tools (tool wd-shex-infer, job name wd-shex-infer-35) appears to have died with SIGKILL (exit_status 137) for no reason I can discern (nothing suspicious in its stdout/stderr) [21:37:29] WikidataFacts: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [21:37:29] (03CR) 10jerkins-bot: [V: 04-1] Make template addition dependent on starting parameter [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460716 (https://phabricator.wikimedia.org/T204351) (owner: 10Lokal Profil) [21:37:46] perhaps it was killed manually by someone? is there a way to find out? (IRC logs doesn’t show anything special) [21:38:35] (the job’s line from the accounting file: https://gist.github.com/lucaswerkmeister/fb6e129467aa41cef9a179d45840c2e9) [21:40:13] Just now? [21:40:37] no, around 13:27 UTC today, I just noticed it now [21:41:25] could it have run out of memory? would that result in SIGKILL? [21:42:32] Hmm possible. I cannot check now, but I can have a look to see if the OOM killer went after it [21:42:48] yes, it can, basically [21:43:24] oh, the kernel OOM killer? [21:43:29] ouch [21:46:38] (03PS2) 10Lokal Profil: Add tests for images_of_monuments_without_id [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460712 [21:52:02] (03PS2) 10Lokal Profil: Tidy up docstrings, string formats and linebreaks. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460715 [21:57:34] (03PS2) 10Lokal Profil: Make template addition dependent on starting parameter [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460716 (https://phabricator.wikimedia.org/T204351) [22:12:17] (03CR) 10Lokal Profil: "see comment on follow-up patch" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459405 (owner: 10Lokal Profil) [22:16:56] Will check soon. [23:22:23] WikidataFacts: Interesting, that node had some segfaults, but not at that time. Probably not related. [23:22:40] Poking around to see if there's any reason I can find [23:23:35] I honestly don't see much. I'm not aware of any maintenance. [23:23:39] ok, thanks! [23:24:18] it is a job that potentially uses a lot of memory, so if that results in OOM instead of some kind of ENOMEM, that could be the reason [23:24:40] is it okay if I retry the job and see if it goes through this time? (perhaps there’ll be less activity at nighttime…) [23:25:56] That node is still scheduled, nothing weird [23:26:21] I run a job that uses a ton of memory. It's python, though and the err log was pretty clear that it ran out of mem. [23:26:32] May as well [23:26:33] :) [23:26:38] ok :) [23:26:43] If you need to, you can increase the vmem ask on jsub [23:27:20] I already am, -mem 8g :D [23:27:21] I do this for my memory hog job: `jsub -l h_vmem=3145728k /data/project/cdnjs/update-index.sh` [23:27:23] haha [23:27:28] That's pretty intense [23:27:36] yeah :/ [23:27:41] Well, good luck. I didn't find a log of an OOM, but it could have been [23:27:45] at that size [23:28:01] I’ll try again and see what happens… thanks anyways! [23:28:17] Total mem on an exec node: `7.8G` [23:28:26] They are around 8G in general [23:28:29] ah [23:28:31] That's what free -h gives [23:28:37] So you may be hitting the upper limit [23:29:00] There's swap [23:29:03] so I might be hitting system OOM before the per-process resource limit? [23:29:09] But if there's other stuff, you never know [23:29:11] (or however the grid engine implements it… cgroups?) [23:29:28] grid engine doesn't do cgroups out of the box...it's kind of old [23:29:41] (actually it could, but I'd be seriously surprised) [23:30:09] The grid engine limits we have set up probably are based purely on "slots" and free mem at the time of submit [23:30:20] To grid engine, I think these nodes have 8G each [23:30:33] They have a LOT of swap to keep it alive as well, but you never know. [23:30:43] You could hit the kernel OOM if other jobs are running [23:30:45] Quite possible [23:31:16] repeated job started [23:31:34] I probably won’t be awake when it finishes (one way or another), but I’ll check tomorrow [23:31:40] Good luck though. If it's a genuine scaling issue, then we might want to do a task and think about it a bit more...maybe it just needs something heavier in a VPS or something [23:31:46] Ok. :) [23:31:50] Hope it goes well [23:31:54] yeah, I was on the fence whether my tool should use VPS or Toolforge [23:32:01] perhaps I should’ve gone for VPS [23:32:03] thanks!