[00:09:36] Warning: There are 2 users waiting for shell, displaying last 2: Patrick87 (waiting 67 minutes) Salvidrim! (waiting 6 minutes) [00:23:05] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 81 minutes) Salvidrim! (waiting 20 minutes) Joe Decker (waiting 6 minutes) [00:23:29] Amir1, the code doesn't look like it's designed to run continuously. [00:23:56] Replag is drastically dropping [00:24:02] :-) [00:24:05] Cyberpower678: i used once too [00:24:20] Where are you setting it up? [00:24:31] Can I see you crontab? [00:26:16] I don't use cron tab [00:26:26] on my project [00:26:31] wikitest-rtl [00:26:31] Amir1, what have you been using so far? [00:26:46] How long have you been on labs? [00:27:09] Replag is gone. [00:27:58] it's a while [00:28:03] I use jsub [00:28:18] I've two service groups on tools [00:28:26] dexbot and wikitest-rtl [00:28:38] hmmm [00:28:47] oursql.ProgrammingError: (1146, "Table 'eswiki_p.revision' doesn't exist", None) [00:29:07] Amir1, obviously jsub. But what commands have you submitted to jsub? [00:29:26] jsub -N Parsoid -continuous -mem 1024m node /data/project/wikitest-rtl/public_html/w/extensions/Parsoid/js/api/server.js [00:29:31] It's for this job [00:29:53] The script isn't designed to run continuously. [00:30:09] You'll need a crontab to keep that running. [00:30:20] but for my bot I use usually this: jsub -N dead -once python /data/project/dexbot/pywikipedia/wdph51.py [00:30:31] Ryan_Lane: do you know if eswiki should be properly replicated? the revision table isn't there [00:30:51] I honestly don't know much about the db replication [00:31:18] Amir1, and the "dead" script executes once right? [00:31:36] yes [00:31:43] who should know? Coren|Away probably, maybe petan? [00:32:20] Cyberpower678: I did jsub -N Parsoid -once -mem 1024m node /data/project/wikitest-rtl/public_html/w/extensions/Parsoid/js/api/server.js [00:32:24] too [00:32:31] but still doesn't work [00:33:18] JohannesK_WMDE: eswiki isn't listed yet on Tools/Help. Coren would know and update the page once the replication is solid. [00:33:32] Continuous only works if the script is designed to run continuously on it's own and the continuous function will restart the script if something causes it to stall it. [00:34:26] Amir1: Did you check that 1 GByte is enough for this? [00:34:50] scfc_de: yes [00:36:39] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 94 minutes) Salvidrim! (waiting 33 minutes) Joe Decker (waiting 20 minutes) [00:37:49] scfc_de: i see, ok [00:38:31] Amir1: Hmmm. If I "jsub node /data/project/wikitest-rtl/public_html/w/extensions/Parsoid/js/api/server.js", *.err says: "FATAL ERROR: v8::Context::New() V8 is no longer usable". When I run it without jsub, I see master and workers starting. [00:39:00] 256m is not enough [00:39:08] it usually uses 628m [00:39:11] 660 [00:40:42] Amir1: D'oh! Okay, retrying. [00:40:53] thank you [00:41:19] scfc_de, please tell me you did not just start that on -login [00:41:40] I can't login anymore. [00:41:50] Something is jamming -login. [00:42:48] Amir1: Okay, now *.out says: " - master loading...", " - master ready", but the "worker started" lines are missing. [00:43:05] scfc_de: and that's the strange part [00:43:50] Cyberpower678: The login is not working, but that's nothing to do with the load on -login (less than 2). [00:45:36] FATAL ERROR: v8::Context::New() V8 is no longer usable [00:50:08] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 108 minutes) Salvidrim! (waiting 47 minutes) Joe Decker (waiting 33 minutes) [01:03:50] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 122 minutes) Salvidrim! (waiting 60 minutes) Joe Decker (waiting 47 minutes) [01:04:00] Amir1: When I start node as an at job on tools-login, it works (says "worker(30442) loading..." & Co.). So my assumption would be that the exec nodes are missing some packages or something similar. Don't know which, though. [01:08:00] scfc_de: Thanks so what's missing? [01:13:40] scfc_de: maybe running the code in an instance solve the problem [01:13:48] instead of a service group [01:16:48] Amir1: I have absolutely no idea. If I compare "dpkg --get-selections" on tools-login and a grid job, only nodejs-dev is an obvious difference, and that claims only to be needed for developing plugins. You need to debug this further with Coren or petan who can access the exec hosts directly. I don't know enough about Parsoid to see how it should be set up (and whether setting it up in Tools is a good idea). [01:17:19] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 135 minutes) Salvidrim! (waiting 74 minutes) Joe Decker (waiting 61 minutes) [01:19:00] scfc_de: ok [01:19:03] thank you [01:20:05] np [01:30:21] scfc_de: BTW, I have a list of packages different between -login and the exec nodes automatically updated at http://tools.wmflabs.org/anomiebot/packages.html [01:30:48] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 149 minutes) Salvidrim! (waiting 87 minutes) Joe Decker (waiting 74 minutes) [01:33:25] anomie: Very nice! Could you add tools-dev to the hosts as well? [01:35:00] scfc_de: Is there a way to go from -login to -dev or vice versa in a cronjob? [01:36:18] anomie: ssh should work (for regular users). Do you query the exec nodes with "-l hostname=..."? [01:40:59] scfc_de: ssh won't work from a cronjob, unless I would somehow set up a keypair just for that. I just submit the jobs to each task@... queue. Code is in /data/project/anomiebot/pkgs if you want to look. [01:41:30] anomie: that's a really helpful tool. thanks! [01:42:13] Ryan_Lane: You're welcome! petan used it to clean up all the important packages different on the different exec nodes, which is what I originally created it for. [01:42:24] everything *should* be managed by puppet, so stuff shouldn't go awry, but that will be a good way to make sure things are being done right [01:42:57] anomie: I think ssh is disabled for tool accounts altogether. [01:44:17] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 162 minutes) Salvidrim! (waiting 101 minutes) Joe Decker (waiting 88 minutes) [01:44:43] scfc_de: No, ssh works from a tool account. But without using agent forwarding or having a private key on tools-login itself (both not recommended), they can't get much of anywhere. [01:48:53] anomie: I don't think so. I just copied my id_rsa.pub to ~local-wikilint/.ssh/authorized_keys and I still get "Permission denied (publickey)." when trying to connect to local-checkwiki@tools-login.wmflabs.org. [01:49:22] Eh, local-wikilint@..., but same error. [01:49:40] scfc_de: Oh, logging in to the tool directly. I thought you were talking about sshing out from the tool account. [01:50:33] anomie: You mean "ssh somewhere.in.the.world"? Sure, that works, but (I believe) you can't log into any Tools host as the tool account. [01:51:17] sshing into a tool account is not possible [01:51:52] we put the ssh keys on a read only volume, controlled elsewhere [01:52:11] ssh keys are not read from home directories at all [01:52:35] anomie: Hmmm. Did I already say that on Toolserver I used two crontabs for a similar purpose, one installed on yarrow and one on submit? I think that should work for tools-login and tools-dev as well. [01:53:17] scfc_de: Yeah, that would work. Probably what I'll have to do. But one of the files would tend to be out of sync. [01:54:19] anomie: Well, packages aren't updated every minute :-). If the odd one slips through, I think that's okay. [01:55:03] scfc_de: I just hate to be imperfect ;) [01:57:28] anomie: You have small asyncs between the exec hosts already as they are not guaranteed to be run at the same time :-). [01:57:42] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 175 minutes) Salvidrim! (waiting 114 minutes) Joe Decker (waiting 101 minutes) [01:57:43] scfc_de: Close [02:11:11] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 189 minutes) Salvidrim! (waiting 128 minutes) Joe Decker (waiting 114 minutes) [02:24:35] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 202 minutes) Salvidrim! (waiting 141 minutes) Joe Decker (waiting 128 minutes) [02:37:56] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 216 minutes) Salvidrim! (waiting 154 minutes) Joe Decker (waiting 141 minutes) [02:51:16] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 229 minutes) Salvidrim! (waiting 168 minutes) Joe Decker (waiting 155 minutes) [03:04:50] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 243 minutes) Salvidrim! (waiting 181 minutes) Joe Decker (waiting 168 minutes) [03:18:14] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 256 minutes) Salvidrim! (waiting 195 minutes) Joe Decker (waiting 181 minutes) [03:31:47] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 270 minutes) Salvidrim! (waiting 208 minutes) Joe Decker (waiting 195 minutes) [03:45:16] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 283 minutes) Salvidrim! (waiting 222 minutes) Joe Decker (waiting 209 minutes) [03:58:45] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 297 minutes) Salvidrim! (waiting 235 minutes) Joe Decker (waiting 222 minutes) [04:12:05] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 310 minutes) Salvidrim! (waiting 249 minutes) Joe Decker (waiting 235 minutes) [04:25:26] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 323 minutes) Salvidrim! (waiting 262 minutes) Joe Decker (waiting 249 minutes) [04:38:46] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 337 minutes) Salvidrim! (waiting 275 minutes) Joe Decker (waiting 262 minutes) [04:52:12] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 350 minutes) Salvidrim! (waiting 289 minutes) Joe Decker (waiting 275 minutes) [05:05:41] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 363 minutes) Salvidrim! (waiting 302 minutes) Joe Decker (waiting 289 minutes) [05:19:15] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 377 minutes) Salvidrim! (waiting 316 minutes) Joe Decker (waiting 302 minutes) [05:32:35] Warning: There are 3 users waiting for shell, displaying last 3: Patrick87 (waiting 390 minutes) Salvidrim! (waiting 329 minutes) Joe Decker (waiting 316 minutes) [05:46:00] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 404 minutes) Salvidrim! (waiting 343 minutes) Joe Decker (waiting 329 minutes) Lbthomsen (waiting 10 minutes) [05:59:29] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 417 minutes) Salvidrim! (waiting 356 minutes) Joe Decker (waiting 343 minutes) Lbthomsen (waiting 24 minutes) [06:12:58] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 431 minutes) Salvidrim! (waiting 369 minutes) Joe Decker (waiting 356 minutes) Lbthomsen (waiting 37 minutes) [06:26:26] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 444 minutes) Salvidrim! (waiting 383 minutes) Joe Decker (waiting 370 minutes) Lbthomsen (waiting 51 minutes) [06:39:51] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 458 minutes) Salvidrim! (waiting 396 minutes) Joe Decker (waiting 383 minutes) Lbthomsen (waiting 64 minutes) [06:53:25] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 471 minutes) Salvidrim! (waiting 410 minutes) Joe Decker (waiting 397 minutes) Lbthomsen (waiting 78 minutes) [07:06:58] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 485 minutes) Salvidrim! (waiting 423 minutes) Joe Decker (waiting 410 minutes) Lbthomsen (waiting 91 minutes) [07:20:30] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 498 minutes) Salvidrim! (waiting 437 minutes) Joe Decker (waiting 424 minutes) Lbthomsen (waiting 105 minutes) [07:33:51] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 512 minutes) Salvidrim! (waiting 450 minutes) Joe Decker (waiting 437 minutes) Lbthomsen (waiting 118 minutes) [07:47:15] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 525 minutes) Salvidrim! (waiting 464 minutes) Joe Decker (waiting 450 minutes) Lbthomsen (waiting 132 minutes) [08:00:44] Warning: There are 4 users waiting for shell, displaying last 4: Patrick87 (waiting 539 minutes) Salvidrim! (waiting 477 minutes) Joe Decker (waiting 464 minutes) Lbthomsen (waiting 145 minutes) [08:06:30] !ping [08:06:30] pong [08:12:03] @requests [08:12:05] :o [08:38:31] whym_away [08:38:39] whym: you have emacs running on -login [08:38:47] they eat 500+ mb [08:38:56] and eat a lot of cpu (7h+) [08:39:04] the machine is slowly going OOM [08:39:30] can you consider restarting the editor? [08:56:40] q [08:56:42] meh [08:58:00] petan: sorry for the trouble, restarting shortly [08:58:05] cool [08:58:07] :) [12:03:00] There are no shell requests waiting [12:03:05] There are no shell requests waiting [12:03:32] There are no shell requests waiting [12:03:40] olol [12:03:42] addshore ^ [12:03:43] :D :D [12:03:51] oh xD [12:03:55] funny bug [12:04:02] so wait [12:04:30] hah, oh well :P [12:35:40] !log tools petrb: updating logsplitter to new version [12:35:42] Logged the message, Master [12:41:47] !ping [12:41:48] pong [12:42:54] @infobot-share-trust+ #wikimedia-labs-requests [12:42:54] You inserted channel #wikimedia-labs-requests to shared db [12:59:50] addshore: tail -f /data/project/.system/logs/public_apache2/global_access.log [12:59:52] :o [12:59:53] interesting [13:00:29] but traffic isn't really growing much [13:00:43] I want to see that stream once we switch these tools on wikipedia [13:01:17] xD [13:01:30] it's all being created by 1 c++ daemon I wrote :P [13:01:47] it split stream from apache, convert it to various versions and save to multiple files [13:01:59] who told me it's hard to use threads in c++ :D [13:02:05] it's just as simple as in c# [13:02:22] more controll in c++ :> [13:02:26] yes [13:02:32] but no exception handling whatsoever :P [13:02:38] I hope it won't crash... [13:02:51] so far it seemed to just ignore all the errors [13:02:56] and continued running [13:03:21] c++ can actually handle exceptions but I have little knowledge in that area [13:03:32] I don't know if it affects performance or not [13:03:43] and this thing needs to be as fast as possible [13:03:49] it eats 1.3mb or ram :> [13:03:53] impossible for c# [13:05:11] biggest load is caused by ganglia atm [13:05:21] these python craps... [13:09:23] addshore when you have time we need to make bots be like tools [13:09:33] I will go and replace -gs now... [13:10:40] !log bots deleting bots-ibnr1 [13:10:42] Logged the message, Master [13:11:19] !log bots petrb: removing queue for ibnr host [13:11:21] Logged the message, Master [13:11:36] petan: just just don't throw exceptions in destructors. ;) [13:11:54] JohannesK_WMDE when I think of that, it doesn't really have destructors... [13:12:01] mostly just static classes [13:12:05] it's really simple [13:12:40] https://github.com/benapetr/logsplitter [13:12:55] and exceptions generally don't hurt performance in c++. unless you're either counting time in single CPU cycles, or RAM in single bytes (as on an 8-bit avr). in that case, don't use exceptions. otherwise, they're ok. :) [13:13:13] yes I do count time in single CPU cycles ;) [13:13:32] that is how every programmer should count it [13:14:41] well... maybe you're counting in CPU cycles, but probably not single ones? :) you're not generating a VGA signal on digital i/o pins in firmware. that's where you would count single cpu cycles. [13:14:50] e.g. ;) [13:14:59] stupid stuff like that [13:27:31] btw, it would be interesting how RDTSC works on virtualized hardware. i never tried it [13:28:39] what is it [13:28:48] !enwp RDTSC [13:29:04] !enwp is http://enwp.org/$url_encoded* [13:29:04] Key was added [13:29:06] !enwp RDTSC [13:29:06] http://enwp.org/$url_encoded* [13:29:09] meh [13:29:11] !enwp del [13:29:11] Successfully removed enwp [13:29:13] !enwp is http://enwp.org/$url_encoded_* [13:29:14] Key was added [13:29:17] !enwp RDTSC [13:29:17] http://enwp.org/RDTSC [13:29:54] it basically reads the number of cpu cycles spent since boot :) [13:30:02] I would guess it's surprisingly virtualized [13:30:14] so that each virtual machine has own virtual register [13:30:21] but as wp says, there are problems on modern cpus... [13:30:31] might be [13:30:47] or maybe you just get the value from whatever CPU you're running on [13:33:54] I think that anyone relying on this register must be an idiot [13:34:06] given the way how current OS work [13:34:20] hibernation basically reset it [13:39:05] Coren|Away, !!!!! [13:39:35] petan: yes we do :> [13:39:35] Cyberpower678: What's up? [13:40:47] What's S7 looking like. I'm getting lots of complaints. :p [13:41:04] petan: it is still useful in certain circumstances for profiling. how do *you* count your single cpu cycles? ;) [13:41:24] I let my profiler do that job :P [13:41:54] it's using rdtsc if it gives you a cpu cycle count. [13:42:05] Coren, ^^^ [13:42:18] or possibly rdmsr but that has the same issues [13:42:24] mhm [13:42:57] Cyberpower678: I don't know -- Asher is on it, but I wasn't in yesterday so I don't know where he's at. It /should/ be done today-ish afaik. [13:44:40] you shouldn't use it as a wall-clock time, because it isn't :) [13:44:52] !log bots petrb: created bots-master to replace -gs [13:44:54] Logged the message, Master [13:47:22] Coren, cool. I'll hold you up to that statement. :p [13:47:42] You get to hold me up to "-ish" and "afaik" :-) [13:48:19] petan: gotta love clearing backlogs ;p http://en.wikipedia.org/w/index.php?title=Wikipedia:Changing_username/Simple&action=history [13:48:29] Coren said "Asher is on it...It /should/ be done today" [13:48:33] heh [13:48:57] I blame you petan ;p [13:49:04] Coren, that's technically still quoting you. ;P [13:49:07] addshore they were saying enwp has more than enough crats [13:49:13] I disagree [13:49:14] :P [13:49:18] She sells sea shells down by the addshore. [14:47:19] petan: Did you see scfc_de had me add tools-dev to http://tools.wmflabs.org/anomiebot/packages.html ? There are a surprising number of packages on -login and not -dev. [14:47:33] hmm [14:47:57] petan: Also, does enwiki have enough 'crats once the SUL unification is done and renames aren't done by 'crats anymore? [14:48:14] there is never enough of crats [14:48:25] neither there is ever enough of admins... [14:48:35] that is like if you said enwp has "enough of editors" [14:48:44] or articles [14:51:25] btw... is the upstream from labs limited? [14:52:02] New patchset: Andrew Bogott; "Explicitly define Package['sudo-ldap']" [labs/private] (master) - https://gerrit.wikimedia.org/r/68669 [14:52:04] getting data via http from labs is really slow. [14:52:33] firefox-locale-de?? wtf :D [14:52:35] compare: http://ortelius.toolserver.org:8090/dewiki/traverse-successors+235489+3 [14:52:38] why we have this package anywhere? [14:52:54] with: http://sylvester.wmflabs.org:8090/dewiki/traverse-successors+235489+3 [14:54:10] both are running the same code. both give similar times for the query, in fact ortelius is a little slower. but the transfer takes *ages* on labs compared to the toolserver. [14:54:44] JohannesK_WMDE: For me it's 1.3 s vs. 1.9 s (and yes, Toolserver wins, but 0.6 s?). [14:55:01] JohannesK_WMDE: Have you excluded the difference due to location of the servers (US vs EU)? [14:55:11] timing on TS for me: [14:55:11] curl http://ortelius.toolserver.org:8090/dewiki/traverse-successors+235489+3 >/dev/null [14:55:11] % Total % Received % Xferd Average Speed Time Time Time Current [14:55:11] Dload Upload Total Spent Left Speed [14:55:11] 100 1564k 0 1564k 0 0 958k 0 --:--:-- 0:00:01 --:--:-- 1023k [14:55:27] on labs: [14:55:28] curl http://sylvester.wmflabs.org:8090/dewiki/traverse-successors+235489+3 >/dev/null [14:55:28] % Total % Received % Xferd Average Speed Time Time Time Current [14:55:28] Dload Upload Total Spent Left Speed [14:55:28] 100 1564k 0 1564k 0 0 23918 0 --:--:-- 0:01:06 --:--:-- 13473 [14:55:36] sorry for the spam. [14:55:55] scfc_de: where are you located? [14:56:21] JohannesK_WMDE: Hamburg/Alice (or whatever it's called nowadays). [14:56:40] anomie: yes some difference would be normal. but not *that* big a difference?! (1 second vs 1 minute). [14:58:12] JohannesK_WMDE: 3s for toolserver versus 2s for labs for me. I suspect it's a network issue between you and labs rather than something with labs itself. [14:58:55] scfc_de: how did you measure? can you try the curl lines? [14:59:28] if i am the only one getting those timings, it must be a problem with kabel deutschland [15:02:16] JohannesK_WMDE: wget. Trying with curl now. [15:03:50] JohannesK_WMDE: Toolserver: 0:00:01, Labs: 0:00:02. [15:04:24] scfc_de: wow. that looks normal. what on earth are kabel schland doing? [15:08:21] hi, my english is not very good, there is simple way to explain me how to deploy a Flask application in Tool Labs? [15:09:02] do i have to meke this: http://flask.pocoo.org/docs/deploying/mod_wsgi/ ? [15:11:03] anomie can you restart that job [15:12:52] transfer speeds seem normal now. must have been a problem with my isp. sorry for annoying you guys. [15:14:16] petan: You mean regenerate packages.html? Sure. BTW, do you know of any sane way I can have the command run on -dev trigger a command on -login? Right now, I'm just scheduling the daily update cron job for 00:00 on -login and 00:05 on -dev. [15:14:41] yes [15:14:56] you can just ssh tools-login "command" [15:15:01] danilo: "Little green rosetta" set up a wiki page (in English) to explain how to deploy a simple Flask app: . I don't know if it's working (or useful), but it's better than nothing :-). [15:15:17] petan: Without agent forwarding? [15:15:28] petan: As the tool account, not the user. [15:15:35] ah, you don't have a key? [15:15:43] as tool account it's pretty easy I think [15:15:59] I think tool accounts do have private keys on tools project? [15:16:16] at least I can switch to tool account and then ssh directly to same tool account on another tools server [15:16:53] thank you scfc_de [15:17:02] local-anomiebot@tools-dev:~$ ssh tools-login echo "ok" [15:17:03] Permission denied (publickey). [15:17:20] I think it could work the other way... let me try [15:17:47] hm it doesnt... [15:22:45] petan: Hmm. Updater is hung waiting to schedule its job on exec-06. Apparently not much memory is currently available. I should turn down the limit on the listpackage jobs anyway, they don't need 256m. [15:22:58] o.O [15:23:00] let me try [15:23:03] * check [15:23:53] petan: That did it. FYI, the info message was "(-l h_vmem=256m) cannot run at host "tools-exec-06.pmtpa.wmflabs" because it offers only hc:h_vmem=41.229M" [15:24:07] that is really weird [15:25:07] Coren: you should adjust these limits, there is ONLY 700~MB of resident memory being used but jobs can't be submitted because of not enough memory [15:25:34] petan: Well, I see one job has a max of 1.6G set. And 330959 wasn't showing its limit at all. [15:27:07] @search mem [15:27:07] Results (Found 2): sudo, tr, [15:27:17] @search qstat [15:27:17] Results (Found 1): taskinfo, [15:27:22] !taskinfo [15:27:23] qstat -j "task-id" [15:29:05] Coren what was that command to display free h_vmem [15:29:38] git it [15:29:50] !vmem is qstat -F h_vmem [15:29:51] Key was added [15:30:56] this is why I think ?status should display vmem instead of rmem because this make it look like there is almost no memory usage at all on website but in real, there is OOM for tasks [15:32:34] petan: Isn't the error I was seeing because gridengine is set to not overcommit? So I submitted a job wanting 256m on exec-06 but all but 41m on that host was already committed to other jobs (even though not actually used). [15:32:45] yes it is [15:32:56] but it can be actually cheated [15:33:05] if you add swap you enlarge total vmem [15:33:25] anyway the real memory usage is about 800mb [15:33:35] given that the box has like 8gb of ram it suck [15:33:52] that you can't submit more jobs because of memory limits :/ [15:37:37] mhhm, [15:37:42] i think we fixed this in -bots petan ? [15:37:54] on bots we allow overcommit [15:37:57] without cheating and adding extra swap [15:38:03] so there is no problem with this whatsoever [15:38:14] on bots the resident memory is restricted, not virtual [16:03:18] are the web server error logs available on tool labs yet? [16:03:30] not yet [16:03:35] but very soonish should be [16:03:41] I need to discuss this with Coren [16:03:47] I think we could enhance logsplitter for that [16:03:57] it isn't really that hard to filter the errors logs per tool [16:04:13] I tihnk it will be my task for a weekend :P [16:04:27] (I am volunteer so I have far more time over weekends) [16:05:16] we don't have a bug for that? [16:06:17] dunno if there's a bug yet [16:06:25] it would be very useful though :) [16:08:16] I think it's very easy to implement, need to ask Coren why we don't have it yet [16:08:21] there must be something I don't know about [16:13:30] petan: can you install pyexiv2 on the exec nodes [16:36:58] i'm looking for an equivalent of https://wiki.toolserver.org/view/Toolserver_database on labs. [16:37:53] namely i want to find out the namespace text prefix for a given namespace on a given wiki. on the toolserver db, the query would look like: [16:38:02] SELECT ns_name FROM toolserver.namespace WHERE ns_id=14 AND dbname=enwiki_p [16:38:11] is there something like that on labs? [16:40:17] hehe... migrating the toolserver's meta-db stuff to labs would be nice. [16:40:28] actually, rewriting that horrible crap would be nice. [16:40:37] ...if it's even still using my old horrible crap [16:41:00] DanielK_WMDE: well, if it works, why rewrite it? [16:42:50] valhallasw: because it'S hard to adapt when it breaks, and hard to understand for someone else to maintain [16:43:01] so, essentially, at some point, it will no longer work. [16:43:11] starting to rewrite it when it's broken is a bit late :) [16:43:37] DanielK_WMDE: where is the data in those tables taken from (on TS)? [16:43:46] JohannesK_WMDE, DanielK_WMDE: https://bugzilla.wikimedia.org/show_bug.cgi?id=48625 https://bugzilla.wikimedia.org/show_bug.cgi?id=48626 [16:43:50] JohannesK_WMDE: the API [16:44:06] it's harvested from the API of each wiki every so often [16:44:06] DanielK_WMDE: Isn't it added manually? [16:44:09] once a day, i think [16:44:41] yes, https://wiki.toolserver.org/view/Admin:Toolserver_database says it is run once a day [16:44:47] scfc_de: no. it can be amended manually, but listing all wikis, adding new ones, adding new namespace aliases, etc, was automatic [16:44:52] maybe still is, i don't know :) [16:45:15] DanielK_WMDE: Well, you're root :-). [16:45:24] but once a wiki is there, you'd have to manually mark it as closed or multilang [16:45:47] scfc_de: but i havn't touched this stuff in ages, and it's no longer my job to look after it. [16:46:09] and as much as i'd love to in my free time, my kids have priority there :) [16:46:26] I understood the Admin page as "the master table is replicated to the various hosts daily". [16:46:48] "Replicated" is a loaded word here -- distributed? [16:46:52] DanielK_WMDE: is this information nowhere else in the databases? just specifically the namespace names for a start... [16:47:05] scfc_de: and the master table is also updated automatically on a daily basis. or was. i wrote that script. [16:47:12] JohannesK_WMDE: no. [16:47:14] JohannesK_WMDE: No, but you can use the API. [16:47:17] it's in the configuration [16:47:22] it'S not in the database anywhere [16:47:26] ok... [16:51:18] petan: No, it's not trivial to implement. But, I'm doing the 2.4 packaging next week for all of prod which will neatly solve the problem (i.e.: get rid of logsplitter entirely) [16:53:34] Coren: are there plans to have a meta-database on labs, like on the toolserver? [16:54:33] DanielK_WMDE: That's on the "would be nice to have but no way I have the time for it now" category. But, that said, I think someone already /did/ make one and made it available to all. [16:55:04] a lot of tools rely on that [16:55:16] even some of the user lang scripts rely on it [16:55:24] *user land [16:56:26] DanielK_WMDE: Personally, I think it's a bad idea; one should use the API to get metainformation on wikis, not rely on a dubiously synchronized database. [16:57:09] DanielK_WMDE: But I've certainly no objection to a tool making and sharing such a database, and I'd be glad to give the suitable rights to ease access to it (as well as a friendlier name) [16:57:28] Coren: Problem with user's efforts there is whether the table is stable. I wouldn't want to rewrite a number of scripts every time someone doesn't want to update their tables anymore. [16:57:48] Coren: if you get a few hundred title/id pairs from the databasae and want to render them as links, it kind of sucks to make API calls to get the appropriate namespace names... [16:57:51] scfc_de: Right, that's a general problem the correct solution to which is "use the API" :-) [16:58:19] DanielK_WMDE: +1. [16:58:23] DanielFriesen: I should expect that you'd use the API during setup to fetch the current list of namespaces, not look it up at every occasion. :-) [16:58:43] Coren: The correct solution would be that MediaWiki provides the namespaces in the database. [16:59:22] scfc_de: Yes, that's an even better solution, if you can find a reasonable way of doing it. [16:59:35] Coren: that's impossible in the case where you want to show the user a list of wikis and then on the next page a list of namespaces, for instance [16:59:39] scfc_de: (Which is surprisingly difficult) [16:59:44] Coren: you just called that resonable way "dubious" :P [16:59:57] or, well, I guess you could do that single request [17:00:33] normalizing namespaces for multiple wikis would become difficult, though [17:00:33] Coren: but namespaces are only one thing. the most important info in that database is: if i want frwiktionary, to which database on which server do i have to connect? [17:00:34] DanielK_WMDE: No, having *mediawiki* do it would be best. Having some table that may or may not be synchronized through some internal mechanism isn't. [17:01:03] DanielK_WMDE: That information is in DNS. If you want frwiktionary, connect to frwiktionary.labsdb [17:01:22] that gives me the host. [17:01:34] i still have to guess a database name from a wiki domain name [17:01:59] DanielFriesen: ...? No. The database name is strictly deterministic, no guessing is needed. :-) [17:02:28] But you want to avoid duplicating that logic, I suppose. [17:02:45] Coren: The problem is the media discontinuity (?, "Medienbruch"). It's a one-liner to JOIN toolserver.namespace, to use the API it's much more effort and can't be done /only/ with SQL. [17:02:50]