[06:02:02] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/440811 (owner: 10L10n-bot) [07:45:06] !log wikibase-registry Ran update.php to fix problems adding references [07:45:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikibase-registry/SAL [09:38:05] (03CR) 10MarcoAurelio: "recheck" [labs/striker] - 10https://gerrit.wikimedia.org/r/440758 (owner: 10MarcoAurelio) [09:40:17] (03CR) 10jerkins-bot: [V: 04-1] Use #acl*repository-admins instead of #repository-admins [labs/striker] - 10https://gerrit.wikimedia.org/r/440758 (owner: 10MarcoAurelio) [09:42:17] (03CR) 10MarcoAurelio: "I think the same is happening at I7bf4e5dc." [labs/striker] - 10https://gerrit.wikimedia.org/r/421670 (https://phabricator.wikimedia.org/T190543) (owner: 10BryanDavis) [09:47:27] (03PS1) 10Giuseppe Lavagetto: Add token for kubernetes CI [labs/private] - 10https://gerrit.wikimedia.org/r/440836 [09:47:52] (03CR) 10Giuseppe Lavagetto: [V: 032 C: 032] Add token for kubernetes CI [labs/private] - 10https://gerrit.wikimedia.org/r/440836 (owner: 10Giuseppe Lavagetto) [15:20:29] !help I created a Wikimedia Developer Account on page https://toolsadmin.wikimedia.org/register/ (I'd like to develop a Wikidata bot), with username "Olem", linked to my Wikimedia unified account "Olem". However, i entered the password in my Password manager... and I forgot to save! Reset password page (https://wikitech.wikimedia.org/wiki/Special:PasswordReset) says "There is no user by the name 'Olem'". However, I cannot crea [15:20:29] olem: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [15:22:12] olem: this is a known problem. If you can create a phabricator task about this and assign it to me I should be able to help get your developer account attached to wikitech so that the password reset works for you. [15:27:44] bd808: Thank you! I've created a Phabricator account with my Wikimedia unified account. I'm currently waiting for approval by an administrator. I'll create a task when my account is approved. [15:27:51] I can approve that now [15:28:06] olem: Phab account approved [15:34:02] Reedy: Thanks! [15:36:50] The high level problem is T174469 [15:36:51] T174469: LDAP account that is not attached on wikitech has no means for password reset - https://phabricator.wikimedia.org/T174469 [15:40:12] bd808: if I'm trying to run a Python script on Grid Engine and I'm getting GCC errors of all things, what do I even do? [15:40:25] * hare goes to retrieve the exact error [15:40:59] "libgcc_s.so.1 must be installed for pthread_cancel to work" [15:46:45] missing library? [15:48:28] as a generic not helpful response, deploying dependencies with your script to a cloud is basically containers. Do we have support yet for people to upload their own containers? [15:49:49] ebernhardson: there's no bring-your-own-container support for docker, but with running scripts on the grid engine you can set up a venv and install dependencies in there [15:50:27] though it's worth nothing that i'm not on a traditional venv here – this is running on Miniconda because there are some weird dependencies [15:51:02] hare: well, the problem you are running into though is that your python dependencies require system level dependencies when they compile [15:52:05] right, hence miniconda (which is basically python + fancy binaries) [15:52:29] what's interesting is that this all works when i run the script interactively on the bastion (bad harej!), but not when run on the grid engine [15:52:42] Which is pretty consistent with my overall experience with Grid Engine [15:52:54] i'm not surprised the machines have different stuff installed. [15:53:44] But should it matter, if I have what's effectively a venv that brings its own dependencies? [15:54:00] (I use the venv-specific Python executable, not the general one) [15:56:19] it should matter, because the thing your script is complaining about can't come from venv, miniconda, or whatever else [15:57:05] it has to come from the system. Potentially installing the `libgcc1` package to all grid-engine hosts would allow your job to continue. But in the longer term this is a game of wack-a-mole and not really a great solution [15:57:27] (we play that game of wack-a-mole on the hadoop cluster too though ...) [15:57:50] And in general, you shouldn't have to install random stuff to fulfill random tenant needs [16:05:13] srish_aka_tux__: running late from my last meeting. I will join you soon :) [16:14:27] hare: can this run under kubernetes? [16:15:05] It probably could. It's not a webservice but would probably still benefit from being on a k8s pod. [16:25:01] hare: fyi, https://wikitech.wikimedia.org/wiki/Help:Toolforge/FAQ#Why_am_I_getting_errors_about_libgcc_s.so.1_must_be_installed_for_pthread_cancel_to_work? [16:25:14] for that specific error [16:25:27] Ooh, thank you [16:26:39] ooooh, its the weird grid engine class of errors, "give me more memory" [16:28:46] Wait that's it? Boy talk about a useless error message. [16:29:05] What's the default? [16:29:34] 500mb [16:29:43] 512mb, actually [16:30:38] and the parameter I want is -mem 1G or something like that? [16:30:52] exactly that should work [16:31:38] hare: and at least its giving you an error, mono bots just get stuck at 100% cpu if you don't give enough memory :D [16:31:53] Computers. [16:32:52] Okay, now I put the thing on the job grid. Let's monitor it. [16:32:53] "working hard? or hardly working?" [16:36:02] hardly working hard [16:36:25] working hard at hardly working [16:37:05] what would be nice would be a page like https://tools.wmflabs.org/admin/oge/status but for a specific task number so I can periodically refresh a page and see if my job is dead [16:38:46] a web interface for `watch qstat -J jobid`? [16:40:18] I suppose, though that works too (it has to be a lowercase j though) [17:51:49] hare: I would *love* to have a good tool for looking up the grid engine accounting data for all the jobs started by a tool. Using the grid engine cli tools for this can be a really horrible experience today in part because we do not rotate the data files often enough and this leads to serious IOPS load on the NFS server. [17:52:49] the https://tools.wmflabs.org/grid-jobs/ uses some interesting hacks and a lot of caching to try and work around this problem, but does not report much on the actual job level status [18:04:08] bd808: Hi, Do you have some time ? [20:04:08] Neha16: I'm around now if you still have questions. Emails always work too. :) [21:59:17] !log wpx Added BryanDavis (self) as admin to work on T197517 [21:59:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wpx/SAL [21:59:19] T197517: Create redirect from partnermetrics.wmflabs.org to Toolforge - https://phabricator.wikimedia.org/T197517 [21:59:54] bd808: note that I've since moved the redirect logic to wpx-redirects-01 [22:00:19] (wpx-prod-01 is a dinosaur and my eventual plan is to delete it as soon as I move things off of it) [22:00:54] toolforge dog food tastes like hard work [22:01:24] Toolforge is not the magic solution to all problems ;) [22:01:57] My hope is that it's at least the solution for its advertised use cases [22:02:26] * bd808 tries to get horizon to realize that he is now a wpx admin [22:03:09] ah ha! The magic 32 project in drop down limit is hiding it from me [22:03:34] * bd808 shakes fist at impedance mismatch with upstream developer assumptions [22:03:57] "No one will ever administer more than 32 projects in Horizon." -- Bill Gates, 1984 [22:05:53] !log wpx Removed proxy config for https://partnermetrics.wmflabs.org/ [22:05:54] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wpx/SAL [22:06:15] !log redirects Added proxy config for https://partnermetrics.wmflabs.org/ (T197517) [22:06:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Redirects/SAL [22:06:17] T197517: Create redirect from partnermetrics.wmflabs.org to Toolforge - https://phabricator.wikimedia.org/T197517 [22:07:10] !log wpx Removed BryanDavis (self) from project (T197517) [22:07:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wpx/SAL [22:13:02] thank you! [22:13:29] hare: I made a note on the task of a bit of work you will need to do in the tool still [22:13:51] the redirect project's magic is not complete for your use case [22:16:13] hare: also, thank you for giving me an easy thing to do! I'm tired of all the hard things today [22:16:22] :D