[04:05:20] hello everyone [04:05:35] I have a question. [04:06:14] How I can Upload .png on on tool at Toolforge [04:44:14] !log codesearch restarted hound-search service (T188477), also I restarted hound-extensions earlier (T188450) [04:44:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Codesearch/SAL [04:44:18] T188477: CodeSearch: Add Extension:UploadWizard - https://phabricator.wikimedia.org/T188477 [04:44:18] T188450: CodeSearch: SocialProfile extension is not being indexed by Codesearch - https://phabricator.wikimedia.org/T188450 [04:45:20] Jayprakash12345: from where to toolforge? [04:45:26] your local computer? [04:45:57] or a remote repository? [04:50:16] zhuyifei1999: From Local Computer [09:08:28] hello! puppet is disabled on labstore1007 since 5 days ago with the message "testing" [09:08:36] just FYI in case it's been forgotten :) [09:15:55] !help My public ssh key is up and the access request was accepted, but I can't log into login.tools.wmflabs.org . Any idea why? I can login via ssh with keys in other systems, so it should not be a problem in the ssh client [09:15:55] Sigfrido: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [10:02:54] Hi all, I have a question [10:03:08] I tried "webservice --backend=gridengine stop" [10:03:22] and then "webservice --backend=kubernetes start" [10:03:35] but I get: "You should stop that webservice by issuing: [10:03:35] webservice --backend=gridengine stop [10:03:35] And then start it again with backend kubernetes by issuing: [10:03:35] webservice --backend=kubernetes start" [10:16:24] oops [10:18:09] now I'm using webservice --backend=kubernetes nodejs shell [10:18:12] and it works [11:05:42] Hi! Please deploy MediaWiki 1.31.0-wmf.23 [11:06:15] Hi! Please deploy MediaWiki 1.31.0-wmf.23 [11:08:49] joaquinito01: you can see here the versions and its schedule: https://tools.wmflabs.org/versions/ [11:08:59] but this is the wrong channel about that [11:10:12] more info: https://phabricator.wikimedia.org/T183962 [11:10:18] Hi! Please deploy MediaWiki 1.31.0-wmf.23 in group 1, see the old version: https://www.mediawiki.org/wiki/Special:Version [11:11:36] again, this channel has nothing to do with mediawiki deployments [11:12:06] but if you need help or report problems, there is #wikimedia-tech [11:21:21] cannot open ‘/public/dumps/public/enwiktionary/20180220/enwiktionary-20180220-pages-articles.xml.bz2’ for reading: Operation not permitted [11:21:24] why? [11:22:53] I cannot do that from ssh to etytree-b.etytree.eqiad.wmflabs [11:25:50] joaquinito01 can be kicked, it's a bot. [11:34:24] can anyone help me set up a nodejs app on tool labs? [11:50:11] Hej hej. I created a tool on toolforge and it does not let my connect to the DB with the credentials given... :-/ [11:50:42] Can anyone help? ;-) [11:51:52] ( ERROR 1045 (28000): Access denied for user 's53668'@'10.68.16.44' (using password: YES) ) [12:38:14] ... :-) [12:47:45] bd808: ^ *poke* [13:31:51] CFisch_remote: what is the tool? [13:32:20] tools.wmde-inline-movedparagraphs [13:32:21] Ester: sure, if I remember correctly you're having difficulty with npm [13:32:27] chicocvenancio: \ [13:32:44] CFisch_remote: mysql doesn't want you to move the paragraphs [13:32:49] :-D [13:32:54] seems so [13:40:30] CFisch_remote: there is something wrong with the user for that tool, could you create a phab task? [13:40:55] okay, sure [13:45:44] chicocvenancio: https://phabricator.wikimedia.org/T188508 feel free to add tags / subscribers [15:03:18] Technical Advice IRC meeting starting in 60 minutes in channel #wikimedia-tech, hosts: @addshore & @Christoph_Jauera_(WMDE) - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting [15:25:50] Ester: for that operation not permitted thing, try rebooting [15:26:31] I encountered it before, it’s some sort of NFS client ID collision [15:59:31] zhuyifei1999_:thanks [16:03:15] Ester_: how is the node.js tool going? [16:03:21] Do you need help? [16:03:55] Hi! thanks! I followed your instructions, now I'm trying to do npm install and I'm fixing a bug [16:04:01] will keep you posted. Thanks a lot! [16:04:56] you're welcome [16:06:32] I tried to install and I get two warnings : [16:06:33] fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version. [16:06:38] webpack-jquery-ui@1.0.0 No repository field. [16:07:04] besides this, how do I arrage folders? [16:07:12] and how do I start the app? [16:07:25] I saw on the documentation that I should create a folder www [16:07:41] and inside www a folder js with the javascript file [16:08:35] but my package structure is a bit different as I have a webpack.config.js file and a folder src [16:08:53] within src I have the javascript code [16:09:21] to start the app, the best way would be to setup the folder structure as k8s expects it and run `webservice --backend=kubernettes nodejs start` [16:09:40] *`webservice --backend=kubernetes nodejs start` (single t) [16:10:01] and webpack currently writed to the dist foder [16:10:26] *writes [16:10:32] *folder (sorry) [16:11:04] 'fs: re-evaluating native module sources is not supported. If you are using the graceful-fs module, please update it to a more recent version.' <= iirc it's some version incompatibilities [16:12:14] in my package.json file: "start": "./node_modules/.bin/webpack-dev-server --inline --progress" [16:12:54] Ester_: for the folder structure as long as www/js/server.js is able to load your main script and execute the server it's okay [16:13:17] the webservice code do not currently read package.json [16:13:46] yes, I see you have no server.js [16:13:52] right [16:13:54] I don't [16:13:56] * chicocvenancio trying to understand the code [16:14:55] webpack creates a file at ./dist/bundle.js [16:15:21] and copies .src/html/index.html into ./dist/index.html [16:15:37] is this actually to be served by node? [16:15:38] ./dist/index.html in turn calls bundle.js [16:15:59] I think bundle.js is js to be called client-side [16:16:27] chicocvenancio: same question :) [16:16:41] yep, https://webpack.js.org/ confirms it [16:17:11] so, Ester_ you only need the node part to build the bundle, that is done [16:17:52] you will need to do it again if the code changes (might be good to leave that as cron job or something later) [16:18:14] to serve the static html and js you can use lighttpd if you want. probably easiest to setup [16:18:59] webpack is fundamentally a static site generator, but it does have a "live" mode that it typically only used for local development. [16:19:33] can you pont me to docs on lighttpd? [16:19:37] thanks a lot guys! [16:20:01] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web/Lighttpd [16:20:10] great [16:21:02] bd808: I'm wondering, can tools-static serve the site if it's 100% static? [16:21:11] if you put your html+js+css into $HOME/public_html and run `webservice --backend=kubernetes php5.6 start` you are pretty much done [16:22:10] zhuyifei1999_: yes. the $HOME/www/static directory is served as https://tools-static.wmflabs.org/$TOOL/ [16:22:44] the webserver for this is nginx, but there is no way to do per-tool/directory configuration changes [16:23:27] we can symlink :) [16:24:28] yes, symlink farms should work [16:29:51] Ester_: feel free to ping me if you need help with any of that [16:30:41] so do I put my js css and html in public html? [16:30:54] chicocvenancio: thanks [16:31:22] yes, an index.html that includes the bundle.js script will work [17:01:15] Ester_: you may also be able to symlink public_html to dist [17:30:54] ok I have set webpack output directly to thw public_html folder [17:31:12] I'm trying webservice --backend=kubernetes php5.6 start [17:31:24] but I get "Looks like you already have another webservice running, with a gridengine backend [17:31:24] You should stop that webservice by issuing: [17:31:24] webservice --backend=gridengine stop [17:31:24] And then start it again with backend kubernetes by issuing: [17:31:24] webservice --backend=kubernetes start" [17:31:31] I logged out [17:31:35] and logged in again [17:32:33] zhuyifei1999_: what do you mean by "try rebooting? [17:32:36] " [17:32:56] Ester_: rebooting was for the VPS error [17:33:13] you can go to horizon and ask to soft reboot the instance [17:33:45] hmmm... ok [17:34:00] Ester_: for that webservice, did you stop your currently running webservice? [17:34:00] Ester_: what's your tool name? I've seen this problem before, but I'd like to take a look at the current state before I tell you how to fix it [17:34:13] bd808: tools.etytree [17:34:15] etytree is the name of the tool [17:34:26] yes I did [17:34:31] I think [17:34:39] webservice --backend=kubernetes stop [17:35:15] yeah. this is the goofy state that can happen. [17:35:16] I guess bd808 will debug it :) /me is on mobile rn [17:35:46] `qstat` shows no grid jobs. `kubctl get pods` shows no running pods. [17:35:54] one more question: do I have to put the package in the www/js folder? [17:36:00] and run nom install from there? [17:36:13] in kubernetes? [17:36:18] *npm [17:36:24] bd808: `webservice status` says it is not running as well [17:36:56] !lot tools.etytree Deleted $HOME/service.manifest [17:37:00] !log tools.etytree Deleted $HOME/service.manifest [17:37:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.etytree/SAL [17:37:16] !log tools.etytree Started with `webservice --backend=kubernetes php5.6 start` [17:37:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.etytree/SAL [17:37:29] Ester_: it should be running now [17:37:50] I can see that [17:37:55] it is almost working [17:38:04] there areother things that do not work :) [17:38:06] the "trick" is deleting the service.manifest file. That seems to clear up the confusion that the webservice watchdog can have [17:38:09] anyways I have a meeting now [17:38:20] can I do this myself? [17:38:30] Ester_: yes [17:38:31] (delete the manifest file) [17:38:35] sure, but hopefully you don't see that error again [17:38:40] oh ok [17:38:47] ok guys thanks a lot [17:38:52] but you should only do it when webservice says that something is running but it is really not [17:38:54] I'll ping you later or tomorrow [17:39:26] chicocvenancio: we need to start those docs about node things. :) [17:39:57] especially this use case of using node tools to generate content rather than running an active node webservice [17:43:50] bd808: on stewardbots we have those service.{log,maniphest,access} files; if we get rid of them, do they recreate themselves or would the tool break? [17:44:19] Hauskatze: if you get rid of service.manifest the web interface will break [17:44:45] ah oh [17:44:45] removing service.log while the webservice is running probably won't end well either [17:44:57] so better to webservice stop first [17:45:00] yeah [17:45:15] webservice restar alone would suffice? [17:45:19] *restart [17:45:39] I think last time I restarted it was using 'trusty' [17:45:54] no, sadly. To clean up old logs you really need to stop the webservice, move or delete the logs, and then start it back up [17:45:54] anyway, just a question, I don't want to do nothing for now [19:21:34] hi guys, i'm having a little problem with utf-8 encoding with my python script in tool forge [19:21:49] does anyone know about it? [19:22:20] the thing is: when I run my code in my toolforge account [19:22:27] tool account [19:22:44] "UnicodeEncodeError: 'ascii' codec can't encode character '\xe0' in position 41: ordinal not in range(128)" [19:23:00] when i try to print a dictionary that contains arrays of strings [19:23:04] mmecor: what is tool? where are you running it? [19:23:20] wcdo account [19:23:54] it must be there: /data/project/wcdo/ [19:24:12] where are you seeing the error? [19:24:26] mmecor: the errors says it, there is a mix of ascii and utf8 there- probably python2, which was horrible to handle those [19:24:39] when i run it in my terminal just for quick feedback and when i sned the task in the output .err [19:24:46] i'm using python3 [19:24:49] oh [19:24:55] jynus: since its in print it can be just the locale set for the terminal [19:24:56] and i put utf-8 at the header [19:25:01] but the very weird thing is [19:25:05] that in my laptop, when i run the script [19:25:07] there is no problem [19:25:15] in both cases i am using python3 [19:25:22] echo $LANG ? [19:25:25] hence my question of where you are running this, grid/bastion/k8s have different locale setup [19:25:38] chicocvenancio: how can i check? [19:25:51] what did you do to get the error? [19:25:51] echo $LANG [19:25:51] en_US.UTF-8 [19:26:59] i load a dataframe from .csv file, i check some rows, and i put them into arrays in a dictionary [19:27:02] then i print hte dictionary [19:27:10] this is what it is in my .py file [19:27:32] https://pastebin.com/s3ANePca [19:27:38] mmecor: I see no k8s pods, no grid jobs and no python terminal or processes opened for that tool in tools-bastion-03 [19:28:04] nope, wait a sec, i'll generate a job [19:28:09] and you'll see the .err [19:28:23] wait [19:28:37] so you're sending it to the grid? [19:28:47] i'm doing it right now [19:28:59] 1794535 0.30000 prova tools.wcdo r 02/28/2018 19:28:38 task@tools-exec-1432.tools.eqi 1 [19:29:22] this is a known bug, give me a minute to get the docs [19:30:01] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#An_error_with_"ascii"_codepage,_"file_not_found",_or_UnicodeEncodeError [19:30:39] simplest workarround is to add `-v LC_ALL=en_US.UTF-8` to jsub [19:31:47] the task is still in the grid [19:31:56] with no output, that's strange [19:32:11] qstat tells it is running [19:33:11] chicocvenancio: i'll try to add that in the command, but in any case, it is also failing when i run the script in the termianl (not sending in the grid... that i know i sholdn't do normally, but i just do it for quick stuff, nto really heavy) [19:35:30] ok, the prova.err came out from the grid job [19:35:31] with the error [19:36:10] I can run the script in the bastion without error [19:36:17] how are you connecting to bastion? [19:36:22] mmecor: ^ [19:36:31] first i login with my username [19:36:40] to @tools-login.wmflabs.org [19:36:44] then i become the tool [19:36:47] if your terminal has ascii set this will be a problem with bastion login as well [19:36:50] then i activate the venv [19:36:58] mmecor: what program do you use to login? [19:37:11] chicocvenancio: but when i run it locally in my mac-python3 there is no problem [19:37:17] so i guess it is not my terminal [19:37:27] oh, yes, its a mac [19:37:42] mmm [19:38:08] but in the mac works and not in the linux from the servers, shouldn't it be the other way around? [19:38:20] when i "echo $LANG" [19:38:33] in my mac, the response is nothing an empty line [19:38:36] could this be the problem? [19:38:46] i mean in a terminal i just opened [19:39:14] $LANG is the system locale, it might not match a particular terminal encoding [19:39:22] with the tool account i'm logged in bastion at "(venv)tools.wcdo@tools-bastion-03" [19:39:54] i have english as my primary language [19:40:08] have you tried sending it to the grid with `-v LC_ALL=en_US.UTF-8` [19:40:49] i just did now [19:41:02] it says running,.... let's see [19:42:52] I think https://stackoverflow.com/a/4606636/3930971 might be the answer for the mac terminal, but I'm not sure [19:43:33] i have utf-8 enabled [19:43:37] in the mac terminal [19:44:11] does a simple `echo à` work ? [19:44:24] yup [19:44:36] i run the script in my mac and it works [19:44:40] when logged in to toolforge as well? [19:44:57] ahhh [19:44:59] right [19:45:01] not when logged in [19:45:32] does it work with your user, before the become wcdo? [19:45:37] in any other terminal tab [19:45:51] nope, not with my user [19:46:27] this quite weird, all this used to work one year and half ago. it was the same laptop and operating system [19:49:42] oh [19:49:52] chicocvenancio: that's bad, adding that line in the jsub didn't work [19:50:02] the same error appeared in the .err [19:57:52] !log tools.stewardbots New maintainer: rxy [19:57:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [20:00:03] chicocvenancio: the more i check things i can do about setting the terminal into utf-8, the less i understand. everything seems ok. in the different configurations there are the right lines [20:00:47] the problem is the terminal enconding [20:00:57] let me see what happens if I run this in k8s [20:01:42] https://pastebin.com/JbiR6F3k [20:01:49] this is what the locale says when i'm in toolforge [20:02:58] so, apparently, there is utf-8, but then i cannot echo à [20:03:27] I can, in the same env [20:04:10] which is why I'm assuming the "not working in bastion" part to be related to the ssh in mac os [20:05:40] chicocvenancio: i now found the way to print 'echo à' [20:05:46] https://blog.remibergsma.com/2012/07/10/setting-locales-correctly-on-mac-osx-terminal-application/ [20:05:53] "Escape non-ASCII input" [20:05:59] i checked this checkbox [20:06:11] and it seems that i can print in the toolforge in bastion [20:06:18] but the script gives the same error [20:06:23] in bastion? [20:06:28] yes [20:06:46] `python3 ccc_selection.py`? [20:06:51] yes [20:07:56] https://www.irccloud.com/pastebin/ScqU6TDk/expected_output [20:08:06] ok, now i unchecked 'set locale environment variables on startup' [20:08:27] chicocvenancio: that's what's supposed to be printed, yes [20:08:39] i unchecked that and checked the 'escape non-ascii input' [20:08:48] and i don't know exactly what is going on but works! [20:09:05] thanks! :) [20:09:09] now let's try the jsub [20:09:09] lol, thats the best kind of debugging [20:10:06] thanks for help :) [20:10:48] pointing the mac out has been key haha [20:11:24] where are you placing the `-v LC_ALL=en_US.UTF-8`? [20:11:37] many problems are related to that... i guess that if i using ubuntu i would come across very similar problems. been in mac for 5 years and i can't remember how it was with ubuntu :) [20:11:41] nope, chicocvenancio [20:11:45] i'm trying first without that line [20:12:16] that won't work, unless you set locale in the prova.sh script [20:12:21] what i'm seeing is that although it says running, it usually takes longer now to start running [20:12:45] chicocvenancio: how can i set the locale in the script? [20:12:56] but one thing I was doing wrong is to place -v LC_ALL=en_US.UTF-8 at the end of the jsub command [20:13:08] it must come before the prova.sh part [20:13:14] ah, before [20:13:17] i was putting it after too [20:17:05] the bash script prova.sh could have a `export LC_ALL=en_US.UTF-8` that would solve things for the grid as well [20:17:20] chicocvenancio: inside the prova.sh? [20:18:16] yes, it has to be before the python line, ideally on the second line [20:19:25] ohhh, I forgot, but `export PYTHONIOENCODING=UTF-8`probably solves it for bastion as well [20:19:36] export PYTHON... where? [20:19:46] we have 3 options here i see [20:19:57] first the param. in the jsub [20:20:00] although you might get weird characters in the terminal if you had not solved the macos config [20:20:01] second in the bash script [20:20:04] third...? [20:21:04] those are the options for the grid [20:21:19] there is a phabricator task somewhere with a really detailed analysis that zhuyifei1999_ did on python locale issues and the grid... [20:21:31] for the bastion problem you already solved it with the macos config [20:21:56] but export PYTHONIOENCODING=UTF-8 will let the script run even without that fix [20:21:56] yes, bastion's solved [20:22:09] chicocvenancio: where should i put this line? [20:22:57] this one or one of the merged tasks of https://phabricator.wikimedia.org/T60784 [20:23:18] * zhuyifei1999_ goes back to doing Commons POTY stuff [20:23:25] * chicocvenancio is again too slow [20:23:54] mmecor: never mind that line, as long as bastion doesn't give you unicodedecode errors again [20:24:14] chicocvenancio: bastion is ok [20:24:24] then that line is not needed [20:24:33] now i was looking for a fix in the script for the jobs [20:25:04] although it seems that adding -v LC_ALL=en_US.UTF-8 before the .py works [20:25:20] either `-v LC_ALL=en_US.UTF-8` in the jsub command or `export LC_ALL=en_US.UTF-8`in the bash script [20:25:56] export LC_ALL=en_US.UTF-8 in the bash? [20:25:57] trying that [20:31:21] chicocvenancio: yay, it woked with the 'export LC_ALL' in the second line of the bash script [20:31:28] so that's the easiest part [20:31:33] the easiest way i mean :) [20:31:43] thanks! [20:31:59] cool, you're welcom [20:32:07] do you know how come it takes longer now to run the tasks? [20:32:11] or was it just my perception? [20:32:40] a silly script that in terminal runs in a sec, when i send it.. it takes around a min although it says running [20:35:30] grid jobs times are quite unpredictable, tbh I don't understand it fully [20:35:52] ok :) [20:43:01] The grid scheduler has to do a lot of work to setup for running a job and may need to wait for resources (RAM, CPU) to be available. It is a system designed for consistency and efficient resource utilization rather than real-time responsiveness [20:46:24] i see [22:06:02] (03Draft1) 10Paladox: Send all notifications for refs/meta/config to #wikimedia-releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/415461 [22:06:04] (03PS2) 10Paladox: Send all notifications for refs/meta/config to #wikimedia-releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/415461 [22:06:49] (03CR) 10Chad: Send all notifications for refs/meta/config to #wikimedia-releng (031 comment) [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/415461 (owner: 10Paladox) [22:07:31] (03PS3) 10Paladox: Send all notifications for refs/meta/config to #wikimedia-releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/415461 [22:07:41] (03CR) 10Paladox: Send all notifications for refs/meta/config to #wikimedia-releng (031 comment) [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/415461 (owner: 10Paladox) [22:57:02] !log quarry killed two IO-intensive query saves [22:57:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [22:57:17] thanks zhuyifei1999_ [22:57:51] yeah, when people run queries like https://quarry.wmflabs.org/query/25152 :( [22:58:23] SELECT *; [22:58:24] wat [22:58:29] also, lol [22:58:42] (03CR) 10Chad: [C: 032] Send all notifications for refs/meta/config to #wikimedia-releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/415461 (owner: 10Paladox) [22:58:54] thanks no_justification :) [22:59:09] (03Merged) 10jenkins-bot: Send all notifications for refs/meta/config to #wikimedia-releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/415461 (owner: 10Paladox) [22:59:19] (03CR) 10jenkins-bot: Send all notifications for refs/meta/config to #wikimedia-releng [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/415461 (owner: 10Paladox) [23:00:16] Does it auto deploy? [23:02:39] Docs say yes [23:02:48] Ok, thanks. [23:02:50] * paladox tests [23:02:59] by editing some random config file (as a test) [23:03:29] (03Draft1) 10Paladox: Test: DO NOT MERGE [labs/tools/wikibugs2] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/415476 [23:03:49] (03Abandoned) 10Paladox: Test: DO NOT MERGE [labs/tools/wikibugs2] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/415476 (owner: 10Paladox)