[05:45:14] It was announced that webservices of tools hosted on toolforge.org needs to be migrated to the domain toolforge.org. According to this, do I need to chang the url https://tools-static.wmflabs.org/cdnjs/ajax/libs/twitter-bootstrap/4.0.0/js/bootstrap.min.js to something else? [05:50:39] no, tools-static.wmflabs.org is staying the way it is for the moment [05:51:22] https://wikitech.wikimedia.org/wiki/News/Toolforge.org provides more details [08:19:22] !log tools.zppixbot restarting to apply changes for T254348 [08:19:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [08:19:24] T254348: ZppixBot-test starts improperly on python 3.7 - https://phabricator.wikimedia.org/T254348 [08:22:15] Reception123: logs are green [08:23:13] Reception123: worked! [08:29:23] :) [08:30:28] !log tools.zppixbot tools.zppixbot@interactive:~$ python3.7 -m venv zbpy37 [08:30:29] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [08:30:57] morning hauskatze [08:31:33] Morning sir [08:31:54] * RhinosF1 wonders how much he can break today [09:00:59] !log tools.stewardbots Fix flake8 errors 6fec91d [09:01:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [09:02:15] hauskatze: can you also rebase https://gerrit.wikimedia.org/r/#/c/labs/tools/stewardbots/+/600355/ please [09:05:30] looks like the flake8 fix broke something [09:11:43] be more specific please [09:16:20] I can't, I'm not sure what's going on [09:16:34] Reverted the commit locally and tried to restart the bot, no luck either [09:16:44] So maybe it's something else [09:16:51] I've asked Martin to take a look [09:17:36] hmm interesting [09:21:36] oh, it joined this time [09:21:51] probably needs a bit more time to kill previous IRC sessions [09:33:35] I've created a venv and am now getting '-bash: /data/project/zppixbot/zbpy37/bin/pip3.7: /data/project/zppixbot/zbpy37/bin/python3.7: bad interpreter: No such file or directory' [09:36:13] I installed the venv using this [09:36:16] https://www.irccloud.com/pastebin/oqtoJChf/ [09:37:03] RhinosF1: try using "python -m venv" instead of "python3.7 -m venv" [09:38:56] Majavah: /usr/bin/python: No module named venv [09:39:10] https://www.irccloud.com/pastebin/J0gnY7I6/ [09:39:45] any recent changes that might affect how crons are run on a cloud VPS? i doubt it but I have the weirdest issue.. something that worked for years and now my cron stops running.. but also not all of them [09:39:54] maybe "python3"? I remember having a similar issue and solving it by using some other name of the python binary [09:40:43] Majavah: that created [09:41:59] https://www.irccloud.com/pastebin/A1HHV4Xh/ [09:42:09] Majavah: ^ same error now after creation [09:42:42] try using "pip3" instead of "pip3.7" [09:44:10] Majavah: "-bash: /data/project/zppixbot/zbpy37/bin/pip3: /data/project/zppixbot/zbpy37/bin/python3.7: bad interpreter: No such file or directory" [09:44:27] what files /are/ in that bin directory? :/' [09:44:59] are you running all those files inside a `webservice shell`? [09:45:06] because I think the bastions are still on python3.5 [09:45:14] running all those *commands, I mean [09:45:41] Lucas_WMDE: I'm creating the venv in webservice shell [09:45:51] then activate the venv to run that [09:46:26] you can't use py3.7 in bastions [09:47:01] https://www.irccloud.com/pastebin/KCfievm6/ [09:47:04] Majavah: ^ [09:48:06] does that command work in `webservice shell` [09:49:28] https://www.irccloud.com/pastebin/UZZZk11D/ [09:49:37] Majavah: like ^? no [09:50:05] activate the venv first before you try to use it [09:50:51] Majavah: yes it does [09:51:14] that's the issue then as Lucas suggested [09:51:28] Majavah: I guess I might have to use 3.5 for this script then [09:51:30] bastions only have py3.5 so you can't use py3.7 inside them [09:51:34] yeah unfortunately [09:52:32] !log tools.zppixbot manually install https://git.io/JfDWA [09:52:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [09:53:59] * RhinosF1 waits for git [09:54:05] this ran in 2s locally [09:56:44] Majavah: Running my script and getting Unable to fetch in submodule path 'puppet' now (https://gerrit.wikimedia.org/r/#/c/labs/tools/ZppixBot/+/603373/10/deploy.py where testrepo is https://github.com/RhinosF1/testrepo) [09:58:23] I think I fixed that [09:58:37] it was as I killed the script as it seemed to be hanging [09:58:47] no [09:58:50] https://www.irccloud.com/pastebin/mcWse82u/ [10:01:37] * RhinosF1 tries another submodule [10:04:50] !log tools.zppixbot auto-update@website: Synced website repo in 1.4s [10:04:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [10:05:01] yey, it worked with that [10:05:03] weird [10:05:17] Reception123: yey!!!!! [10:07:34] !log tools.zppixbot auto-update@website: Synced website repo in 1.0s [10:07:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [10:07:49] !log tools.zppixbot auto-update@website: Synced website repo in 1.0s [10:07:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [10:07:59] no, stupid script [10:08:24] !log tools.zppixbot auto-update@website: Synced website repo in 1.0s [10:08:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [10:08:31] No, you haven't [10:10:47] * RhinosF1 will fix that [10:12:13] ^ found the bug [10:19:02] yes! Fixed! [10:24:03] * RhinosF1 has to run the script twice more, one should output and one not. I have fixed the bug [10:24:19] no, just once, it will not output [10:25:01] It worked! [10:25:57] Bug fixed, I will have 1 more test when added to cron and then will be done [12:16:32] !log tools.zppixbot replace crontab with "*/15 0,8-23 * * * /usr/bin/jsub -N cron-27 -once -quiet /data/project/zppixbot/bin/python3 /data/project/zppixbot/ZppixBot/deploy.py > /data/project/zppixbot/deploy.log" [12:16:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [12:28:17] * RhinosF1 is testing that and then should be done !log'ing until tonight [12:32:51] !help why does dologmsg not work via jsub ^ [12:32:51] If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-kanban [12:33:33] I'm not sure how that works [12:33:42] !log tools.zppixbot auto deploy failed to log but I just had a patch ran to update the website [12:34:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [12:34:34] arturo: it's a script installed on every tool. Running from my user does ^ but via the cron that just ran it was mute. [12:35:09] anything on the grid logs? [12:36:26] arturo: looking at cat *.err I see "sh: 1: dologmsg: not found" [12:36:50] so it looks as if jsub doesn't have access to it / whatever user the cron runs as [12:37:30] perhaps it is installed only on the bastions [12:37:49] perhaps, does that need a task [12:44:08] petan: around seen as you manage wm-bot? [12:50:56] arturo: should I create a task? [13:17:23] petan, arturo: https://phabricator.wikimedia.org/T254747 [15:27:51] According to Horizon our instance wikispeech-tts-dev belongs to a security group with port 10000 allowed from 0.0.0.0/0. How do I reach this from WAN? It's an HTTP service, if that helps. [15:30:03] WAN as in internet [15:32:58] Would it perhaps be easier if I setup an nginx (or something) that proxy passed the request so it seems to be port 80? [15:35:05] kalle: does it have a public IP or just an internal IP? [15:35:23] (what project is this?) [15:38:08] andrewbogott: No floating IP assigned. I could assign, but it feels like it wouldn't be needed for this? The project is https://www.mediawiki.org/wiki/Wikispeech [15:39:56] You're going to need either a public IP or to route it via some proxy if you want it to be externally accessible [15:40:30] kalle: you need either to use a shared web proxy -- https://wikitech.wikimedia.org/wiki/Help:Using_a_web_proxy_to_reach_Cloud_VPS_servers_from_the_internet -- or a public ip. If it is http based, the shared proxy is almost always the right thing to use [15:42:23] Thanks all! I'm sure I'll manage getting it up from this. [15:48:11] kalle — sorry, in a meeting now but if you ping me later I can help you work through it [15:57:47] !log tools.zppixbot auto-update@website: Synced website repo in 1.5s [15:57:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [15:58:03] RhinosF1: ^ [15:58:10] it works [15:59:33] MacFan4000: yey! [17:13:09] !log monitoring switching pontoon-puppet-01 over to a clean upstream puppet branch to get puppet working. Existing changes are in a branch named 'godog' in /var/lib/git/operations/puppet [17:13:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Monitoring/SAL [17:14:42] !log tools.zppixbot restarted for code changes [17:14:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [17:18:41] !log monitoring moved puppetmaster back to the 'godog' branch and leaving things broken because this is much weirder than I expected [17:18:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Monitoring/SAL [17:21:56] !log tools.zppixbot-test restarted for code changes [17:21:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL [17:23:17] !log tools.zppixbot re-enable reload [17:23:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [17:24:47] !log tools.zppixbot-test re-enable reload [17:24:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL [18:16:04] !log tools.zppixbot auto-update@website: Synced website repo in 43.s [18:16:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [18:26:26] MacFan4000: 43s? [18:26:31] That's slow [18:29:04] RhinosF1: welcome to shared NFS storage :) [18:29:26] bd808: it's should be about 2D [18:29:29] 2s [18:29:36] Or it was earlier [18:29:47] Maybe it's doing it's stupid hanging thing [18:30:50] RhinosF1: is is all very, very variable. It depends on what else is happening on both the bastion that is being used and the backend NFS server [18:31:32] * RhinosF1 shrugs [18:33:08] * RhinosF1 needs to actually make that 1dp [18:33:14] Not 43.s [18:38:07] !log Deployment-prep Rebooting deployment-logstash0[2,3] [18:38:08] hauskatze: Unknown project "Deployment-prep" [18:38:15] !log deployment-prep Rebooting deployment-logstash0[2,3] [18:38:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [21:22:30] !log tools.zppixbot-test deleted sopel deployment to switch to py3.7 START -- T254246 [21:22:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL [21:22:33] T254246: Upgrade ZppixBot docker image to python 3.7 - https://phabricator.wikimedia.org/T254246 [21:22:40] Texas: ^ should be disconnected now [21:22:46] k [21:30:03] !log tools.zppixbot-test recreate deployment in py3.7 for T254246 END [21:30:05] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL [21:30:06] T254246: Upgrade ZppixBot docker image to python 3.7 - https://phabricator.wikimedia.org/T254246 [21:30:32] Texas: we should be back, please ping when bot joins and type .version to verify [21:31:11] ok