[09:23:49] !log tools `aborrero@cloud-cumin-01:~$ sudo cumin --force -x 'O{project:tools name:tools-sge[bcew].*}' 'rm /etc/apt/preferences.d/* ; run-puppet-agent'` (T247422) [09:23:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:23:53] T247422: Update Tesseract on Toolforge to v4.1.0 - https://phabricator.wikimedia.org/T247422 [09:35:14] !log tools `aborrero@cloud-cumin-01:~$ sudo cumin --force -x 'O{project:tools name:tools-sge[bcew].*}' 'rm /etc/apt/sources.lists.d/kubeadm-k8s-component-repo.list ; rm /etc/apt/sources.list.d/repository_thirdparty-kubeadm-k8s-1-15.list ; run-puppet-agent'` (T247422) [09:35:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:35:18] T247422: Update Tesseract on Toolforge to v4.1.0 - https://phabricator.wikimedia.org/T247422 [09:48:18] !help if I run, kubectl delete deployment --all it will recreate and restart? [09:48:18] If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-kanban [09:48:26] same as delete pods does? [09:50:09] !log tools `aborrero@cloud-cumin-01:~$ sudo cumin --force -x 'O{project:tools name:tools-sge[bcew].*}' 'apt-get install tesseract-ocr -t stretch-backports -y'` (T247422) [09:50:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:50:16] T247422: Update Tesseract on Toolforge to v4.1.0 - https://phabricator.wikimedia.org/T247422 [09:51:05] RhinosF1: I don't think so. The deployment is the object ensuring pods are running, right? You can check upstream k8s docs to be sure [09:52:19] arturo: I'll try deleting the pods and see if this fixes the weirdness of one of our bots then [09:53:03] !log tools.zppixbot-test webservice stop && kubectl delete pods --all & webservice start due to memory improperly loaded [09:53:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL [09:57:11] arturo: where I might the error log for a kubectl pod? It's now showing sopeltest.bot-6b45cd8d5-gqf25 0/1 Error 3 2m21s [09:59:49] !log tools now running tesseract-ocr v4.1.1-2~bpo9+1 in the Toolforge grid (T247422) [09:59:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:59:55] T247422: Update Tesseract on Toolforge to v4.1.0 - https://phabricator.wikimedia.org/T247422 [10:00:34] RhinosF1: try something like `kubectl logs` [10:02:06] !log tools.zppixbot-test drop abuse.py [10:02:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL [10:02:41] arturo: that's correct but nothing obvious in it [10:06:36] !log tools.zppixbot-test all services rebooted [10:06:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot-test/SAL [10:08:12] !log tools.zppixbot drop abuse.py & reboot [10:08:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [10:09:31] arturo: weird, seems happy now [10:14:13] * RhinosF1 hopes that this plan works and he doesn't have to kill anything [16:36:37] !log paws cleaned up the old DNS entries for the external LBs that have been off for a year [16:36:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [16:38:48] !log paws deleting the old shut-down VMs from the last effort to rebuild paws T211096 [16:38:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [16:38:51] T211096: PAWS: Rebuild and upgrade Kubernetes - https://phabricator.wikimedia.org/T211096 [18:59:13] !log paws created anti-affinity group "controlplane" T211096 [18:59:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [18:59:17] T211096: PAWS: Rebuild and upgrade Kubernetes - https://phabricator.wikimedia.org/T211096 [19:56:22] I see etherpad was updated -- which means all the previous work on etherpad is now lost forever? [20:00:48] acagastya: how do you mean? "previous work" as in Etherpad-Lite code base? As in content of a specific Etherpad? [20:01:20] andre__ I was talking about etherpad.wikimedia.org [20:01:42] acagastya, yeah. Me too. [20:01:53] and I said I don't understand what you mean by "lost forever". [20:02:15] When I visited it today, I saw new UI which was good for a change. But then, I opened a pad I was working on, and now it shows a blank. [20:03:24] My Etherpads are not blank when it comes to content [20:03:34] got a public URL? [20:04:53] though in general, anyone can edit any public Etherpad anyway. Always been like that. Hence anyone could also blank an Etherpad anyway. Hence not necessarily related to any software update. [20:05:06] https://etherpad.wikimedia.org/p/closestblackhole [20:05:47] It's not blank for me [20:05:49] acagastya, that is NOT empty. [20:05:59] Caching screwing things up? [20:06:05] Try private browsing/incognito [20:06:07] O_o ? [20:06:22] * acagastya tries to access Firefox Focus. [20:06:33] There's 55 lines [20:06:34] "Hello. How are you doing? [20:06:34] Thank you for agreeing to discuss the recent discovery of the closest black hole." [20:06:35] etc [20:06:50] Drat. It does work in incognito. [20:07:01] Darned cache invalidation. [20:07:07] In any case, backups can save your day. [20:07:25] Well, at least I have access to it now. [20:09:30] Try force refeshing it in your normal browser etc [20:35:28] Is there an extension for wikimarkup highlight for VS Code? [20:39:21] A quick google suggests yes [20:39:22] https://marketplace.visualstudio.com/items?itemName=jakeboone02.mediawiki [20:39:37] https://marketplace.visualstudio.com/items?itemName=RoweWilsonFrederiskHolme.wikitext [20:43:41] * bd808 is reminded that he would like a pygments plugin that does wikitext [20:53:08] aye, t'would be nice to have wikitext syntax highlighting in Extension:SyntaxHighlight and Phabricator [20:56:54] https://phabricator.wikimedia.org/T29828 [20:56:58] Bug is 9 years old [21:00:04] lulz. I wonder if folks got bogged down in trying to make it too fancy? [21:00:49] Reedy: I am not sure if those two are Wikimedians. It would be great if mediawiki releases one. [21:01:03] Why? [21:01:08] If there's one that works fine, use it [21:01:13] We don't need to reinvent the wheel [21:19:02] The wheel seems not to be working ideally. [21:19:15] And we can just fork this metaphorical wheel and improve it. [21:20:56] acagastya: have at it. I do not think that any Wikimedia Foundation team will take that on if that is what you are hoping for. [21:36:02] !log paws created paws-k8s-control-1/2/3 with appropriate sec group and server group T211096 [21:36:05] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [21:36:05] T211096: PAWS: Rebuild and upgrade Kubernetes - https://phabricator.wikimedia.org/T211096 [21:36:52] Have you even tried them? Have you tried reporting bugs to their authors? [21:38:19] I just tried it. [21:38:50] There is a user who is complaining about a deleted page. [21:39:31] They are not okay with me sending them the recent version. [21:39:44] And so I am putting the history in a git repo and sending them so they can access the history. [22:02:51] On toolforge, how should I add something to my PATH environment that the webservice will actually see? [22:04:33] what are you trying to do? [22:05:52] My Python webservice calls an executable inside my homedir, I'm trying to help my webservice find that executable [22:06:42] I can just use an absolute path, but I'd be surprised if there wasn't a way to just add it to my PATH [22:07:53] Hmm, maybe in .profile [22:08:57] there's two main ways to do it [22:10:07] one is to use sys.path.append [22:10:44] ST47: is your webservice running on the job grid or Kubernetes? [22:10:50] bd808: kubernetes [22:10:59] the other is to extract the Kubernetes configuration file for your webservice and an `env` parameter to the pod template [22:11:04] ST47: there's no way to adjust $PATH there [22:11:48] a hack could be done with manual editing of the deployment template in theory, but it would not survive a hard restart cycle [22:12:19] !log paws created paws-k8s-haproxy-1/2 with antiaffinity group T211096 [22:12:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [22:12:22] T211096: PAWS: Rebuild and upgrade Kubernetes - https://phabricator.wikimedia.org/T211096 [22:12:34] I'd suggest just using sys.path.append(os.environ["HOME"]) [22:13:02] okay dokey, I'll just do sys.path.append() then, probably in app.py [22:35:53] !log paws created paws-k8s-worker-1/2/3/4 T211096 [22:35:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [22:35:56] T211096: PAWS: Rebuild and upgrade Kubernetes - https://phabricator.wikimedia.org/T211096