[00:01:50] !log tools.jouncebot Restart to deploy ba7998f (T243394) [00:01:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.jouncebot/SAL [08:29:54] Hello. During last year I using command like [08:29:55] ssh -N -4 -L 4711:enwiki.analytics.db.svc.eqiad.wmflabs:3306 ruswebs@tools-login.wmflabs.org [08:29:57] and after go to database ruwiki_p. Some days ago, one or more, this database disappeared and I see only database enwiki_p. Anybody know, how can I find database ruwiki_p? [08:45:27] @kergma_sas: The wiki replicas have been upgraded and you now need to connect to the correct wiki name. See https://wikitech.wikimedia.org/wiki/News/Wiki_Replicas_2020_Redesign for details, which was announced multiple times on the cloud-announce@ mailing list. [09:15:01] !log paws added user `taavi` (Majavah) as projectadmin [09:15:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [09:17:53] !log paws set `profile::wmcs::kubeadm::docker_vol: false` on ingress nodes T282087 [09:17:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [09:17:55] T282087: Support cinder or expanded ephemeral disk worker nodes on Toolforge Kubernetes - https://phabricator.wikimedia.org/T282087 [10:19:33] stashbot: [10:19:40] stashbot: ? [10:19:55] stashbot: ping [10:20:23] !log admin rebooted cloudgw1002 (active) thus causing a failover to cloudgw1001 [10:20:36] I figured [10:22:14] !log admin rebooted cloudgw1002 (active) thus causing a failover to cloudgw1001 [10:22:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:46:22] hi all! I'm having an issue with the Web Proxies of one project, which tags or who should I ping on the phab task: https://phabricator.wikimedia.org/T282532 ? [10:48:27] dsaez: already replied on the task, but my guess is that you forgot to adjust the security group rules [10:50:19] aaaha [10:51:05] Where can I check/configure that? [10:51:35] https://wikitech.wikimedia.org/wiki/Help:Using_a_web_proxy_to_reach_Cloud_VPS_servers_from_the_internet#Security_groups, tl;dr is horizon, network -> security groups [10:51:57] let me see [10:54:54] Hi, if it would be great if people in WMCS take a look at this patch https://gerrit.wikimedia.org/r/c/operations/puppet/+/686353/ [11:04:50] Amir1: done [11:05:10] arturo: Thanks! Awesome [11:05:17] 1 down, 121 to go [11:05:22] heh [11:05:33] I have some to bother you more later [11:24:24] cool [11:27:36] arturo: so the thing is that I'm no SRE. I can't merge/deploy the patch. Do you know how can I get it merged? [11:30:04] I can do that, or godog if he's interested [11:32:29] patch LGTM but I have no horse in the open ssh session information race :) [11:35:53] merged ✅ [11:38:54] Thanks! [12:19:58] just wanted to provide feedback that I resized an instance just now and it worked as expected, nicely done [12:35:56] !log tools.admin restart dead webservice [12:35:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.admin/SAL [14:48:06] https://phabricator.wikimedia.org/T282557 looks like I'm not the only one who didn't bother to check for 2020 replica compatibility [14:48:18] though I only broke a replag checker [16:29:46] !log tools carefully shutdown tools-k8s-haproxy-2 T252239 [16:29:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [16:29:51] T252239: Rebuild tools-k8s-haproxy-* as an anti-affinity server group - https://phabricator.wikimedia.org/T252239 [16:32:39] !log tools carefully shutdown tools-k8s-haproxy-1 T252239 [16:32:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [16:49:44] !log tools creating tools-checker-04 with buster T278540 [16:49:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [16:49:48] T278540: Toolforge: migrate checker server to Debian Buster - https://phabricator.wikimedia.org/T278540 [16:58:04] !log tools add tools-checker-04 to toollabs::checker_hosts hiera key T278540 [16:58:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [16:58:11] T278540: Toolforge: migrate checker server to Debian Buster - https://phabricator.wikimedia.org/T278540 [17:12:56] !log tools add tools-checker-04 as a grid submit host T278540 [17:13:00] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [17:13:00] T278540: Toolforge: migrate checker server to Debian Buster - https://phabricator.wikimedia.org/T278540 [17:14:46] !log tools move floating ip 185.15.56.61 to tools-checker-04 [17:14:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [17:17:20] !log shutdown and delete tools-checker-03 T278540 [17:17:21] Majavah: Unknown project "shutdown" [17:17:26] !log tools shutdown and delete tools-checker-03 T278540 [17:17:29] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [18:00:15] !log admin adding 'trove' service project in advance of deploying trove in eqiad1 [18:00:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [18:15:02] does toolforge permit installing a system package? [18:15:54] I think you'd have to ask a root person in here [18:16:09] But which package out of interest [18:16:41] some headless browser, maybe chromium or firefox [18:18:14] proc: you can't. we (roots) can install packages on all grid worker nodes if there's a request on https://phabricator.wikimedia.org/project/profile/3978/. the kubernetes cluster doesn't really have a solution for apt packages (yet). [18:18:35] so basically not possible? [18:19:10] well, i guess it depends, would you install a browser if i made a request there (lol)? [18:22:14] I'd like a task, mostly because I'll need someone from the cloud services team (or with otherwise full root on all prod instances) to actually merge the puppet patch [18:22:47] (assuming we have some suitable headless browser available in debian main) [18:25:22] opened [18:27:00] Majavah: We... should. As we use some in CI [18:27:10] (mw browser test stuff) [18:28:25] I don't see any dedicated headless packages for chromium/firefox in debian repos or apt.wm.o, let me see if I can dig what ci uses [18:30:10] https://github.com/wikimedia/integration-config/blob/7102ead5fea3416e3bea463b4c41f9e222bfdd90/dockerfiles/node10-test-browser/Dockerfile.template#L8 [18:30:36] * Majavah worried about the amount of dependencies including xorg/gtk [19:00:25] proc: you should add concrete use cases for a headless browser to your ticket at T282597. [19:00:26] T282597: Headless browser - https://phabricator.wikimedia.org/T282597 [19:01:30] bd808: the use case is https://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval/ProcBot_8. no api endpoints exist for editing abusefilters. so i decided to mimic the http requests, but the approach turns up a few caveats and is slightly less maintainable, so i figure a web scraping approach will work better [19:03:12] i can add that to the ticket [19:09:44] proc: all code running in wmcs/toolforge needs to be available under a foss license, are you sure you want to run that task on toolforge? [19:09:58] firefox/chromium are foss right? [19:10:55] https://www.mozilla.org/en-US/foundation/licensing/ [19:11:01] yep, but reading that brfa I got the feeling the bot itself is not [19:11:06] why? [19:11:13] oh [19:11:14] uhhh [19:14:33] i can probably write the bot in a way where releasing the task's code wouldn't provide much information about what it's doing [19:15:51] i could also selfhost it like with the rest of procbot but tbh i dont want to use like 300mb of my space with browser dependencies [19:47:10] 300mb is not a lot of space... [20:11:20] proc: this may sound wild, but I think you should work on adding Action API actions to manager the abusefilters. [20:12:06] building out the functionality of the API to deal with the problem you bot wants to deal with is valuable to all wikis, not just enwiki. [20:14:01] https://phabricator.wikimedia.org/T213037 [20:14:17] been stalled since 2019 [20:16:18] AntiComposite: stalled in that nobody is working on it, or stalled in that there is some kind of blocking action being made to keep the code from being written? [20:16:54] I think it's just part of a dependancy tree, but not marked as such [20:17:12] the patch that it was stalled on was abandoned, and the person working on it now isn't [20:17:21] https://phabricator.wikimedia.org/T213037#6476357 [20:17:25] >I think this is going to be quite easy to implement after the architecture review part. Unsure if there are resources for the actual implementation of the module as part of the project. [20:18:30] working on MediaWiki is maybe not as self-directed and quickly fulfilling as working on a bot, but in the long run code in MediaWiki is likely to help many more people [20:18:54] * bd808 says as a person who ran away from MediaWiki code to do other things for the movement [20:49:22] bd808: it’s far more time consuming to make a change to core code [20:49:45] than to just mimic the web flow with requests or just use selenium [20:50:43] it's extension code ;) [20:50:51] but screen scraping breaks [20:53:24] the code I whipped up for the screen scraping has some safeguards to not save if it seems the output is unexpected. The web requests version also works I think but I don’t maintain the code for the actual task that needs to be done to the specific filter, which is also in a different Lang, so it’s more irritating to keep it in sync [20:55:22] Basically trying to be as lazy as possible [21:01:00] https://bash.toolforge.org/quip/AVWoDg8ZgCrwkbTdmcjL -- my past musing on screen scraping :) [21:18:37] proc: I once downloaded the firefox binary and it worked in Toolforge