[00:26:57] Hi, I need my 2fa reset. I got a new phone. I am still logged in on my tablet. I don’t have my recovery codes [00:31:10] !help [00:31:10] MacFan4000: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [01:04:23] Hello? [01:38:22] (03PS1) 10GoranSMilovanovic: Initial [labs/tools/WiktionaryCognateDashboard] - 10https://gerrit.wikimedia.org/r/451561 [01:40:02] (03CR) 10GoranSMilovanovic: [V: 032 C: 032] Initial [labs/tools/WiktionaryCognateDashboard] - 10https://gerrit.wikimedia.org/r/451561 (owner: 10GoranSMilovanovic) [01:42:47] !log deployment-prep T201518: ORES, fawiki wp10, misc updates [01:42:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [01:42:59] T201518: ORES deployment (Early August) - https://phabricator.wikimedia.org/T201518 [10:27:43] !log tools T201602 upgrade packages from trusty-updates [10:27:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [10:27:49] T201602: toolforge: round of package upgrades - https://phabricator.wikimedia.org/T201602 [10:30:50] !log tools T201602 upgrade packages from jessie-wikimedia [10:30:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [10:33:05] I’m getting “MySQL server has gone away” errors in my tool (errno 32, broken pipe), and I’m wondering about the best way to fix that wihout making the DBAs sad ;) [10:33:17] I’m currently initializing the DB connection at startup and that’s it [10:33:25] should I keep that approach, and try to reconnect when the connection dies [10:33:48] or instead open new connections on each web request and close them when the request finishes, to avoid stale connections hanging around? [10:40:23] !log tools T201602 upgrade packages from jessie-backports (excluding python-designateclient) [10:40:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [10:40:29] T201602: toolforge: round of package upgrades - https://phabricator.wikimedia.org/T201602 [10:41:25] WikidataFacts: I understand your problem, but can't properly answer your question [10:41:54] I would say stale connections hanging around is totally something to avoid [10:42:19] I can answer that [10:42:23] it is documented [10:42:49] ah [10:42:51] thanks jynus [10:42:51] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connection_handling_policy ? [10:42:52] WikidataFacts: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connection_handling_policy [10:42:56] yep [10:43:00] * arturo reading [10:43:02] amazing how I only needed to know it was documented ^^ [10:43:03] for webrequests [10:43:04] thanks jynus [10:43:15] it is highly recommended to open a connection each time [10:43:15] (I was searching for the error message, that didn’t bring up anything) [10:43:19] this is for several reasons [10:43:28] 1- idle connections are killed automtaically [10:43:43] 2-backends can change due to proxy balancing/maintenance [10:43:57] so you want to open a new connection normally to get the freshest conffiguration [10:44:16] 3-supporing pool of connections would overload the highly shared db infrastrcture [10:44:27] we have thousands of db users, so we have to be conservative [10:45:05] this would be different if you had dedicated resources (a db just for you), but that is not the case for toolsdb/wikireplicas [10:45:14] WikidataFacts: does it make sense? [10:45:22] totally, thank you [10:45:48] regarding retries, if you do, retry once to reconnect, then fail [10:56:07] MacFan4000: we would have to follow these steps: https://wikitech.wikimedia.org/wiki/Password_reset#Reset_two_factor_authentication [11:06:53] jynus: implemented at https://phabricator.wikimedia.org/R2390:fc35a05df86cf2d57961a145a7fbe58eab99f6dc, thanks again for your help! [11:08:33] you are welcome! [12:23:04] arturo: ok, I’ll make a file on a bastion [12:23:41] On tools-bastion-03 my home dir doesn’t exist [12:24:07] I haven’t become a tool at all [12:24:34] I can’t create the dir [12:28:01] mmm [12:28:15] and what if you become a tool? [12:31:37] Still can’t create it [12:34:38] Hi [12:34:40] MacFan4000 what is the error message? [12:36:40] https://www.irccloud.com/pastebin/fcIbzud4 [12:36:58] It’s cause /home is owned by root [12:36:58] I see a `legofan400` home directory which belongs to the `macfan4000` user [12:37:06] Oh [12:37:23] Guess my dir was never renamed [12:39:23] [12:40:16] https://www.irccloud.com/pastebin/UhnKc8bv [12:40:30] that's legit [12:42:14] MacFan4000: don't try to write anything outside your home [12:42:43] and /home is not a home of an user ... @_@ [12:42:56] so, for example, this works for me [12:43:43] `cd && touch test` as your macfan4000 user [12:43:56] ^^^ MacFan4000 try that please :-P [12:45:48] Maybe an odd request, and also a long shot, but would anyone here have time to help with deploying a tool on a horizon project instance? The developers who worked on it don't have the time, and I'm fumbling around in the dark trying to set it up. It's using docker, and I've managed to get it running, but don't seem to be able to hook everything up correctly for external web access. [12:46:42] Samwalton9: with horizon project instance I understand a Cloud VPS instance/virtual machine, right? [12:46:50] Yep [12:48:24] Specifically I'm trying to deploy https://github.com/Samwalton9/WikiCiteVis via the wikicitevis project. [12:49:05] for inbound connections we would need either a NAT or a proxy [12:49:26] (if I understand correctly) [12:52:11] So I think I have a web proxy set up [12:52:53] From my limited understanding I think the issue is most likely in the docker and file structure setup somewhere [12:54:43] Samwalton9: you set the web proxy, using horizon, right? let me jump to your project and review the setup [12:54:51] Yeah [12:54:54] Thanks :) [12:57:05] ok, so your docker setup should listen in this URL: http://10.68.19.194:8080 [12:58:16] is this file how your are configuring docker? https://github.com/Samwalton9/WikiCiteVis/blob/master/docker-compose.yml [12:58:31] bc it contains not a single mention to :8080 [12:58:44] Samwalton9: ^^^ [12:58:53] Yeah I think the only things I tried to change were some nginx config files that were pointing to localhost [12:59:25] Hmm, I think I went with 8080 because of https://github.com/Samwalton9/WikiCiteVis/blob/370d364eaee9fbedc92cc4dc1e5e256fa0d39587/.env, but perhaps it should be 8000 then? [13:00:01] well, I would say the proxy backend and the port you are actually using in docker should match [13:00:24] (i.e, the number in horizon and your docker config) [13:00:48] That would make sense - let me play around with that [13:01:07] ok, let us know [13:01:17] y [13:01:38] hey Murbaut, do you need some help? [13:01:57] no, thanks very much [13:02:07] ok :-) [13:02:35] arturo: /home/legofan4000/2fa.txt [13:03:13] Sorry for delay but my WiFi connection is having issues [13:03:20] MacFan4000: ok :-) [13:05:40] MacFan4000: what is your wikitech username? [13:05:42] Hmm. Modified all 8000 ports to 8080, and added the horizon IP everywhere it seemed it should be, gunicorn now reporting it's listening on :8090 for some reason. [13:06:10] One of the devs is going to give me some notes on what needs doing, so I think I'll wait for that. Thanks for the help so far arturo :) [13:06:27] you are welcome Samwalton9 :-) [13:07:22] MacFan4000 [13:07:47] OATHAuth disabled for MacFan4000 <--- done :-) [16:04:40] (03PS211) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [16:08:56] (03CR) 10jerkins-bot: [V: 04-1] Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [16:10:54] * Reedy wonders why they haven't just merged that commit yet [16:11:33] 211 patch sets wow [16:11:40] (03CR) 10Ricordisamoa: [C: 04-2] "PS211 adds help text to invalid entity inputs" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [16:12:13] And nearly 3 years [16:13:16] (03CR) 10Reedy: "Can I ask... Why are you just doing commits/amends onto this change set... It seems well beyond an initial commit now, and going on 3 year" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [16:13:39] (03PS212) 10Ricordisamoa: Initial commit [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 [16:13:44] @seen Framawiki [16:13:44] zhuyifei1999_: I have never seen Framawiki [16:16:19] can someone take a brief look at T201625 ? it’s not urgent, I just want to know if I added the right projects so the right people will see it :) [16:16:20] T201625: Review Wikidata Shape Expressions Inference tool regarding Toolforge Rule 6 - https://phabricator.wikimedia.org/T201625 [16:25:45] I will let you know when I see WikidataFacts and I will deliver that message to them [16:25:45] @notify WikidataFacts I /think/ 'The tool only allows authenticated users ... attributed to the user who started it' is okay, although I can't say for sure [16:31:32] (03CR) 10Ricordisamoa: [C: 04-2] "@Reedy" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [16:32:21] (03CR) 10Ricordisamoa: [C: 04-2] "PS212 removes unused defaultdict import" [labs/tools/wikidata-slicer] - 10https://gerrit.wikimedia.org/r/241296 (owner: 10Ricordisamoa) [17:56:29] zhuyifei1999_: sure :) [17:56:39] ok :) [18:01:04] I'm splitting the db out of the web server [18:40:52] framawiki: do you think we should make mwoauth a submodule? [18:41:03] I can't find the APT package for it [18:41:18] (submodule as in git) [18:42:09] i don't really like git submodules, but if it's necessary... [18:42:23] why not moving all py dependencies to a venv ? [18:42:56] probably not easy to puppetize [18:43:43] I really wonder how ORES does this [18:43:54] halfak|Lunch: [18:44:32] how does ORES deal with pip packages not in apt? [18:45:00] zhuyifei1999_: Ah yeah good question. By distributing a submodule of wheels in our deployment repo. Let me send links, [18:45:55] https://gerrit.wikimedia.org/r/plugins/gitiles/research/ores/wheels/+/master [18:46:40] okay. how do they get applied? [18:47:24] Here’s how they’re built: https://github.com/wikimedia/mediawiki-services-ores-deploy/blob/master/Makefile [18:47:34] They’re applied at runtime with virtualenv, [18:47:49] https://github.com/wikimedia/mediawiki-services-ores-deploy/blob/master/scap/scripts/fetch_check.sh [18:50:37] I don't think changing the out deployment mechanism to scap is do-able in the near-term without investing a lot of time on the setup [18:51:06] but thanks for the pointers. I'm looking into it [19:12:55] Can anyone help me figure out why labtestweb2001 is still trying to send writes to a read-only database master? it's spamming logstash [19:43:27] twentyafterfour, I think the labtest keystone DB is read-only [19:43:34] because it lives in codfw [19:43:50] or something [19:44:35] right I know why the db is read only. I guess I'm trying to figure out how to get labtestweb to stop connecting to it [19:44:49] I guess I can just ignore it for now [19:47:16] twentyafterfour, oh I shouldn't think you'd be able to do that properly [19:50:52] maybe you could try telling mediawiki that the wiki is read only [19:51:15] but I don't know if that'd get everything to stop [19:52:08] real solution really is just to use a DB host that isn't read-only [19:55:12] framawiki: I'm thinking of generating requireemets.txt using https://docs.pipenv.org/advanced/#generating-a-requirements-txt, considering we are no longer pinned by the version provided by APT and can choose whichever version we want [19:55:24] twentyafterfour, Krenair: yeah, we need to point labtestweb at a db server in eqiad rather than codfw. I think there is an open ticket about that [19:57:32] at some point I lost access to my labtestwiki account but couldn't get it restored because the db is readonly :( [20:13:25] zhuyifei1999_: yes, why not [20:13:37] k :) [20:16:42] will take a close look this weekend [23:53:26] !help Is anyone around to help debug a prominent tool labs project? (not mine) [23:53:26] huh_: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [23:55:29] The problem is with WikiMiniAtlas. If you don't know what that is, go to the top right corner of any location article on en.wikipedia (and a bunch of other wikis) and click the globe icon [23:55:39] I became aware of this issue from seeing https://meta.wikimedia.org/wiki/Wikimedia_Forum#WikiMiniAtlas [23:56:27] WikiMiniAtlas used to display links to nearby articles on the map, e.g. see https://upload.wikimedia.org/wikipedia/commons/b/b6/Wma_webglbuildings.png [23:57:06] Now it doesn't, because the server-side database query fails. [23:58:04] This bug was reported in December 2017 (https://github.com/dschwen/wikiminiatlas/issues/27) and the OP of the Wikimedia Forum section says they have tried to contact the author multiple times since then but has gotten no response. (Granted, I'm guilty of this too)