[09:35:36] !log devtools set phabricator::vcs::address::v6 to fe80 local address to fix puppet error on phabricator-stage-1001 [09:35:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Devtools/SAL [09:40:07] !log devtools set missing (and new) profile::tlsproxy::envoy::capitalize_headers: true to fix puppet errors [09:40:09] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Devtools/SAL [10:00:41] !log devtools - phabricator-stage-1001: replace deployment-tin.deployment-rep with deploy-1002.devtools in deployment-cache/.config [10:00:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Devtools/SAL [11:17:26] Hi, as a noobie with little to none background with servers (and especially lighttpd), I have a question: I have built a small tool called "wiki-tennis" which runs its webservice via https://wiki-tennis.toolforge.org/. The webservice runs a series of Python scripts and weboutput is created by Flask, i.e. via app.py with routes. I have changed my [11:17:27] $HOME/.lighttpd.conf to alloe for server-status and server-statistics according to https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web/Lighttpd#Enable_status_and_statistics, however, returns a "not found" page. Any suggestions? Thanks [11:18:02] let me see [11:18:16] well, give me a second, I'm finishing writing an email [11:18:25] no worries! [11:33:20] mad_melone: sorry if this questions is obvious, but did you try restarting the webservice after adding the new config? [11:33:30] yes [11:33:40] also I detected some blank lines issues in the file, probably due to copy/paste from wikitech [11:33:49] I just fixed the file, can you please restart it again? [11:33:53] that could well be [11:33:56] give me a sec [11:34:35] also, I don't like how this log looks like: [11:34:38] https://www.irccloud.com/pastebin/ZYiytOMY/ [11:35:07] I restarted. but it seems it didn't work [11:36:45] also, I don't understand the log you posted :( again, I am not really experienced [11:36:59] therefore, already thanks for your help [11:37:54] no problem, lets keep trying [11:39:13] wait this is using kubernetes, does the docker image include lighttpd? [11:40:46] i followed https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Python_(uWSGI) to setup the webservice [11:41:17] which I guess includes kubernetes [11:46:04] brb [11:48:47] back [11:52:03] mad_melone: how do you start your webservice? [11:52:26] webservice --backend=kubernetes python3.7 start [11:55:07] mad_melone: I think you are using uwsgi rather than lighttpd [11:55:15] ok [11:55:16] lighttpd would be used for php webservices [11:55:21] I think you are using https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Default_uwsgi_configuration [11:56:18] ok [11:56:48] so there is no way of getting server-statistics and server-status (just for the information it provides)? [11:59:17] ok, I googled it, probably don't need it anyways. However, I learned quite a bit in the last 30 minutes, so thank you so much for taking your time with me [12:00:39] mad_melone: would this help you? https://tools.wmflabs.org/k8s-status/namespaces/tool-wiki-tennis/ [12:00:48] in concrete: https://grafana-labs.wikimedia.org/d/toolforge-k8s-namespace-resources/kubernetes-namespace-resources?var-namespace=tool-wiki-tennis [12:01:57] that's really cool, thank you! I will bookmark right away [12:02:54] 👍 [12:09:38] Getting 504 gateway timeout from https://toolforge.org/ [12:12:42] Krinkle: same here [12:15:36] seems to be only affecting that particular URL, other webservices seem to be working just fine [12:15:58] honestly I don't know if we even gave support for that URL anywhere [12:40:15] Krinkle: better now? [12:41:18] !log tools.fourohfour adding a new ingress 'default-route-toolforge.org' to redirect `http://toolforge.org` to the wikitech front page (T234617) [12:41:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.fourohfour/SAL [12:41:22] T234617: Toolforge. introduce new domain toolforge.org - https://phabricator.wikimedia.org/T234617 [12:43:00] !log tools.fourohfour adding a new ingress 'default-route-www.toolforge.org' to redirect `http://www.toolforge.org` to the wikitech front page (T234617) [12:43:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.fourohfour/SAL [12:45:19] we can later on decide which is the "bestest" target for that URL, but meanwhile at least it lands in a sensible web page [13:40:25] arturo: thx :) [14:01:08] It’s the 15th June when the switch to redirecting to toolname.toolforge.org happens, correct? [14:10:43] Also, what’s happening to the maintainer emails [14:10:57] Which are tools.x@tools.wmflabs.org [14:20:09] !log tools.zppixbot update gerrit description for labs/tools/zppixbot - T250074 [14:20:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [14:20:13] T250074: Switch ZppixBot to use zppixbot.toolforge.org - https://phabricator.wikimedia.org/T250074 [14:24:51] RhinosF1: https://wikitech.wikimedia.org/wiki/News/Toolforge.org#Exceptions_to_service_migration [14:25:26] arturo: ah [15:07:18] !log admin restart memcached on labwebs to increase cache size T145703 [15:07:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [15:34:12] can someone help me move my bot to Kubernetes please? [15:36:38] !help [15:36:38] Examknow: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [15:36:52] hi Examknow ! [15:37:05] what is the name of your tool? [15:37:06] hello arturo [15:37:09] xslack [15:38:03] Examknow: have you tried the migration before? what went wrong? [15:38:19] arturo: I am running an eggdrop IRC bot on toolforge and zppix told me I need to be running it on kubernetes rather than bastions [15:38:57] oh the bastion is not the right place to run anything, that's for sure [15:39:02] ok [15:39:08] how do I move it [15:39:29] is your tool a webservice? [15:39:39] It has a webservice end [15:39:44] ok [15:40:01] Examknow: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes#Kubernetes_continuous_jobs -- that would be a good place to start reading about hosting an irc bot on the Toolforge Kubernetes cluster [15:40:15] bd808: thank you [16:52:37] question about the service template files – are we supposed to write those by hand (or adapt the sample) or is `webservice` also going to write them at some point, like service.manifest? [16:53:01] (I don’t see any template files in the wdmm tool so I assume webservice doesn’t write them at the moment at least) [16:55:07] lucaswerkmeister: for now they are a by hand thing [16:55:37] I feel like I should have added something in `webservice` to create them, so that could still happen [16:56:11] something like `webservice --backend=... --canonical python3.7 save-template` maybe [16:56:30] ok, thanks [16:56:50] then I’ll try putting a template file in the source code directory and see if symlinking it into ~ works [16:57:18] you can almost just copy $HOME/service.manifest to $HOME/service.template to do this. The difference I think is that service.manifest uses "web" where service.template uses "type". [17:03:39] why does pip need like ten seconds to tell me that --reinstall and --update are not valid options grmblgrmbl [17:03:42] (it’s --upgrade) [17:09:43] lucaswerkmeister: NFS :( [17:09:56] oh [17:10:10] right, if it’s a venv, all of pip comes from NFS I guess… [17:10:22] pretty much [17:10:24] and not some nice /usr place [17:10:28] meh, ok [17:12:21] !log tools.pagepile-visual-filter deployed 9666236 (toolforge.org, Python 3.7) [17:12:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.pagepile-visual-filter/SAL [17:13:29] I noticed that NoScript treats *.toolforge.org as a single domain right now [17:13:34] should that be split by tool? [17:13:42] I assume NoScript bases that decision on the Public Suffix List [17:14:56] oh, apparently it was done already? maybe I just need to wait a bit longer for NoScript to catch up https://phabricator.wikimedia.org/T168677 [17:15:24] yeah, a.rturo got that patch merged upstream I think [17:15:51] https://github.com/publicsuffix/list/commit/f71184ff9cd3e8632ba16d4ea148bcd8de42fcce [17:16:14] yeahr, and last noscript psl update was slightly before that https://github.com/hackademix/noscript/commit/8acd1551d777fda8ee70c92101640414f0053f24 [17:29:19] ok, first tool migrated successfully, I think [17:29:23] an OAuth one next, I guess [17:30:16] lucaswerkmeister: poke me when you need your new grant approved [17:30:23] will do, thanks! [17:31:40] * bd808 makes a note to look in the db and see how many OAuth grants point at tools.wmflabs.org [17:44:14] bd808: please approve https://meta.wikimedia.org/wiki/Steward_requests/Miscellaneous#OAuth_approval_request_for_SpeedPatrolling :) [17:44:31] (I wasn’t sure if I should still go through the stewards requests page or not) [17:46:40] lucaswerkmeister: it doesn't hurt anything, but I'm not sure any stewards are actually patrolling this stuff these days. [17:46:46] {{done}} by the way [17:46:49] thnaks [17:46:51] *thanks [17:46:59] then I’ll add that to the news page later [17:47:29] (currently doing venv re-init because this is another tool that I hadn’t migrated to Python 3.7 yet) [17:47:44] you are the best person for taking that time :) [17:48:13] 😊 [17:48:22] php 5.5 and python 3.4 are our most used Kubernetes runtimes right now [17:49:00] and joakino will be getting an intro to the Toolforge community in helping deprecate them :) [17:49:09] good luck ^^ [17:49:15] php 5.5 is truly ancient, jeez [17:49:32] its brand new in wiki-years lucaswerkmeister [17:49:49] not so long ago that we had to support it in MediaWiki ourselves… [17:50:04] I think it was 5.6 that brought non-primitive parameter types? that was so nice [17:50:32] * bd808 is a lover of duck-typing [17:52:06] !log tools.speedpatrolling deployed 6434f58a84 (toolforge.org, Python 3.7) [17:52:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.speedpatrolling/SAL [17:52:21] and everything seems to work! I can log in and patrol edits, at least [17:52:51] thanks bd808! [17:53:00] cool! [17:53:25] lucaswerkmeister: now you can stop filing bugs about XSS exploits ;) [17:53:40] sshhhhhhh :D [18:00:45] lucaswerkmeister: FWIW I just removed the reference to the steward noticeboard from the OAuth docs over the weekend [18:00:54] (well, some of the docs, I imagine) [18:01:05] ah, ok [18:01:28] almost no one used it, and there wasn't any good reason to, the extension has its own request queue and notification system [18:01:30] it’s still in the message on top of the registration page (https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration/propose), that’s where I usually follow it [18:01:42] ok, I didn’t know that [18:01:57] well, it says ask there if nothing happened for a few days [18:02:04] ah, ok [18:02:11] I guess I skim over that part, sorry [18:02:29] tgr: I don't suppose you have interface admin rights on meta to fix that message override? [18:02:40] I wouldn't expect people to re-read it every time they make a new tool :) [18:02:47] * bd808 guesses he could go ask to collect a new hat [18:03:13] it's a borderline staff responsibility so I just edit it as a staff member [18:03:24] I don't think it needs any fixing, though [18:03:54] (well-actually, interface editor) [18:04:27] * bd808 does not have the staff group and is not too sad about that [18:04:50] I see you have hacked that message before though, so do you will :) [18:05:38] *do as you will [18:06:33] anyways, feel free to revert my edit to the news page in that case, since I misunderstood “the usual process” ^^ [18:07:24] I got it because of Superprotect being introduced but fortunately did not have to use it to anything related to it [18:07:32] probably jumped a huge bullet there [18:09:00] yeah, that was a bad time to have the staff right :( [18:09:17] * bd808 almost got tricked into writing the superprotect config patch [18:11:52] probably the only commit in Wikimedia history with an "I wrote this under duress" summary [18:11:55] fun times [18:12:26] it was quite a day [18:17:59] !log tools.github-pr-closer updating from Gerrit [18:18:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.github-pr-closer/SAL [18:36:09] bd808, is there a group we can delegate OAuth request approval to? [18:36:43] bd808, tgr: also I can possibly fix any messages? I hold global editinterface [18:37:07] what change is being proposed exactly [18:50:28] oh I see that if you have your language set to plain en you get a different notice [18:50:38] with a link to SRM [18:59:12] Krenair: there's an oauthadmin group, but in practice it's pretty much me and Bryan [18:59:40] a long time ago we talked to stewards and they felt this is too technical for them [18:59:58] we could probably recruit people from the developer community if we put some effort into it [19:00:21] do we have documentation for what things to look for when reviewing these requests? [19:01:14] Krenair: I feel like we wrote up some stuff years ago when we were trying to convince the stewards to do it. Not sure where on meta we put that though. [19:01:47] mostly reasonable perms, TLS, and ideally open source of the app [19:03:45] that sounds fine [19:04:41] https://meta.wikimedia.org/wiki/Special:ListUsers/oauthadmin hm [19:05:02] tgr: I could be definitely interested in that of that's even a possibility [19:06:16] that was https://meta.wikimedia.org/wiki/Requests_for_comment/OAuth_handover [19:07:01] and https://meta.wikimedia.org/wiki/OAuth_app_guidelines [19:07:30] and https://meta.wikimedia.org/wiki/MediaWiki:Mwoauthconsumerregistration-propose-text which is the one people actually read [19:08:27] some of it will have to be updated for OAuth 2 (especially the parts around keeping the secret secret) [19:08:42] there's a couple of proposed consumers here, 9th and 12th: https://meta.wikimedia.org/wiki/Special:OAuthListConsumers?name=&publisher=&stage=0 [19:09:17] I tend to ignore everything that has 'test' in the name [19:09:28] tgr, it's what people read if they left their interface language set to en :) - I don't see it at https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration/propose [19:10:02] unapproved consumers still work for the user who proposed them, so that's usually enough for testing [19:10:40] My main contribution to cleaning up the workboard is approving the localhost requests once in a while :) [19:11:15] system message fallback rules always confuse me [19:11:20] Krenair: is there a magic trick to make the en-gb message follow the en message? [19:11:25] shouldn't local overrides work for all languages? [19:11:28] Don't have an override :P [19:11:36] tgr: https://meta.wikimedia.org/wiki/Special:OAuthConsumerRegistration/propose?uselang=en-gb [19:12:14] https://github.com/wikimedia/mediawiki-extensions-OAuth/blob/master/i18n/en-gb.json#L9 [19:12:14] I don't think there is any cascade for the MediaWiki: namespace overrides, but I might be wrong about that [19:12:31] s/z/s/ [19:12:56] those darn zeds [19:13:34] You could put the override in WikimediaMessages... And then a translation can exist easily [19:14:10] this is T3495, I can never remember which is the direction it is broken in [19:14:11] T3495: Improve message source fallback flow - https://phabricator.wikimedia.org/T3495 [19:17:17] Reedy, but then one would have to make a patch every time it needs changed [19:17:25] Sure [19:17:32] But at least it gets translated and rolled out [19:18:30] or not, compare https://translatewiki.net/wiki/MediaWiki:Mwoauthconsumerregistration-propose-text/en with https://translatewiki.net/wiki/MediaWiki:Mwoauthconsumerregistration-propose-text/en-gb [19:18:48] I got nerd sniped into working on a language inheritance chain bug a few years ago. Not going to trick me with that again. ;) [19:19:26] that bug actually does not seem hard [19:20:12] I traced the code paths at some point, never got around to write a patch for it [19:20:26] but it was fairly self-contained [19:21:30] the one I worked on was about Special:MyLanguage not doing the same things as the json message lookups [19:22:20] because of course MediaWiki has 2-N parallel implementations for this kind of stuff ;) [19:22:29] https://translatewiki.net/w/i.php?title=MediaWiki:Mwoauthconsumerregistration-propose-text/en-gb&diff=prev&oldid=9375746 [19:22:37] en-gb gets full of non en-gb translations [19:22:50] I went on a spree deleting over the weekend [19:22:59] Removing 700+ wrong or identical as en translations just from MW core [21:09:11] Hi, can an labsadmin take a look at https://wikistream.wmflabs.org/ ? It's throwing a 503 [21:10:28] or "cloudadmin" now? [21:11:48] Sagan, are you a member of that project or a user? Typically you'd want to contact the actual people who run the project [21:12:10] andrewbogott: https://openstack-browser.toolforge.org/project/wikistream says you :P [21:12:32] yeah, I'm in all the old ones because I made 'em :( [21:12:39] heh [21:12:44] So looks like edsu and noone [21:12:48] otherwhise it's edsu, but not really active [21:12:59] (here the user talk: https://en.wikipedia.org/wiki/User_talk:Edsu#Wikistream_2) [21:14:09] I mean 503 must not be a big problem, so it would be nice if someone can take a look at restart it maybe?= [21:14:28] I'm logged in, what would I restart? [21:14:35] I could also reboot the whole VM [21:14:57] I'm not sure, what kind of service. but reboot could work too [21:15:38] !log wikistream rebooting host ws-web [21:15:39] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikistream/SAL [21:15:45] huh, no apache, no nginx, no uwsgi [21:15:54] I restarted varnish but that didn't seem to do much [21:16:09] now it's 502 xD [21:22:52] reboot seems to have (eventually) resolved the problem [21:24:41] I also pestered edsu on wiki [21:25:14] nice, works now for me [21:25:45] andrewbogott: thx for the fix :) [21:25:52] sure [21:26:03] do you think that project has lots of users these days, or is it just you? [21:26:29] I wonder if we should have some way of excluding cloudadmins from the project admin lists of individual projects, except those projects where it makes sense [21:26:43] hm, someone else notified me that it's down, so at least two ;) [21:28:09] Krenair: really I should clean up my routines so that I can do things without actually having a lurking membership [21:28:28] :) [21:29:23] well, or, Horizon should clean up their game so there's inheritable adminship [21:29:29] s/horizon/keystone/ [21:56:36] what's a normal loading time for this listing? took almost 2 full minutes - https://openstack-browser.toolforge.org/server/ [22:02:03] DSquirrelGM, I expect it'll take a while as it has to send a ton of queries to nova/keystone [22:02:35] there's no single API call you can make to get all instances across labs, you have to get a list of all projects, get a session in each project, and get all instances in each project [22:03:14] didn't know if there was some issue going on, or if that was an expected delay time [22:06:56] it's expected to be slow to load [22:07:16] it should be fast for the second visit DSquirrelGM :) That page makes a really large number of serial requests to the OpenStack APIs [22:07:37] the results are cached for a while (30 minutes?) [22:20:22] !log tools.lexeme-forms deployed ab7f751ba6 (edit mode) [22:20:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [22:24:31] !log tools.lexeme-forms deployed 2fe2118d4e (python3.7) [22:24:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL