[02:31:26] !help I'm trying to restart a job but can't [02:31:26] rschen7754: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [02:31:31] tools.usrd-tools@tools-sgebastion-07:~$ qmod -r 2409949 [02:31:31] denied: host "tools-sgebastion-07.tools.eqiad.wmflabs" is not an admin host [02:32:15] is it possible to not change how labs works every 6 months so i have to relearn everything?> [02:43:23] rschen7754: I'll look into the admin host error. In the mean time, I don't see the job 2409949 running. Were you able to work around your issue? [03:02:35] per the man page it looks like rescheduling jobs with qmod now requires admin / manager privileges. I've updated the toolforge docs at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#Restarting_jobs_%28Deprecated%29 [08:28:30] !log tools.newusers Switched over to use new k8s cluster [08:28:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.newusers/SAL [10:55:49] !log tools drop jessie VM instances tools-prometheus-{01,02} which were shutdown (T238096) [10:55:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [10:55:56] T238096: Toolforge: prometheus: refresh setup - https://phabricator.wikimedia.org/T238096 [12:27:25] Hi everyone. I'm struggling with migrating my tool's celery job to the new kubernetes cluster. After updating the deployment yml file, I'm getting this error from kubectl create: `Error from server (Forbidden): error when creating "/data/project/wikigrade/wikigrade-celery.yml": deployments.apps is forbidden: User "wikigrade" cannot create resource "deployments" in API group "apps" in the namespace "wikigrade"` [12:31:38] okay, never mind - prefixing the namespace with `tool-` fixed it. [12:42:16] 🎉 [12:53:31] I had a similar issue [12:58:11] I tried to add some more info to the news page, maybe it helps [15:29:26] bd808: block: https://wikitech.wikimedia.org/wiki/Special:Contributions/Jimbo_Is_A_thicc_boi [15:31:33] charming, this time they’ve vandalized my user page too [15:33:08] !help wikitech vandalism ^ (I think this command also pings some admins) [15:33:08] Lucas_WMDE: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [15:33:31] Looking into it [15:33:51] hm, this one is persistent [15:33:52] thanks bstorm_ [15:34:30] Blocked them. [15:34:37] Haven't reverted vandalism [15:37:16] Not too many. I can do that by hand [15:37:24] I’ve reverted some, but can’t delete pages that were created [15:38:11] to revert do you just undo their edit? [15:38:20] that’s what I’m doing [15:38:27] and then adjust the edit summary so the user name isn’t in it [15:39:40] Did a mass delete on anything they created [15:39:52] thanks [15:39:53] I don't remember if wikitech has rollback links for adinistrators - but that won't help if it's a new page [15:39:55] 👍🏻 [15:42:13] I think that’s everything, I don’t see any “current”s left on the contributions page [15:44:42] Same [15:44:46] Thanks for the help [15:44:56] And raising the alarm :) [16:03:00] * bd808 sighs about boring vandal edit [16:06:20] the saddest thing is, it isn't even new text [16:06:22] bd808: if it was you who revdeleted the username, you probably also want to delete the comment of https://wikitech.wikimedia.org/wiki/Special:Diff/1854323, that was before I realized I should remove the name from the edit summaries [16:07:05] Lucas_WMDE: *nod* I can get that. [16:07:13] thanks [16:10:50] for future reference, would you prefer I send reports here, or the main #wikimedia channel, for wikitech? [16:11:58] That's a good question. I won't see them in the #wikimedia channel, but there may be other admins there who would? [16:12:09] I know the WMCS team is most likely to see reports here [16:12:54] DSquirrelGM: here is probably best. Wikitech is relatively isolated from the the main wiki patrolling process [16:57:25] !log deployment-prep deploying ores a6f4f14 [16:57:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [18:11:26] !log admin shutdown cloudvirt1016 for hardware maintenance T241882 [18:11:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [18:11:30] T241882: cloudvirt1016 crash - https://phabricator.wikimedia.org/T241882 [18:45:07] !log tools.mabot Deploy 454b2d5 [18:45:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.mabot/SAL [18:51:49] !log tools.fourohfour Updated Ingress default-route to redirect https://tools.wmflabs.org/ to https://tools.wmflabs.org/admin/ [18:51:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.fourohfour/SAL [18:58:27] that reminds me, is it possible to set up a default page to use in any tool's subdirectory to show if an index.* file isn't in that subdirectory of public_html ? Seems like it'd be a useful configuration option to have [18:59:08] I guess that would be a custom error handler? https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web/Lighttpd#Header,_mimetype,_character_encoding,_error_handler [19:00:50] (I have no idea if the “outdated information follows” warning on that page also applies to the error handler option… try it out?) [20:07:26] bd808: hi. Is it possible to get a bit more insight as to why tools.wmflabs.org/stewardbots renders HTTP Error 500 ? [20:08:21] hauskatze: I'm in a meeting, but I would suggest checking $HOME/error.log [20:08:46] ack, checking [20:08:47] thanks [20:10:00] sigh, composer error [20:10:03] okay [21:09:19] !log toolsbeta upgraded toollabs-webservice package for stretch toolsbeta to 0.59 T244293 T244289 T234617 T156626 [21:09:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [21:09:29] T244293: Add a function to webservice called "migrate" that will push a tool from the old cluster on Kubernetes to the new one - https://phabricator.wikimedia.org/T244293 [21:09:29] T244289: Improve limit range management in webservice for Kubernetes - https://phabricator.wikimedia.org/T244289 [21:09:29] T234617: Toolforge. introduce new domain toolforge.org - https://phabricator.wikimedia.org/T234617 [21:09:29] T156626: k8s webservice restart failure with `ValueError: get() more than one object; use filter` - https://phabricator.wikimedia.org/T156626 [23:07:48] !log toolsbeta upgraded toollabs-webservice for stetch toolsbeta to 0.60 T244611 [23:07:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [23:07:51] T244611: Regression in webservice 0.59 due to deprecation change - https://phabricator.wikimedia.org/T244611