[08:41:13] !log tools upgrade prometheus-02 to prometheus 2.6 - T215272 [08:41:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [08:41:17] T215272: Upgrade Prometheus to 2.6 in deployment-prep and tools - https://phabricator.wikimedia.org/T215272 [13:23:06] Does anyone know who I can talk to about Hue issues? [13:23:47] @edsanders: hue? [13:24:51] https://hue.wikimedia.org/ [13:25:29] edsanders: #wikimedia-analytics probably [13:25:42] thanks [16:33:06] /usr/local/sbin/tools-clush-generator is spamming cron with some key again [16:33:34] andrewbogott: I think you looked at this last time? I'm sorry to randomly ping but I searched and searched in phab and failed to find the old ticket... [16:34:05] apergos: I'll look [16:34:13] thanks, I really appreciate it [16:34:48] Cron just fyi [16:35:08] apergos: I only see a single message, are you seeing lots? [16:35:14] no, just the one [16:35:22] but the leak of the key is the thing that concerns me [16:35:35] even though it is likely not archived anywhere now [16:35:42] oh! It's the observer pass [16:35:46] yep [16:35:46] that's public [16:35:58] it is? well nm then [16:36:24] Thank you for noticing though. Probably that could/should be moved to a config file to avoid people being alarmed :) [16:37:02] heh [16:39:00] apergos: T215532 [16:39:01] T215532: Move tools-clush-generator password to a config file - https://phabricator.wikimedia.org/T215532 [16:39:09] that one [16:39:16] why could I not find it >_< [16:39:21] oh, that's a new one maybe [16:39:44] subscribed, thank you! [16:39:58] yeah, it' s because I just made it [16:40:49] heh [16:41:06] there was some old ticket about thi observer-thingie password, I swear [16:41:18] anyways it doesn't matter now, this will be the last of it [16:43:38] and there's a second spam so I guess a few mght come in until the job is happy again [16:48:20] apergos: maybe you're remembering this one: T210404 [16:50:32] the tools-generator script connects to cloudcontrol1003, which is the active server... and we just shutdown cloudcontrol1004. it shouldn't have failed but maybe we had a hiccup of sorts [16:51:15] YES that's the one [16:51:28] and now I'll never find it again :-D [16:52:31] lol [16:53:18] I was talking about this with bstorm_ the other day, what a coincidence (re: using cmd line arguments vs config files) [17:03:35] yeah we really (as an org) gotta not pass in passwords on the command line like that, it's just an accident waiting to happen :-D [17:03:43] agreed [17:03:55] all right, off to staff breakfast thingie, see you in the usual channels! [17:34:06] o/ Who here can approve https://toolsadmin.wikimedia.org/tools/membership/status/471? [17:34:49] who's on call this week? (is that an "on call" thing?) [17:35:20] worst case scenario we'll get bd808 to do it :) [17:35:39] Dom Walden, that's the new contractor? [17:35:57] harej: Yeah. QA. [17:37:54] andrewbogott: I believe ^^^ [17:38:06] yep, it's me [17:39:05] Niharika: done, sorry for the delay [17:53:44] andrewbogott: Thank you! [20:18:17] !log tools cleared mail queue on tools-mail-02 [20:18:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [22:21:23] Hi all :) Just received the warning about migration from Trusty to Stretch. Looking at https://wikitech.wikimedia.org/wiki/News/Toolforge_Trusty_deprecation#Move_a_grid_engine_webservice -- the section about how to move the webservice -- if it does not work with stretch, how do I move it back to trusty? If it does not work, ideally I would like to have the ability to run the main instance on Trusty and a test instance on Stretch [22:21:23] simultaneously - is this possible? [22:22:00] Sveta: good questions! [22:22:26] Moving from Stretch back to Trusty is currently as easy as stopping the Stretch job and starting it again on the Trusty grid [22:23:20] We don't have a way for a single tool to run multiple webservices, so running on both Trusty and Stretch at the same time is not possible under a single tool account [22:23:46] This would be nice to have, in the long run, right? [22:24:07] In the long run trusty is dead [22:24:43] I mean when stretch becomes obsoleted and there is a NewThing, it would be nice to have a transition period, when I can run the tool on both stretch and the NewThing, to implement bugfixes without causing downtime to my tool. [22:26:02] Sveta: so the "easy" way to do that today is to make a second tool account for your "test" code to run under. We have quite a few examples of that today in tools like https://tools.wmflabs.org/admin-beta/ [22:26:07] Sveta: yes, I agree, sorry. Just had to seize the Keynes quote [22:26:44] The "hard" way is creating a deployment k8s file and testing locally with curl? [22:27:01] the annoying part of making a -beta tool is that today there is no easy way to clean up the -beta when you are done using it [22:36:42] !log tools.paws Move webservice from Trusty to Stretch [22:36:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.paws/SAL [22:38:30] !log tools.paws Move webservice from Stretch to Kubernetes [22:38:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.paws/SAL [22:39:52] chicocvenancio: ^ the paws webservice is moved [22:53:16] bd808: great! Thanks [22:59:34] Trying to move my crontabs from trusty to stretch results in '"-":34: bad day-of-week. [22:59:34] errors in crontab file, can't install. [22:59:35] ' [23:00:40] DatGuy: interesting. which tool? I would like to see what is making our crontab wrapper sad in your existing config [23:00:48] datbot [23:03:25] DatGuy: I think its the "1/1" on line 35 of that file. That doesn't seem like a valid day of week specifier to me [23:03:45] that file == ~/crontab.backup for datbot [23:04:39] that cron line seems to want to run a job at midnight on the 12th of January each year? [23:05:15] changing "1/1" to "*" should fix it [23:05:28] seems like it, don't exactly remember. should be fine to just change it to 0 0 1 * [23:06:02] that whole like kind of seems like nonsense... [23:06:33] yeah thinking about it it really does [23:07:01] "? * echo ''" as a command won't do anything useful as far as I can tell [23:07:10] it did seem to clear the file though [23:07:50] 0 0 1 * echo '' > /data/project/datbot/Tasks/wikiproject/pageschecked.txt [23:07:52] would that work? [23:12:39] you need to specify minute, hour, day_of_month, and day_of_week ... so you need one more * somewhere [23:13:16] https://pastebin.com/hSB0u9TX [23:13:17] seems fine? [23:13:52] DatGuy: yeah. that should run at midnight on the 1st of each month [23:14:24] thanks [23:14:45] crontabs are somehow always confusing [23:48:39] !log tools Updated DNS to make tools-trusty.wmflabs.org and trusty.tools.wmflabs.org CNAMEs for login-trusty.tools.wmflabs.org [23:48:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL