[08:03:33] hi folks, we have a grafana upgrade pending and cloudmetrics hosts are still to be upgraded -- let me know what you think ! https://phabricator.wikimedia.org/T263747 [08:50:12] godog: is there any backwards compatibility breakage? [08:53:01] dcaro: not afaik, the upgrade on grafana.w.o happened already and save for editing the home dashboard there wasn't anything else to be done [08:53:41] godog: nice, what do you need from us to go ahead? (/me is not familiar yet with the processes xd) [08:56:12] dcaro: basically that's an heads up the upgrade needs to happen, if you have time/space to upgrade the package yourselves I'd appreciate it, if not that's fine too [08:56:27] !log cloudvirt-canary canary instances seem to be stuck, looking (T275111) [08:56:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Cloudvirt-canary/SAL [08:56:30] T275111: [cloudvirt-canary]Canaries are not going through - https://phabricator.wikimedia.org/T275111 [08:56:59] godog: okok, what's the process, just apt-get upgrade? [08:58:21] dcaro: yeah, I usually do a backup of the sqlite db in /var/lib/grafana, then apt install grafana to be precise, unless you want to upgrade all packages [08:58:38] ack [08:59:45] godog: https://etherpad.wikimedia.org/p/WMCS-2021-02-18-grafana-update <- this? [08:59:53] !log tools.stewardbots restart both stewardbot and sulwatcher manually [08:59:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [09:00:56] is everything ok with wmcs network? many irc bots hosted there have been having some issues today [09:02:37] dcaro: yeah exactly, I've added the copy/paste commands [09:06:02] ack thanks [09:06:48] np -- thank you ! LMK if I can help [10:12:06] !log admin Upgrading grafana on cloudmetrics1001 (T275116) [10:12:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:12:11] T275116: [cloudmentrics] upgrade grafana - https://phabricator.wikimedia.org/T275116 [10:14:56] !log admin Upgrading grafana on cloudmetrics1002 (T275116) [10:14:59] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:16:24] !log admin Rebooting cloudmetrics1002 to apply new kernel (T275116) [10:16:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:25:24] !log admin Rebooting cloudmetrics1001 to apply new kernel (T275116) [10:25:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:25:28] T275116: [cloudmentrics] upgrade grafana - https://phabricator.wikimedia.org/T275116 [10:35:38] godog: done, let me know if you need anything else :) [10:58:35] dcaro: amazing, thank you so much! [11:07:11] Hey, what does `is_sensitive` mean in `meta_p.wiki`? [12:24:32] !log testlabs delete couple of VMs no longer in use (arturo-puppetmaster, arturo-cloudgw-test) [12:24:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Testlabs/SAL [12:55:32] https://github.com/wikimedia/puppet/blob/279db5b46befe4bd3a66b092e003c62e35ea47ac/modules/profile/files/wmcs/db/wikireplicas/maintain-meta_p.py#L356 [12:56:08] Not very clear... [13:34:40] on toolforge, I get "No module named 'pip'" when I try to run "pip install", am I missing something? [13:36:23] on the bastion or on a kubernetes pod? virtualenv or not? [13:37:30] oops, I just realized I was supposed to open the shell for kubernetes [13:37:33] nothing to see here :) [14:50:46] !log admin rebooting cloudnet1004 for T271058 [14:50:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [14:50:51] T271058: cloudnet1004/cloudnet1003: network hiccups because broadcom driver/firmware problem - https://phabricator.wikimedia.org/T271058 [14:54:18] kostajh: that gave me an idea... as I've seen that issue happen to more that one person (including myself) (T275135) [14:54:18] T275135: [toolforge] create a wrapper/relpace pip on the vms system to add hints on how to use it for the webservices - https://phabricator.wikimedia.org/T275135 [15:00:41] dcaro: nice idea! [15:23:11] Urbanecm: from what I could find in phab (wikitech doesn't say), it seems to be about case sensitivity in titles/links T69476 [15:23:12] T69476: Add wiki title case sensitivity flag (is_sensitive) to meta_p.wiki to support jbo.wp and wiktionary tools - https://phabricator.wikimedia.org/T69476 [15:23:46] ah! thanks joakino [15:24:03] I interpreted it as "has sensitive content" (ie. private), but wasn't able to find actually private wikis there... [15:25:07] yeah it definitely sounds like that, I thought the same too [15:31:20] here is the crux: https://gerrit.wikimedia.org/g/operations/puppet/+/4c0b0ea76628c999c04c9ae5f517b7e66dca76c2/modules/profile/files/wmcs/db/wikireplicas/maintain-meta_p.py#281 [15:32:20] > "case sensitivity of titles isn't in a .dblist, nor is it exposed through the API so we have to hardcode it here to match what is in InitialiseSettings.php" [15:37:55] hm, isn’t it the "case" field of siprop=namespaces? https://en.wiktionary.org/w/api.php?action=query&meta=siteinfo&siprop=namespaces&formatversion=2 [15:49:55] also, if that list in puppet is meant to reflect $wgCapitalLinks, then according to InitialiseSettings.php it’s missing szywiki (T237369) [15:49:55] T237369: Create Sakizaya Wikipedia - https://phabricator.wikimedia.org/T237369 [15:53:05] out of date lists in wikimedia? never [15:53:31] oh, there’s also a "case" in siprop=general https://en.wiktionary.org/w/api.php?action=query&meta=siteinfo&formatversion=2 which matches $wgCapitalLinks more closely I guess [15:54:05] and git blame traces that back to at least 2014 ^^ [15:57:52] meta_p is cargo cult data from toolserver [16:06:03] if anyone would like to review https://gerrit.wikimedia.org/r/c/operations/puppet/+/665115/ :P [19:16:04] I'd appreciate if a wikitech content admin looked at https://wikitech.wikimedia.org/wiki/Special:Contributions/Rizky_Yp, I think some of the pages they created should be deleted but they're just reverting my {{delete}} tags [20:18:50] thank you bd808 [20:19:11] not sure what was going on there, but it looked very strange [20:21:10] not sure either, some of it just looked like someone throwing new words together to sound like they know what it means [20:44:38] !log tools.lexeme-forms deployed 23ccbcf6f6 (work around T272319) [20:44:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [20:45:41] !log tools.lexeme-forms deployed a0ba7b84ab (quickfix) [20:45:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL