[02:02:31] PROBLEM - Puppet errors on tools-exec-1436 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [02:31:12] 10Striker, 10Epic, 10Patch-For-Review: Manage shared tool accounts via Striker - https://phabricator.wikimedia.org/T149458#2753308 (10bd808) [02:32:34] 10Striker, 10Epic, 10Patch-For-Review, 10cloud-services-team (FY2017-18): Manage shared tool accounts via Striker - https://phabricator.wikimedia.org/T149458#3337590 (10bd808) [02:33:06] 10Striker, 10Epic, 10Patch-For-Review, 10cloud-services-team (FY2017-18): Manage shared tool accounts via Striker - https://phabricator.wikimedia.org/T149458#2753308 (10bd808) p:05Triage>03Normal [02:37:30] RECOVERY - Puppet errors on tools-exec-1436 is OK: OK: Less than 1.00% above the threshold [0.0] [03:08:28] !log striker Fixed firewall issue by applying profile::mediawiki::deployment::server instead of role::deployment_server [03:08:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL [03:23:46] 10Labs, 10Striker, 10Operations, 10LDAP: Store Wikimedia unified account name (SUL) in LDAP directory - https://phabricator.wikimedia.org/T148048#3337599 (10bd808) @faidon, @Volans, and I talked about this at the Vienna hackathon. Moving this data from #striker's local DB to LDAP would be a useful step in... [06:30:20] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1412 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:30:58] PROBLEM - Puppet errors on tools-exec-1434 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [07:10:19] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1412 is OK: OK: Less than 1.00% above the threshold [0.0] [07:10:57] RECOVERY - Puppet errors on tools-exec-1434 is OK: OK: Less than 1.00% above the threshold [0.0] [11:50:10] 10Tool-Labs-tools-stewardbots, 10WorkType-NewFunctionality: Let stewardbot display temporary rightchanges too - https://phabricator.wikimedia.org/T164204#3337845 (10MarcoAurelio) [12:22:39] 10Labs-project-other, 10Math: Instances in math project show high system CPU usage - https://phabricator.wikimedia.org/T160824#3337871 (10Physikerwelt) p:05Triage>03Normal [12:23:51] 10Labs-project-other, 10Math: Instances in math project show high system CPU usage - https://phabricator.wikimedia.org/T160824#3112076 (10Physikerwelt) @Hcohl we need to take care of updating the drmf server. I suggest to rename drmf-beta to drmf and to start a new drmf-beta instance. What do you think? [12:31:16] Is the IBM Public License accepted on labs too? [14:07:06] 10Tool-Labs-tools-Other: svgtranslate tool-Not Working - https://phabricator.wikimedia.org/T164275#3337961 (10Jarry1250) Excessive warnings fixed in https://github.com/Jarry1250/labs-svgtranslate/commit/f4c75bc43d71dbaaf316e1596292ad97b651052d Actual oAuth error has me stumped though -- was working in January [14:15:03] !log video Updating code on encoding0{1..3} and frontend to 1e60ecc [14:15:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Video/SAL [14:23:23] !log video Depool v2c on encoding0{2,3} [14:23:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Video/SAL [14:24:08] !log video Repool v2c on encoding0{1,2} [14:24:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Video/SAL [14:25:02] !log video Restart v2c frontend [14:25:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Video/SAL [16:01:39] !log tools.nlwikibots changed mailing list settings to allow emails from non-subscribers [16:01:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.nlwikibots/SAL [16:04:28] 10Labs: Request increased quota for git labs project - https://phabricator.wikimedia.org/T163213#3338054 (10Paladox) @bd808 sorry for late reply. Since this task was filled now the wikimedia ai team have begun using icinga2 using https://gerrit-icinga.wmflabs.org on labs. [16:12:40] hello [16:12:53] cannot connect ot my replica databases on labs [16:13:10] is there a problem there? [16:15:10] it says [16:15:11] ERROR 1045 (28000): Access denied for user 'superyetkin'@'10.68.23.58' (using password: YES) [16:16:48] Superyetkin: definitely wrong username [16:17:07] Superyetkin: use your shell username (found in wikitech preferences) [16:17:10] did you use the shortcut like $ sql enwiki [16:17:30] Zppix: no it should be something like u [16:17:53] oh yes sorry i forgot db was uid ... [16:18:18] mysql -u superyetkin -p [16:18:22] like this? [16:19:17] no, which database are you connecting to? [16:19:47] s51698 [16:19:54] or trwiki_p replica [16:20:18] I used to connect without any problem [16:20:30] but today, it is rejecting the connection [16:20:31] $ sql trwiki [16:21:16] ok, sql trwiki worked [16:21:36] but when I try to connect with Mysql Workbench, it does not accept my password [16:21:57] the username and password are stored in your replica.my.cnf [16:22:18] it's not your ldap password [16:24:51] after become superyetkin ? [16:25:13] it is weird, used to accept the same before [16:25:29] `become` switch to a service account from user account [16:25:34] but both should work [16:27:54] does not work [16:28:43] what do you mean by 'does not work'? what username did you use? [16:29:01] it should be in the form of u like u3306 [16:29:16] I do this [16:29:21] or s51201 [16:29:22] mysql -u superyetkin -p [16:29:41] still wrong username. see replica.my.cnf file [17:02:02] 10Labs, 10Commons: high commonswiki_p replag on labs - https://phabricator.wikimedia.org/T167120#3338221 (10Framawiki) 05Open>03Resolved a:03Framawiki ``` MariaDB [commonswiki_p]> SELECT lag FROM heartbeat_p.heartbeat JOIN meta_p.wiki ON CONCAT(shard, '.labsdb') = slice WHERE dbname = 'commonswiki'; +---... [17:49:22] 10Labs-project-other, 10Math: Instances in math project show high system CPU usage - https://phabricator.wikimedia.org/T160824#3338436 (10Hcohl) I need to check to make sure that beta is in a perfect final state before I give my permission. I will try and do the evaluation this week. [18:35:10] PROBLEM - Puppet errors on tools-exec-1409 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [19:03:46] shinken-wm: try doing just sql enwiki [19:03:50] uhh [19:04:19] Superyetkin ^ (but i guess the user quit) [19:05:08] RECOVERY - Puppet errors on tools-exec-1409 is OK: OK: Less than 1.00% above the threshold [0.0]