[00:06:29] 10Data-Services, 10DC-Ops, 10Operations: Split up labstore external shelf storage available in codfw between labstore2001 and 2 - https://phabricator.wikimedia.org/T171623#3534707 (10madhuvishy) @Papaul poke on this task since it's been ~3 weeks. Let me know if you need anything from me to proceed, thank you! [01:09:46] PROBLEM - Puppet errors on tools-exec-1438 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [02:18:38] 10Tool-stewardbots, 10Need-volunteer, 10User-MarcoAurelio, 10WorkType-Maintenance: SULWatcher: avoid logging account creations more than once - https://phabricator.wikimedia.org/T156546#3534800 (10RuyP) [02:19:43] RECOVERY - Puppet errors on tools-exec-1438 is OK: OK: Less than 1.00% above the threshold [0.0] [02:35:05] 10Data-Services, 10DC-Ops, 10Operations: Split up labstore external shelf storage available in codfw between labstore2001 and 2 - https://phabricator.wikimedia.org/T171623#3534808 (10Papaul) @madhuvishy This task was not assigned to me and it is no way on the ops-codfw work board so I did not know about the... [02:35:58] 10Data-Services, 10DC-Ops, 10Operations: Split up labstore external shelf storage available in codfw between labstore2001 and 2 - https://phabricator.wikimedia.org/T171623#3534809 (10Papaul) a:03Papaul [02:40:43] PROBLEM - Puppet errors on tools-exec-1438 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [03:15:44] RECOVERY - Puppet errors on tools-exec-1438 is OK: OK: Less than 1.00% above the threshold [0.0] [03:43:52] PROBLEM - Puppet errors on tools-exec-1431 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [04:08:55] RECOVERY - Puppet errors on tools-exec-1431 is OK: OK: Less than 1.00% above the threshold [0.0] [06:34:04] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1422 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:39:45] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1416 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:43:10] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1426 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:57:16] PROBLEM - Puppet errors on tools-exec-1420 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:57:40] PROBLEM - Puppet errors on tools-exec-1422 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [07:14:01] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1422 is OK: OK: Less than 1.00% above the threshold [0.0] [07:18:10] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1426 is OK: OK: Less than 1.00% above the threshold [0.0] [07:19:46] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1416 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:14] RECOVERY - Puppet errors on tools-exec-1420 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:42] RECOVERY - Puppet errors on tools-exec-1422 is OK: OK: Less than 1.00% above the threshold [0.0] [18:33:19] 10Tools, 10Toolforge-standards-committee, 10Privacy: Hunt for Toolforge tools that loads resources from third party sites - https://phabricator.wikimedia.org/T172065#3535348 (10yuvipanda) [18:33:21] 10Tools: Tool "masscamps" loads jquery and angularjs from ajax.googleapis.com - https://phabricator.wikimedia.org/T172660#3535345 (10yuvipanda) 05Open>03Resolved a:03yuvipanda I've shut it down. [18:33:33] 10PAWS: Point paws.wmflabs.org to new PAWS setup - https://phabricator.wikimedia.org/T172257#3535349 (10yuvipanda) Done now! \o/ [18:36:06] 10PAWS: Installing R packages fails. Directory not write-able - https://phabricator.wikimedia.org/T173278#3523053 (10yuvipanda) Fixed and deployed. paws.wmflabs.org should have it in about 20 minutes or so. [18:36:13] 10PAWS: Installing R packages fails. Directory not write-able - https://phabricator.wikimedia.org/T173278#3535352 (10yuvipanda) 05Open>03Resolved a:03yuvipanda [18:36:33] 10PAWS: PAWS doesn't display space character - https://phabricator.wikimedia.org/T171820#3477179 (10yuvipanda) What browser are you on? [18:40:04] 10PAWS: Enable downloading notebooks as PDF - https://phabricator.wikimedia.org/T159617#3535375 (10yuvipanda) 05Open>03Resolved a:03yuvipanda I deployed new backend for paws.wmflabs.org now, and it should allow you to download PDFs. Please re-open if it doesn't! [18:40:28] 10PAWS: PAWS loop redirect - https://phabricator.wikimedia.org/T159072#3056065 (10yuvipanda) Heya! This should be fixed now, since I deployed a new backend for paws.wmflabs.org. Let me know if it isn't! [18:41:38] 10PAWS: paws.wmflabs.org should be the public view - https://phabricator.wikimedia.org/T150180#3535380 (10yuvipanda) 05Open>03declined I'm going to say 'too late' and close this, unfortunately. [18:42:54] 10PAWS: PAWS 500 : Internal Server Error - https://phabricator.wikimedia.org/T133486#3535382 (10yuvipanda) 05Open>03Resolved a:03yuvipanda I've switched the backend of paws completely as of today, and that should help fix most of these! [18:45:02] 10PAWS: PAWS loop redirect - https://phabricator.wikimedia.org/T159072#3535385 (10yuvipanda) 05Open>03Resolved