[00:27:00] 10Data-Services, 10cloud-services-team (Kanban), 10Analytics, 10DBA, 10Research: Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#3556374 (10bd808) Consolidating discussion: >>! In T173513#3538175, @bd808 wrote: > The ask here as I... [01:09:40] 10Data-Services, 10cloud-services-team (Kanban), 10Analytics, 10DBA, 10Research: Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#3556395 (10bd808) # Straw dog proposal * Single database named `datasets_p` hosting //N// tables of c... [02:26:44] bd808: I'm wondering, how are meta_p setup on the new servers? [02:28:06] maybe dataset_p can somehow use a similar mechanism [02:34:56] meta_p is generated using a python script that is manually run against each server. [02:35:08] that script just makes some api calls and then stores in the db [02:35:38] that's pretty different than multi-mega/gigabyte tables :) [02:36:09] :( [03:31:59] 10Tools: Rotate /data/project/hay/crawler.log regularly to make it easier to check for crawler issues - https://phabricator.wikimedia.org/T174266#3556403 (10bd808) [03:32:44] PROBLEM - Puppet errors on tools-exec-1437 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [04:27:44] RECOVERY - Puppet errors on tools-exec-1437 is OK: OK: Less than 1.00% above the threshold [0.0] [05:03:21] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1402 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [05:38:22] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1402 is OK: OK: Less than 1.00% above the threshold [0.0] [07:44:45] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1416 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [08:14:46] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1416 is OK: OK: Less than 1.00% above the threshold [0.0] [10:35:37] 10Tool-stewardbots, 10Patch-For-Review, 10WorkType-NewFunctionality: Let stewardbot display temporary rightchanges too - https://phabricator.wikimedia.org/T164204#3556679 (10MarcoAurelio) Tested again, looks good to me as well. I've updated the stewardbot public code. [10:36:32] 10Tool-stewardbots, 10User-MarcoAurelio: Make tools work under the public_html data - https://phabricator.wikimedia.org/T174278#3556680 (10MarcoAurelio) [10:36:44] 10Tool-stewardbots: Make tools work under the public_html data - https://phabricator.wikimedia.org/T174278#3556694 (10MarcoAurelio) [10:37:00] 10Tool-stewardbots: README.md is outdated - https://phabricator.wikimedia.org/T172871#3556695 (10MarcoAurelio) [10:37:07] 10Tool-stewardbots: New page design - https://phabricator.wikimedia.org/T172870#3556696 (10MarcoAurelio) [14:22:01] PROBLEM - Puppet errors on tools-exec-1425 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [14:57:00] RECOVERY - Puppet errors on tools-exec-1425 is OK: OK: Less than 1.00% above the threshold [0.0] [15:50:24] 10Tools: Anonymous contributions are not allowed from your IP address - https://phabricator.wikimedia.org/T174284#3556902 (10Zoranzoki21) [15:58:23] 10Tools: Anonymous contributions are not allowed from your IP address - https://phabricator.wikimedia.org/T174284#3556857 (10Reedy) You probably want to login as `Obaid-bot` via a web browser, and complete the captcha before trying pywikipedia again [15:59:54] 10Toolforge, 10Kubernetes, 10Patch-For-Review: Kubernetes man pages missing from WMF packages - https://phabricator.wikimedia.org/T170346#3556907 (10akosiaris) 05Open>03Resolved a:03akosiaris Just released a new package (1.4.6-7) that has the man pages for kubectl bundled in. man pages for the rest of... [16:16:39] 10Tool-stewardbots: Only parse data from loginwiki - https://phabricator.wikimedia.org/T174287#3556928 (10MarcoAurelio) [16:16:56] 10Tool-stewardbots: SULWatcher: only parse data from loginwiki - https://phabricator.wikimedia.org/T174287#3556941 (10MarcoAurelio) [16:17:57] 10Tool-stewardbots, 10Need-volunteer, 10User-MarcoAurelio, 10WorkType-Maintenance: SULWatcher: avoid logging account creations more than once - https://phabricator.wikimedia.org/T156546#3556943 (10MarcoAurelio) [17:53:32] (03PS1) 10BryanDavis: Guard striker.tools.utils.tools_admin for anons [labs/striker] - 10https://gerrit.wikimedia.org/r/374085 [18:27:18] PROBLEM - Puppet errors on tools-exec-1420 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [18:30:02] 10Tools: Fix obaid's tool by using pywikibot - https://phabricator.wikimedia.org/T174284#3556857 (10Framawiki) Please note that `pywikipedia` is an old version of what is now named #pywikibot-core. You can use the last version on tools by entering `python /shared/pywikipedia/core/pwb.py yourscriptname.py`. It wi... [18:30:05] 10Tools: Fix obaid's tool by using pywikibot - https://phabricator.wikimedia.org/T174284#3557037 (10Framawiki) [18:31:56] 10Tools: Fix obaid's tool by using pywikibot - https://phabricator.wikimedia.org/T174284#3557040 (10Framawiki) FYI: {T154698} [19:07:19] RECOVERY - Puppet errors on tools-exec-1420 is OK: OK: Less than 1.00% above the threshold [0.0] [21:50:45] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1416 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [21:56:49] (03PS1) 10Legoktm: Major revamp [labs/tools/extreg-wos] - 10https://gerrit.wikimedia.org/r/374095 [22:50:46] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1416 is OK: OK: Less than 1.00% above the threshold [0.0] [23:10:08] (03PS1) 10BryanDavis: Enhance workflow for tool labs membership requests [labs/striker] - 10https://gerrit.wikimedia.org/r/374104 (https://phabricator.wikimedia.org/T168027) [23:10:46] 10Striker, 10cloud-services-team (Kanban), 10Patch-For-Review, 10User-bd808: Enhance workflow for tool labs membership requests - https://phabricator.wikimedia.org/T168027#3557353 (10bd808) p:05Triage>03Normal a:03bd808 [23:11:45] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1416 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [23:13:15] 10Striker: Striker error log events not getting into ELK cluster due to UDP truncation of JSON payload - https://phabricator.wikimedia.org/T151422#3557359 (10bd808) [23:51:48] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1416 is OK: OK: Less than 1.00% above the threshold [0.0]