[00:19:42] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Optimize edit count queries in XTools - https://phabricator.wikimedia.org/T163284#3402982 (10kaldari) So now the page loads quickly, but eventually I get an error under Latest global edits saying: `Error querying Global contributions API: timeout`. We should s... [00:32:37] PROBLEM - Puppet errors on tools-exec-1439 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [00:35:49] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Restrict access to users' edit stats unless opted-in - https://phabricator.wikimedia.org/T165401#3402987 (10kaldari) 05Open>03Resolved Looks like it's working now. [00:51:05] 10Tool-Labs-tools-Xtools, 10Community-Tech: Create an XTools logo - https://phabricator.wikimedia.org/T167345#3403010 (10DannyH) [00:51:18] 10Tool-Labs-tools-Xtools, 10Community-Tech: Restrict access to users' edit stats unless opted-in - https://phabricator.wikimedia.org/T165401#3403011 (10DannyH) [00:51:32] 10Tool-Labs-tools-Xtools, 10Community-Tech: Update all of the Xtools with new backend - https://phabricator.wikimedia.org/T165399#3403013 (10DannyH) [00:51:50] 10Tool-Labs-tools-Xtools, 10Community-Tech: Fix "Notice: Undefined index: allusers" in Adminstats when the wiki is unreachable - https://phabricator.wikimedia.org/T165707#3403016 (10DannyH) [00:51:53] 10Tool-Labs-tools-Xtools, 10Community-Tech: Bogus "not a valid project" errors - https://phabricator.wikimedia.org/T168676#3403017 (10DannyH) [01:07:38] RECOVERY - Puppet errors on tools-exec-1439 is OK: OK: Less than 1.00% above the threshold [0.0] [02:04:50] what's the normal setup here for ownership of PHP files in /etc/www/ for apache? [02:05:22] at the moment we've got them owned by whichever team member set up the app, which makes it hard for others to update [02:05:38] PROBLEM - Puppet staleness on tools-worker-1020 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [43200.0] [02:11:42] PROBLEM - Puppet errors on tools-exec-1405 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [02:41:21] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Optimize edit count queries in XTools - https://phabricator.wikimedia.org/T163284#3403178 (10Samwilson) The speed has improved a bit by doing the ~30 queries for global edits in one UNION'd query. Still times out for some contributors, but works for lots. Is t... [02:45:35] 10Wikibugs, 10Wikimedia-Venezuela, 10Wikimedia-VE-Campaigns (S2-2017): Introduce wikibugs in #wikimedia-ve channel - https://phabricator.wikimedia.org/T160973#3403195 (10White-Master) [03:16:42] RECOVERY - Puppet errors on tools-exec-1405 is OK: OK: Less than 1.00% above the threshold [0.0] [03:52:09] PROBLEM - Puppet errors on tools-exec-1427 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [04:17:10] RECOVERY - Puppet errors on tools-exec-1427 is OK: OK: Less than 1.00% above the threshold [0.0] [04:49:28] samwilson: www-data [05:18:23] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [05:53:22] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [06:30:55] PROBLEM - Puppet errors on tools-exec-1436 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:31:58] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1422 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:32:22] PROBLEM - Puppet errors on tools-exec-1414 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:32:56] PROBLEM - Puppet errors on tools-worker-1009 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:33:00] PROBLEM - Puppet errors on tools-flannel-etcd-03 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:33:10] PROBLEM - Puppet errors on tools-k8s-etcd-02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:34:32] PROBLEM - Puppet errors on tools-exec-1417 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [06:34:34] PROBLEM - Puppet errors on tools-checker-01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:35:11] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1411 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:36:08] PROBLEM - Puppet errors on tools-checker-02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:36:51] PROBLEM - Puppet errors on tools-worker-1012 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:36:59] PROBLEM - Puppet errors on tools-prometheus-02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:37:22] PROBLEM - Puppet errors on tools-exec-1401 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:37:22] PROBLEM - Puppet errors on tools-paws-worker-1001 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:37:30] PROBLEM - Puppet errors on tools-exec-1442 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [06:37:42] PROBLEM - Puppet errors on tools-exec-1405 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:38:08] ummm [06:38:11] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1426 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:38:11] PROBLEM - Puppet errors on tools-exec-1427 is CRITICAL: CRITICAL: 37.50% of data above the critical threshold [0.0] [06:38:13] PROBLEM - Puppet errors on tools-exec-1435 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:38:15] PROBLEM - Puppet errors on tools-exec-1438 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:38:35] PROBLEM - Puppet errors on tools-logs-02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:39:05] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1427 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:39:11] PROBLEM - Puppet errors on tools-exec-1410 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:39:23] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1419 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:39:25] PROBLEM - Puppet errors on tools-docker-registry-01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:39:49] PROBLEM - Puppet errors on tools-worker-1025 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:40:01] PROBLEM - Puppet errors on tools-services-01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:40:02] PROBLEM - Puppet errors on tools-exec-1423 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:40:08] PROBLEM - Puppet errors on tools-grid-master is CRITICAL: CRITICAL: 37.50% of data above the critical threshold [0.0] [06:40:08] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1420 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:40:10] PROBLEM - Puppet errors on tools-bastion-02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:40:12] PROBLEM - Puppet errors on tools-worker-1011 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:40:14] PROBLEM - Puppet errors on tools-grid-shadow is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:40:33] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1416 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:40:36] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1401 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:40:51] PROBLEM - Puppet errors on tools-puppetmaster-01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:40:51] PROBLEM - Puppet errors on tools-exec-1421 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:40:57] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1421 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:41:01] PROBLEM - Puppet errors on tools-exec-1418 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:41:05] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1415 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:41:13] PROBLEM - Puppet errors on tools-exec-gift-trusty-01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:41:15] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1405 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:41:55] PROBLEM - Puppet errors on tools-exec-1424 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:42:13] PROBLEM - Puppet errors on tools-k8s-etcd-01 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:42:24] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1413 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:42:44] puppet is erroring out [06:42:44] PROBLEM - Puppet errors on tools-exec-1412 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:42:50] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1428 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:43:00] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1414 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [06:43:08] PROBLEM - Puppet errors on tools-static-10 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:43:21] PROBLEM - Puppet errors on tools-paws-master-01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:43:24] PROBLEM - Puppet errors on tools-exec-1419 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:43:28] PROBLEM - Puppet errors on tools-exec-1426 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:44:09] PROBLEM - Puppet errors on tools-exec-1415 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:44:27] PROBLEM - Puppet errors on tools-bastion-03 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:44:48] PROBLEM - Puppet errors on tools-worker-1003 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:44:52] PROBLEM - Puppet errors on tools-worker-1007 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [06:44:57] PROBLEM - Puppet errors on tools-worker-1026 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:45:06] PROBLEM - Puppet errors on tools-redis-1002 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:45:12] PROBLEM - Puppet errors on tools-exec-1432 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:45:47] PROBLEM - Puppet errors on tools-exec-1416 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:45:52] PROBLEM - Puppet errors on tools-cron-01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:47:05] PROBLEM - Puppet errors on tools-elastic-01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:47:19] PROBLEM - Puppet errors on tools-exec-1402 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:47:39] PROBLEM - Puppet errors on tools-docker-builder-05 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:48:10] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1417 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:48:20] PROBLEM - Puppet errors on tools-worker-1002 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:48:32] PROBLEM - Puppet errors on tools-elastic-03 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:48:57] PROBLEM - Puppet errors on tools-exec-1429 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:49:11] PROBLEM - Puppet errors on tools-worker-1027 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:49:23] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:49:26] PROBLEM - Puppet errors on tools-exec-1433 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:49:45] PROBLEM - Puppet errors on tools-flannel-etcd-02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:50:09] PROBLEM - Puppet errors on tools-worker-1005 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:50:09] PROBLEM - Puppet errors on tools-worker-1004 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:50:25] PROBLEM - Puppet errors on tools-worker-1022 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [06:51:50] PROBLEM - Puppet errors on tools-exec-1425 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:52:21] PROBLEM - Puppet errors on tools-worker-1021 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:52:28] PROBLEM - Puppet errors on tools-proxy-01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:52:30] PROBLEM - Puppet errors on tools-k8s-master-01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:52:30] PROBLEM - Puppet errors on tools-docker-registry-02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:52:36] PROBLEM - Puppet errors on tools-exec-1413 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:52:47] PROBLEM - Puppet errors on tools-worker-1008 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:52:49] PROBLEM - Puppet errors on tools-worker-1006 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:53:01] PROBLEM - Puppet errors on tools-exec-1437 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:53:17] PROBLEM - Puppet errors on tools-webgrid-generic-1402 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:53:25] PROBLEM - Puppet errors on tools-flannel-etcd-01 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:53:39] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1424 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:53:47] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1403 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:53:51] PROBLEM - Puppet errors on tools-worker-1023 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:54:05] PROBLEM - Puppet errors on tools-worker-1001 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:54:20] PROBLEM - Puppet errors on tools-exec-1407 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:54:26] PROBLEM - Puppet errors on tools-exec-1411 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:54:46] PROBLEM - Puppet errors on tools-proxy-02 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:55:16] PROBLEM - Puppet errors on tools-worker-1018 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [06:55:36] PROBLEM - Puppet errors on tools-prometheus-01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:56:02] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1410 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:56:10] PROBLEM - Puppet errors on tools-package-builder-01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:56:53] PROBLEM - Puppet errors on tools-worker-1013 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:57:13] PROBLEM - Puppet errors on tools-exec-1431 is CRITICAL: CRITICAL: 25.00% of data above the critical threshold [0.0] [06:57:23] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1425 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:57:37] PROBLEM - Puppet errors on tools-services-02 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:57:53] PROBLEM - Puppet errors on tools-exec-1403 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [06:57:58] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1407 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:57:58] PROBLEM - Puppet errors on tools-bastion-05 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [06:57:58] PROBLEM - Puppet errors on tools-exec-1420 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [06:58:04] PROBLEM - Puppet errors on tools-webgrid-generic-1403 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:58:16] PROBLEM - Puppet errors on tools-exec-1408 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [06:58:34] PROBLEM - Puppet errors on tools-worker-1010 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:58:36] PROBLEM - Puppet errors on tools-exec-1439 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [06:58:44] PROBLEM - Puppet errors on tools-exec-1440 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [06:58:48] PROBLEM - Puppet errors on tools-exec-1422 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [06:59:10] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1418 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:59:23] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1406 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:59:29] PROBLEM - Puppet errors on tools-exec-1434 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:59:35] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1412 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [06:59:45] PROBLEM - Puppet errors on tools-elastic-02 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [06:59:53] PROBLEM - Puppet errors on tools-mail is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [06:59:55] PROBLEM - Puppet errors on tools-worker-1019 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [07:00:01] PROBLEM - Puppet errors on tools-exec-1428 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [07:00:35] PROBLEM - Puppet errors on tools-exec-1406 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:01:07] PROBLEM - Puppet errors on tools-exec-1409 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:01:14] PROBLEM - Puppet errors on tools-worker-1014 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [07:01:56] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1408 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:01:56] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1408 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:02:33] PROBLEM - Puppet errors on tools-worker-1015 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [07:02:33] PROBLEM - Puppet errors on tools-worker-1015 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [07:02:33] PROBLEM - Puppet errors on tools-worker-1016 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:02:33] PROBLEM - Puppet errors on tools-worker-1016 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:02:41] is fixed - recoveries should come in soon [07:02:45] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1402 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:02:45] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1402 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:03:09] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1409 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [07:03:10] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1409 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [07:03:23] PROBLEM - Puppet errors on tools-worker-1017 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:03:23] PROBLEM - Puppet errors on tools-worker-1017 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [07:03:29] PROBLEM - Puppet errors on tools-webgrid-generic-1404 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:03:29] PROBLEM - Puppet errors on tools-webgrid-generic-1404 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:03:36] PROBLEM - Puppet errors on tools-exec-1404 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:03:36] PROBLEM - Puppet errors on tools-exec-1404 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:03:42] PROBLEM - Puppet errors on tools-redis-1001 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:03:42] PROBLEM - Puppet errors on tools-redis-1001 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:03:44] PROBLEM - Puppet errors on tools-exec-1441 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:03:44] PROBLEM - Puppet errors on tools-exec-1441 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:04:10] PROBLEM - Puppet errors on tools-static-11 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:04:10] PROBLEM - Puppet errors on tools-static-11 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [07:11:03] RECOVERY - Puppet errors on tools-checker-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:11:03] RECOVERY - Puppet errors on tools-checker-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:11:55] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1422 is OK: OK: Less than 1.00% above the threshold [0.0] [07:12:28] RECOVERY - Puppet errors on tools-exec-1414 is OK: OK: Less than 1.00% above the threshold [0.0] [07:12:54] RECOVERY - Puppet errors on tools-worker-1009 is OK: OK: Less than 1.00% above the threshold [0.0] [07:12:58] RECOVERY - Puppet errors on tools-flannel-etcd-03 is OK: OK: Less than 1.00% above the threshold [0.0] [07:13:09] RECOVERY - Puppet errors on tools-k8s-etcd-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:13:09] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1426 is OK: OK: Less than 1.00% above the threshold [0.0] [07:14:23] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1419 is OK: OK: Less than 1.00% above the threshold [0.0] [07:14:26] RECOVERY - Puppet errors on tools-docker-registry-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:14:34] RECOVERY - Puppet errors on tools-checker-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:14:50] RECOVERY - Puppet errors on tools-worker-1025 is OK: OK: Less than 1.00% above the threshold [0.0] [07:15:08] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1411 is OK: OK: Less than 1.00% above the threshold [0.0] [07:15:08] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1420 is OK: OK: Less than 1.00% above the threshold [0.0] [07:15:12] RECOVERY - Puppet errors on tools-worker-1011 is OK: OK: Less than 1.00% above the threshold [0.0] [07:15:18] RECOVERY - Puppet errors on tools-grid-shadow is OK: OK: Less than 1.00% above the threshold [0.0] [07:15:28] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1416 is OK: OK: Less than 1.00% above the threshold [0.0] [07:15:57] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1421 is OK: OK: Less than 1.00% above the threshold [0.0] [07:16:07] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1415 is OK: OK: Less than 1.00% above the threshold [0.0] [07:16:51] RECOVERY - Puppet errors on tools-worker-1012 is OK: OK: Less than 1.00% above the threshold [0.0] [07:16:55] RECOVERY - Puppet errors on tools-exec-1424 is OK: OK: Less than 1.00% above the threshold [0.0] [07:16:59] RECOVERY - Puppet errors on tools-prometheus-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:17:23] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1413 is OK: OK: Less than 1.00% above the threshold [0.0] [07:17:23] RECOVERY - Puppet errors on tools-paws-worker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [07:17:25] RECOVERY - Puppet errors on tools-exec-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [07:17:29] RECOVERY - Puppet errors on tools-exec-1442 is OK: OK: Less than 1.00% above the threshold [0.0] [07:17:41] RECOVERY - Puppet errors on tools-exec-1405 is OK: OK: Less than 1.00% above the threshold [0.0] [07:18:10] RECOVERY - Puppet errors on tools-exec-1435 is OK: OK: Less than 1.00% above the threshold [0.0] [07:18:12] RECOVERY - Puppet errors on tools-exec-1427 is OK: OK: Less than 1.00% above the threshold [0.0] [07:18:14] RECOVERY - Puppet errors on tools-exec-1438 is OK: OK: Less than 1.00% above the threshold [0.0] [07:18:22] RECOVERY - Puppet errors on tools-paws-master-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:18:38] RECOVERY - Puppet errors on tools-logs-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:19:04] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1427 is OK: OK: Less than 1.00% above the threshold [0.0] [07:19:10] RECOVERY - Puppet errors on tools-exec-1410 is OK: OK: Less than 1.00% above the threshold [0.0] [07:19:57] RECOVERY - Puppet errors on tools-worker-1026 is OK: OK: Less than 1.00% above the threshold [0.0] [07:19:59] RECOVERY - Puppet errors on tools-services-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:00] RECOVERY - Puppet errors on tools-exec-1423 is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:06] RECOVERY - Puppet errors on tools-grid-master is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:11] RECOVERY - Puppet errors on tools-bastion-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:13] RECOVERY - Puppet errors on tools-exec-1432 is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:37] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:49] RECOVERY - Puppet errors on tools-puppetmaster-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:20:51] RECOVERY - Puppet errors on tools-exec-1421 is OK: OK: Less than 1.00% above the threshold [0.0] [07:21:01] RECOVERY - Puppet errors on tools-exec-1418 is OK: OK: Less than 1.00% above the threshold [0.0] [07:21:13] RECOVERY - Puppet errors on tools-exec-gift-trusty-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:21:16] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1405 is OK: OK: Less than 1.00% above the threshold [0.0] [07:22:14] RECOVERY - Puppet errors on tools-k8s-etcd-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:22:40] RECOVERY - Puppet errors on tools-docker-builder-05 is OK: OK: Less than 1.00% above the threshold [0.0] [07:22:44] RECOVERY - Puppet errors on tools-exec-1412 is OK: OK: Less than 1.00% above the threshold [0.0] [07:22:50] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1428 is OK: OK: Less than 1.00% above the threshold [0.0] [07:23:05] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1414 is OK: OK: Less than 1.00% above the threshold [0.0] [07:23:06] RECOVERY - Puppet errors on tools-static-10 is OK: OK: Less than 1.00% above the threshold [0.0] [07:23:24] RECOVERY - Puppet errors on tools-exec-1419 is OK: OK: Less than 1.00% above the threshold [0.0] [07:23:28] RECOVERY - Puppet errors on tools-exec-1426 is OK: OK: Less than 1.00% above the threshold [0.0] [07:23:42] RECOVERY - Puppet errors on tools-redis-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [07:24:08] RECOVERY - Puppet errors on tools-exec-1415 is OK: OK: Less than 1.00% above the threshold [0.0] [07:24:10] RECOVERY - Puppet errors on tools-worker-1027 is OK: OK: Less than 1.00% above the threshold [0.0] [07:24:22] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [07:24:25] RECOVERY - Puppet errors on tools-bastion-03 is OK: OK: Less than 1.00% above the threshold [0.0] [07:24:47] RECOVERY - Puppet errors on tools-worker-1003 is OK: OK: Less than 1.00% above the threshold [0.0] [07:24:53] RECOVERY - Puppet errors on tools-worker-1007 is OK: OK: Less than 1.00% above the threshold [0.0] [07:25:07] RECOVERY - Puppet errors on tools-redis-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [07:25:11] RECOVERY - Puppet errors on tools-worker-1005 is OK: OK: Less than 1.00% above the threshold [0.0] [07:25:25] RECOVERY - Puppet errors on tools-worker-1022 is OK: OK: Less than 1.00% above the threshold [0.0] [07:25:45] RECOVERY - Puppet errors on tools-exec-1416 is OK: OK: Less than 1.00% above the threshold [0.0] [07:25:53] RECOVERY - Puppet errors on tools-cron-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:26:49] RECOVERY - Puppet errors on tools-exec-1425 is OK: OK: Less than 1.00% above the threshold [0.0] [07:27:05] RECOVERY - Puppet errors on tools-elastic-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:27:16] RECOVERY - Puppet errors on tools-exec-1402 is OK: OK: Less than 1.00% above the threshold [0.0] [07:27:30] RECOVERY - Puppet errors on tools-proxy-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:27:32] RECOVERY - Puppet errors on tools-k8s-master-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:27:36] RECOVERY - Puppet errors on tools-docker-registry-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:27:44] RECOVERY - Puppet errors on tools-worker-1008 is OK: OK: Less than 1.00% above the threshold [0.0] [07:28:00] RECOVERY - Puppet errors on tools-exec-1437 is OK: OK: Less than 1.00% above the threshold [0.0] [07:28:08] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1417 is OK: OK: Less than 1.00% above the threshold [0.0] [07:28:22] RECOVERY - Puppet errors on tools-worker-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [07:28:35] RECOVERY - Puppet errors on tools-elastic-03 is OK: OK: Less than 1.00% above the threshold [0.0] [07:28:54] RECOVERY - Puppet errors on tools-exec-1429 is OK: OK: Less than 1.00% above the threshold [0.0] [07:29:06] RECOVERY - Puppet errors on tools-worker-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [07:29:26] RECOVERY - Puppet errors on tools-exec-1433 is OK: OK: Less than 1.00% above the threshold [0.0] [07:29:44] RECOVERY - Puppet errors on tools-flannel-etcd-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:30:10] RECOVERY - Puppet errors on tools-worker-1004 is OK: OK: Less than 1.00% above the threshold [0.0] [07:30:36] RECOVERY - Puppet errors on tools-prometheus-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:30:56] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1404 is OK: OK: Less than 1.00% above the threshold [0.0] [07:31:02] RECOVERY - Puppet errors on tools-exec-1430 is OK: OK: Less than 1.00% above the threshold [0.0] [07:31:02] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1410 is OK: OK: Less than 1.00% above the threshold [0.0] [07:32:19] RECOVERY - Puppet errors on tools-worker-1021 is OK: OK: Less than 1.00% above the threshold [0.0] [07:32:24] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1425 is OK: OK: Less than 1.00% above the threshold [0.0] [07:32:41] RECOVERY - Puppet errors on tools-exec-1413 is OK: OK: Less than 1.00% above the threshold [0.0] [07:32:50] RECOVERY - Puppet errors on tools-worker-1006 is OK: OK: Less than 1.00% above the threshold [0.0] [07:33:18] RECOVERY - Puppet errors on tools-webgrid-generic-1402 is OK: OK: Less than 1.00% above the threshold [0.0] [07:33:26] RECOVERY - Puppet errors on tools-flannel-etcd-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:33:40] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1424 is OK: OK: Less than 1.00% above the threshold [0.0] [07:33:46] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1403 is OK: OK: Less than 1.00% above the threshold [0.0] [07:33:50] RECOVERY - Puppet errors on tools-worker-1023 is OK: OK: Less than 1.00% above the threshold [0.0] [07:34:18] RECOVERY - Puppet errors on tools-exec-1407 is OK: OK: Less than 1.00% above the threshold [0.0] [07:34:27] RECOVERY - Puppet errors on tools-exec-1411 is OK: OK: Less than 1.00% above the threshold [0.0] [07:34:45] RECOVERY - Puppet errors on tools-proxy-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:34:45] RECOVERY - Puppet errors on tools-elastic-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:34:53] RECOVERY - Puppet errors on tools-mail is OK: OK: Less than 1.00% above the threshold [0.0] [07:35:15] RECOVERY - Puppet errors on tools-worker-1018 is OK: OK: Less than 1.00% above the threshold [0.0] [07:36:07] RECOVERY - Puppet errors on tools-exec-1409 is OK: OK: Less than 1.00% above the threshold [0.0] [07:36:13] RECOVERY - Puppet errors on tools-package-builder-01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:36:55] RECOVERY - Puppet errors on tools-worker-1013 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:09] RECOVERY - Puppet errors on tools-exec-1431 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:31] RECOVERY - Puppet errors on tools-worker-1016 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:36] RECOVERY - Puppet errors on tools-services-02 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:46] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1402 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:54] RECOVERY - Puppet errors on tools-exec-1403 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:56] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1407 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:56] RECOVERY - Puppet errors on tools-exec-1420 is OK: OK: Less than 1.00% above the threshold [0.0] [07:37:58] RECOVERY - Puppet errors on tools-bastion-05 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:06] RECOVERY - Puppet errors on tools-webgrid-generic-1403 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:14] RECOVERY - Puppet errors on tools-exec-1408 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:22] RECOVERY - Puppet errors on tools-worker-1017 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:28] RECOVERY - Puppet errors on tools-webgrid-generic-1404 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:34] RECOVERY - Puppet errors on tools-worker-1010 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:36] RECOVERY - Puppet errors on tools-exec-1439 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:36] RECOVERY - Puppet errors on tools-exec-1404 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:44] RECOVERY - Puppet errors on tools-exec-1440 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:47] RECOVERY - Puppet errors on tools-exec-1441 is OK: OK: Less than 1.00% above the threshold [0.0] [07:38:49] RECOVERY - Puppet errors on tools-exec-1422 is OK: OK: Less than 1.00% above the threshold [0.0] [07:39:09] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1418 is OK: OK: Less than 1.00% above the threshold [0.0] [07:39:13] RECOVERY - Puppet errors on tools-static-11 is OK: OK: Less than 1.00% above the threshold [0.0] [07:39:23] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1406 is OK: OK: Less than 1.00% above the threshold [0.0] [07:39:29] RECOVERY - Puppet errors on tools-exec-1434 is OK: OK: Less than 1.00% above the threshold [0.0] [07:39:33] RECOVERY - Puppet errors on tools-exec-1417 is OK: OK: Less than 1.00% above the threshold [0.0] [07:39:33] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1412 is OK: OK: Less than 1.00% above the threshold [0.0] [07:39:53] RECOVERY - Puppet errors on tools-worker-1019 is OK: OK: Less than 1.00% above the threshold [0.0] [07:40:01] RECOVERY - Puppet errors on tools-exec-1428 is OK: OK: Less than 1.00% above the threshold [0.0] [07:40:38] RECOVERY - Puppet errors on tools-exec-1406 is OK: OK: Less than 1.00% above the threshold [0.0] [07:40:55] RECOVERY - Puppet errors on tools-exec-1436 is OK: OK: Less than 1.00% above the threshold [0.0] [07:41:17] RECOVERY - Puppet errors on tools-worker-1014 is OK: OK: Less than 1.00% above the threshold [0.0] [07:41:57] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1408 is OK: OK: Less than 1.00% above the threshold [0.0] [07:42:31] RECOVERY - Puppet errors on tools-worker-1015 is OK: OK: Less than 1.00% above the threshold [0.0] [07:43:07] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1409 is OK: OK: Less than 1.00% above the threshold [0.0] [08:10:07] 10Tool-Labs-tools-Xtools, 10Community-Tech-Sprint: Optimize edit count queries in XTools - https://phabricator.wikimedia.org/T163284#3403405 (10kaldari) Hmm, it still seems slower than the old XTool's editcounter (even without loading global edits), but I can't prove that due to query caching). http://xtools.... [08:43:11] Anyone know how to run arbitrary commands non-interactively on the kubernetes webservice? I want to update my python venv without having to run `webservice --backend=kubernetes python shell` each time. I'd love to be able to do something like `webservice --backend=kubernetes python exec somescript.sh` [10:53:33] (03CR) 10Addshore: [C: 032] Add MediaWiki-extensions-Wikibase(.*) to #wikidata-feed [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362374 (owner: 10Addshore) [10:53:34] (03CR) 10Addshore: [C: 032] Send all WMDE-.* phab projects to #wikimedia-de-tech [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362373 (owner: 10Addshore) [10:53:36] (03CR) 10Addshore: [C: 032] Add User-Addshore to ##add [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362962 (owner: 10Addshore) [10:53:54] (03Merged) 10jenkins-bot: Send all WMDE-.* phab projects to #wikimedia-de-tech [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362373 (owner: 10Addshore) [10:53:56] (03Merged) 10jenkins-bot: Add MediaWiki-extensions-Wikibase(.*) to #wikidata-feed [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362374 (owner: 10Addshore) [10:53:58] (03Merged) 10jenkins-bot: Add User-Addshore to ##add [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362962 (owner: 10Addshore) [10:54:03] (03CR) 10jenkins-bot: Send all WMDE-.* phab projects to #wikimedia-de-tech [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362373 (owner: 10Addshore) [10:54:05] (03CR) 10jenkins-bot: Add MediaWiki-extensions-Wikibase(.*) to #wikidata-feed [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362374 (owner: 10Addshore) [10:54:07] (03CR) 10jenkins-bot: Add User-Addshore to ##add [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/362962 (owner: 10Addshore) [10:54:09] (03CR) 10Addshore: [C: 032] change old project names for projects maintained by Zppix and add an additional project [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/359497 (owner: 10Zppix) [10:54:25] (03Merged) 10jenkins-bot: change old project names for projects maintained by Zppix and add an additional project [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/359497 (owner: 10Zppix) [10:54:32] (03CR) 10jenkins-bot: change old project names for projects maintained by Zppix and add an additional project [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/359497 (owner: 10Zppix) [10:56:12] !log tools.wikibugs Updated channels.yaml to: 9e0b6db4e6bdc63808ca8847824b52dff007f9c0 Merge "change old project names for projects maintained by Zppix and add an additional project" [13:15:20] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [14:25:20] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [17:37:46] tarrow: Hi! I don't think the webservice module has that facility now [17:38:26] seems useful though, feel free to open a bug on phab [17:59:07] 10Quarry: Provide a way to hyperlink texts - https://phabricator.wikimedia.org/T74874#771075 (10Framawiki) [[ https://phabricator.wikimedia.org/T74874 | Asked by a user here ]] [20:37:26] *waves* [20:37:46] Where can I find the docs for using the wikimedia docker image repo place thingy? [20:37:49] [= [21:09:29] addshore: hey! I don't know much about it, but what do you mean by use :) [21:09:52] there's some stuff that mentions the registry here - https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Kubernetes [21:10:08] and the image repo is here - https://phabricator.wikimedia.org/diffusion/ODIT/repository/master/ [21:10:24] These images are built using the Dockerfiles in the operations/docker-images/toollabs-images git repository. [21:10:31] ace, thats probably what I was looking for [21:10:43] or something similar [21:10:44] hmm [21:11:57] I wonder if that would be an appropriate place to publish an image including php7 & php phan for use while developing. or not [21:12:47] at a guess I would say yes, hehe [21:12:59] publish for use where? [21:13:19] that docker repo is for toolforge's keuberentes [21:13:30] *kubernetes [21:14:01] I don't know where they are setting up the prod docker repo [21:14:44] bd808: ahh okay, and only for toolforge's keuberentes? I guess thats one of my main questions [21:15:04] yes, only for toolforge. not a general purpose service [21:15:09] ack! [21:15:51] And it seems there is another one for CI ci-staging-docker-registry01.wmflabs.org [21:17:17] I think the CI k8s cluster is mostly copied from the Toolforge one. The local repo is a security thing. The k8s workers are only allowed to load docker images from the paired registry [21:17:39] ooh, and found a production repo of images https://github.com/wikimedia/operations-docker-images-production-images [21:17:43] mostly copied meaning setup in a similar way [22:15:52] PROBLEM - Puppet errors on tools-worker-1007 is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [22:45:04] 10Quarry: Provide a way to hyperlink texts - https://phabricator.wikimedia.org/T74874#3405668 (10MZMcBride) >>! In T74874#3405277, @Framawiki wrote: > [[ https://phabricator.wikimedia.org/T74874 | Asked by a user here ]] Wrong link? [22:50:54] RECOVERY - Puppet errors on tools-worker-1007 is OK: OK: Less than 1.00% above the threshold [0.0] [23:09:43] (03PS1) 10BryanDavis: Wikimedia-cloud: Adjust for renamed tags [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/363287 (https://phabricator.wikimedia.org/T167244) [23:13:01] bd808: so I want to read the apache2 error log on my vps instance... so I tried CDing into the log directory... permission denied... and you can't do "sudo cd"... and I can't su into the root account because this server has a passwordless root account [23:13:03] What do I do now. [23:13:30] harej: `sudo -s` or `sudo su` [23:13:43] well there you go [23:13:45] thanks! [23:13:49] yw [23:13:51] happy meat and explosions day to you [23:14:03] mmmmm meat explosion [23:15:11] alcoholic meat explosion