[03:25:16] PROBLEM - MariaDB Slave Lag: s1 on dbstore1002 is CRITICAL: CRITICAL slave_sql_lag Replication lag: 884.92 seconds [03:41:16] RECOVERY - MariaDB Slave Lag: s1 on dbstore1002 is OK: OK slave_sql_lag Replication lag: 267.98 seconds [05:15:55] 10Operations, 10MediaWiki-Vagrant, 10Patch-For-Review: Import kibana package from jessie into stretch - https://phabricator.wikimedia.org/T183071#3860329 (10bd808) 05Open>03Resolved a:03bd808 [06:49:46] PROBLEM - Check Varnish expiry mailbox lag on cp4026 is CRITICAL: CRITICAL: expiry mailbox lag is 2037681 [07:39:46] RECOVERY - Check Varnish expiry mailbox lag on cp4026 is OK: OK: expiry mailbox lag is 0 [10:48:16] PROBLEM - Check Varnish expiry mailbox lag on cp4021 is CRITICAL: CRITICAL: expiry mailbox lag is 2016089 [11:28:16] PROBLEM - Check Varnish expiry mailbox lag on cp4021 is CRITICAL: CRITICAL: expiry mailbox lag is 2085904 [19:00:36] PROBLEM - puppet last run on db1082 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:01:46] PROBLEM - puppet last run on cp1062 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:02:06] PROBLEM - puppet last run on labvirt1015 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:02:06] PROBLEM - puppet last run on ms-be1034 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:02:27] PROBLEM - puppet last run on restbase1011 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:03:06] PROBLEM - puppet last run on aqs1006 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:03:06] PROBLEM - puppet last run on mw1183 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:03:27] PROBLEM - puppet last run on ms-be1019 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:03:56] PROBLEM - puppet last run on db1073 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:04:16] PROBLEM - puppet last run on silver is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:04:36] PROBLEM - puppet last run on elastic1048 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:04:37] PROBLEM - puppet last run on db1097 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:05:16] PROBLEM - puppet last run on elastic1018 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [19:28:57] RECOVERY - puppet last run on db1073 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [19:29:36] RECOVERY - puppet last run on elastic1048 is OK: OK: Puppet is currently enabled, last run 56 seconds ago with 0 failures [19:29:37] RECOVERY - puppet last run on db1097 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [19:30:16] RECOVERY - puppet last run on elastic1018 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [19:30:36] RECOVERY - puppet last run on db1082 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:31:46] RECOVERY - puppet last run on cp1062 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:32:06] RECOVERY - puppet last run on labvirt1015 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:32:06] RECOVERY - puppet last run on ms-be1034 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:32:27] RECOVERY - puppet last run on restbase1011 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [19:33:06] RECOVERY - puppet last run on aqs1006 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [19:33:06] RECOVERY - puppet last run on mw1183 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [19:33:26] RECOVERY - puppet last run on ms-be1019 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures [19:34:16] RECOVERY - puppet last run on silver is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures [23:48:16] RECOVERY - Check Varnish expiry mailbox lag on cp4021 is OK: OK: expiry mailbox lag is 5