[00:03:08] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:03:28] PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:03:46] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:11:28] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [00:11:46] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [00:13:08] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:16:19] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2a00:d880:5:8ea::ebc7/cpweb [00:16:35] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2604:180:0:33b::2/cpweb [00:18:21] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:18:37] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [04:51:27] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 5874 [05:24:13] RECOVERY - bacula1 Bacula Static Lizardfs1 on bacula1 is OK: OK: Diff, 148627 files, 15.59GB, 2018-10-21 05:22:00 (2.2 minutes ago) [06:05:52] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[php7.2-gettext] [06:07:32] RECOVERY - bacula1 Bacula Static Lizardfs2 on bacula1 is OK: OK: Diff, 113295 files, 9.126GB, 2018-10-21 06:06:00 (1.5 minutes ago) [06:09:38] RECOVERY - bacula1 Bacula Phabricator Static on bacula1 is OK: OK: Diff, 7587 files, 48.46MB, 2018-10-21 06:08:00 (1.6 minutes ago) [06:13:54] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:59:27] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [08:55:44] !m [08:55:47] !m me [08:55:47] <[d__d]> You're doing good work, me! [14:03:50] PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:11:42] RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [14:14:36] PROBLEM - bacula1 Bacula Static Lizardfs2 on bacula1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:15:18] PROBLEM - bacula1 Bacula Misc4 Lizardfs Master on bacula1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:16:37] RECOVERY - bacula1 Bacula Static Lizardfs2 on bacula1 is OK: OK: Diff, 113295 files, 9.126GB, 2018-10-21 06:06:00 (8.2 hours ago) [14:16:55] PROBLEM - bacula1 Bacula Daemon on bacula1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:18:49] RECOVERY - bacula1 Bacula Daemon on bacula1 is OK: PROCS OK: 2 processes with UID = 110 (bacula) [14:20:17] RECOVERY - bacula1 Bacula Misc4 Lizardfs Master on bacula1 is OK: OK: Full, 16 files, 150.1MB, 2018-10-21 04:09:00 (10.2 hours ago) [16:09:26] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 5916 [16:52:09] !log PURGE BINARY LOGS BEFORE '2018-10-21 17:51:00'; on db4 [16:52:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:13:36] [02miraheze/mediawiki] 07paladox pushed 0315 commits to 03REL1_32 [+0/-0/±39] 13https://git.io/fxK1a [18:13:38] [02miraheze/mediawiki] 07AaronSchulz 0313eb2c7 - rdbms: re-add DB domain sanity checks to LoadBalancer Also clean up empty schema handling in DatabaseDomain This reverts commit f23ac02f4fcf156767df66a5df2fa407310fe1d2. Bug: T193565 Change-Id: I95fde5c069f180ca888a023fade25ec81b846d44 [18:13:39] [02miraheze/mediawiki] 07Brad Jorsch 039d957e2 - Add COALESCE for image deletion and imgcomment_description_id I have no idea why this suddenly started raising an error rather than it doing so since February (I0dd7258fe). But this should fix it. Bug: T207419 Change-Id: Id97e1c7c2655d90928c777bc3377e5ea23f49f6b (cherry picked from commit b53f0c278f6cb84c3cbbcb3100231c8c7e2fd870) [18:13:41] [02miraheze/mediawiki] 07legoktm 030ddc252 - Include IP address in "Login for $1 succeeded" log entry Bug: T207540 Change-Id: Iab4f2f2ddc8e64ead2f33356d03fa7beed399415 [18:13:42] [02miraheze/mediawiki] ... and 12 more commits. [18:17:28] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [18:25:50] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_32 [+0/-0/±1] 13https://git.io/fxKMk [18:25:51] [02miraheze/mediawiki] 07paladox 03cb0ce9b - Update MW [18:50:13] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxKMN [18:50:15] [02miraheze/services] 07MirahezeSSLBot 03b5e9fa6 - BOT: Updating services config for wikis [18:58:39] .weather [18:58:40] Yahoo! Weather - Portland, ME, US: Breezy, 7°C (44°F), Humidity: 50%, Near gale 14.8m/s (↘) [19:14:59] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fxKyU [19:15:00] [02miraheze/mediawiki] 07paladox 03ce7e83a - Update TemplateData [19:18:29] [02mw-config] 07Amanda-Catherine opened pull request 03#2533: Add storefront namespace to crazybloxianempireinfowiki T3700 - 13https://git.io/fxKy3 [19:20:29] Amanda-Catherine/mw-config/patch-1/a29a3f8 - Amanda The build passed. https://travis-ci.com/Amanda-Catherine/mw-config/builds/88666443 [19:20:32] [02mw-config] 07paladox closed pull request 03#2533: Add storefront namespace to crazybloxianempireinfowiki T3700 - 13https://git.io/fxKy3 [19:20:33] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxKy8 [19:20:35] [02miraheze/mw-config] 07Amanda-Catherine 034512ce9 - Add storefront namespace to crazybloxianempireinfowiki T3700 (#2533) [21:46:43] [02miraheze/mediawiki] 07paladox pushed 0328 commits to 03REL1_31 [+0/-0/±80] 13https://git.io/fxK5p [21:46:44] [02miraheze/mediawiki] 07Brad Jorsch 031f664ea - WatchedItemStore::countVisitingWatchersMultiple() shouldn't query all titles when asked for none If a caller gives an empty array for $targetsWithVisitThresholds, per the documentation it should be expecting an empty array in return, not a count of watchers for every title in the database. Bug: T204729 Change-Id: I0f25fae301450d077bb30597281aaef0fba209d4 (cherry [21:46:44] picked from commit f5469d36602cb2a95396830b14e9a631d698f3a6) [21:46:46] [02miraheze/mediawiki] 07legoktm 038641df9 - Document removal of CologneBlue & Modern from the tarball Worth noting as a pre-upgrade thing, as trying to run the updater will fail unless it's disabled or downloaded separately. Reported by . Change-Id: I5ea6dc65d683dfceaa25104ca59f2f56c1ab1884 [21:46:47] [02miraheze/mediawiki] 07Brad Jorsch 03590e94d - Fix typo in postgres patch-drop-ar_text.sql I18f1c740 was cherry-picked to REL1_31 but it was broken. Before that got merged to REL1_31, this was already in master, but apparently that was missed when cherry-picking. This should have made 1.31-rc.2. Bug: T205967 Change-Id: I6d479d123848325b8501275ef4b98fc81cd99505 (cherry picked from commit [21:46:47] 7e685a395d4a6e69d904c03f6e847b18795217a9) [21:46:49] [02miraheze/mediawiki] ... and 25 more commits. [21:46:51] Title: [ #909589 - mediawiki: /usr/share/mediawiki/skins/CologneBlue/skin.json, does not exist! [upgrading to 1:1.31.1-2] - Debian Bug report logs ] - bugs.debian.org [21:48:56] !log applying 2 security fixes (https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/468846/ and https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/468849/) [21:48:57] Title: [ Gerrit Code Review ] - gerrit.wikimedia.org [21:48:58] Title: [ Gerrit Code Review ] - gerrit.wikimedia.org [21:49:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:39:03] [02ManageWiki] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vpSns [22:39:04] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fxKbv [22:39:06] [02miraheze/ManageWiki] 07paladox 03c0ad13c - Fix Undefined index [22:39:07] [02ManageWiki] 07paladox opened pull request 03#35: Fix Undefined index - 13https://git.io/fxKbf [22:45:28] [02ManageWiki] 07paladox synchronize pull request 03#35: Fix Undefined index - 13https://git.io/fxKbf [22:45:30] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/fxKbG [22:45:31] [02miraheze/ManageWiki] 07paladox 0393edae3 - Update ManageWikiFormFactory.php