[00:20:14] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxWyg [00:20:15] [02miraheze/services] 07MirahezeSSLBot 03b594ed1 - BOT: Updating services config for wikis [01:06:45] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:8902::f03c:91ff:fe07:444e/cpweb [01:08:45] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [07:12:12] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+1/-0/±3] 13https://git.io/fxlJU [07:12:14] [02miraheze/MirahezeMagic] 07translatewiki 030305313 - Localisation updates from https://translatewiki.net. [07:12:15] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxlJT [07:12:17] [02miraheze/ManageWiki] 07translatewiki 03ac086bb - Localisation updates from https://translatewiki.net. [11:34:01] [02mw-config] 07Amanda-Catherine synchronize pull request 03#2514: Add parameter namespace for ecoepiwiki T3691 - 13https://git.io/fxcYL [11:37:26] [02mw-config] 07MacFan4000 closed pull request 03#2514: Add parameter namespace for ecoepiwiki T3691 - 13https://git.io/fxcYL [11:37:27] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxlBo [11:37:29] [02miraheze/mw-config] 07Amanda-Catherine 0317e5cc2 - Add parameter namespace for ecoepiwiki T3691 (#2514) * Add parameter namespace for ecoepiwiki T3691 * Update LocalSettings.php [12:48:03] [02mw-config] 07Amanda-Catherine opened pull request 03#2515: Set $wgStructuredChangeFiltersShowPreference false T3683 - 13https://git.io/fxlg8 [12:49:22] Amanda-Catherine/mw-config/patch-2/24901f4 - Amanda The build passed. https://travis-ci.com/Amanda-Catherine/mw-config/builds/87624725 [12:50:58] [02mw-config] 07Amanda-Catherine opened pull request 03#2516: Add $wgExtraInterlanguageLinkPrefixes T3683 - 13https://git.io/fxlgM [12:51:06] [02mw-config] 07paladox closed pull request 03#2515: Set $wgStructuredChangeFiltersShowPreference false T3683 - 13https://git.io/fxlg8 [12:51:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxlgy [12:51:09] [02miraheze/mw-config] 07Amanda-Catherine 035bf9768 - Set $wgStructuredChangeFiltersShowPreference false T3683 (#2515) [12:52:28] Amanda-Catherine/mw-config/patch-3/505d51e - Amanda The build passed. https://travis-ci.com/Amanda-Catherine/mw-config/builds/87625109 [12:54:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxl2T [12:54:24] [02miraheze/puppet] 07paladox 033fcb7f5 - Fix dump cron for fortnights Run every 14 days, instead if 15 [12:58:33] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxl2r [12:58:35] [02miraheze/mw-config] 07Amanda-Catherine 03eb2ce3d - Add $wgExtraInterlanguageLinkPrefixes T3683 (#2516) [12:58:38] [02mw-config] 07MacFan4000 closed pull request 03#2516: Add $wgExtraInterlanguageLinkPrefixes T3683 - 13https://git.io/fxlgM [13:15:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxlV6 [13:15:58] [02miraheze/puppet] 07paladox 031bd9f9c - Generate gz.xml for dumps [13:39:34] PROBLEM - test1 Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 8 minutes ago with 0 failures [13:43:34] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [13:59:17] PROBLEM - test1 Current Load on test1 is CRITICAL: CRITICAL - load average: 2.18, 1.44, 0.67 [14:01:15] RECOVERY - test1 Current Load on test1 is OK: OK - load average: 0.83, 1.20, 0.67 [14:14:20] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 5765 [15:14:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxl9t [15:14:29] [02miraheze/puppet] 07paladox 031325ed6 - Update dumps.pp [15:17:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fxl9E [15:17:03] [02miraheze/puppet] 07paladox 0367dda26 - Fix syntax [15:19:50] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:8902::f03c:91ff:fe07:444e/cpweb [15:21:48] RECOVERY - misc1 GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [16:26:46] PROBLEM - misc1 GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:8902::f03c:91ff:fe07:444e/cpweb [16:27:55] Phab doesn't work! [16:30:35] works for me [16:32:04] paladox: ^^ (icingabot quit) [16:32:08] hmm [16:32:13] i got emails saying ns2 is down [16:32:15] and icinga [16:32:17] and phab [16:32:21] but they all work for me [16:32:44] They work for me [16:32:47] Uh [16:32:54] Now logbot [16:33:00] that was me. [16:33:05] re log bot [16:33:08] Ok [16:33:12] i restarted both just in case [16:33:22] !log restarted icingabot and adminbot on misc1 [16:33:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:34:12] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [20:25:07] PROBLEM - db4 Disk Space on db4 is CRITICAL: DISK CRITICAL - free space: / 40292 MB (10% inode=95%); [20:33:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+1/-0/±0] 13https://git.io/fx8ZQ [20:33:10] [02miraheze/puppet] 07paladox 03264eadf - Add new dumpsBackup.sh script Will be used to run wiki dumps. [20:33:12] [02puppet] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbiAS [20:33:13] [02puppet] 07paladox opened pull request 03#878: Add new dumpsBackup.sh script - 13https://git.io/fx8Z5 [20:37:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/fx8nl [20:37:05] [02miraheze/puppet] 07paladox 03b6a3eca - Update dumps.pp [20:37:07] [02puppet] 07paladox synchronize pull request 03#878: Add new dumpsBackup.sh script - 13https://git.io/fx8Z5 [20:45:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/fx8c3 [20:45:09] [02miraheze/puppet] 07paladox 035ab5051 - Update dumpsBackup.sh [20:45:11] [02puppet] 07paladox synchronize pull request 03#878: Add new dumpsBackup.sh script - 13https://git.io/fx8Z5 [20:47:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/fx8cg [20:47:31] [02miraheze/puppet] 07paladox 0379de53a - Update dumpsBackup.sh [20:47:32] [02puppet] 07paladox synchronize pull request 03#878: Add new dumpsBackup.sh script - 13https://git.io/fx8Z5 [21:19:19] [02puppet] 07paladox closed pull request 03#878: Add new dumpsBackup.sh script - 13https://git.io/fx8Z5 [21:19:20] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+1/-0/±1] 13https://git.io/fx8lW [21:19:22] [02miraheze/puppet] 07paladox 037a9df59 - Add new dumpsBackup.sh script (#878) * Add new dumpsBackup.sh script Will be used to run wiki dumps. * Update dumps.pp * Update dumpsBackup.sh * Update dumpsBackup.sh [21:19:23] [02puppet] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbiAS [21:19:25] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-2 [21:22:03] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fx8lo [21:22:04] [02miraheze/puppet] 07paladox 036088202 - set dumpsBackup.sh to mode 0755 [21:38:08] !log rm /mnt/mediawiki-static/dumps/* on mw1 (regenerating current wiki dumps) [21:38:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [21:43:12] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fx88p [21:43:14] [02miraheze/puppet] 07paladox 03683cc40 - Remove mussmanwissenwiki from xml_dumps Wiki no longer exists [21:46:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fx84G [21:46:20] [02miraheze/puppet] 07paladox 03e98286d - nexusaeternia -> nexusaeterniawiki in dumps [22:27:38] !log PURGE BINARY LOGS BEFORE '2018-10-11 23:27:00'; on db4 [22:27:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [22:27:54] PROBLEM - db4 Disk Space on db4 is WARNING: DISK WARNING - free space: / 47593 MB (12% inode=95%);