[02:41:27] <icinga-miraheze>	 PROBLEM - bacula1 Bacula Phabricator Static on bacula1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[02:43:25] <icinga-miraheze>	 PROBLEM - bacula1 Bacula Phabricator Static on bacula1 is WARNING: WARNING: Full, 79592 files, 1.998GB, 2019-06-16 02:18:00 (3.4 weeks ago)
[11:53:43] <icinga-miraheze>	 PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 14 failures. Last run 2 minutes ago with 14 failures. Failed resources (up to 3 shown)
[11:54:00] <icinga-miraheze>	 PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 52 failures. Last run 2 minutes ago with 52 failures. Failed resources (up to 3 shown)
[11:54:05] <icinga-miraheze>	 PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 203 failures. Last run 2 minutes ago with 203 failures. Failed resources (up to 3 shown)
[11:54:06] <icinga-miraheze>	 PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[11:54:17] <icinga-miraheze>	 PROBLEM - elasticsearch1 Puppet on elasticsearch1 is CRITICAL: CRITICAL: Puppet has 19 failures. Last run 2 minutes ago with 19 failures. Failed resources (up to 3 shown)
[11:54:18] <icinga-miraheze>	 PROBLEM - lizardfs1 Puppet on lizardfs1 is CRITICAL: CRITICAL: Puppet has 13 failures. Last run 2 minutes ago with 13 failures. Failed resources (up to 3 shown)
[11:54:19] <icinga-miraheze>	 PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Puppet has 16 failures. Last run 2 minutes ago with 16 failures. Failed resources (up to 3 shown)
[11:54:30] <icinga-miraheze>	 PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 202 failures. Last run 2 minutes ago with 202 failures. Failed resources (up to 3 shown)
[11:54:37] <icinga-miraheze>	 PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[11:54:41] <icinga-miraheze>	 PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 214 failures. Last run 2 minutes ago with 214 failures. Failed resources (up to 3 shown)
[11:54:58] <icinga-miraheze>	 PROBLEM - lizardfs3 Puppet on lizardfs3 is CRITICAL: CRITICAL: Puppet has 14 failures. Last run 3 minutes ago with 14 failures. Failed resources (up to 3 shown)
[11:55:06] <icinga-miraheze>	 PROBLEM - lizardfs2 Puppet on lizardfs2 is CRITICAL: CRITICAL: Puppet has 13 failures. Last run 3 minutes ago with 13 failures. Failed resources (up to 3 shown)
[11:55:07] <icinga-miraheze>	 PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[11:55:08] <icinga-miraheze>	 PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[11:55:13] <icinga-miraheze>	 PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[11:55:15] <icinga-miraheze>	 PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 211 failures. Last run 3 minutes ago with 211 failures. Failed resources (up to 3 shown)
[11:55:17] <icinga-miraheze>	 PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 217 failures. Last run 3 minutes ago with 217 failures. Failed resources (up to 3 shown)
[11:55:31] <icinga-miraheze>	 PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 202 failures. Last run 3 minutes ago with 202 failures. Failed resources (up to 3 shown)
[11:59:02] <icinga-miraheze>	 PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 208 failures. Last run 3 minutes ago with 208 failures. Failed resources (up to 3 shown): File[/etc/rsyslog.conf],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades],File[/etc/apt/apt.conf.d/20auto-upgrades]
[12:04:07] <icinga-miraheze>	 RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 22 seconds ago with 0 failures
[12:04:18] <icinga-miraheze>	 RECOVERY - lizardfs1 Puppet on lizardfs1 is OK: OK: Puppet is currently enabled, last run 31 seconds ago with 0 failures
[12:04:19] <icinga-miraheze>	 RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures
[12:04:37] <icinga-miraheze>	 RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures
[12:04:43] <icinga-miraheze>	 RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures
[12:04:58] <icinga-miraheze>	 RECOVERY - lizardfs3 Puppet on lizardfs3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[12:05:06] <icinga-miraheze>	 RECOVERY - lizardfs2 Puppet on lizardfs2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[12:05:07] <icinga-miraheze>	 RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[12:05:08] <icinga-miraheze>	 RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[12:05:09] <icinga-miraheze>	 RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures
[12:05:13] <icinga-miraheze>	 RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[12:05:19] <icinga-miraheze>	 RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures
[12:05:30] <icinga-miraheze>	 RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures
[12:05:42] <icinga-miraheze>	 RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[12:05:57] <icinga-miraheze>	 RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[12:06:04] <icinga-miraheze>	 RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[12:06:35] <icinga-miraheze>	 RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 46 seconds ago with 0 failures
[12:09:00] <icinga-miraheze>	 RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures
[12:10:12] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjPOy
[12:10:14] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot 0381f7917 - BOT: Updating services config for wikis
[12:24:17] <icinga-miraheze>	 RECOVERY - elasticsearch1 Puppet on elasticsearch1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[13:10:22] <Vees>	 Hi all, I’m doing a climate action project and would like to set the wiki to be edited by members only. How can I create log in details for the members?
[13:10:44] <JohnLewis>	 Vees: you can create an account via Special:CreateAccount on your wiki
[13:11:23] <Vees>	 JohnLewis, thank you!
[13:15:29] <Vees>	 I have already an account. I only want to create login for members to whom I assign the rights for the edit. As in: how can they login so that they can edit the wiki?
[13:17:50] <JohnLewis>	 you need to create them accounts
[13:22:19] <Vees>	 I see. Thank you again
[13:23:13] <Vees>	 I aims to have thousands members, so I’m afraid it’s too much work to create them individual accounts
[18:50:13] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjPlL
[18:50:14] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot 03ae67e5a - BOT: Updating services config for wikis
[20:35:10] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjP8a
[20:35:12] <Not-7290>	 [02miraheze/services] 07MirahezeSSLBot 0341f8080 - BOT: Updating services config for wikis
[21:22:34] <RhinosF1>	 thanks JohnLewis 
[21:22:43] <JohnLewis>	 yw
[22:44:38] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjP4b
[22:44:40] <Not-7290>	 [02miraheze/mw-config] 07paladox 0397678dc - Sitenotice for the enabling of ElasticSearch on all wikis Change-Id: Ib76b008226d7cd6c6d4bcb9ce2215f79cc306fa3
[22:58:29] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjPBe
[22:58:30] <Not-7290>	 [02miraheze/mw-config] 07paladox 0382db9bf - Set wmgUseCirrusSearch to true Change-Id: I9dabe848eff1e4f228f63fa56faffe86a931dcc3
[23:00:26] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjPBf
[23:00:28] <Not-7290>	 [02miraheze/mw-config] 07paladox 0396575ed - Set wmgDisableSearchUpdate for all wikis Change-Id: I3a095a99b1ac07079e2ed93aede951f0052311d2
[23:04:31] <paladox>	 !log running updateSearchIndexConfig.php for all wikis
[23:04:37] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[23:25:14] <icinga-miraheze>	 PROBLEM - mw2 Current Load on mw2 is CRITICAL: CRITICAL - load average: 9.55, 7.47, 5.99
[23:26:59] <paladox>	 huh
[23:27:14] <icinga-miraheze>	 PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 6.82, 7.21, 6.08
[23:30:48] <paladox>	 !log depool mw2
[23:30:54] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[23:32:30] <paladox>	 !log restart php-fpm && nginx on mw2
[23:32:38] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[23:33:06] <icinga-miraheze>	 RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 1.44, 5.10, 5.67
[23:35:14] <paladox>	 !log repool mw2
[23:35:24] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[23:38:58] <icinga-miraheze>	 PROBLEM - mw2 Current Load on mw2 is WARNING: WARNING - load average: 7.11, 6.33, 5.89
[23:39:15] <icinga-miraheze>	 PROBLEM - elasticsearch1 elasticsearch-lb.miraheze.org HTTPS on elasticsearch1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[23:43:13] <icinga-miraheze>	 RECOVERY - elasticsearch1 elasticsearch-lb.miraheze.org HTTPS on elasticsearch1 is OK: HTTP OK: HTTP/1.1 200 OK - 790 bytes in 0.041 second response time
[23:44:50] <icinga-miraheze>	 RECOVERY - mw2 Current Load on mw2 is OK: OK - load average: 5.48, 6.38, 6.10
[23:53:37] <paladox>	 !log cancle updateSearchIndexConfig.php (es OOM)
[23:53:41] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[23:54:06] <Not-7290>	 [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjPB7
[23:54:07] <Not-7290>	 [02miraheze/mw-config] 07paladox 03363cfbe - Revert "Set wmgDisableSearchUpdate for all wikis" This reverts commit 96575ed3f8631007e32e98c3bd5f8df7ac22603a.
[23:57:00] <icinga-miraheze>	 PROBLEM - mw1 Disk Space on mw1 is WARNING: DISK WARNING - free space: / 8422 MB (10% inode=99%);
[23:57:31] <paladox>	 hmm
[23:59:01] <icinga-miraheze>	 RECOVERY - mw1 Disk Space on mw1 is OK: DISK OK - free space: / 11871 MB (15% inode=99%);
[23:59:06] <paladox>	 !log rm undefined.log*
[23:59:12] <paladox>	 !log that's for mw1
[23:59:14] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master
[23:59:18] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master