[00:02:39] PROBLEM - mon1 Current Load on mon1 is WARNING: WARNING - load average: 3.78, 3.17, 2.58 [00:03:17] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 43.06, 35.43, 26.19 [00:04:34] RECOVERY - mon1 Current Load on mon1 is OK: OK - load average: 2.83, 3.07, 2.62 [00:10:58] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.93, 21.95, 23.43 [00:22:27] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.41, 17.74, 20.07 [00:34:01] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 22.96, 22.66, 21.12 [00:38:02] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 11.76, 17.93, 19.65 [00:44:14] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is WARNING: WARNING - NGINX Error Rate is 44% [00:46:20] [02mw-config] 07paladox closed pull request 03#3364: Move ContactPage to ManageWikiExtensions; Move ContactPage's RecipientUser config to ManageWikiSettings; which defaults to the wiki requester's username - 13https://git.io/JTpOg [00:46:21] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-1/±5] 13https://git.io/JksoD [00:46:23] [02miraheze/mw-config] 07Universal-Omega 03f8e6193 - Move ContactPage to ManageWikiExtensions; Move ContactPage's RecipientUser config to ManageWikiSettings; which defaults to the wiki requester's username (#3364) * Move ContactPage configs to LS * Delete ContactPage.php * Update LocalExtensions.php * Move ContactPage to MWE/config to MWS * Migrate ContactPage to MWE * migrate ContactPage config to MWS * [00:46:23] migrate ContactPage config to MWS * add $wmgContactPageRecipientUser default value * fix missing "," * fix default value for $wmgContactPageRecipientUser * Update default for $wmgContactPageRecipientUser * rename variable * rename variable * rename variable * add ->getName() * add ->getName(); change type to "user" * Wiki requester not founder * Wiki requester not founder [00:47:49] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+1/-0/±5] 13https://git.io/JksoQ [00:47:51] [02miraheze/mw-config] 07paladox 03196a45b - Revert "Move ContactPage to ManageWikiExtensions; Move ContactPage's RecipientUser config to ManageWikiSettings; which defaults to the wiki requester's username (#3364)" This reverts commit f8e6193b6f419668df08533da2a9d5f3816e8f03. [00:48:10] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is CRITICAL: CRITICAL - NGINX Error Rate is 64% [00:50:07] PROBLEM - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is WARNING: WARNING - NGINX Error Rate is 46% [00:52:05] RECOVERY - cp6 HTTP 4xx/5xx ERROR Rate on cp6 is OK: OK - NGINX Error Rate is 39% [00:54:07] PROBLEM - db7 Check MariaDB Replication on db7 is CRITICAL: MariaDB replication - both - CRITICAL - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 106s [00:56:06] RECOVERY - db7 Check MariaDB Replication on db7 is OK: MariaDB replication - both - OK - Slave_IO_Running state : Yes, Slave_SQL_Running state : Yes, Seconds_Behind_Master : 1s [00:58:10] [02mw-config] 07Universal-Omega opened pull request 03#3447: Migrate ContactPage to MWS/MWE - 13https://git.io/JksKl [01:01:17] [02mw-config] 07Universal-Omega synchronize pull request 03#3447: Migrate ContactPage to MWS/MWE - 13https://git.io/JksKl [01:04:26] [02mw-config] 07paladox closed pull request 03#3447: Migrate ContactPage to MWS/MWE - 13https://git.io/JksKl [01:04:28] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-1/±5] 13https://git.io/JksKh [01:04:29] [02miraheze/mw-config] 07Universal-Omega 03df13b38 - Migrate ContactPage to MWS/MWE (#3447) * Move ContactPage configs to LS * Delete ContactPage.php * Update LocalExtensions.php * Move ContactPage to MWE/config to MWS * Migrate ContactPage to MWE * migrate ContactPage config to MWS * migrate ContactPage config to MWS * add $wmgContactPageRecipientUser default value * fix missing "," * fix default value for [01:04:29] $wmgContactPageRecipientUser * Update default for $wmgContactPageRecipientUser * rename variable * rename variable * rename variable * add ->getName() * add ->getName(); change type to "user" * Wiki requester not founder * Wiki requester not founder * Move to LocalWiki.php * Moved [01:05:01] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+1/-0/±5] 13https://git.io/Jks6f [01:05:02] [02miraheze/mw-config] 07paladox 034a9444a - Revert "Migrate ContactPage to MWS/MWE (#3447)" This reverts commit df13b38bb6806e866d1ddcb8be8bc756fd8eff06. [01:12:04] [02mw-config] 07Universal-Omega opened pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [01:28:39] [02mw-config] 07The-Voidwalker commented on pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/JksiN [01:58:47] [02mw-config] 07Universal-Omega synchronize pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [01:59:57] [02mw-config] 07Universal-Omega synchronize pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [02:00:19] [02mw-config] 07Universal-Omega synchronize pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [02:01:42] [02mw-config] 07Universal-Omega synchronize pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [02:10:16] [02mw-config] 07Universal-Omega synchronize pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [02:12:51] [02mw-config] 07Universal-Omega synchronize pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [04:44:52] [02mw-config] 07Universal-Omega opened pull request 03#3449: Fix Medik configs; unclosed code tag - 13https://git.io/JksAA [04:51:22] [02mw-config] 07Pix1234 closed pull request 03#3449: Fix Medik configs; unclosed code tag - 13https://git.io/JksAA [04:51:24] [02miraheze/mw-config] 07Pix1234 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/Jksxi [04:51:25] [02miraheze/mw-config] 07Universal-Omega 0336e2bee - Fix medik configs (#3449) [04:51:37] Thanks @Zppix [04:51:43] Sorry for the mistake. [06:34:38] [02mw-config] 07Universal-Omega synchronize pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [14:11:35] * hispano76 greetings [14:54:29] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 23.09, 21.34, 18.18 [14:56:25] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 15.90, 18.84, 17.63 [16:02:01] hi @Hispano76 🙂 [16:02:54] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.62, 20.11, 17.72 [16:04:49] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 17.49, 19.50, 17.79 [17:42:12] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 24.23, 19.96, 16.94 [17:44:07] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.69, 19.54, 17.14 [18:37:06] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 25.61, 21.77, 17.72 [18:39:01] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.22, 21.27, 18.02 [18:43:23] [02mw-config] 07paladox closed pull request 03#3448: Migrate ContactPage to MWS/MWE - 13https://git.io/Jks60 [18:43:24] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-1/±5] 13https://git.io/JkZ2S [18:43:26] [02miraheze/mw-config] 07Universal-Omega 032178066 - Migrate ContactPage to MWS/MWE (#3448) * Move ContactPage configs to LS * Delete ContactPage.php * Update LocalExtensions.php * Move ContactPage to MWE/config to MWS * Migrate ContactPage to MWE * migrate ContactPage config to MWS * migrate ContactPage config to MWS * add $wmgContactPageRecipientUser default value * fix missing "," * fix default value for [18:43:26] $wmgContactPageRecipientUser * Update default for $wmgContactPageRecipientUser * rename variable * rename variable * rename variable * add ->getName() * add ->getName(); change type to "user" * Wiki requester not founder * Wiki requester not founder * Move to LocalWiki.php * Moved * fix * fix * fix * fix * rename * fix * fix * fix * fix * fix * fix * fix * fix [18:44:34] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+1/-0/±5] 13https://git.io/JkZ2b [18:44:36] [02miraheze/mw-config] 07paladox 03d782099 - Revert "Migrate ContactPage to MWS/MWE (#3448)" This reverts commit 2178066981125a6585501cf4b9607969caf41dc5. [18:44:48] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 12.84, 18.25, 17.97 [18:46:40] PROBLEM - test2 MediaWiki Rendering on test2 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.025 second response time [18:48:43] PROBLEM - test2 Puppet on test2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 6 minutes ago with 0 failures [18:48:54] paladox: why test2 critical? [18:49:14] because there's a failure in a commit. [18:49:46] Ok [18:50:03] I guess it's what you reverted so it can't hit prod [18:50:54] yes [18:52:41] RECOVERY - test2 MediaWiki Rendering on test2 is OK: HTTP OK: HTTP/1.1 200 OK - 18951 bytes in 1.644 second response time [18:52:44] RECOVERY - test2 Puppet on test2 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [18:53:16] [02mw-config] 07Universal-Omega opened pull request 03#3450: Migrate ContactPage to MWS/MWE - 13https://git.io/JkZa0 [18:54:11] [02landing] 07WerySkok synchronize pull request 03#46: Russian translation. - 13https://git.io/JksuA [18:55:25] [02mw-config] 07paladox closed pull request 03#3450: Migrate ContactPage to MWS/MWE - 13https://git.io/JkZa0 [18:55:27] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-1/±5] 13https://git.io/JkZai [18:55:28] [02miraheze/mw-config] 07Universal-Omega 03480cfee - Migrate ContactPage to MWS/MWE (#3450) * Move ContactPage configs to LS * Delete ContactPage.php * Update LocalExtensions.php * Move ContactPage to MWE/config to MWS * Migrate ContactPage to MWE * migrate ContactPage config to MWS * migrate ContactPage config to MWS * add $wmgContactPageRecipientUser default value * fix missing "," * fix default value for [18:55:28] $wmgContactPageRecipientUser * Update default for $wmgContactPageRecipientUser * rename variable * rename variable * rename variable * add ->getName() * add ->getName(); change type to "user" * Wiki requester not founder * Wiki requester not founder * Move to LocalWiki.php * Moved * fix * fix * fix * fix * rename * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix [19:01:48] [02landing] 07paladox closed pull request 03#46: Russian translation. - 13https://git.io/JksuA [19:01:49] :) [19:01:49] [02miraheze/landing] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkZVT [19:01:51] [02miraheze/landing] 07WerySkok 03c3eec6a - Russian translation. (#46) * Added Russian translation. * Removed spaces after dots. * Removed comma * typo [19:05:01] [02mw-config] 07Universal-Omega opened pull request 03#3451: Fix contactpage extension name - 13https://git.io/JkZVl [19:05:44] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 13.76, 9.08, 5.16 [19:06:08] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 35.99, 26.93, 20.54 [19:07:44] PROBLEM - cp7 Current Load on cp7 is WARNING: WARNING - load average: 4.41, 7.47, 5.07 [19:08:03] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 16.19, 23.28, 20.01 [19:09:45] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 1.83, 5.41, 4.60 [19:10:01] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 9.96, 18.90, 18.80 [19:13:40] Hello Driolar! If you have any questions, feel free to ask and someone should answer soon. [19:14:19] [02mw-config] 07Universal-Omega opened pull request 03#3452: Remove config, meant for extension developers only and has no impact here - 13https://git.io/JkZVx [19:14:20] Hi Driolar [19:14:42] I have actived PDF Handler extension. However, I believe it does not work. [19:15:14] Explain further? [19:17:16] [02mw-config] 07paladox closed pull request 03#3452: Remove config, meant for extension developers only and has no impact here - 13https://git.io/JkZVx [19:17:17] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkZwt [19:17:19] [02miraheze/mw-config] 07Universal-Omega 033b7ac00 - Remove config (#3452) [19:17:33] [02mw-config] 07paladox commented on pull request 03#3451: Fix contactpage extension name - 13https://git.io/JkZwq [19:18:28] [02mw-config] 07Universal-Omega commented on pull request 03#3451: Fix contactpage extension name - 13https://git.io/JkZwZ [19:18:57] In https://wpe.miraheze.org/wiki/Global_Voting_Platform, the last item should display multiview. But instead, only shows link to File:ProjectBrief.pdf. Even more, this link is displayed in Chrome only if I am logged in! In other browsers, the link is always shown. [19:18:58] [ Global Voting Platform - World Parliament Experiment ] - wpe.miraheze.org [19:20:35] Yeah that shouldn't happen [19:21:14] Universal_Omega: ^ [19:22:16] Looking now. [19:23:17] I can view the page in Vivaldi (latest build), a Chromium-based web browser. I have only tested it logged out. [19:24:05] PDF loads fine, too. [19:24:52] Yeah, I am not sure what the issue is either. [19:25:10] I have tried with Opera, Edge and Chrome (download today). [19:29:02] I have now installed Vivaldi and tried the page. I see the link, but it doesn't show as multiview (that is the purpose of PDFHandler extension). What do you mean with "PDF loads fine"? [19:32:38] PROBLEM - cloud2 Current Load on cloud2 is CRITICAL: CRITICAL - load average: 40.35, 29.53, 21.93 [19:34:15] PROBLEM - munwiki.info - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for munwiki.info could not be found [19:38:07] PROBLEM - cp7 Current Load on cp7 is CRITICAL: CRITICAL - load average: 11.82, 8.90, 5.71 [19:40:03] RECOVERY - cp7 Current Load on cp7 is OK: OK - load average: 4.55, 6.79, 5.29 [19:40:47] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://git.io/JkZrF [19:40:49] [02miraheze/puppet] 07Reception123 03819e6be - add universalomega to mw-admins [19:40:51] [02puppet] 07Reception123 created branch 03Reception123-patch-1 - 13https://git.io/vbiAS [19:40:53] [02puppet] 07Reception123 opened pull request 03#1516: add universalomega to mw-admins - 13https://git.io/JkZrb [19:41:09] RECOVERY - munwiki.info - reverse DNS on sslhost is OK: rDNS OK - munwiki.info reverse DNS resolves to cp6.miraheze.org [19:42:53] [02puppet] 07Reception123 closed pull request 03#1516: add universalomega to mw-admins - 13https://git.io/JkZrb [19:42:55] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkZof [19:42:56] [02miraheze/puppet] 07Reception123 03a2f73e2 - add universalomega to mw-admins (#1516) [19:47:30] Reception123: how do we do mail aliases now? [19:48:04] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 19.10, 22.22, 23.60 [19:48:05] RhinosF1: oh, I asked John [19:48:15] as I don't know the progress made with the LDAP migration [19:49:20] Yeah I tried to work it out earlier but it wasn't obvious and unsurprisingly docs are wrong [19:49:52] Requests isn't pinned on jobrunner1 is it? [19:52:21] Universal_Omega: you wanna do you your first task? [19:53:01] !log created mail account for Universal Omega [19:53:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:53:46] !log added Universal Omega to @miraheze GH project [19:53:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:56:01] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 12.61, 15.33, 19.76 [20:01:52] [02mw-config] 07paladox commented on pull request 03#3451: Fix contactpage extension name - 13https://git.io/JkZK0 [20:05:17] !log created LDAP account for Universal Omega [20:05:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:06:56] *user Universal_Omega give 4 [20:06:56] Universal_Omega granted ChanOp access for #miraheze. [20:07:58] [02mw-config] 07JohnFLewis commented on pull request 03#3451: Fix contactpage extension name - 13https://git.io/JkZKd [20:09:20] [02mw-config] 07paladox commented on pull request 03#3451: Fix contactpage extension name - 13https://git.io/JkZKx [20:10:02] [02mw-config] 07paladox closed pull request 03#3451: Fix contactpage extension name - 13https://git.io/JkZVl [20:10:04] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkZ6J [20:10:05] [02miraheze/mw-config] 07Universal-Omega 03a500541 - Fix contactpage extension name (#3451) [20:12:23] !log added universal omega to tech@ alias on misc1 [20:12:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:14:03] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 20.81, 20.23, 19.46 [20:14:11] !log /usr/bin/newaliases on misc1 [20:14:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:15:08] Reception123: can you run 'pip3 show requests' and 'pip3 check' on at least jobrunner1 and paste the output [20:15:17] ok [20:15:44] I wanna see if we also got broke in the same way bots did [20:16:00] reception@jobrunner1:~$ sudo pip3 show requests [20:16:00] sudo: pip3: command not found [20:16:02] ^ RhinosF1 hmm [20:18:04] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 18.17, 19.19, 19.23 [20:19:51] Reception123: /usr/bin/pip3 maybe [20:20:13] Or maybe it has to go via apt [20:20:20] doesn't like that it either [20:20:24] oh [20:21:07] Reception123: is there a way to show a package version in Apt? [20:21:18] If so look for python3-requests [20:21:41] But I assume apt blocks you installing incompatible packages so won't be an issue [20:21:44] paladox: ^ [20:21:54] ? [20:21:57] python3-requests is already the newest version (2.21.0-1). [20:22:09] (I ran a "simulation" install) [20:22:10] Reception123: ah ok [20:22:15] we don't use pip packages as far as i'm aware. [20:22:28] paladox: does apt just refuse to install things that are incompatible? [20:23:16] apt won't install anything that isn't either in the dist (for the os you have) and also requires a package version that either doesn't exist or is not available for that os [20:23:55] paladox: I'm gonna assume then it won't be affected by some combinations of requests & urllib3 not working together then [20:24:05] nope [20:25:32] * RhinosF1 is glad he doesn't have to spend another 30-40 minutes unbreaking that [21:11:16] Universal_Omega: try oping in here or #miraheze-offtopic [21:11:22] .op [21:11:22] Attempting to OP... [21:11:39] .op [21:11:50] Yey! [21:16:15] .deop [21:17:05] .deop MirahezeBot [22:22:04] [02mw-config] 07Hispano76 opened pull request 03#3453: temporary removal of configurations per T6444 - 13https://git.io/JkZyN [22:22:49] PROBLEM - wiki.autocountsoft.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.autocountsoft.com All nameservers failed to answer the query. [22:24:13] [02mw-config] 07Universal-Omega reviewed pull request 03#3453 commit - 13https://git.io/JkZSf [22:28:42] [02mw-config] 07Hispano76 reviewed pull request 03#3453 commit - 13https://git.io/JkZS4 [22:29:28] RECOVERY - wiki.autocountsoft.com - reverse DNS on sslhost is OK: rDNS OK - wiki.autocountsoft.com reverse DNS resolves to cp7.miraheze.org [22:33:33] [02mw-config] 07Universal-Omega closed pull request 03#3453: temporary removal of configurations per T6444 - 13https://git.io/JkZyN [22:33:35] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JkZS1 [22:33:36] [02miraheze/mw-config] 07Hispano76 03271eef9 - temporary removal of configurations per T6444 (#3453) [23:04:37] PROBLEM - cloud2 Current Load on cloud2 is WARNING: WARNING - load average: 21.44, 19.14, 17.01 [23:06:45] PROBLEM - test2 APT on test2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:08:28] RECOVERY - cloud2 Current Load on cloud2 is OK: OK - load average: 19.95, 19.43, 17.59 [23:10:46] RECOVERY - test2 APT on test2 is OK: APT OK: 14 packages available for upgrade (0 critical updates).