[00:25:28] PROBLEM - Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 50 seconds ago with 0 failures [00:25:35] [02miraheze/ManageWiki] 07MacFan4000 pushed 032 commits to 03master [+0/-0/±3] 13https://git.io/fNTd7 [00:25:37] [02miraheze/ManageWiki] 07MacFan4000 03aeeebca - Revert "allow URLs in helptips" This reverts commit 755fc1539b301d8fa291eff255c0f176e8509aa7. [00:25:38] [02miraheze/ManageWiki] 07MacFan4000 03944b8bc - Link to mw.org pages [00:29:20] [02miraheze/ManageWiki] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTFk [00:29:22] [02miraheze/ManageWiki] 07MacFan4000 036b0c472 - Add i18n [00:32:32] [02miraheze/mediawiki] 07MacFan4000 pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNTFB [00:32:34] [02miraheze/mediawiki] 07MacFan4000 035ba69a3 - Update mw [00:34:13] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTFr [00:34:14] [02miraheze/mw-config] 07MacFan4000 037d5a14e - mw config [00:34:27] PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [00:48:08] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTbZ [00:48:09] [02miraheze/ManageWiki] 07paladox 036b9b072 - fix syntax [00:48:49] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNTbc [00:48:50] [02miraheze/mediawiki] 07paladox 03d61c495 - Update ManageWiki [00:52:28] RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:53:53] !log sudo -u www-data php rebuildLocalisationCache.php --wiki test1wiki on mw* [00:53:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [01:06:05] [02miraheze/ManageWiki] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTNc [01:06:07] [02miraheze/ManageWiki] 07MacFan4000 03adcb831 - Update en.json [01:07:06] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTNl [01:07:07] [02miraheze/ManageWiki] 07paladox 034398174 - Update SpecialManageWiki.php [01:08:22] [02miraheze/mediawiki] 07MacFan4000 pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNTNB [01:08:24] [02miraheze/mediawiki] 07MacFan4000 03a8d5719 - Uodate mw [01:10:24] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTNu [01:10:26] [02miraheze/ManageWiki] 07paladox 03976879f - Update SpecialManageWiki.php [01:11:09] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNTN2 [01:11:10] [02miraheze/mediawiki] 07paladox 03f5dd736 - Update ManageWiki [01:13:34] miraheze/ManageWiki/master/976879f - paladox The build passed. https://travis-ci.org/miraheze/ManageWiki/builds/402470549 [01:32:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTAI [01:32:53] [02miraheze/puppet] 07paladox 0303f7d77 - Update rewrite.py [02:03:26] RECOVERY - Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [02:42:09] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTpI [02:42:10] [02miraheze/ManageWiki] 07paladox 03dc16f2e - Add isset check for $ext['skin'] [02:42:45] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNTpq [02:42:47] [02miraheze/mediawiki] 07paladox 038da2e7d - Update ManageWiki [02:52:06] [02miraheze/CreateWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNTpa [02:52:08] [02miraheze/CreateWiki] 07paladox 030f1b9f7 - Add a isset check to getSettingsValue [02:52:45] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNTpw [02:52:47] [02miraheze/mediawiki] 07paladox 03dce7307 - Update CreateWiki [02:56:06] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNTpo [02:56:08] [02miraheze/mediawiki] 07paladox 03fc9ad9d - Update DynamicPageList3 [03:05:28] PROBLEM - Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [04:38:28] PROBLEM - Disk Space on db4 is WARNING: DISK WARNING - free space: / 76924 MB (20% inode=95%): [04:50:48] PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.54, 3.19, 2.88 [04:54:47] RECOVERY - Current Load on swift1 is OK: OK - load average: 3.01, 3.12, 2.92 [05:02:47] PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.52, 3.40, 3.11 [05:11:25] paladox: is findInactiveWikis.php ran automatically? (I remember some plans about that) [05:54:26] PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [06:02:27] RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:22:06] [02miraheze/ManageWiki] 07Reception123 pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/fNkIV [07:22:08] [02miraheze/ManageWiki] 07Reception123 03c490f02 - revert back to stable Per John [07:22:10] ^ JohnLewis [07:23:51] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNkIi [07:23:52] [02miraheze/mediawiki] 07Reception123 03e743067 - Update ManageWiki [07:24:28] PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [07:27:10] !log sudo -u www-data php rebuildLocalisationCache.php --wiki test1wiki on mw* [07:27:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [07:32:26] RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:43:49] JohnLewis: not sure why links are still appearing https://meta.miraheze.org/wiki/Special:ManageWiki/metawiki [07:43:50] Title: [ Permission error - Miraheze Meta ] - meta.miraheze.org [07:43:57] since I went back to the change you made before [07:44:27] PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [07:45:42] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNktt [07:45:43] [02miraheze/mw-config] 07Reception123 0317a5285 - Revert "mw config" This reverts commit 7d5a14e18829365414435ab472a882b7c1958c77. [07:53:08] ^ paladox not sure why the change isn't reverting [08:37:17] Reception123: run the lc? Also it’s because of the change [08:37:27] Chace [08:37:32] I did [08:37:43] cache [08:37:56] Reception123: cache :) [08:38:25] paladox: I did and the links are still there for me [08:39:26] Hmm [08:41:32] Reception123: check on mw* that it pulled your change [08:41:45] And also updated the submodule [08:41:58] Though this could be varnish cache [08:47:39] Reception123: bbl. though if this is cache to confirm follow https://meta.miraheze.org/w/index.php?title=Tech:Varnish&mobileaction=toggle_view_desktop#One-off_purges_.28bans.29 [08:47:41] Title: [ Tech:Varnish - Miraheze Meta ] - meta.miraheze.org [08:48:05] ban req.http.Host == meta.miraheze.org [08:48:14] JohnLewis: is that correct ^^ [08:48:26] Bbl [08:49:01] paladox: ok [08:49:13] paladox: also, https://phabricator.miraheze.org/T3091#63121 [08:49:14] Title: [ ⚓ T3091 Better design for ManageWiki with separate collapsible parts for each thing ] - phabricator.miraheze.org [08:49:23] so on test1 please leave the change, so we can finish working on that [08:50:01] Ok [08:50:11] paladox: hmm yes the change wasn't pulled [08:50:21] paladox: so do you think you could do a landing page? [10:12:27] RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:44:27] PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [10:52:28] RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:57:28] PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 5535 [11:34:28] PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [11:42:27] RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:04:26] PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [12:24:11] Hello [12:36:26] PROBLEM - HTTP 4xx/5xx ERROR Rate on cp4 is WARNING: WARNING - NGINX Error Rate is 44% [12:38:26] RECOVERY - HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 5% [12:55:28] RECOVERY - JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [13:44:26] PROBLEM - Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Apt_key[Add key: F86AA916A2195E121AEDB11437BBEE3F7AD95B3F from Apt::Source grafana_apt] [13:52:27] RECOVERY - Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:34:27] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNk5K [14:34:28] [02miraheze/mediawiki] 07paladox 033592a20 - Update RandomGameUnit to the master branch This is recommended by mw:social tools for stable mw releases. [14:42:46] PROBLEM - Current Load on swift1 is WARNING: WARNING - load average: 3.60, 3.29, 2.95 [14:44:46] RECOVERY - Current Load on swift1 is OK: OK - load average: 2.79, 3.05, 2.90 [15:07:43] [02miraheze/CreateWiki] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNkA4 [15:07:45] [02miraheze/CreateWiki] 07paladox 03654e0e5 - Change o to 0 [15:07:47] JohnLewis ^^ [15:09:02] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNkAV [15:09:04] [02miraheze/mediawiki] 07paladox 039483f6d - Update CreatePage [15:09:48] !log sudo -u www-data php /srv/mediawiki/w/maint*/rebuildLocalisationCache.php --wiki test1wiki on mw* [15:09:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [15:11:26] uh [15:11:29] i did the wrong ext [15:11:30] meh [15:12:19] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNkxJ [15:12:21] [02miraheze/mediawiki] 07paladox 035db2e7e - Update CreateWiki [15:20:10] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNkxF [15:20:12] [02miraheze/mediawiki] 07paladox 035dca27a - Update DynamicPageList3 [15:23:27] RECOVERY - Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures [15:25:27] RECOVERY - Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:39:11] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_31 [+0/-0/±1] 13https://git.io/fNkji [15:39:13] [02miraheze/mediawiki] 07paladox 03229bbe2 - Update DynamicPageList3 [15:52:26] PROBLEM - JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 2659 [17:17:28] RECOVERY - Puppet on test1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [17:21:49] !log rm -rf /srv/mediawiki on test1 [17:21:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [17:25:28] PROBLEM - Puppet on test1 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [17:30:10] MacFan4000: uh I forgot, what was that page that listed all the pages that need to be marked as translations? [17:30:34] Reception123: Special:PageTranslation [17:30:43] yeah, thanks :) [17:33:26] PROBLEM - MediaWiki Rendering on test1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 MediaWiki configuration Error - 1272 bytes in 0.022 second response time [17:33:40] Notice: Applied catalog in 583.50 seconds [17:33:44] paladox: here's our timeout [17:33:54] ok [17:40:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNI3U [17:40:08] [02miraheze/mw-config] 07paladox 038dae5ce - Change wmgUseEditCount to wmgUseEditcount [17:40:10] Reception123 ^^ [17:40:24] ok great :) [17:40:25] thanks [17:41:14] Error: /Stage[main]/Mediawiki/Git::Clone[MediaWiki core]/Exec[git_pull_MediaWiki core]/returns: change from notrun to 0 failed: Command exceeded timeout [17:41:17] ^ paladox this happens [17:41:25] but then when I git pull in /w it says "already up to date" [17:41:26] hmm [17:41:30] Reception123 yeh [17:41:35] test1 does not have enough cores [17:41:40] to do it within the timeout period [17:41:44] Warning: /Stage[main]/Mediawiki::Extensionsetup/Exec[Math texvccheck]: Skipping because of failed dependencies [17:41:55] paladox: yeah, but I already did git pull manually [17:41:55] Reception123 rm -rf /srv/mediawiki/w [17:42:13] ok [17:42:18] git clone -b REL1_31 https://github.com/miraheze/mediawiki.git w --recursive [17:42:18] Title: [ GitHub - miraheze/mediawiki: The collaborative editing software that runs Wikipedia. ] - github.com [17:42:57] PROBLEM - Current Load on test1 is CRITICAL: CRITICAL - load average: 2.79, 2.13, 1.34 [17:43:17] paladox: haha by accidentally writing w instead of "cd w" I discovered a new command [17:43:27] PROBLEM - MediaWiki Rendering on test1 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 199 bytes in 0.028 second response time [17:43:31] lol Reception123 what new command? [17:43:35] w :P [17:44:56] Reception123 oh w is not a command [17:44:57] that's the path [17:44:57] PROBLEM - Current Load on test1 is WARNING: WARNING - load average: 1.35, 1.80, 1.32 [17:45:39] paladox: no, try it :D [17:45:48] oh -w? [17:45:50] no [17:45:50] just w [17:45:58] git clone -b REL1_31 https://github.com/miraheze/mediawiki.git w --recursive [17:45:59] Title: [ GitHub - miraheze/mediawiki: The collaborative editing software that runs Wikipedia. ] - github.com [17:46:00] ? [17:46:03] w --recursive? [17:46:06] Reception123 ^^ [17:46:20] https://www.irccloud.com/pastebin/zvas12j5/ [17:46:22] Title: [ Snippet | IRCCloud ] - www.irccloud.com [17:46:22] ^ paladox [17:46:23] type fucking w paladox in shel [17:46:32] ll [17:46:39] it's that easy :D [17:46:57] RECOVERY - Current Load on test1 is OK: OK - load average: 1.04, 1.53, 1.27 [17:47:16] oh [17:47:21] https://www.irccloud.com/pastebin/loodl7Df/ [17:47:22] Title: [ Snippet | IRCCloud ] - www.irccloud.com [17:47:23] i thought he was talking about git JohnLewis [17:47:30] I was just scrolling back to find something and then I see that [17:47:38] ^ paladox [17:47:43] uh [17:48:06] It reveals who is logged in, and also their IP [17:48:42] and load avg + uptime [17:48:53] for each user [17:48:58] but uptime does this too [17:49:02] yeah [17:49:30] echo "Reception123 test" | wall [17:49:43] Reception123 try that :D [17:50:02] heh [17:50:05] there must be many of these commands [17:50:17] .mangle [17:50:24] MacFan4000: Acceptance 123 says that "many of these directives are worthy" [17:50:25] MacFan4000: I just recieved email saying that my request (T3336) went from resolved to declined. When checking Special:ManageWiki I see that the (succesful) changes from yesterday were rolled back. [17:50:52] Not me, rather JohnLewis to blame [17:51:00] Fritigern: Hi. Unfortunately, the changes were not successful as many of the extensions did not work with the links. [17:51:09] JohnLewis didn't decline the task, he reverted the change as it broke other things [17:51:27] Most of them worked and some had name conflicts [17:51:28] It is possible that this could be done sometime in the future, but since it doesn't work very well now, it was decided that it will be declined [17:51:40] Yeah, but we can't have most working and not some [17:51:48] unless we could somehow modify those manually [17:51:57] We should do it right [17:52:02] It wasn’t breaking anything paladox those were only warnings [17:52:03] otherwise it's just hacks [17:52:03] on top of hacks [17:52:13] The question is why some did not work [17:52:15] MacFan4000: yeah, but we don't want some of the links not to work [17:52:24] I think it was your change that broke it paladox [17:52:24] MacFan4000 they were log spam and also JohnLewis said it didn't work correctly. [17:52:29] i didn't check my self. [17:52:39] MacFan4000 um your change broke it [17:52:48] if it was producing that amount of log spam [17:52:51] in the first place [17:53:02] It was good that some of them worked, but all of them working is what we want to achieve. [17:53:16] As I said before, anyone is free to pick up the task again [17:53:20] I think your change made the extension unusable [17:53:56] I think that this may be a case of "too many cooks spoil the soup" [17:54:06] I never broke anything except links [17:54:14] MacFan4000: which change? [17:54:16] MacFan4000 still it caused the log spam sooo my change may have broke it [17:54:27] but i do not want the logs to be spammed hiding other stuff. [17:54:33] Stop bickering. Be professional. [17:54:42] ^^ [17:54:46] There's no need to get into a "who's change broke more" discussion here :P [17:55:10] Whatever the changes were, the current version is the stable one and the one that works properly. [17:55:56] The re4al question is not "who proke it" but "why do most links work and some don't?" as well as "how can the logspam be pevented?". [17:56:25] i think the way to resolve this is to add a linkPage param [17:56:40] so that when defined it adds the links to that ext which defined it. [17:58:07] ^ was my plan and my cha change that was ignored [17:58:27] the label-message parameter is i18n string and parameter, and label is raw only [17:59:29] Which is why I said we couldn’t do it [18:01:29] if we do ‘label-message’ => [‘managewiki-extension/skin-name’, $ext[‘linkPage’], $ext[‘name’]] I think it just may work [18:01:43] that will fail..... [18:01:43] Along with that if statement [18:01:53] yeh if statement [18:02:08] It’s i18n string followed by two parameters [18:02:38] 14:01 Along with that if statement [18:02:51] Actually no [18:03:50] Just do the parameter change, add i18n, and do a lot of changes in ManageWiki.php [18:04:04] I think it would work paladox [18:04:49] I think it's easier to just link [[M:Extensions]] rather than all this work [18:05:17] ^ that too I guess maybe? [18:05:45] Everything is linked there, so that's a great page people can consult to see extensions [18:06:00] That’s not always up to date though [18:07:14] Well we could just remember to update when installing an extension [18:07:30] It's easier than making a complicated edit to MW [18:08:33] At this point what I am saying, means changing two lines in one file adding two in another and a bunch in mw-config [18:09:11] If you can make it work. Then do it [18:09:45] My close yesterday on the task was basically a “Ive done this if someone does x” and my x is what you’re suggesting [18:10:35] The task is done either way as I did it yesterday in a different way pending upstream [18:17:24] also, everything in /srv/mediawiki/w on test1 is owned by root, can someone please fix that? [18:17:32] Otherwise I can't edit [18:18:28] geez 9.9k jobs [18:19:17] oh, uh, nvm, I see that people are still doing stuff [18:21:44] yeah puppet is failing because of perms, so ops needs to change the perms on /srv/mediawiki/w [18:22:58] Reception123: ^ your fault [18:23:56] Notice: /Stage[main]/Mediawiki/Git::Clone[MediaWiki core]/Exec[git_pull_MediaWiki core]/returns: error: cannot open .git/FETCH_HEAD: Permission denied [18:24:04] Yeah, though test1 is meant for testing [18:24:20] No access atm, paladox can you chown ^ [18:24:25] well, anyway thats why puppet is failing [18:24:32] Yeah [18:26:27] Reception123 still, tsk tsk! [18:28:54] the folder itself is owned by www-data, but its contents are owned by root [18:30:11] When I was mw-admins people left files owned by root so [18:30:12] .. [18:30:22] But yeah should've done www-data [18:30:33] Reception123 Where? [18:30:53] /srv/mediawiki/w [18:31:00] Reception123 i ment which server? [18:31:08] test1 paladox [18:31:10] ok [18:31:11] thanks [18:31:34] done [18:31:35] fixed [18:32:04] Reception123: hush, I did what I want back then okay because it was mostly be doing it anyway xD [18:32:38] JohnLewis huh? [18:33:16] Paladox: nothing [18:33:32] ok [18:33:52] paladox: 14:30 <+Reception123> When I was mw-admins people left files owned by root so [18:34:19] oh o [18:34:21] ok [18:39:27] RECOVERY - Puppet on test1 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures [18:41:27] PROBLEM - MediaWiki Rendering on test1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 624 bytes in 0.365 second response time [18:42:51] !log macfan@test1:/srv/mediawiki/w/maintenance$ sudo -u www-data php rebuildLocalisationCache.php --wiki test1wiki [18:42:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [18:52:58] PROBLEM - Current Load on test1 is WARNING: WARNING - load average: 1.75, 1.56, 1.04 [18:56:56] PROBLEM - Current Load on test1 is CRITICAL: CRITICAL - load average: 2.34, 1.86, 1.27 [18:58:56] RECOVERY - Current Load on test1 is OK: OK - load average: 1.41, 1.68, 1.27 [18:59:26] PROBLEM - Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [19:01:00] 8k jobs [19:01:05] This is annoying now [19:04:09] JohnLewis add more runners [19:05:14] The job runner isn’t the issue [19:05:39] Overpowering stuff because of inefficiency is always bad [19:07:27] RECOVERY - Puppet on test1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:09:27] RECOVERY - MediaWiki Rendering on test1 is OK: HTTP OK: HTTP/1.1 200 OK - 30105 bytes in 0.971 second response time [19:47:30] paladox: https://meta.miraheze.org/w/index.php?diff=54695&oldid=54660&rcid=143524 [19:47:32] Title: [ Difference between revisions of "Stewards' noticeboard" - Miraheze Meta ] - meta.miraheze.org [19:47:52] * paladox looks [19:48:41] JohnLewis fixed [19:48:47] !log swift post -r '.r:*' jawp2chwiki-mw [19:48:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [19:49:43] Kay [19:59:30] You haven’t replied to the user btw paladox [20:00:04] JohnLewis done [20:03:26] PROBLEM - Puppet on test1 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 8 minutes ago with 0 failures [20:07:11] Not specified? Who disabled it? ^^ [20:07:26] PROBLEM - MediaWiki Rendering on test1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 222 bytes in 0.045 second response time [20:08:24] JohnLewis not me [20:10:55] [11-Jul-2018 20:10:48 Etc/Utc] PHP Parse error: syntax error, unexpected ':', expecting ')' in /srv/mediawiki/config/ManageWiki.php on line 6 [20:11:16] someone touched it at -rw-r--r-- 1 www-data www-data 32K Jul 11 20:04 /srv/mediawiki/config/ManageWiki.php [20:11:21] Its me [20:12:07] In general for all then: when disabling puppet, provide a reason please. Even your name is enough please. [20:12:48] MacFan4000 they are all in literels [20:12:51] should be '' [20:13:17] well it's test server [20:13:23] so im going back to socialprofile [20:18:53] acc'd working on it [20:19:04] s/acc/ack [20:19:05] MacFan4000 meant to say: ack'd working on it [20:49:58] paladox: ^^ I'm still seeing the same error, but I replaced ' with " [20:50:08] hmm [20:50:30] i can still see the problems in ManageWiki.php [20:50:33] vi highlighting off [20:50:39] indicating syntax error [20:50:48] what [20:50:56] Also I use nano and not vi/vim [20:51:31] paladox: ^^ [20:51:46] MacFan4000 huh, is your keyboard set funny? [20:52:28] uh [20:52:37] PROBLEM - MediaWiki Rendering on mw3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:52:48] I don't understand what the syntax error is [20:53:27] PROBLEM - Varnish Backends on cp5 is CRITICAL: 2 backends are down. mw1 mw2 [20:53:47] PROBLEM - GDNSD Datacenters on misc1 is CRITICAL: CRITICAL - 6 datacenters are down: 107.191.126.23/cpweb, 2604:180:0:33b::2/cpweb, 81.4.109.133/cpweb, 2a00:d880:5:8ea::ebc7/cpweb, 172.104.111.8/cpweb, 2400:8902::f03c:91ff:fe07:444e/cpweb [20:53:51] um [20:53:55] why is mw* down [20:53:57] PROBLEM - Current Load on misc4 is CRITICAL: CRITICAL - load average: 4.89, 3.24, 1.70 [20:54:07] PROBLEM - MediaWiki Rendering on mw2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:54:12] uh crap [20:54:27] PROBLEM - Varnish Backends on cp2 is CRITICAL: 1 backends are down. mw2 [20:55:07] timed out [20:55:18] is it because of increased traffic? [20:55:27] RECOVERY - Varnish Backends on cp5 is OK: All 3 backends are healthy [20:55:44] 16:52 <+MacFan4000> I don't understand what the syntax error is [20:55:47] RECOVERY - GDNSD Datacenters on misc1 is OK: OK - all datacenters are online [20:55:51] We have increased traffic?? Didn’t know a World Cup exit could do that [20:55:55] lol [20:56:01] MacFan4000 it's fixed now [20:56:11] MacFan4000 but yes invalid syntax [20:56:16] `` is not a string [20:56:20] that is literels [20:56:26] i may have misplet it [20:56:27] RECOVERY - HTTP 4xx/5xx ERROR Rate on cp4 is OK: OK - NGINX Error Rate is 10% [20:56:34] and same goes for the back ticks. [20:56:48] JohnLewis i am throwing out ideas :) [20:56:49] but I replaced " with ' [20:56:53] but it is timing out [20:56:55] in php-fpm [20:57:11] meaning in creased connection (i got that from #wikimedia-operations) [20:57:12] today! [20:57:21] MacFan4000 for some you didn't do them all [20:57:28] maybe it's your client? [20:57:52] https://test1.miraheze.org/wiki/Special:ManageWiki [20:57:53] Title: [ Internal error - Test1 Wiki ] - test1.miraheze.org [20:57:55] parse error anyways [20:57:58] Or its just me missing some by mistake? :P [20:58:46] !log restarted php7.2-fpm on mw2 [20:58:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [20:59:32] /wiki/Special:ManageWiki ParseError from line 154 of /srv/mediawiki/w/extensions/ManageWiki/includes/SpecialManageWiki.php: syntax error, unexpected ''default'' (T_CONSTANT_ENCAPSED_STRING), expecting ')' [20:59:37] MacFan4000 ^^ [20:59:58] RECOVERY - MediaWiki Rendering on mw2 is OK: HTTP OK: HTTP/1.1 200 OK - 30102 bytes in 0.091 second response time [21:00:06] MacFan4000 uh yeh it's your client [21:00:10] RECOVERY - GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [21:00:11] literels again [21:00:28] RECOVERY - Varnish Backends on cp2 is OK: All 3 backends are healthy [21:02:00] ack'd [21:03:12] fixed https://test1.miraheze.org/wiki/Special:ManageWiki [21:03:13] Title: [ Permission error - Test1 Wiki ] - test1.miraheze.org [21:03:19] it's very slow tonight [21:03:22] test1 [21:03:22] ssh [21:05:26] PROBLEM - Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:05:28] PROBLEM - Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:05:29] And on wiki i18n now created [21:05:39] so, now just correct a few urls, and done [21:06:27] ⧼managewik-skin-name⧽ [21:06:33] MacFan4000 remember to check the logs [21:06:34] too [21:06:38] when saving [21:13:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNIVQ [21:13:10] [02miraheze/puppet] 07paladox 038dcf43d - Increasing php-fpm resources Will revert this if it has a negative impact. [21:13:10] JohnLewis ^^ [21:13:37] for incrased demand! (i have no proof that it's increased demand) we should probaley have grafana have a dashboard for traffic. [21:17:27] RECOVERY - Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [21:23:27] RECOVERY - Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [21:26:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fNIrT [21:26:29] [02miraheze/puppet] 07paladox 033fbce07 - Update jobrunner.json.erb [21:28:17] paladox: why? [21:28:24]