[00:11:17] !log running hack script for Cargo move [00:11:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log, Master [00:23:23] [02miraheze/mw-config] 07JohnFLewis pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjvbM [00:23:25] [02miraheze/mw-config] 07JohnFLewis 0336ac00e - change cargoDB name [07:49:49] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [07:50:34] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [07:50:48] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [07:51:09] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [07:53:43] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown) [07:54:02] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown) [07:54:08] PROBLEM - elasticsearch1 Puppet on elasticsearch1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown) [07:54:09] PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 2 minutes ago with 4 failures. Failed resources (up to 3 shown) [07:54:22] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 2 minutes ago with 6 failures. Failed resources (up to 3 shown) [07:54:23] PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 2 minutes ago with 6 failures. Failed resources (up to 3 shown) [07:54:54] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 359 failures. Last run 3 minutes ago with 359 failures. Failed resources (up to 3 shown) [07:55:11] PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Puppet has 5 failures. Last run 3 minutes ago with 5 failures. Failed resources (up to 3 shown) [07:55:15] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 30 failures. Last run 3 minutes ago with 30 failures. Failed resources (up to 3 shown) [07:55:18] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 3 minutes ago with 10 failures. Failed resources (up to 3 shown) [07:55:33] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 213 failures. Last run 2 minutes ago with 213 failures. Failed resources (up to 3 shown): File[antiguabarbudacalypso.com_private],File[spiral.wiki],File[spiral.wiki_private],File[permanentfuturelab.wiki] [07:55:57] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 346 failures. Last run 3 minutes ago with 346 failures. Failed resources (up to 3 shown): File[spiral.wiki_private],File[permanentfuturelab.wiki],File[permanentfuturelab.wiki_private],File[private.revi.wiki] [07:57:42] PROBLEM - cp5 Puppet on cp5 is CRITICAL: CRITICAL: Puppet has 111 failures. Last run 3 minutes ago with 111 failures. Failed resources (up to 3 shown): File[antiguabarbudacalypso.com],File[antiguabarbudacalypso.com_private],File[spiral.wiki],File[spiral.wiki_private] [08:02:07] RECOVERY - elasticsearch1 Puppet on elasticsearch1 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures [08:03:43] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [08:04:01] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [08:04:09] RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [08:04:22] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:04:23] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [08:04:54] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:05:11] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:05:14] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:05:17] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:05:32] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 46 seconds ago with 0 failures [08:05:40] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 1028 [08:05:41] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures [08:06:02] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:27:25] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [08:54:32] hi [08:55:41] got more tech queries [08:55:46] 1. How do i infobox [08:56:55] 2. Is an image copyrighted if it's of a person or thing the photographer did not make? [08:57:34] that's it [09:05:10] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjvhh [09:05:11] [02miraheze/services] 07MirahezeSSLBot 03264c635 - BOT: Updating services config for wikis [12:44:02] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 9 failures. Last run 2 minutes ago with 9 failures. Failed resources (up to 3 shown) [12:44:08] PROBLEM - elasticsearch1 Puppet on elasticsearch1 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 2 minutes ago with 4 failures. Failed resources (up to 3 shown) [12:44:09] PROBLEM - bacula1 Puppet on bacula1 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 2 minutes ago with 3 failures. Failed resources (up to 3 shown) [12:44:22] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 2 minutes ago with 6 failures. Failed resources (up to 3 shown) [12:44:23] PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown) [12:44:25] PROBLEM - lizardfs1 Puppet on lizardfs1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown) [12:44:54] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 362 failures. Last run 3 minutes ago with 362 failures. Failed resources (up to 3 shown) [12:45:01] PROBLEM - lizardfs2 Puppet on lizardfs2 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown) [12:45:11] PROBLEM - db4 Puppet on db4 is CRITICAL: CRITICAL: Puppet has 5 failures. Last run 3 minutes ago with 5 failures. Failed resources (up to 3 shown) [12:45:15] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 35 failures. Last run 3 minutes ago with 35 failures. Failed resources (up to 3 shown) [12:45:18] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 12 failures. Last run 3 minutes ago with 12 failures. Failed resources (up to 3 shown) [12:45:34] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 187 failures. Last run 2 minutes ago with 187 failures. Failed resources (up to 3 shown): File[/etc/nginx/mime.types],File[/etc/nginx/fastcgi_params],File[antiguabarbudacalypso.com],File[antiguabarbudacalypso.com_private] [12:45:43] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 4 minutes ago with 2 failures. Failed resources (up to 3 shown) [12:46:06] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 321 failures. Last run 3 minutes ago with 321 failures. Failed resources (up to 3 shown): File[antiguabarbudacalypso.com],File[antiguabarbudacalypso.com_private],File[spiral.wiki],File[spiral.wiki_private] [12:46:28] umm paladox [12:47:42] PROBLEM - cp5 Puppet on cp5 is CRITICAL: CRITICAL: Puppet has 105 failures. Last run 3 minutes ago with 105 failures. Failed resources (up to 3 shown): File[/etc/nginx/mime.types],File[/etc/nginx/fastcgi_params],File[antiguabarbudacalypso.com],File[antiguabarbudacalypso.com_private] [12:48:01] I think something broke miraheze servers [12:48:07] Reception123: paladox MacFan4000 [12:52:07] RECOVERY - elasticsearch1 Puppet on elasticsearch1 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [12:53:01] RECOVERY - lizardfs2 Puppet on lizardfs2 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [12:53:11] RECOVERY - db4 Puppet on db4 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [12:53:14] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [12:53:17] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 18 seconds ago with 0 failures [12:53:31] whatever puppet did it fixed itself *shrug* [12:53:43] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [12:54:01] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:54:09] RECOVERY - bacula1 Puppet on bacula1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:54:10] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [12:54:11] if puppet dies on a bunch of things, it's probably garbage collection [12:54:20] (and expected) [12:54:22] Voidwalker: it went as high as 362 [12:54:22] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:54:23] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:54:24] RECOVERY - lizardfs1 Puppet on lizardfs1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:54:28] Voidwalker: i've never seen it before [12:54:52] Voidwalker: and that would be some very very blunt garbage collection [12:54:54] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:55:00] yeah, that'd be GC, usually happens late in the day [12:55:32] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:55:41] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [12:55:51] jesus icinga-miraheze we get it [13:08:33] PROBLEM - mw3 JobQueue on mw3 is CRITICAL: JOBQUEUE CRITICAL - job queue greater than 300 jobs. Current queue: 813 [13:15:49] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 36 seconds ago with 0 failures [13:16:15] RECOVERY - mw3 JobQueue on mw3 is OK: JOBQUEUE OK - job queue below 300 jobs [13:16:21] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 52 seconds ago with 0 failures [13:16:53] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [13:16:58] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [14:29:07] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [14:35:13] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures [15:45:43] * Hispano76_ saluda [17:51:12] Hello Guest50834! If you have any questions feel free to ask and someone should answer soon. [19:51:29] Hi! Here is the list of currently open high priority tasks on Phabricator [19:51:36] No updates for 25 days - https://phabricator.miraheze.org/T4128 - Fix SSH not working after reboot for various servers - authored by Southparkfan, assigned to None [19:51:43] No updates for 3 days - https://phabricator.miraheze.org/T4024 - Install elasticsearch - authored by Southparkfan, assigned to Southparkfan [22:04:54] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Puppet has 355 failures. Last run 2 minutes ago with 355 failures. Failed resources (up to 3 shown) [22:05:15] PROBLEM - misc1 Puppet on misc1 is CRITICAL: CRITICAL: Puppet has 26 failures. Last run 2 minutes ago with 26 failures. Failed resources (up to 3 shown) [22:05:18] PROBLEM - misc4 Puppet on misc4 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 2 minutes ago with 10 failures. Failed resources (up to 3 shown) [22:05:58] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Puppet has 350 failures. Last run 2 minutes ago with 350 failures. Failed resources (up to 3 shown): File[private.revi.wiki_private],File[publictestwiki.com],File[publictestwiki.com_private],File[wiki.valentinaproject.org] [22:06:03] PROBLEM - misc2 Puppet on misc2 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 3 minutes ago with 10 failures. Failed resources (up to 3 shown) [22:06:22] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Puppet has 6 failures. Last run 3 minutes ago with 6 failures. Failed resources (up to 3 shown) [22:06:24] PROBLEM - misc3 Puppet on misc3 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 3 minutes ago with 10 failures. Failed resources (up to 3 shown) [22:07:33] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Puppet has 229 failures. Last run 3 minutes ago with 229 failures. Failed resources (up to 3 shown): File[permanentfuturelab.wiki_private],File[private.revi.wiki],File[private.revi.wiki_private],File[publictestwiki.com] [22:07:42] PROBLEM - cp5 Puppet on cp5 is CRITICAL: CRITICAL: Puppet has 108 failures. Last run 3 minutes ago with 108 failures. Failed resources (up to 3 shown): File[spiral.wiki],File[spiral.wiki_private],File[permanentfuturelab.wiki],File[permanentfuturelab.wiki_private] [22:07:50] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Puppet has 352 failures. Last run 2 minutes ago with 352 failures. Failed resources (up to 3 shown): File[private.revi.wiki_private],File[publictestwiki.com],File[publictestwiki.com_private],File[wiki.valentinaproject.org] [22:08:30] PROBLEM - mw3 Puppet on mw3 is CRITICAL: CRITICAL: Puppet has 354 failures. Last run 3 minutes ago with 354 failures. Failed resources (up to 3 shown): File[permanentfuturelab.wiki_private],File[private.revi.wiki],File[private.revi.wiki_private],File[publictestwiki.com] [22:08:44] PROBLEM - test1 Puppet on test1 is CRITICAL: CRITICAL: Puppet has 353 failures. Last run 2 minutes ago with 353 failures. Failed resources (up to 3 shown): File[private.revi.wiki],File[private.revi.wiki_private],File[publictestwiki.com],File[publictestwiki.com_private] [22:09:02] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Puppet has 359 failures. Last run 3 minutes ago with 359 failures. Failed resources (up to 3 shown): File[antiguabarbudacalypso.com],File[antiguabarbudacalypso.com_private],File[spiral.wiki],File[spiral.wiki_private] [22:10:12] [02miraheze/services] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/fjfYy [22:10:13] [02miraheze/services] 07MirahezeSSLBot 03a155460 - BOT: Updating services config for wikis [22:13:14] RECOVERY - misc1 Puppet on misc1 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [22:13:17] RECOVERY - misc4 Puppet on misc4 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [22:14:01] RECOVERY - misc2 Puppet on misc2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:14:02] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures [22:14:22] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:14:23] RECOVERY - misc3 Puppet on misc3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:14:54] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:15:32] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:15:41] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [22:16:32] RECOVERY - mw3 Puppet on mw3 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [22:16:54] RECOVERY - test1 Puppet on test1 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures [22:17:08] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:18:13] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:53:05] hi [23:53:42] at me if any of yall're up and good to answer tech questions