[00:06:54] (03PS2) 10Filippo Giunchedi: Move hhvm_exporter to its own package [software/hhvm_exporter] - 10https://gerrit.wikimedia.org/r/322371 (https://phabricator.wikimedia.org/T147423) [00:06:56] (03PS2) 10Filippo Giunchedi: Debian packaging [software/hhvm_exporter] - 10https://gerrit.wikimedia.org/r/322372 (https://phabricator.wikimedia.org/T147423) [00:31:17] RECOVERY - Esams HTTP 5xx reqs/min on graphite1001 is OK: OK: Less than 1.00% above the threshold [250.0] [01:47:56] PROBLEM - Postgres Replication Lag on maps2002 is CRITICAL: CRITICAL - Rep Delay is: 1804.625613 Seconds [01:48:56] RECOVERY - Postgres Replication Lag on maps2002 is OK: OK - Rep Delay is: 13.22737 Seconds [01:56:26] PROBLEM - puppet last run on lvs3001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [02:16:20] !log l10nupdate@tin scap sync-l10n completed (1.29.0-wmf.3) (duration: 05m 27s) [02:16:39] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:20:37] !log l10nupdate@tin ResourceLoader cache refresh completed at Sat Nov 19 02:20:37 UTC 2016 (duration 4m 18s) [02:21:01] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:24:26] RECOVERY - puppet last run on lvs3001 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [03:03:36] PROBLEM - puppet last run on rcs1002 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [03:22:56] PROBLEM - MariaDB Slave Lag: s1 on dbstore1002 is CRITICAL: CRITICAL slave_sql_lag Replication lag: 788.04 seconds [03:31:36] RECOVERY - puppet last run on rcs1002 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [03:39:16] RECOVERY - cassandra-b CQL 10.192.16.187:9042 on restbase2010 is OK: TCP OK - 0.036 second response time on 10.192.16.187 port 9042 [03:39:56] RECOVERY - MariaDB Slave Lag: s1 on dbstore1002 is OK: OK slave_sql_lag Replication lag: 92.23 seconds [03:48:09] 06Operations, 10Domains, 10Phabricator, 10Traffic: short URL for phabricator - https://phabricator.wikimedia.org/T151094#2807477 (10mmodell) Sure it's possible but it would not be very practical. What is the purpose of a shorter url? Just to save a few keystrokes? Here's a shorter url: http://bit.ly/2g5h... [03:56:16] PROBLEM - puppet last run on graphite1003 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [04:00:15] 06Operations, 10Domains, 10Phabricator, 10Traffic: short URL for phabricator - https://phabricator.wikimedia.org/T151094#2807784 (10greg) 05Open>03declined I see no reason to have a short url for phabricator. It's an unneeded maintenance burden. [04:24:16] RECOVERY - puppet last run on graphite1003 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [04:48:55] PROBLEM - puppet last run on etherpad1001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [05:16:56] RECOVERY - puppet last run on etherpad1001 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [05:31:56] PROBLEM - puppet last run on cp1048 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [05:43:16] PROBLEM - puppet last run on db1001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [05:59:56] RECOVERY - puppet last run on cp1048 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [06:11:16] RECOVERY - puppet last run on db1001 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [06:21:16] PROBLEM - puppet last run on labvirt1012 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [06:42:52] 06Operations, 10Domains, 10Phabricator, 10Traffic: short URL for phabricator - https://phabricator.wikimedia.org/T151094#2807477 (10Dzahn) Possible, yes, but I would not recommend multiple URLs to get to the same resource. At the same time, perhabs predicting myself, i would recommed for the use of the gen... [06:45:49] 06Operations, 10Domains, 10Phabricator, 10Traffic: short URL for phabricator - https://phabricator.wikimedia.org/T151094#2807831 (10Dzahn) please see T44085 ("Wikimedia needs a URL shortener (tracking)") and T108557 ("Review and deploy UrlShortener extension to Wikimedia wikis") and their subtasks and re... [06:49:16] RECOVERY - puppet last run on labvirt1012 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [06:59:47] 06Operations, 10Domains, 10Phabricator, 10Traffic: short URL for phabricator - https://phabricator.wikimedia.org/T151094#2807838 (10Dzahn) https:://w.wiki [07:00:44] 06Operations, 10Domains, 10Phabricator, 10Traffic: short URL for phabricator - https://phabricator.wikimedia.org/T151094#2807840 (10Dzahn) ^ this is a working URL that gets you to https://meta.wikimedia.org/wiki/Special:UrlShortener why it does not link here is another question. [07:10:51] 06Operations, 10Domains, 10Phabricator, 10Traffic: short URL for phabricator - https://phabricator.wikimedia.org/T151094#2807843 (10Dzahn) we got the .wiki domains back in T88873 which was a long ticket which was a continuation of an RT ticket and was pretty much declined. (just full disclosure here). [09:22:57] PROBLEM - puppet last run on analytics1036 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [09:44:05] 06Operations, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Asynchronous processing in production: one queue to rule them all - https://phabricator.wikimedia.org/T149408#2807912 (10Nemo_bis) Using custom undefined jargon makes sure that no users (or non-WMF persons in general) can be involv... [09:51:57] RECOVERY - puppet last run on analytics1036 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [10:07:29] PROBLEM - Host ms-be2025 is DOWN: PING CRITICAL - Packet loss = 100% [10:51:09] PROBLEM - Text HTTP 5xx reqs/min on graphite1001 is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [1000.0] [10:53:09] RECOVERY - Text HTTP 5xx reqs/min on graphite1001 is OK: OK: Less than 1.00% above the threshold [250.0] [10:59:09] PROBLEM - Text HTTP 5xx reqs/min on graphite1001 is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [1000.0] [11:01:09] RECOVERY - Text HTTP 5xx reqs/min on graphite1001 is OK: OK: Less than 1.00% above the threshold [250.0] [11:59:40] (03CR) 10MarcoAurelio: Remove FlaggedRevs autopromotion function at eowiki (031 comment) [mediawiki-config] - 10https://gerrit.wikimedia.org/r/322262 (https://phabricator.wikimedia.org/T150591) (owner: 10MarcoAurelio) [12:20:33] (03PS1) 10Alex Monk: Kill role::beta::uploadservice [puppet] - 10https://gerrit.wikimedia.org/r/322403 [12:20:47] bah, stupid submodules [12:21:13] (03PS2) 10Alex Monk: Kill role::beta::uploadservice [puppet] - 10https://gerrit.wikimedia.org/r/322403 [12:27:34] (03PS1) 10Alex Monk: Kill role::beta::bastion [puppet] - 10https://gerrit.wikimedia.org/r/322404 [12:37:22] (03PS1) 10Alex Monk: Kill role::beta::trebuchet_testing [puppet] - 10https://gerrit.wikimedia.org/r/322405 [12:39:23] (03PS1) 10Alex Monk: Kill beta::config [puppet] - 10https://gerrit.wikimedia.org/r/322406 [12:40:21] (03CR) 10jenkins-bot: [V: 04-1] Kill role::beta::bastion [puppet] - 10https://gerrit.wikimedia.org/r/322404 (owner: 10Alex Monk) [12:41:18] 12:39:29 stderr: fatal: Could not read from remote repository. [12:41:21] 12:39:29 Please make sure you have the correct access rights [12:41:21] 12:39:29 and the repository exists. [12:41:22] wut [12:41:29] (03CR) 10Alex Monk: "recheck" [puppet] - 10https://gerrit.wikimedia.org/r/322404 (owner: 10Alex Monk) [12:50:11] (03PS1) 10Alex Monk: Kill beta::deployaccess [puppet] - 10https://gerrit.wikimedia.org/r/322407 [12:54:24] (03PS5) 10XXN: Fixes for namespace definitions for some Romanian (ro) projects. [mediawiki-config] - 10https://gerrit.wikimedia.org/r/321121 [13:01:38] (03PS6) 10XXN: Fixes for namespace definitions for some Romanian (ro) projects. [mediawiki-config] - 10https://gerrit.wikimedia.org/r/321121 [13:04:50] (03CR) 10XXN: "As I stated above, "Discuţie_Book" is un-necessary and was introduced by mistake probably, thus I changed Discuţie_Book to Book_talk." [mediawiki-config] - 10https://gerrit.wikimedia.org/r/321121 (owner: 10XXN) [13:09:18] (03PS7) 10XXN: Fixes for namespace definitions for some Romanian (ro) projects. [mediawiki-config] - 10https://gerrit.wikimedia.org/r/321121 [13:10:59] PROBLEM - puppet last run on analytics1049 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [13:11:12] (03PS1) 10Alex Monk: beta::autoupdater: Stop wmf-beta-mwconfig-update being a template just to get the staging dir [puppet] - 10https://gerrit.wikimedia.org/r/322408 [13:13:04] (03CR) 10jenkins-bot: [V: 04-1] beta::autoupdater: Stop wmf-beta-mwconfig-update being a template just to get the staging dir [puppet] - 10https://gerrit.wikimedia.org/r/322408 (owner: 10Alex Monk) [13:14:23] (03PS2) 10Alex Monk: beta::autoupdater: Stop wmf-beta-mwconfig-update being a template just to get the staging dir [puppet] - 10https://gerrit.wikimedia.org/r/322408 [13:26:16] 06Operations, 10Domains, 10Phabricator, 10Traffic: short URL for phabricator - https://phabricator.wikimedia.org/T151094#2808005 (10Aklapper) We have `[[phab:T123456]]` on wikis already which is short enough. In general, a rationale is welcome in task descriptions //why// something is requested. [13:29:06] (03CR) 10Nemo bis: [C: 04-1] "Old URLs must continue to function, please add the aliases." [mediawiki-config] - 10https://gerrit.wikimedia.org/r/321121 (owner: 10XXN) [13:38:59] RECOVERY - puppet last run on analytics1049 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [13:56:09] PROBLEM - puppet last run on puppetmaster1002 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:03:02] (03PS1) 10Alex Monk: mediawiki: move redirects site from prod_sites to sites [puppet] - 10https://gerrit.wikimedia.org/r/322413 [14:24:10] RECOVERY - puppet last run on puppetmaster1002 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures [14:34:49] (03PS1) 10Alex Monk: beta sites: copy zerowiki config from prod [puppet] - 10https://gerrit.wikimedia.org/r/322416 [14:34:51] (03PS1) 10Alex Monk: beta: configure loginwiki the same as prod [puppet] - 10https://gerrit.wikimedia.org/r/322417 [14:34:53] (03PS1) 10Alex Monk: beta: Use wikimedia-common.incl for testwiki [puppet] - 10https://gerrit.wikimedia.org/r/322418 [14:34:55] (03PS1) 10Alex Monk: beta: copy metawiki config from prod instead of having our own [puppet] - 10https://gerrit.wikimedia.org/r/322419 [14:36:10] PROBLEM - puppet last run on mc1003 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [14:39:53] (03PS1) 10Alex Monk: Remove bits.wikimedia.org apache config [puppet] - 10https://gerrit.wikimedia.org/r/322420 [15:04:10] RECOVERY - puppet last run on mc1003 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [15:11:19] (03PS1) 10Alex Monk: Remove apache-level HTTPS redirects [puppet] - 10https://gerrit.wikimedia.org/r/322423 [15:21:07] (03PS1) 10Alex Monk: beta: bring remnant.conf closer to prod version [puppet] - 10https://gerrit.wikimedia.org/r/322424 [15:36:10] (03PS1) 10Alex Monk: Consolidate all of the simple wikimedia.org VHosts into two [puppet] - 10https://gerrit.wikimedia.org/r/322425 [16:27:16] PROBLEM - Esams HTTP 5xx reqs/min on graphite1001 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [1000.0] [16:31:16] PROBLEM - Esams HTTP 5xx reqs/min on graphite1001 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [1000.0] [16:36:17] PROBLEM - Esams HTTP 5xx reqs/min on graphite1001 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [1000.0] [16:49:16] PROBLEM - Esams HTTP 5xx reqs/min on graphite1001 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [1000.0] [17:05:16] PROBLEM - salt-minion processes on thumbor1002 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [17:05:46] PROBLEM - dhclient process on thumbor1002 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [17:06:36] RECOVERY - dhclient process on thumbor1002 is OK: PROCS OK: 0 processes with command name dhclient [17:08:16] RECOVERY - salt-minion processes on thumbor1002 is OK: PROCS OK: 1 process with regex args ^/usr/bin/python /usr/bin/salt-minion [18:46:06] PROBLEM - High lag on wdqs1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [1800.0] [18:48:06] PROBLEM - High lag on wdqs1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [1800.0] [19:04:07] PROBLEM - High lag on wdqs1001 is CRITICAL: CRITICAL: 73.33% of data above the critical threshold [1800.0] [19:04:56] PROBLEM - High lag on wdqs1002 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [1800.0] [19:05:10] ^ should we be worried? [19:05:14] no [19:05:21] whats going on with them [19:05:34] I guess they're busy [19:06:04] Reedy i've never seen them report problems before its usually always a db or labs or something prod relatecd [19:08:44] (03PS1) 10Ori.livneh: Changes to coal-web.py should refresh the coal webapp [puppet] - 10https://gerrit.wikimedia.org/r/322432 (https://phabricator.wikimedia.org/T131820) [19:31:06] PROBLEM - High lag on wdqs1001 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [1800.0] [19:33:06] RECOVERY - High lag on wdqs1001 is OK: OK: Less than 30.00% above the threshold [600.0] [19:53:16] 07Puppet, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2808344 (10hashar) p:05Triage>03High [19:53:27] 07Puppet, 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2736876 (10hashar) [19:54:14] 07Puppet, 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2736876 (10hashar) @mobrovac here is the task related to the instance... [19:54:26] 07Puppet, 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2808351 (10hashar) [20:01:23] 07Puppet, 10Continuous-Integration-Infrastructure, 06Labs, 10Labs-Infrastructure, 07Beta-Cluster-reproducible: New instance have broken puppet configuration when using puppetmaster standalone - https://phabricator.wikimedia.org/T148929#2808357 (10hashar) Note that once one has deleted `/var/lib/puppet/ss... [20:13:56] RECOVERY - High lag on wdqs1002 is OK: OK: Less than 30.00% above the threshold [600.0] [20:57:26] anything i can help with at the moment? [20:57:34] -operations wise [20:59:16] PROBLEM - puppet last run on silver is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [21:02:23] (03CR) 10Aaron Schulz: Remove FlaggedRevs autopromotion function at eowiki (031 comment) [mediawiki-config] - 10https://gerrit.wikimedia.org/r/322262 (https://phabricator.wikimedia.org/T150591) (owner: 10MarcoAurelio) [21:05:36] (03PS1) 10ArielGlenn: move IncrDumpLib to miscdumpslib and rename classes and methods accordingly [dumps] - 10https://gerrit.wikimedia.org/r/322450 (https://phabricator.wikimedia.org/T133547) [21:05:39] (03PS1) 10ArielGlenn: move config defaults to a separate method [dumps] - 10https://gerrit.wikimedia.org/r/322451 (https://phabricator.wikimedia.org/T133547) [21:05:41] (03PS1) 10ArielGlenn: move some adds/changes-specific code out of miscdumpslib [dumps] - 10https://gerrit.wikimedia.org/r/322452 (https://phabricator.wikimedia.org/T133547) [21:05:43] (03PS1) 10ArielGlenn: move some methods into miscdumpslib that will be reused for other misc dumps [dumps] - 10https://gerrit.wikimedia.org/r/322453 (https://phabricator.wikimedia.org/T133547) [21:05:57] (03CR) 10jenkins-bot: [V: 04-1] move IncrDumpLib to miscdumpslib and rename classes and methods accordingly [dumps] - 10https://gerrit.wikimedia.org/r/322450 (https://phabricator.wikimedia.org/T133547) (owner: 10ArielGlenn) [21:06:00] (03CR) 10jenkins-bot: [V: 04-1] move some adds/changes-specific code out of miscdumpslib [dumps] - 10https://gerrit.wikimedia.org/r/322452 (https://phabricator.wikimedia.org/T133547) (owner: 10ArielGlenn) [21:06:02] (03CR) 10jenkins-bot: [V: 04-1] move config defaults to a separate method [dumps] - 10https://gerrit.wikimedia.org/r/322451 (https://phabricator.wikimedia.org/T133547) (owner: 10ArielGlenn) [21:06:06] (03CR) 10jenkins-bot: [V: 04-1] move some methods into miscdumpslib that will be reused for other misc dumps [dumps] - 10https://gerrit.wikimedia.org/r/322453 (https://phabricator.wikimedia.org/T133547) (owner: 10ArielGlenn) [21:11:17] (03PS2) 10ArielGlenn: move some adds/changes-specific code out of miscdumpslib [dumps] - 10https://gerrit.wikimedia.org/r/322452 (https://phabricator.wikimedia.org/T133547) [21:11:19] (03PS2) 10ArielGlenn: move some methods into miscdumpslib that will be reused for other misc dumps [dumps] - 10https://gerrit.wikimedia.org/r/322453 (https://phabricator.wikimedia.org/T133547) [21:11:21] (03PS2) 10ArielGlenn: move IncrDumpLib to miscdumpslib and rename classes and methods accordingly [dumps] - 10https://gerrit.wikimedia.org/r/322450 (https://phabricator.wikimedia.org/T133547) [21:11:23] (03PS2) 10ArielGlenn: move config defaults to a separate method [dumps] - 10https://gerrit.wikimedia.org/r/322451 (https://phabricator.wikimedia.org/T133547) [21:11:26] (03PS1) 10ArielGlenn: fix new flake8 whines [dumps] - 10https://gerrit.wikimedia.org/r/322484 [21:11:32] hahaha [21:12:03] that'll teach it to whine at me for flake8 being upgraded and rejecting my new changes based on code in the repo I didn't touch [21:28:16] RECOVERY - puppet last run on silver is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [21:48:21] apergos lol you crashed grrrit-wm with all your patches [21:48:33] proud of it too :-P [21:48:36] it was only 4 [21:48:43] lol [21:48:49] then there's some new version of flake 8 that choked on the code already in the repo [21:48:52] so then it was 5 :-P [21:49:17] apergos you break it you buy it [21:50:20] apergos i belive it may have hit a known issue with the npm package for irc, anyways its uptime was far better then prevously [21:50:38] almost a week [21:50:45] paladox can you tell me what issue this in in the gerrit chan pls [21:52:32] at midnight I"m not buyin nothin [21:55:57] lol [21:59:22] apergos you caused so many errors it had to create a file called apergos's errorlog and its about 1tb xD [21:59:40] poor thing :-P [22:02:39] apergos you caused flood protection to go off lol [22:02:45] eh no [22:02:51] gerrit-wm did that all by itself [22:03:16] I idn't tell it to use a new flake8 on existing code and whine about new errors for every change I wanted to submit [22:03:28] oh [22:03:46] the log is too big for me to see why it failed [22:03:50] :-D [22:03:58] well I added a patch to the existing code [22:04:12] for the new "expected 2 lines, found 1" messages [22:04:18] problem sovled [22:04:20] *solved [22:04:33] Oh [22:04:36] there were 6-7 files with one such error each [22:05:07] oh [22:05:21] really it's no big deal, it was just funny [22:05:32] yep [22:10:22] PROBLEM - puppet last run on cp3033 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [22:10:49] grrrit-wm: status [22:11:26] excuse paladox i think hes gona bit tipsy [22:11:34] lol nope [22:24:24] ATTENTION grrit-wm will be done for an unexpected maintence (sorry!) [22:24:31] down* [22:30:25] (03CR) 10Paladox: "recheck" [puppet] - 10https://gerrit.wikimedia.org/r/322357 (owner: 10Paladox) [22:32:34] (03PS10) 10Paladox: Gerrit: Enable concurrent collector [puppet] - 10https://gerrit.wikimedia.org/r/316983 (https://phabricator.wikimedia.org/T148478) [22:32:48] (03CR) 10Paladox: [C: 031] "Testing" [puppet] - 10https://gerrit.wikimedia.org/r/316983 (https://phabricator.wikimedia.org/T148478) (owner: 10Paladox) [22:32:55] (03CR) 10Paladox: [C: 04-1] Gerrit: Enable concurrent collector [puppet] - 10https://gerrit.wikimedia.org/r/316983 (https://phabricator.wikimedia.org/T148478) (owner: 10Paladox) [22:32:57] (03CR) 10Paladox: Gerrit: Enable concurrent collector [puppet] - 10https://gerrit.wikimedia.org/r/316983 (https://phabricator.wikimedia.org/T148478) (owner: 10Paladox) [22:39:24] RECOVERY - puppet last run on cp3033 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [22:39:58] (03PS1) 10Hashar: contint: basic role class for logstash [puppet] - 10https://gerrit.wikimedia.org/r/322488 [22:40:57] (03CR) 10Hashar: "Instance is at : https://horizon.wikimedia.org/project/instances/d718d1a0-08e7-4544-b1da-e226336db495/" [puppet] - 10https://gerrit.wikimedia.org/r/322488 (owner: 10Hashar) [22:42:43] (03CR) 10Hashar: "And I probably dont need Logstash at all bah..." [puppet] - 10https://gerrit.wikimedia.org/r/322488 (owner: 10Hashar) [22:42:48] doesnt jenkins or something create a commit in phab for whatever jenkins-bot merges in gerrit? [22:48:02] yes [22:49:38] (03CR) 10BryanDavis: [C: 04-1] Adding nick change functionality automatically (032 comments) [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [22:51:40] (03PS5) 10Zppix: Adding nick change functionality automatically [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [22:51:54] bd808 ^ [22:52:20] paladox weird one of my own repos (zppixbot) on gerrit didnt get a commit from the bot on phab [22:52:37] Zppix, check the status [22:53:02] zuul shows it empty when i checked few mins ago [22:53:04] Zppix https://phabricator.wikimedia.org/diffusion/LTZB/manage/status/ [22:53:09] i meant on phabricator [22:53:21] its been 54m [22:53:25] but its all green [22:53:29] 54m freq [22:53:40] Yep [22:53:52] but no commit from a recent merge via jenkins [22:53:53] -bot [22:53:55] It should hopefully update in a min [22:54:00] Update Frequency 54 m, 23 s · Learn More [22:54:05] Updates OK Last updated Sat, Nov 19, 9:57 PM (56 m, 10 s ago). [22:55:11] it was merged at 4:15 according to the gerrit change [22:55:32] there it goes [22:55:51] Oh [22:56:17] (03CR) 10BryanDavis: "recheck" [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [22:56:48] (03CR) 10jenkins-bot: [V: 04-1] Adding nick change functionality automatically [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [22:59:54] bd808 i cant understand why it failed? [23:00:16] Zppix you have whitespaces [23:00:20] 4 spaces on line 77 [23:00:33] more on line 71 [23:00:59] (03CR) 10Paladox: Adding nick change functionality automatically (032 comments) [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [23:03:05] ok [23:03:43] i thought it needed a blank line between Def [23:06:01] Yes, but not a whitespace [23:06:32] Zppix ^^ [23:09:23] (03PS6) 10Zppix: Adding nick change functionality automatically [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [23:11:24] (03CR) 10Paladox: "recheck" [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [23:11:54] (03CR) 10jenkins-bot: [V: 04-1] Adding nick change functionality automatically [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [23:12:02] (03CR) 10Paladox: Adding nick change functionality automatically (031 comment) [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [23:16:34] Zppix ^^ [23:16:50] (03PS7) 10Zppix: Adding nick change functionality automatically [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [23:17:00] god damn TOX is needy [23:17:06] Ok [23:17:32] (03CR) 10Paladox: "recheck" [wikimedia/bots/jouncebot] - 10https://gerrit.wikimedia.org/r/322037 (https://phabricator.wikimedia.org/T150916) (owner: 10Gerrit Patch Uploader) [23:30:58] PROBLEM - puppet last run on mc1024 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [23:41:48] (03PS1) 10ArielGlenn: start moving adds/changes methods out to incr_dumps module [dumps] - 10https://gerrit.wikimedia.org/r/322491 (https://phabricator.wikimedia.org/T133547) [23:42:15] (03CR) 10jenkins-bot: [V: 04-1] start moving adds/changes methods out to incr_dumps module [dumps] - 10https://gerrit.wikimedia.org/r/322491 (https://phabricator.wikimedia.org/T133547) (owner: 10ArielGlenn) [23:58:57] RECOVERY - puppet last run on mc1024 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures