[00:00:06] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 707.599976 [00:00:36] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 678.366638 [00:01:16] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 722.466675 [00:01:16] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 752.599976 [00:06:06] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [00:07:16] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [00:07:16] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [00:07:36] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [00:11:17] hoo: select count(*) from logging where log_title like "EditWikiSets/0"; says 10 still [00:11:34] there are 304 entries that still point to EditWikiSets in general [00:12:42] legoktm: Oh crap [00:13:47] Guess I'm going to leave those alone as they are legacy [00:14:01] Even EditWikiSets/0? [00:14:09] mh [00:14:36] I think we should fix the /0 ones, and not the others [00:14:57] mh, will talk to Chris about these first [00:15:11] ok [00:19:46] PROBLEM - Puppet freshness on elastic1017 is CRITICAL: Last successful Puppet run was Sat 07 Jun 2014 21:19:14 UTC [00:28:11] hoo, legoktm, right, and perhaps leave them with EditWikiSets but with the corrected number [01:02:29] (03PS1) 10Hoo man: Allow searching NS_MODULE with mwgrep [operations/puppet] - 10https://gerrit.wikimedia.org/r/138216 [01:22:39] (03PS2) 10Hoo man: Allow searching NS_MODULE with mwgrep [operations/puppet] - 10https://gerrit.wikimedia.org/r/138216 [01:26:36] PROBLEM - HTTP 5xx req/min on tungsten is CRITICAL: CRITICAL: 14.29% of data exceeded the critical threshold [500.0] [01:39:36] RECOVERY - HTTP 5xx req/min on tungsten is OK: OK: Less than 1.00% data above the threshold [250.0] [01:52:46] PROBLEM - Puppet freshness on labstore1001 is CRITICAL: Last successful Puppet run was Tue 03 Jun 2014 16:21:49 UTC [01:59:58] Hi, anybody around? Looking for some advice on how to find all project API URLs using project names... Thanks in advance! [02:00:22] AndyRussG: ?? [02:00:52] Hi Betacommand... It's for a patch I'm working on for Wikimetrics [02:01:01] AndyRussG, what format do you have exactly? [02:01:17] Hi Krenair [02:02:04] I just have the string names of all the projects, as in "enwiki", "anwiki", "akwiktionary", etc [02:02:18] okay, you have the wiki IDs (in WMF's case, basically just database names) [02:02:20] Use https://meta.wikimedia.org/w/api.php?action=sitematrix [02:02:24] As pulled for example from https://noc.wikimedia.org/conf/s3.dblist [02:02:47] You can specify format=json to get it in JSON format or format=jsonfm to get it formatted nicely as JSON [02:04:07] Oooo fantastic, that's exactly what I need :) [02:05:45] Krenair: thanks so much!! Also thanks Betacommand :) [02:05:50] you're welcome [02:05:57] 'project names' isn't particularly helpful [02:06:02] AndyRussG: np [02:06:42] AndyRussG: Krenair said what I meant :P [02:07:02] You could have 'English Wikipedia', enwiki, en-wp, enwikipedia, or probably many other different formats [02:07:15] Right [02:07:36] Yeah wasn't quite sure what to call the strings that I had 8p [02:08:17] describing the format would have helped [02:09:13] K, I'll know for next time :) [02:09:37] On Wikimetrics on-site doc they're just called "language or project codes": https://metrics.wmflabs.org/cohorts/upload [02:11:07] But yeah, as Wikimetrics normally pulls via SQL, what it usually needs are database names [02:11:22] (now adding something that'll get some info via a Web API instead) [02:11:51] Thanks again! [02:11:59] Krenair: Heya [02:12:10] hi [02:12:56] ori was nice to merge a change of mine in WikimediaEvents for jqmigrate tracking (like we have to mwdeprecate), but I think he forgot that it wasn't merged in core yet. is that a change you'd feel comfortable revieweing? [02:13:06] https://gerrit.wikimedia.org/r/#/c/133235/ [02:13:30] (whenever a reference is made to e.g. $.browser it not only logs to console but emits mw.track, like we do for accessing mw.user.name already) [02:15:13] !log LocalisationUpdate completed (1.24wmf7) at 2014-06-08 02:14:10+00:00 [02:15:23] Logged the message, Master [02:17:12] Krinkle, the WikimediaEvents depends on this and will break stuff if it's not present? [02:17:36] No, tracking is all soft dependencies. Listens for whatever is there based in string values. [02:17:39] No method signatures [02:17:50] So it will listen for jqmigrate events that simply won't yet happen. [02:18:10] it's all push based too, so it won't even pull or poll, totally cheap and harmless. [02:18:11] ah right, yeah I see the commit [02:19:00] Krenair: Betacommand: is the Web API address for all WMF wikis always going to "base_url_whatever/w/api.php" ? [02:19:15] I believe so [02:19:27] One thing I'm worried for though is that this references mw.* from within jquery files. That in itself is not an issue, except that this particular jquery/* file is actually loaded _before_ mediawiki.js. In fact, right before it (in the same request even), so it's not like anything could ever call into this without it being there. [02:20:15] I guess the only way to fetch it would be to dive into some config files from noc.wikimedia.org [02:20:36] AndyRussG: There have been a few exceptions in the past. I'd say the path to api.php (unlike any other path) is considered itself an API that will never break (like domain names and cool uris not changing). It's bound to keep working for any one that works now. [02:20:46] AndyRussG: its ..org/w/api.php [02:20:47] However you should avoid assuming it will work for any wiki. [02:21:15] at least for wmf wikis [02:21:18] e.g. you can store urls to api.php, I wouldn't store just the domainname and construct it on the fly. Depends on what your source is. [02:21:34] Betacommand, well yeah, but some projects do their multilingual stuff all in a single wiki [02:21:39] Krinkle: correct and if its a wmf wiki or not [02:21:41] for example mediawiki and wikidata [02:21:46] Be careful there with exceptions like metawiki (like enwiki) where it is short for wikimedia.org, not wikipedia.org, and stuff like mediawiki.org and wikimediafoundation.org [02:21:53] Krenair: example? [02:22:02] for example mediawiki and wikidata [02:22:16] Betacommand: What is your data source for constructing the urls, sitematrix? [02:22:20] or the meta_p database? [02:22:35] We're helping AndyRussG [02:22:46] He's looking at the dblist files and then going to sitematrix [02:22:48] Krinkle: years and years of using them :p [02:23:38] Krinkle, does the metawiki_p db have this info in...? [02:23:53] Hm.. avoid using the dblist files for anything. they can be out of sync and are easy to misintrpet (they'll also change from time to time in their format or location). sitematrix should be usable on its own. I guess you're just looking at the dblist, not using them in the application? [02:24:05] Krenair: not metawiki_p, meta-P [02:24:07] Krenair: meta_p is a special labs database [02:24:08] tool labs database [02:24:22] meta_p.wiki [02:24:28] oh, interesting [02:24:31] MMm well the dblist usage is what's already there in wikimetrics [02:24:54] | dbname | lang | name | family | url | size | slice | is_closed | has_echo | has_flaggedrevs | has_visualeditor | has_wikidata | [02:24:54] | aawiki | aa | Wikipedia | wikipedia | http://aa.wikipedia.org | 1 | s3.labsdb | 1 | 1 | 0 | 1 | 1 | [02:25:09] https://git.wikimedia.org/blob/operations%2Fmediawiki-config.git/e6bf6beb80d69110f91817fe05d7c2d9def43f85/wmf-config%2FCommonSettings.php#L275 [02:25:35] be careful of entries like centralauth: [02:25:35] | centralauth | en | NULL | centralauth | NULL | 1 | s7.labsdb | 0 | 0 | 0 | 0 | 0 | [02:25:50] | mediawikiwiki | en | MediaWiki | special | http://www.mediawiki.org | 1 | s3.labsdb | 0 | 1 | 1 | 1 | 0 | [02:26:20] Ew, what? [02:26:24] Who decided to put that in there? [02:26:41] well, it's useful [02:26:48] ... not really [02:26:53] it's not a wiki [02:26:53] meta_p is essentially what tools use to know what database to query [02:26:57] centralauth is a database [02:27:03] yes but not a wiki [02:27:07] and one needs to know which slice it is on [02:27:11] true [02:27:16] but it's database info, not wiki info [02:27:20] not strictly anyway [02:27:21] they have to be told about meta_p, it's not unreasonable that they should also know about centralauth if they need it [02:27:38] I know [02:27:40] http://wmbug.com/65789 [02:27:53] https://github.com/Krinkle/toollabs-base/blob/master/src/LabsDB.php#L213-L230 [02:28:24] !log LocalisationUpdate completed (1.24wmf8) at 2014-06-08 02:27:21+00:00 [02:28:29] Logged the message, Master [02:33:07] Really mainly important for wikis that folks will want to use Wikimetrics on [02:39:07] Wikimetrics is just taking from https://noc.wikimedia.org/conf/s{1-7}.dblist [02:41:05] https://git.wikimedia.org/blob/analytics%2Fwikimetrics/eb3ccb8854ded032fd195b35e06c48e51762d376/wikimetrics%2Fdatabase.py#L47 [02:43:06] The list as it's munged by WM is actually in the html source of https://metrics.wmflabs.org/cohorts/upload, in the data-source attribute of [02:43:31] So I'll just go with what's there and use sitematrix to grab the base URLs [02:56:28] Oh mow, I just found the best thing since sliced bread [02:56:32] rmate and rsub [02:56:46] So in there the types of sites are wikipedia, wikibooks, wikimedia, wikinews, wikiquote, wikisource, wikiversity, wikivoyage, wiktionary [02:56:54] remote editing of files over ssh in your local text editor. Without any slow mounting of ssh drives or ftp hoops. [02:57:16] right from the command line, like vim, and it opens up your local editor and keeps STDIN blocked until it is ready [02:57:23] perfect [02:57:29] absolu friggin lutely [02:58:11] nice! [02:58:45] Krinkle, can you add me to the reviewer list for that commit? [03:00:19] done [03:02:38] ok. I'll look at it later [03:20:46] PROBLEM - Puppet freshness on elastic1017 is CRITICAL: Last successful Puppet run was Sat 07 Jun 2014 21:19:14 UTC [03:22:34] !log LocalisationUpdate ResourceLoader cache refresh completed at Sun Jun 8 03:21:28 UTC 2014 (duration 21m 27s) [03:22:39] Logged the message, Master [04:00:37] PROBLEM - Disk space on dataset1001 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [04:53:46] PROBLEM - Puppet freshness on labstore1001 is CRITICAL: Last successful Puppet run was Tue 03 Jun 2014 16:21:49 UTC [06:01:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:03:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:05:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:07:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:09:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:11:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:13:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:15:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:17:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:19:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:21:28] PROBLEM - Puppet freshness on elastic1017 is CRITICAL: Last successful Puppet run was Sat 07 Jun 2014 21:19:14 UTC [06:21:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:23:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:25:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:27:28] PROBLEM - Puppet freshness on thallium is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 05:58:32 UTC [06:28:38] RECOVERY - Puppet freshness on thallium is OK: puppet ran at Sun Jun 8 06:28:31 UTC 2014 [07:53:55] PROBLEM - Puppet freshness on labstore1001 is CRITICAL: Last successful Puppet run was Tue 03 Jun 2014 16:21:49 UTC [08:20:55] PROBLEM - RAID on searchidx1001 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [08:21:45] RECOVERY - RAID on searchidx1001 is OK: OK: optimal, 1 logical, 4 physical [09:21:55] PROBLEM - Puppet freshness on elastic1017 is CRITICAL: Last successful Puppet run was Sat 07 Jun 2014 21:19:14 UTC [10:28:25] PROBLEM - Disk space on analytics1020 is CRITICAL: DISK CRITICAL - free space: /var/lib/hadoop/data/e 74667 MB (3% inode=99%): /var/lib/hadoop/data/d 102908 MB (5% inode=99%): /var/lib/hadoop/data/k 117588 MB (6% inode=99%): /var/lib/hadoop/data/c 99479 MB (5% inode=99%): /var/lib/hadoop/data/j 117619 MB (6% inode=99%): /var/lib/hadoop/data/f 86033 MB (4% inode=99%): [10:37:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 48.633335 [10:38:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [10:54:55] PROBLEM - Puppet freshness on labstore1001 is CRITICAL: Last successful Puppet run was Tue 03 Jun 2014 16:21:49 UTC [10:56:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 387.733337 [10:56:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1645.533325 [10:57:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 2592.866699 [10:57:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [10:57:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [10:58:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:13:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1437.733276 [11:13:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1573.866699 [11:13:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1456.900024 [11:14:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:14:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1467.266724 [11:14:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:15:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:17:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:29:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1078.06665 [11:30:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1160.533325 [11:30:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1277.533325 [11:30:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1228.366699 [11:34:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:34:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:35:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:36:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:50:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1314.699951 [11:50:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1316.5 [11:50:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1105.966675 [11:51:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1044.533325 [11:56:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:56:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:57:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [11:57:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:12:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 887.599976 [12:13:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1003.200012 [12:14:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1027.233276 [12:14:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1030.06665 [12:20:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:20:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:20:06] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:21:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:22:55] PROBLEM - Puppet freshness on elastic1017 is CRITICAL: Last successful Puppet run was Sat 07 Jun 2014 21:19:14 UTC [12:37:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1230.400024 [12:38:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1162.199951 [12:38:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1291.0 [12:39:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1160.599976 [12:44:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:44:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:46:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:46:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [12:57:35] PROBLEM - Varnishkafka Delivery Errors on cp4004 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1714.333374 [12:58:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1330.93335 [12:59:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 604.333313 [12:59:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 598.366638 [13:00:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1059.300049 [13:01:35] RECOVERY - Varnishkafka Delivery Errors on cp4004 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:02:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1027.599976 [13:03:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 929.799988 [13:03:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 866.666687 [13:03:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:05:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 930.833313 [13:05:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1553.733276 [13:06:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1418.533325 [13:07:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 618.599976 [13:07:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 604.599976 [13:08:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:08:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 649.93335 [13:09:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:09:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 642.93335 [13:10:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:10:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1460.300049 [13:11:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:11:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:11:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1446.0 [13:11:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:12:35] PROBLEM - Varnishkafka Delivery Errors on cp4001 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1642.599976 [13:12:45] PROBLEM - Varnishkafka Delivery Errors on cp1065 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 332.066681 [13:13:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:13:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:14:35] PROBLEM - Varnishkafka Delivery Errors on cp4004 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1644.633301 [13:16:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 655.200012 [13:16:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 626.93335 [13:16:35] RECOVERY - Varnishkafka Delivery Errors on cp4001 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:16:35] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1575.5 [13:16:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:17:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:17:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:17:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 651.866638 [13:17:45] PROBLEM - Varnishkafka Delivery Errors on cp1055 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 413.466675 [13:17:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:18:35] RECOVERY - Varnishkafka Delivery Errors on cp4004 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:18:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1596.400024 [13:18:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 815.5 [13:18:55] PROBLEM - Varnishkafka Delivery Errors on cp1054 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 399.433319 [13:19:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 622.333313 [13:19:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:19:35] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:20:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:20:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 578.266663 [13:23:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:24:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1052.733276 [13:24:35] PROBLEM - Varnishkafka Delivery Errors on cp4003 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1674.599976 [13:24:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1391.966675 [13:25:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1047.633301 [13:25:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 646.733337 [13:25:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1675.43335 [13:25:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1663.166626 [13:26:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:26:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:27:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 250.833328 [13:27:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:27:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:28:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 923.400024 [13:28:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1558.866699 [13:28:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 278.966675 [13:28:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 274.766663 [13:28:35] RECOVERY - Varnishkafka Delivery Errors on cp4003 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:29:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:29:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1214.599976 [13:29:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 753.0 [13:29:35] PROBLEM - Varnishkafka Delivery Errors on cp4001 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1488.06665 [13:29:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1524.333374 [13:29:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 245.199997 [13:29:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:29:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:30:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 465.233337 [13:30:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:31:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:31:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:32:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:32:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 267.333344 [13:32:35] PROBLEM - Varnishkafka Delivery Errors on cp4004 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1427.199951 [13:32:36] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1559.06665 [13:32:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 272.5 [13:33:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:34:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 306.733337 [13:34:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:34:35] RECOVERY - Varnishkafka Delivery Errors on cp4001 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:35:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:35:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:36:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:36:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 285.466675 [13:36:36] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:36:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1762.43335 [13:37:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:37:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 277.266663 [13:37:35] RECOVERY - Varnishkafka Delivery Errors on cp4004 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:37:55] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 465.799988 [13:38:45] RECOVERY - Varnishkafka Delivery Errors on cp1065 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:38:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 232.633331 [13:39:35] PROBLEM - Varnishkafka Delivery Errors on cp4003 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1749.56665 [13:40:45] RECOVERY - Varnishkafka Delivery Errors on cp1055 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:41:05] RECOVERY - Varnishkafka Delivery Errors on cp1054 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:41:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:41:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 262.0 [13:41:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:41:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1543.266724 [13:41:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1557.199951 [13:41:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:42:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 235.46666 [13:43:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:43:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 284.666656 [13:43:35] RECOVERY - Varnishkafka Delivery Errors on cp4003 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:43:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 137.899994 [13:43:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 666.900024 [13:44:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:44:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 290.933319 [13:45:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1771.599976 [13:45:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:45:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:46:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1219.133301 [13:46:36] PROBLEM - Varnishkafka Delivery Errors on cp4001 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1570.56665 [13:46:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:46:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 140.600006 [13:47:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1173.800049 [13:47:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:48:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:48:35] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1628.233276 [13:48:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 135.566666 [13:49:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:49:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 268.666656 [13:49:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1521.400024 [13:50:35] PROBLEM - Varnishkafka Delivery Errors on cp4004 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1338.733276 [13:50:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:50:45] RECOVERY - Varnishkafka Delivery Errors on cp1053 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:51:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1146.300049 [13:51:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:51:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 400.0 [13:51:35] RECOVERY - Varnishkafka Delivery Errors on cp4001 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:52:06] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1060.300049 [13:53:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:53:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:53:35] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:53:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1763.866699 [13:54:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:54:35] PROBLEM - Varnishkafka Delivery Errors on cp4003 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 2081.466553 [13:54:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:54:55] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:55:36] RECOVERY - Varnishkafka Delivery Errors on cp4004 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:55:46] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:55:55] PROBLEM - Puppet freshness on labstore1001 is CRITICAL: Last successful Puppet run was Tue 03 Jun 2014 16:21:49 UTC [13:56:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:57:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:57:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:58:35] RECOVERY - Varnishkafka Delivery Errors on cp4003 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:58:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:59:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:59:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [13:59:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 185.5 [13:59:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 997.06665 [13:59:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 134.199997 [13:59:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1676.033325 [13:59:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 330.533325 [14:00:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:00:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 134.600006 [14:00:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:00:56] PROBLEM - Puppet freshness on db1007 is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 11:00:14 UTC [14:02:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 239.866669 [14:02:45] PROBLEM - Varnishkafka Delivery Errors on cp1065 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 110.366669 [14:02:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 417.399994 [14:03:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 191.800003 [14:03:45] PROBLEM - Varnishkafka Delivery Errors on cp4001 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1490.0 [14:03:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:04:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:04:35] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1701.033325 [14:04:45] PROBLEM - Varnishkafka Delivery Errors on cp1055 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 202.366669 [14:04:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:04:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:05:05] PROBLEM - Varnishkafka Delivery Errors on cp1054 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 172.399994 [14:05:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 375.533325 [14:05:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:05:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:07:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1303.699951 [14:07:45] PROBLEM - Varnishkafka Delivery Errors on cp4004 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1558.099976 [14:07:45] RECOVERY - Varnishkafka Delivery Errors on cp4001 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:07:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 963.366638 [14:08:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1282.900024 [14:08:16] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 202.166672 [14:08:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:09:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:10:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 2039.833374 [14:10:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 220.899994 [14:11:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:11:46] RECOVERY - Varnishkafka Delivery Errors on cp4004 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:11:46] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 117.76667 [14:12:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1150.733276 [14:12:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 267.0 [14:12:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 205.733337 [14:13:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:13:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:13:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:13:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:14:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1295.333374 [14:14:55] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 245.600006 [14:15:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 291.466675 [14:15:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:16:25] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 277.166656 [14:16:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 236.333328 [14:16:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1167.199951 [14:17:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:17:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1563.300049 [14:17:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 66.866669 [14:17:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:18:09] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:18:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 190.133331 [14:18:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:18:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 142.366669 [14:19:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1926.800049 [14:19:45] PROBLEM - Varnishkafka Delivery Errors on cp4001 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1921.56665 [14:20:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:21:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:21:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 182.066666 [14:22:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:22:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 225.833328 [14:22:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1337.633301 [14:22:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:22:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:23:45] RECOVERY - Varnishkafka Delivery Errors on cp4001 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:23:45] PROBLEM - Varnishkafka Delivery Errors on cp4004 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1703.866699 [14:23:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:24:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:25:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 105.26667 [14:26:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1580.733276 [14:26:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1035.099976 [14:26:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1654.633301 [14:26:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 381.933319 [14:27:45] RECOVERY - Varnishkafka Delivery Errors on cp4004 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:28:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1404.733276 [14:28:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:29:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:31:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:31:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:31:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:31:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:32:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:32:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:33:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1143.56665 [14:33:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:33:45] PROBLEM - Varnishkafka Delivery Errors on cp4001 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1263.900024 [14:33:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 509.700012 [14:34:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1184.900024 [14:34:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:34:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:34:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1262.400024 [14:34:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 773.93335 [14:35:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:35:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:36:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 275.133331 [14:36:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 118.966667 [14:36:45] RECOVERY - Varnishkafka Delivery Errors on cp4001 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:36:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 115.26667 [14:37:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 137.066666 [14:37:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:37:45] PROBLEM - Varnishkafka Delivery Errors on cp4004 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1106.633301 [14:37:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 155.566666 [14:38:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:38:45] RECOVERY - Varnishkafka Delivery Errors on cp4004 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:38:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 80.23333 [14:38:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:39:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 270.133331 [14:39:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:39:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:39:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:39:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:39:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 834.799988 [14:40:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:40:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:40:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:41:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 554.93335 [14:43:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1481.300049 [14:43:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 944.5 [14:44:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 289.433319 [14:46:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1413.0 [14:46:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 77.133331 [14:46:45] PROBLEM - Varnishkafka Delivery Errors on cp4001 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 37.266666 [14:46:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:47:45] RECOVERY - Varnishkafka Delivery Errors on cp4001 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:47:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:47:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:47:55] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:48:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:48:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 219.866669 [14:48:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 243.333328 [14:48:45] RECOVERY - Varnishkafka Delivery Errors on cp1055 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:48:45] RECOVERY - Varnishkafka Delivery Errors on cp1053 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:48:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 212.53334 [14:49:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 196.433334 [14:49:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 215.199997 [14:49:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:50:05] RECOVERY - Varnishkafka Delivery Errors on cp1054 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:50:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1266.733276 [14:50:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 86.166664 [14:50:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 472.133331 [14:51:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:51:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:51:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:52:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1374.133301 [14:52:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 232.53334 [14:52:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 360.733337 [14:53:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1129.133301 [14:53:45] RECOVERY - Varnishkafka Delivery Errors on cp1065 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:54:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:55:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:55:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 273.933319 [14:56:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:56:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 181.666672 [14:56:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:56:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 213.933334 [14:56:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:56:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:56:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 192.866669 [14:57:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:57:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 152.833328 [14:57:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:58:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:58:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 328.733337 [14:58:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [14:59:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1464.633301 [15:00:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1545.5 [15:00:35] RECOVERY - Puppet freshness on db1007 is OK: puppet ran at Sun Jun 8 15:00:26 UTC 2014 [15:00:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:00:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 130.633331 [15:01:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:01:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 131.166672 [15:02:12] how helpful [15:04:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1424.900024 [15:04:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:04:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:04:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:04:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:05:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:05:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 291.733337 [15:05:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:07:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:07:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:07:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 35.966667 [15:07:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:08:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 82.099998 [15:08:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:08:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 79.566666 [15:09:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:09:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 85.23333 [15:10:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:10:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:11:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 225.633331 [15:11:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1375.199951 [15:11:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1318.266724 [15:11:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:11:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:12:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 108.23333 [15:13:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 333.166656 [15:13:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 238.133331 [15:14:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 115.566666 [15:14:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 160.46666 [15:14:55] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 135.699997 [15:15:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 114.066666 [15:15:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:15:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:15:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 133.933334 [15:16:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1982.900024 [15:16:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:16:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:16:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:16:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 474.733337 [15:16:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 201.199997 [15:17:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:17:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:17:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 195.199997 [15:17:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 48.266666 [15:18:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:18:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 303.0 [15:18:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1423.199951 [15:18:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:20:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:20:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 253.100006 [15:20:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:20:45] PROBLEM - Varnishkafka Delivery Errors on cp1055 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 113.0 [15:21:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1584.166626 [15:21:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:22:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 247.46666 [15:22:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:23:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 303.933319 [15:23:36] i don't even know what Varnishkafka is, but i do know that it's continuously shitting itself [15:23:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 228.0 [15:23:55] PROBLEM - Puppet freshness on elastic1017 is CRITICAL: Last successful Puppet run was Sat 07 Jun 2014 21:19:14 UTC [15:24:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:25:45] PROBLEM - Varnishkafka Delivery Errors on cp1065 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 77.366669 [15:26:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:26:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:27:05] PROBLEM - Varnishkafka Delivery Errors on cp1054 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 106.199997 [15:27:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 48.966667 [15:28:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:28:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 206.96666 [15:28:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:28:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 59.066666 [15:28:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:29:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1497.0 [15:29:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1434.366699 [15:29:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 731.56665 [15:29:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 83.300003 [15:30:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 87.800003 [15:30:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 344.933319 [15:31:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 322.666656 [15:31:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1782.666626 [15:31:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:31:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 358.933319 [15:31:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:31:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:32:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 78.099998 [15:33:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:33:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:34:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:34:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:34:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 190.066666 [15:34:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:34:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 96.133331 [15:35:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:36:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:36:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 60.133335 [15:37:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:37:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 226.333328 [15:38:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1553.966675 [15:38:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:39:16] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:39:16] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:39:16] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:39:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:39:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 466.466675 [15:41:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 291.266663 [15:41:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:42:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:42:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 232.633331 [15:43:46] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:43:46] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 240.800003 [15:45:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:45:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:45:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 72.76667 [15:46:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1853.266724 [15:46:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1906.733276 [15:46:46] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:47:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 2062.56665 [15:47:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 218.866669 [15:48:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 97.133331 [15:49:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:49:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:49:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 75.800003 [15:50:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:50:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 29.066668 [15:50:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.133335 [15:51:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:51:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 116.300003 [15:51:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:51:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 289.466675 [15:51:27] mark___: can someone please address this spam ^ ? [15:51:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.433334 [15:51:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 108.76667 [15:51:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:52:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:52:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 129.133331 [15:53:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 205.666672 [15:53:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:54:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1685.766724 [15:54:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 312.100006 [15:55:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 47.566666 [15:56:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 88.533333 [15:56:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:56:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.033333 [15:57:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 373.200012 [15:57:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:57:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:58:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:58:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [15:58:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:00:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:00:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:00:55] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:01:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:01:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:01:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 387.666656 [16:02:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1400.533325 [16:02:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1635.300049 [16:02:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1397.599976 [16:02:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:03:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1056.833374 [16:04:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:04:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 253.766663 [16:05:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 245.46666 [16:06:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:06:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 305.899994 [16:06:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:06:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:07:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:07:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:07:25] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:07:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:07:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:08:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 94.833336 [16:09:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 199.833328 [16:09:45] RECOVERY - Varnishkafka Delivery Errors on cp1053 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:09:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 284.033325 [16:10:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:10:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1692.733276 [16:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:10:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:12:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 323.899994 [16:12:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 133.899994 [16:12:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:12:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 320.033325 [16:12:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 44.366665 [16:12:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 30.166666 [16:12:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 48.0 [16:13:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:14:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:16:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 138.333328 [16:16:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 75.23333 [16:17:45] RECOVERY - Varnishkafka Delivery Errors on cp1055 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:17:46] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.900002 [16:18:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1384.5 [16:18:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1792.866699 [16:18:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1511.400024 [16:18:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.099998 [16:20:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 362.933319 [16:21:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:21:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:22:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:22:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 343.933319 [16:22:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 114.23333 [16:23:06] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:23:06] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:23:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 29.0 [16:23:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 379.633331 [16:23:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:23:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:23:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 292.200012 [16:25:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:25:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:25:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:25:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:25:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:26:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1684.733276 [16:26:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:27:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 121.800003 [16:29:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:29:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 104.866669 [16:29:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:30:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:30:45] RECOVERY - Varnishkafka Delivery Errors on cp1065 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:30:55] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 116.26667 [16:31:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:32:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:33:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1862.699951 [16:33:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 726.133362 [16:33:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1625.900024 [16:33:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 86.366669 [16:33:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:33:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:34:05] RECOVERY - Varnishkafka Delivery Errors on cp1054 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:34:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 71.300003 [16:34:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 273.033325 [16:35:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:36:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:36:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:36:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 33.200001 [16:36:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 30.533333 [16:36:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 43.866665 [16:36:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:37:06] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:37:15] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:37:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 96.699997 [16:37:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 213.96666 [16:38:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:39:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:39:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 118.333336 [16:39:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.633335 [16:40:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 285.033325 [16:40:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:40:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:41:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1710.900024 [16:41:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:41:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 147.96666 [16:42:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 112.966667 [16:42:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.633335 [16:43:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 578.466675 [16:44:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 66.23333 [16:44:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 264.566681 [16:45:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:46:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:47:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 253.0 [16:47:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1375.766724 [16:47:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:47:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 337.833344 [16:48:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 10.7 [16:48:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 209.133331 [16:48:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:48:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:48:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:49:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:49:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1671.900024 [16:49:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:49:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:50:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:50:45] PROBLEM - Varnishkafka Delivery Errors on cp1055 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 113.066666 [16:50:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:51:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 375.166656 [16:52:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:53:05] PROBLEM - Varnishkafka Delivery Errors on cp3020 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 132.399994 [16:53:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 207.399994 [16:53:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 404.100006 [16:53:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 36.066666 [16:53:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:53:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:54:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:54:05] RECOVERY - Varnishkafka Delivery Errors on cp3020 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:54:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 135.53334 [16:54:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.599998 [16:55:05] PROBLEM - Varnishkafka Delivery Errors on cp3022 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1991.366699 [16:56:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:56:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 852.93335 [16:56:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:56:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:56:55] PROBLEM - Puppet freshness on labstore1001 is CRITICAL: Last successful Puppet run was Tue 03 Jun 2014 16:21:49 UTC [16:57:05] RECOVERY - Varnishkafka Delivery Errors on cp3022 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:58:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:59:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [16:59:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 193.399994 [17:00:16] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:01:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:01:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:01:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:01:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 76.333336 [17:01:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 43.266666 [17:02:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:02:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 23.266666 [17:02:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 21.433332 [17:02:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 143.199997 [17:04:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 409.0 [17:04:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1425.533325 [17:04:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 116.166664 [17:04:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 340.033325 [17:04:45] PROBLEM - Varnishkafka Delivery Errors on cp1065 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 68.666664 [17:04:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 36.766666 [17:05:25] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:06:09] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:06:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 143.899994 [17:07:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:08:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.333332 [17:09:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:10:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:10:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:10:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 79.0 [17:10:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 5.8 [17:12:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 61.333332 [17:12:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 48.333332 [17:12:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 219.96666 [17:12:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 107.599998 [17:14:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 102.73333 [17:14:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:15:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 277.5 [17:16:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 615.666687 [17:17:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 167.0 [17:17:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:17:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:17:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:18:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:18:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:19:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:19:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:19:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:19:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:20:05] PROBLEM - Varnishkafka Delivery Errors on cp1054 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 98.666664 [17:20:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 159.133331 [17:20:55] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:21:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1312.93335 [17:21:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:21:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 147.53334 [17:22:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 25.666666 [17:22:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:22:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 139.53334 [17:23:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:23:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:24:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:24:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:24:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 75.23333 [17:25:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 139.46666 [17:26:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.599998 [17:26:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 116.333336 [17:27:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:27:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:27:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 703.56665 [17:28:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:29:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:29:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.733334 [17:29:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 13.6 [17:29:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 36.666668 [17:29:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 170.366669 [17:30:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:30:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:30:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 57.900002 [17:31:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:31:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 38.166668 [17:31:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 17.233334 [17:32:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 488.200012 [17:32:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 191.899994 [17:33:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:34:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:35:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:35:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 152.566666 [17:36:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 707.466675 [17:36:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 30.066668 [17:37:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 149.399994 [17:38:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:38:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 119.933334 [17:38:46] RECOVERY - Varnishkafka Delivery Errors on cp1053 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:38:46] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:39:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:39:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:40:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:41:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:41:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:42:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:42:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:42:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 64.900002 [17:43:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 47.900002 [17:43:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 59.799999 [17:45:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.666668 [17:45:16] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 145.933334 [17:46:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 500.866669 [17:46:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:46:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 52.266666 [17:46:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:47:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:47:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:47:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:48:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:49:45] RECOVERY - Varnishkafka Delivery Errors on cp1055 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:49:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 330.399994 [17:50:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 184.866669 [17:50:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 66.933334 [17:51:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:51:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 145.933334 [17:51:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:51:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:51:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 187.133331 [17:51:55] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 124.933334 [17:52:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 2028.833374 [17:52:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 86.5 [17:53:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 460.299988 [17:53:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:54:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:54:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 127.533333 [17:55:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 134.066666 [17:56:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.533333 [17:57:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:58:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [17:59:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 32.900002 [17:59:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 27.133333 [18:00:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:00:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:00:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 34.833332 [18:00:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 47.466667 [18:01:05] PROBLEM - Disk space on virt1000 is CRITICAL: DISK CRITICAL - free space: / 2161 MB (3% inode=89%): [18:01:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:01:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:02:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:02:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 286.866669 [18:04:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:04:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:04:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:05:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 100.966667 [18:05:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 32.900002 [18:05:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 29.6 [18:06:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 175.833328 [18:07:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1489.300049 [18:07:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:07:16] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:08:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:08:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:09:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 209.899994 [18:09:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:10:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 125.033333 [18:10:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 147.766663 [18:12:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 422.466675 [18:12:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:12:45] RECOVERY - Varnishkafka Delivery Errors on cp1065 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:13:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:13:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 72.199997 [18:13:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 87.566666 [18:13:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 60.666668 [18:13:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 188.0 [18:13:46] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 86.26667 [18:14:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 62.566666 [18:15:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:16:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:16:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:17:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:18:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 55.933334 [18:18:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 39.766666 [18:18:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:18:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:19:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:20:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 12.133333 [18:22:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 649.766663 [18:22:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:23:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:23:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 140.933334 [18:23:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 98.199997 [18:23:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 69.666664 [18:23:45] PROBLEM - Varnishkafka Delivery Errors on cp1055 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 96.566666 [18:24:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 61.033333 [18:24:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:24:55] PROBLEM - Puppet freshness on elastic1017 is CRITICAL: Last successful Puppet run was Sat 07 Jun 2014 21:19:14 UTC [18:25:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:26:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 114.76667 [18:26:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 37.666668 [18:26:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:26:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:27:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:27:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 233.933334 [18:27:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.700001 [18:27:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:29:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:29:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 161.53334 [18:30:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:30:25] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:30:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 28.833334 [18:30:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 27.066668 [18:31:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:31:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 14.8 [18:32:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.333332 [18:34:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 365.200012 [18:34:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:34:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:34:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:34:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 117.26667 [18:37:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:37:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 185.166672 [18:37:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 52.0 [18:38:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1005.599976 [18:38:16] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:38:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 116.400002 [18:39:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:40:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:41:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:41:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 148.833328 [18:41:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 23.299999 [18:42:25] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 68.866669 [18:42:55] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:43:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:43:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 107.066666 [18:44:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 83.866669 [18:44:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:44:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 262.066681 [18:44:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:45:45] PROBLEM - Varnishkafka Delivery Errors on cp1065 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 66.533333 [18:46:05] RECOVERY - Varnishkafka Delivery Errors on cp1054 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:46:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 5.933333 [18:46:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 26.666666 [18:47:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 69.5 [18:47:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:48:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:48:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:49:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:50:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 105.23333 [18:50:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:51:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 104.033333 [18:51:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 94.76667 [18:51:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:52:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:52:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 146.5 [18:52:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:52:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:52:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:52:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:53:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:53:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 132.899994 [18:54:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 872.133362 [18:54:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 189.699997 [18:54:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 103.333336 [18:55:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:56:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 182.600006 [18:57:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:58:25] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:58:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:58:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 80.166664 [18:59:05] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [18:59:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 68.699997 [19:00:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:02:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:02:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 167.866669 [19:02:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:02:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 17.5 [19:02:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 290.0 [19:03:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:03:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 115.099998 [19:03:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:04:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 61.733334 [19:05:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:05:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:05:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 29.133333 [19:06:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 1.366667 [19:06:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:06:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:06:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 19.466667 [19:07:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 25.9 [19:07:45] RECOVERY - Varnishkafka Delivery Errors on cp1053 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:07:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 173.899994 [19:07:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:08:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:09:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 265.299988 [19:10:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:10:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 100.133331 [19:11:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:11:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 37.400002 [19:12:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 483.766663 [19:12:05] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 61.833332 [19:12:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 106.599998 [19:12:55] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 161.933334 [19:14:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 114.666664 [19:15:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 70.933334 [19:15:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:16:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 99.633331 [19:16:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:16:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 110.666664 [19:16:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 133.53334 [19:19:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:19:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.033333 [19:19:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 53.133335 [19:20:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 192.366669 [19:21:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 189.600006 [19:22:45] RECOVERY - Varnishkafka Delivery Errors on cp1055 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:22:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:23:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 103.066666 [19:24:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 50.133335 [19:24:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:25:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 482.166656 [19:25:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 56.900002 [19:26:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:27:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:27:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:27:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:28:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:28:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:28:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:29:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:29:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:29:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:30:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:30:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:31:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:31:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:32:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 375.666656 [19:32:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:32:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 80.633331 [19:33:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:35:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:35:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:36:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 178.266663 [19:37:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.799999 [19:37:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 176.399994 [19:37:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 243.46666 [19:38:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:39:05] PROBLEM - Varnishkafka Delivery Errors on cp1054 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 74.333336 [19:39:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 144.066666 [19:39:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 23.6 [19:39:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 141.300003 [19:39:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:40:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 53.599998 [19:40:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.333332 [19:40:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:41:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 916.133362 [19:41:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 51.733334 [19:41:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 146.800003 [19:41:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 37.766666 [19:42:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:43:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:43:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:43:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:43:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 23.0 [19:43:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 37.400002 [19:45:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 14.6 [19:45:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 27.766666 [19:46:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 72.133331 [19:46:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:47:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 152.800003 [19:48:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:48:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 30.533333 [19:48:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:49:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:49:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 155.03334 [19:49:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 423.066681 [19:51:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 55.299999 [19:51:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:52:45] RECOVERY - Varnishkafka Delivery Errors on cp1065 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:53:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 463.133331 [19:54:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:54:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:55:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 144.733337 [19:55:45] PROBLEM - Varnishkafka Delivery Errors on cp1055 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 109.699997 [19:56:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:56:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 193.899994 [19:56:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 134.066666 [19:56:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:57:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:57:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:57:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:57:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 119.933334 [19:57:55] PROBLEM - Puppet freshness on labstore1001 is CRITICAL: Last successful Puppet run was Tue 03 Jun 2014 16:21:49 UTC [19:58:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 509.366669 [19:58:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [19:58:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.0 [20:00:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 36.633335 [20:02:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:03:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:03:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:03:55] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:04:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:05:16] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.333332 [20:05:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 190.666672 [20:06:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:06:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 56.866665 [20:06:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:07:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 22.533333 [20:07:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 36.933334 [20:08:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 48.133335 [20:09:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:09:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 84.833336 [20:09:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:09:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:10:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 107.066666 [20:10:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:10:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:11:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 132.633331 [20:11:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 36.366665 [20:12:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:13:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:14:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 241.399994 [20:15:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 866.200012 [20:15:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 54.466667 [20:15:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:15:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:15:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:16:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 41.666668 [20:17:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 182.366669 [20:17:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 121.599998 [20:18:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 7.833333 [20:20:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:20:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:21:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:22:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:22:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:22:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 53.700001 [20:23:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:23:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:24:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:24:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 18.433332 [20:25:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 73.5 [20:25:45] PROBLEM - Varnishkafka Delivery Errors on cp1065 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 68.5 [20:25:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 20.6 [20:25:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:26:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:26:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:27:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:27:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:27:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 57.933334 [20:28:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 60.599998 [20:28:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:29:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 23.9 [20:29:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:30:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:30:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:30:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 22.333334 [20:30:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 59.366665 [20:30:55] PROBLEM - Puppet freshness on db1007 is CRITICAL: Last successful Puppet run was Sun 08 Jun 2014 17:30:04 UTC [20:31:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 51.833332 [20:31:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 232.233337 [20:32:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 261.666656 [20:32:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 86.866669 [20:32:55] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 206.733337 [20:33:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:33:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 110.533333 [20:33:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 37.866665 [20:34:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 36.533333 [20:34:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 174.266663 [20:35:45] RECOVERY - Varnishkafka Delivery Errors on cp1053 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:35:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 428.733337 [20:37:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.299999 [20:38:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 218.133331 [20:39:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:39:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:40:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 185.46666 [20:40:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:41:59] (03PS1) 10Matanya: access_new_install: minor lint [operations/puppet] - 10https://gerrit.wikimedia.org/r/138277 [20:42:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 63.866665 [20:42:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:44:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:44:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 168.133331 [20:44:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 99.833336 [20:44:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 42.333332 [20:46:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.333332 [20:46:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:47:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 94.0 [20:47:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:47:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:48:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 803.133362 [20:48:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:48:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:48:46] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:49:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:49:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:49:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:49:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:50:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 197.03334 [20:53:07] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 25.233334 [20:53:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 51.133335 [20:53:45] RECOVERY - Varnishkafka Delivery Errors on cp1055 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:54:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:54:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 43.533333 [20:54:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:54:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:54:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:55:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:56:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 64.066666 [20:56:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:57:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:57:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:57:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [20:57:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.299999 [20:58:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 269.066681 [20:59:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 69.23333 [20:59:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 8.9 [21:00:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 113.5 [21:00:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 25.066668 [21:00:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 114.133331 [21:02:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:03:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:03:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:03:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 166.53334 [21:05:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 455.899994 [21:05:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 57.099998 [21:05:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:06:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 168.566666 [21:07:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 218.433334 [21:08:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 170.800003 [21:08:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 17.1 [21:09:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 41.166668 [21:09:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:10:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:11:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:11:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 162.899994 [21:11:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:11:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 22.0 [21:11:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:11:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 7.133333 [21:11:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:13:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:13:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:14:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 50.333332 [21:14:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:14:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:15:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 50.733334 [21:15:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:16:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:16:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 54.066666 [21:16:25] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:17:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 17.033333 [21:18:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 19.933332 [21:18:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:18:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 163.0 [21:19:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:20:05] RECOVERY - Varnishkafka Delivery Errors on cp1054 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:21:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:21:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 59.799999 [21:22:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 93.400002 [21:22:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 539.033325 [21:22:55] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:23:06] dear icinga-wm, you're a broken record [21:23:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:24:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.799999 [21:25:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 259.399994 [21:25:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 56.333332 [21:25:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:25:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 29.9 [21:25:45] PROBLEM - Varnishkafka Delivery Errors on cp1055 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 152.633331 [21:25:55] PROBLEM - Puppet freshness on elastic1017 is CRITICAL: Last successful Puppet run was Sat 07 Jun 2014 21:19:14 UTC [21:26:25] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.200001 [21:27:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 51.0 [21:29:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:29:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:29:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 41.400002 [21:29:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:31:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:31:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 43.0 [21:31:45] RECOVERY - Varnishkafka Delivery Errors on cp1065 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:32:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:32:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 172.600006 [21:32:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 16.033333 [21:33:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 67.199997 [21:34:06] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:34:26] (03PS1) 10Matanya: gitblit: firewall already on node level [operations/puppet] - 10https://gerrit.wikimedia.org/r/138278 [21:35:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 32.066666 [21:35:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:36:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:37:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:37:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 194.800003 [21:38:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 194.833328 [21:38:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 124.800003 [21:39:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 111.5 [21:39:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:39:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:40:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:40:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:41:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 26.033333 [21:42:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 231.666672 [21:42:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:42:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:43:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:43:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:43:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:43:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:45:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 101.099998 [21:46:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 70.599998 [21:47:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:47:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 66.333336 [21:47:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:49:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:49:16] (03PS1) 10Matanya: puppetproxy: remove firewall [operations/puppet] - 10https://gerrit.wikimedia.org/r/138279 [21:50:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:50:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 31.933332 [21:50:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:50:30] (03PS2) 10Matanya: puppetproxy: remove firewall [operations/puppet] - 10https://gerrit.wikimedia.org/r/138279 [21:50:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:50:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:50:55] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 140.266663 [21:51:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:51:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 39.900002 [21:52:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:52:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:52:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 85.333336 [21:53:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 20.366667 [21:54:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 75.333336 [21:54:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 14.333333 [21:55:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 526.166687 [21:55:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 43.0 [21:55:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 48.200001 [21:56:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 68.199997 [21:56:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 2.733333 [21:57:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 172.899994 [21:57:29] (03PS1) 10Matanya: labsbastion: lint [operations/puppet] - 10https://gerrit.wikimedia.org/r/138280 [21:57:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [21:57:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 39.5 [21:58:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 113.26667 [21:59:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 232.833328 [21:59:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:00:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 62.099998 [22:00:25] RECOVERY - Puppet freshness on db1007 is OK: puppet ran at Sun Jun 8 22:00:16 UTC 2014 [22:00:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 98.333336 [22:01:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 142.066666 [22:01:45] RECOVERY - Varnishkafka Delivery Errors on cp1053 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:02:45] PROBLEM - Varnishkafka Delivery Errors on cp1065 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 65.066666 [22:03:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:03:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:03:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 64.066666 [22:03:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:04:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:04:26] (03PS1) 10Matanya: labslamp: includes in a sane way [operations/puppet] - 10https://gerrit.wikimedia.org/r/138281 [22:04:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 9.133333 [22:05:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 186.46666 [22:05:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 181.100006 [22:07:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:07:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:08:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:09:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:09:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 62.266666 [22:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:10:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:10:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:11:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 163.53334 [22:11:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:12:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 570.033325 [22:13:02] (03PS1) 10Matanya: labs: lint, clean and remove heartbleed leftover [operations/puppet] - 10https://gerrit.wikimedia.org/r/138282 [22:13:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 80.666664 [22:13:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:14:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:14:15] (03CR) 10jenkins-bot: [V: 04-1] labs: lint, clean and remove heartbleed leftover [operations/puppet] - 10https://gerrit.wikimedia.org/r/138282 (owner: 10Matanya) [22:14:42] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:15:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:15:55] Sunday: a day of rest for everyone except software. [22:16:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 64.666664 [22:16:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:16:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 211.699997 [22:17:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 40.866665 [22:18:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.366665 [22:18:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:19:05] PROBLEM - Varnishkafka Delivery Errors on cp1054 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 67.400002 [22:19:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:19:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 104.066666 [22:20:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:20:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 10.333333 [22:21:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:21:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 31.9 [22:21:45] RECOVERY - Varnishkafka Delivery Errors on cp1055 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:22:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 24.4 [22:22:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 42.333332 [22:22:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:23:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:23:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:24:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 47.400002 [22:24:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:25:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 55.633335 [22:26:06] ugh [22:26:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 51.333332 [22:26:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 43.666668 [22:27:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:28:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 79.199997 [22:29:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 446.133331 [22:29:05] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:29:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 182.266663 [22:29:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:30:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 153.5 [22:31:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:31:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 160.666672 [22:31:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:32:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 177.333328 [22:33:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 389.633331 [22:33:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:34:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 8.666667 [22:34:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:34:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 25.833334 [22:34:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:35:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 56.822224 [22:35:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 60.566666 [22:36:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:36:17] seriously [22:37:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:38:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:38:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 28.333334 [22:39:05] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:39:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:39:25] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:40:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 23.1 [22:40:45] RECOVERY - Varnishkafka Delivery Errors on cp1067 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:40:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 116.466667 [22:41:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 46.133335 [22:41:49] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.766666 [22:43:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:43:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 20.066668 [22:43:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:44:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:44:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 41.266666 [22:45:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:45:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:45:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:45:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:45:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 5.4 [22:46:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 563.333313 [22:46:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 63.933334 [22:46:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:46:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:47:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:48:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 309.433319 [22:49:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 21.0 [22:49:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 14.533334 [22:50:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 49.866665 [22:51:45] PROBLEM - Varnishkafka Delivery Errors on cp1055 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 135.333328 [22:51:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:53:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:53:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:53:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:53:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:54:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:55:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:56:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 92.933334 [22:56:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 194.46666 [22:56:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 192.399994 [22:57:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 70.599998 [22:57:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 62.200001 [22:57:45] PROBLEM - Varnishkafka Delivery Errors on cp1067 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 42.733334 [22:58:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:58:06] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [22:58:16] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 92.73333 [22:58:16] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 215.399994 [22:58:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 20.866667 [22:58:55] PROBLEM - Puppet freshness on labstore1001 is CRITICAL: Last successful Puppet run was Tue 03 Jun 2014 16:21:49 UTC [22:59:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 55.066666 [22:59:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 24.666666 [23:00:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:01:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:02:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 28.6 [23:02:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 472.866669 [23:02:46] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 85.033333 [23:03:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 90.400002 [23:04:16] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 224.800003 [23:04:45] RECOVERY - Varnishkafka Delivery Errors on cp1065 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:05:05] PROBLEM - Varnishkafka Delivery Errors on cp1066 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 163.699997 [23:05:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:06:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:06:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:07:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 90.199997 [23:08:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:08:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:09:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:09:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:10:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 50.200001 [23:10:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:10:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:10:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:11:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 21.466667 [23:12:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 10.066667 [23:13:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 45.333332 [23:14:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:15:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:15:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:15:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:16:05] PROBLEM - Varnishkafka Delivery Errors on cp3021 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 486.600006 [23:16:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:17:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 58.933334 [23:17:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:18:05] RECOVERY - Varnishkafka Delivery Errors on cp3021 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:18:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 92.26667 [23:19:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:19:15] PROBLEM - Varnishkafka Delivery Errors on cp1069 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 315.233337 [23:19:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:19:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 19.133333 [23:20:15] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 225.699997 [23:20:25] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 57.799999 [23:20:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:20:45] RECOVERY - Varnishkafka Delivery Errors on cp1053 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:20:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:21:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 195.733337 [23:22:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 221.133331 [23:22:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 36.466667 [23:23:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 60.666668 [23:23:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:24:15] RECOVERY - Varnishkafka Delivery Errors on cp1069 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:24:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 86.26667 [23:26:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:27:10] !log icinga has been shitting in the channel for 9+ hours (before I went to bed) about Varnishkafka, nothing noted in SAL. Here be a note about it. [23:27:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 79.466667 [23:27:18] Logged the message, Master [23:28:15] RECOVERY - Varnishkafka Delivery Errors on amssq56 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:28:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 95.866669 [23:28:45] PROBLEM - Varnishkafka Delivery Errors on cp4002 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 17.633333 [23:29:45] PROBLEM - Varnishkafka Delivery Errors on cp4018 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 20.966667 [23:30:15] PROBLEM - Varnishkafka Delivery Errors on amssq52 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 85.199997 [23:31:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:31:35] RECOVERY - Varnishkafka Delivery Errors on amssq58 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:31:45] PROBLEM - Varnishkafka Delivery Errors on amssq62 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 53.533333 [23:32:45] PROBLEM - Varnishkafka Delivery Errors on cp1065 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 78.566666 [23:32:45] PROBLEM - Varnishkafka Delivery Errors on cp4010 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 25.633333 [23:33:15] PROBLEM - Varnishkafka Delivery Errors on amssq49 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 55.333332 [23:33:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:33:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:34:25] PROBLEM - Varnishkafka Delivery Errors on amssq55 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 76.866669 [23:35:15] RECOVERY - Varnishkafka Delivery Errors on amssq51 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:36:05] RECOVERY - Varnishkafka Delivery Errors on amssq53 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:36:45] RECOVERY - Varnishkafka Delivery Errors on cp4009 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:38:45] RECOVERY - Varnishkafka Delivery Errors on cp3011 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:39:15] PROBLEM - Varnishkafka Delivery Errors on amssq57 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 54.0 [23:39:15] RECOVERY - Varnishkafka Delivery Errors on amssq54 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:39:45] RECOVERY - Varnishkafka Delivery Errors on cp4008 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:40:05] PROBLEM - Varnishkafka Delivery Errors on amssq56 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 58.433334 [23:40:05] RECOVERY - Varnishkafka Delivery Errors on cp3019 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:41:45] RECOVERY - Varnishkafka Delivery Errors on cp1055 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:42:16] PROBLEM - Varnishkafka Delivery Errors on cp1057 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 205.333328 [23:42:35] PROBLEM - Varnishkafka Delivery Errors on amssq58 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 40.066666 [23:44:05] PROBLEM - Varnishkafka Delivery Errors on cp3019 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 34.799999 [23:44:15] PROBLEM - Varnishkafka Delivery Errors on cp1070 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 265.100006 [23:44:45] PROBLEM - Varnishkafka Delivery Errors on cp1056 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 266.333344 [23:45:45] PROBLEM - Varnishkafka Delivery Errors on cp1053 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 176.5 [23:45:45] RECOVERY - Varnishkafka Delivery Errors on cp4017 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:46:15] RECOVERY - Varnishkafka Delivery Errors on amssq50 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:46:15] PROBLEM - Varnishkafka Delivery Errors on amssq51 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 40.233334 [23:47:35] RECOVERY - Varnishkafka Delivery Errors on amssq59 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:47:45] PROBLEM - Varnishkafka Delivery Errors on cp4009 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 47.599998 [23:47:45] PROBLEM - Varnishkafka Delivery Errors on cp3011 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 86.933334 [23:48:05] RECOVERY - Varnishkafka Delivery Errors on cp1066 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:48:45] RECOVERY - Varnishkafka Delivery Errors on cp4016 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:50:05] PROBLEM - Varnishkafka Delivery Errors on amssq53 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 73.466667 [23:50:15] RECOVERY - Varnishkafka Delivery Errors on amssq52 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:51:15] PROBLEM - Varnishkafka Delivery Errors on amssq54 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 41.400002 [23:51:45] RECOVERY - Varnishkafka Delivery Errors on amssq62 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:53:15] RECOVERY - Varnishkafka Delivery Errors on cp1057 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:53:45] RECOVERY - Varnishkafka Delivery Errors on cp4002 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:54:15] RECOVERY - Varnishkafka Delivery Errors on cp1070 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:54:45] RECOVERY - Varnishkafka Delivery Errors on cp4010 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:54:45] PROBLEM - Varnishkafka Delivery Errors on cp4008 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 59.066666 [23:54:45] RECOVERY - Varnishkafka Delivery Errors on cp4018 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:55:15] RECOVERY - Varnishkafka Delivery Errors on amssq49 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:55:25] RECOVERY - Varnishkafka Delivery Errors on amssq55 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:55:45] RECOVERY - Varnishkafka Delivery Errors on cp1056 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:57:45] PROBLEM - Varnishkafka Delivery Errors on cp4017 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 32.066666 [23:58:15] PROBLEM - Varnishkafka Delivery Errors on amssq50 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 35.133335 [23:59:15] RECOVERY - Varnishkafka Delivery Errors on amssq57 is OK: kafka.varnishkafka.kafka_drerr.per_second OKAY: 0.0 [23:59:35] PROBLEM - Varnishkafka Delivery Errors on amssq59 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 38.133335 [23:59:45] PROBLEM - Varnishkafka Delivery Errors on cp4016 is CRITICAL: kafka.varnishkafka.kafka_drerr.per_second CRITICAL: 55.5