[11:37:27] 10serviceops, 10Operations, 10Traffic, 10Patch-For-Review: Certificate *.wikipedia.org valid until 2020-06-20 - https://phabricator.wikimedia.org/T251726 (10Dzahn) >>! In T251726#6103487, @Vgutierrez wrote: > Currently we're using the LE unified cert on the US DCs (codfw, eqiad and ulsfo). LE certs are va... [13:32:01] 10serviceops, 10Operations, 10Traffic: Certificate *.wikipedia.org valid until 2020-06-20 - https://phabricator.wikimedia.org/T251726 (10BBlack) >>! In T251726#6108138, @Dzahn wrote: > Even if we still use non-LE certs in some DCs i believe this is ok since we should also have other monitoring for the expira... [13:49:04] 10serviceops, 10Operations, 10Traffic: Certificate *.wikipedia.org valid until 2020-06-20 - https://phabricator.wikimedia.org/T251726 (10Dzahn) Or we could make a new Icinga check that isn't check_http for a specific service but runs openssl directly on the cert file in the private repo and has a generic nam... [15:09:36] 10serviceops, 10Operations, 10Traffic: Certificate *.wikipedia.org valid until 2020-06-20 - https://phabricator.wikimedia.org/T251726 (10Vgutierrez) >>! In T251726#6108138, @Dzahn wrote: > Even if we still use non-LE certs in some DCs i believe this is ok since we should also have other monitoring for the ex... [16:07:48] docker-report-releng is dead on deneb for some time. is it a known issue? [16:17:02] 10serviceops, 10Mobile-Content-Service, 10Page Content Service, 10Patch-For-Review, 10Product-Infrastructure-Team-Backlog (Kanban): Migrate mobileapps to k8s and node 10 - https://phabricator.wikimedia.org/T218733 (10Mholloway) [16:21:28] <_joe_> shdubsh: no, but I'm unsure what the status of deneb/boron is tbh [16:21:33] 10serviceops, 10Operations: Chaos Engineering - Stop for x hours one or more mc10xx memcached shards - https://phabricator.wikimedia.org/T251378 (10colewhite) p:05Triage→03Medium [16:22:31] _joe_: ok, I'll try to restart it [16:23:47] <_joe_> shdubsh: not advisable, we should understand what fails first [16:23:59] <_joe_> it's a systemd timer, so it only starts at given times [16:24:09] <_joe_> I'll take a look tomorrow [16:27:40] ack [16:29:43] 10serviceops, 10Operations: docker-reporter-releng-images failed on deneb - https://phabricator.wikimedia.org/T251918 (10colewhite) [16:29:49] 10serviceops, 10Operations: docker-reporter-releng-images failed on deneb - https://phabricator.wikimedia.org/T251918 (10colewhite) p:05Triage→03Medium [17:50:54] 10serviceops, 10Operations: docker-reporter-releng-images failed on deneb - https://phabricator.wikimedia.org/T251918 (10hashar) @MoritzMuehlenhoff filed a similar task previously. Do you have details as to why the releng/bazel:0.4.0 fails? [17:51:41] 10serviceops, 10Operations, 10Continuous-Integration-Config: docker-reporter-releng-images failed on deneb - https://phabricator.wikimedia.org/T251918 (10hashar) [18:04:42] 10serviceops, 10Anti-Harassment, 10Operations, 10Traffic: Add IP Info (ASN & Geolocation) to requests to MediaWiki - https://phabricator.wikimedia.org/T251933 (10CDanis) [18:05:18] 10serviceops, 10Anti-Harassment, 10Operations, 10Traffic: Add IP Info (ASN & Geolocation) to requests to MediaWiki - https://phabricator.wikimedia.org/T251933 (10dbarratt) [19:20:34] 10serviceops, 10observability, 10Sustainability (Incident Prevention): Add alert for app servers in prod serving outdated MediaWiki branches - https://phabricator.wikimedia.org/T242023 (10colewhite) [19:58:40] 10serviceops, 10Anti-Harassment, 10Operations, 10Traffic: Add IP Info (ASN & Geolocation) to requests to MediaWiki - https://phabricator.wikimedia.org/T251933 (10colewhite) p:05Triage→03Medium