[00:53:51] PROBLEM - Postgres Replication Lag on maps1005 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 13987535072 and 764 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [00:54:17] PROBLEM - Postgres Replication Lag on maps1003 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 3576425008 and 172 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [00:54:17] PROBLEM - Postgres Replication Lag on maps1010 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 5103079152 and 244 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [00:54:41] PROBLEM - Postgres Replication Lag on maps1002 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 2445420464 and 122 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [00:54:49] PROBLEM - Postgres Replication Lag on maps1001 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 4666067848 and 231 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [00:54:53] PROBLEM - Postgres Replication Lag on maps1007 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 10792535216 and 569 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [00:54:55] PROBLEM - Postgres Replication Lag on maps1008 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 8559739136 and 445 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [00:55:15] PROBLEM - Postgres Replication Lag on maps1006 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 9241073000 and 475 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [00:59:33] RECOVERY - Postgres Replication Lag on maps1002 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 195568 and 135 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:02:23] RECOVERY - Postgres Replication Lag on maps1003 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 364072 and 305 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:02:55] RECOVERY - Postgres Replication Lag on maps1001 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 448816 and 337 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:05:37] RECOVERY - Postgres Replication Lag on maps1010 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 30000 and 498 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:07:49] RECOVERY - Postgres Replication Lag on maps1007 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 372352 and 632 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:07:51] RECOVERY - Postgres Replication Lag on maps1008 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 275560 and 633 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:08:09] RECOVERY - Postgres Replication Lag on maps1006 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 85912 and 651 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:09:29] PROBLEM - Postgres Replication Lag on maps2009 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 1349874552 and 75 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:09:51] PROBLEM - Postgres Replication Lag on maps2008 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 304172536 and 23 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:10:51] PROBLEM - Postgres Replication Lag on maps2010 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 839729176 and 55 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:11:05] PROBLEM - Postgres Replication Lag on maps2005 is CRITICAL: POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB template1 (host:localhost) 581183088 and 35 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:18:55] RECOVERY - Postgres Replication Lag on maps2010 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 0 and 129 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:19:09] RECOVERY - Postgres Replication Lag on maps2005 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 582504 and 142 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:19:09] RECOVERY - Postgres Replication Lag on maps2009 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 582504 and 142 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [01:19:33] RECOVERY - Postgres Replication Lag on maps2008 is OK: POSTGRES_HOT_STANDBY_DELAY OK: DB template1 (host:localhost) 354648 and 166 seconds https://wikitech.wikimedia.org/wiki/Postgres%23Monitoring [02:07:17] (03PS1) 10TrainBranchBot: Branch commit for wmf/1.36.0-wmf.24 [core] (wmf/1.36.0-wmf.24) - 10https://gerrit.wikimedia.org/r/652274 [02:41:50] (03PS2) 10DannyS712: Branch commit for wmf/1.36.0-wmf.24 [core] (wmf/1.36.0-wmf.24) - 10https://gerrit.wikimedia.org/r/652274 (https://phabricator.wikimedia.org/T267417) (owner: 10TrainBranchBot) [02:41:58] (03Abandoned) 10DannyS712: Branch commit for wmf/1.36.0-wmf.24 [core] (wmf/1.36.0-wmf.24) - 10https://gerrit.wikimedia.org/r/652274 (https://phabricator.wikimedia.org/T267417) (owner: 10TrainBranchBot) [02:48:55] PROBLEM - dump of analytics_meta in eqiad on alert1001 is CRITICAL: Last dump for analytics_meta at eqiad (db1108.eqiad.wmnet:3352) taken on 2020-12-29 02:19:50 is 1 GB, but previous one was 2 GB, a change of 28.2% https://wikitech.wikimedia.org/wiki/MariaDB/Backups%23Alerting [06:54:53] PROBLEM - Prometheus jobs reduced availability on alert1001 is CRITICAL: job=atlas_exporter site=eqiad https://wikitech.wikimedia.org/wiki/Prometheus%23Prometheus_job_unavailable https://grafana.wikimedia.org/d/NEJu05xZz/prometheus-targets [06:56:27] RECOVERY - Prometheus jobs reduced availability on alert1001 is OK: All metrics within thresholds. https://wikitech.wikimedia.org/wiki/Prometheus%23Prometheus_job_unavailable https://grafana.wikimedia.org/d/NEJu05xZz/prometheus-targets [09:00:19] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir2002 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86382 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:00:31] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir5002 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86371 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:00:45] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir1002 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86356 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:00:57] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir1001 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86344 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:01:05] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir3001 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86337 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:01:11] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir2001 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86331 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:01:11] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir4002 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86331 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:01:15] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir3002 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86326 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:01:19] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir4001 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86322 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:01:31] PROBLEM - HTTPS non-canonical-redirect-6 on ncredir5001 is CRITICAL: SSL CRITICAL - OCSP staple validity for wikipedia.fi has 86311 seconds left https://wikitech.wikimedia.org/wiki/Ncredir [09:12:10] (03PS1) 10Elukey: role::analytics_cluster::hadoop::ui: set analytics-hive endpoint [puppet] - 10https://gerrit.wikimedia.org/r/652394 [09:12:40] (03CR) 10Elukey: [C: 03+2] role::analytics_cluster::hadoop::ui: set analytics-hive endpoint [puppet] - 10https://gerrit.wikimedia.org/r/652394 (owner: 10Elukey) [09:33:59] PROBLEM - MediaWiki exceptions and fatals per minute on alert1001 is CRITICAL: 169 gt 100 https://wikitech.wikimedia.org/wiki/Application_servers https://grafana.wikimedia.org/d/000000438/mediawiki-alerts?panelId=2&fullscreen&orgId=1&var-datasource=eqiad+prometheus/ops [09:35:37] RECOVERY - MediaWiki exceptions and fatals per minute on alert1001 is OK: (C)100 gt (W)50 gt 31 https://wikitech.wikimedia.org/wiki/Application_servers https://grafana.wikimedia.org/d/000000438/mediawiki-alerts?panelId=2&fullscreen&orgId=1&var-datasource=eqiad+prometheus/ops [10:00:03] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4021 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86398 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:03] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4030 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86398 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:03] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1078 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86398 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:09] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2028 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86392 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:09] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2030 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86392 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:09] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2029 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86392 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:09] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2035 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86392 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:09] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2033 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86392 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:09] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2037 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86392 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:09] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2037 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86392 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:11] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2027 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86391 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:11] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2028 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86391 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:11] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2036 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86390 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:19] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2035 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86383 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:19] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2038 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86383 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:19] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2038 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86383 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:21] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1088 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86382 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:21] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2039 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86380 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:21] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2039 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86380 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:23] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2042 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86378 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:25] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2040 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86376 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:25] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2034 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86376 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:27] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2031 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86374 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:27] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2041 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86374 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:29] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1079 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86373 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:29] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1088 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86373 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:29] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1076 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86373 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:29] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1082 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86372 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:29] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1087 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86372 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:33] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2032 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86370 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:33] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2030 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86369 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:33] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4024 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86368 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:37] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1090 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86365 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:37] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1077 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86365 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:37] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4022 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86364 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:37] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4023 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86364 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:37] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4026 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86364 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:45] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1082 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86356 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:45] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1084 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86356 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:45] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1090 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86356 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:49] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2042 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86354 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:49] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1080 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86352 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:51] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4027 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86350 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:53] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4023 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86350 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:53] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1089 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86348 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:53] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1078 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86348 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:55] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1083 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86346 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:55] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1087 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86346 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:55] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1085 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86346 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:00:57] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1084 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86344 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:03] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4026 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86338 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:03] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4024 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86338 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:03] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4030 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86338 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:03] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4025 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86338 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:03] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4021 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86338 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:05] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4028 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86337 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:05] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4028 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86337 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:05] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4031 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86336 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:05] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4029 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86336 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:05] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4032 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86336 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:06] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1085 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86336 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:13] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1086 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86329 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:15] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1081 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86327 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:15] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1089 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86326 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:15] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1083 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86326 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:21] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2034 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86320 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:21] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2041 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86320 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:21] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2029 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86320 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:21] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2031 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86320 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:21] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2032 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86320 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:21] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2036 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86320 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:23] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1075 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86319 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:25] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2040 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86317 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:25] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1079 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86317 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:25] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4025 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86316 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:25] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4022 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86316 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:27] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4031 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86315 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:27] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp4032 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86315 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:29] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4027 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86314 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:29] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1081 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86313 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:29] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1077 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86312 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:29] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1086 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86312 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:31] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1075 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86311 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:33] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp1080 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86309 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:33] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp1076 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86309 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:33] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp4029 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86309 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:39] PROBLEM - ats-tls HTTPS en.wikipedia.org ECDSA on cp2027 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86302 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [10:01:39] PROBLEM - ats-tls HTTPS en.wikipedia.org RSA on cp2033 is CRITICAL: SSL CRITICAL - OCSP staple validity for en.wikipedia.org has 86302 seconds left https://wikitech.wikimedia.org/wiki/HTTPS [14:19:37] (03PS2) 10Reedy: Remove comment about "Same regex as above in https_recv_redirect" [puppet] - 10https://gerrit.wikimedia.org/r/606457 [14:19:41] (03PS3) 10Reedy: Remove comment about "Same regex as above in https_recv_redirect" [puppet] - 10https://gerrit.wikimedia.org/r/606457 [14:20:13] (03PS4) 10Reedy: Remove comment about "Same regex as above in https_recv_redirect" [puppet] - 10https://gerrit.wikimedia.org/r/606457 [14:25:29] (03PS2) 10Reedy: Remove old list of sites listed on miscweb [puppet] - 10https://gerrit.wikimedia.org/r/627551 [15:42:18] 10Operations, 10MediaWiki-extensions-Score, 10Security-Team, 10Wikimedia-General-or-Unknown, and 2 others: Extension:Score / Lilypond is disabled on all wikis - https://phabricator.wikimedia.org/T257066 (10Ipatrol) tstarling, do you know why it currently works so long as the vorbis=1 parameter is removed?... [15:45:49] !log restarting acme-chief on acmechief1001 [15:45:51] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:47:35] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir5001 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 580346 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [15:48:01] !log triggering a puppet run on cp nodes [15:48:03] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:48:29] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2031 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583892 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:48:31] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2029 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583890 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:48:49] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2027 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583873 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:48:57] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2030 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583864 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:48:59] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2028 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583864 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:48:59] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2029 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583863 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:48:59] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2028 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583863 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:48:59] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2027 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583863 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:49:13] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2034 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583849 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:49:15] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2031 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583846 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:49:17] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2030 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583845 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:49:17] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2032 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583845 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:49:55] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4030 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583807 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:50:19] 10Operations, 10MediaWiki-extensions-Score, 10Security-Team, 10Wikimedia-General-or-Unknown, and 2 others: Extension:Score / Lilypond is disabled on all wikis - https://phabricator.wikimedia.org/T257066 (10Ipatrol) Also, I need to know what "time" means here, because the above fact means that it's possible... [15:50:33] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1082 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583768 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:50:33] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2041 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583768 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:50:33] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2040 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583768 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:50:43] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1078 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583758 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:50:49] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4023 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583752 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:50:51] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2042 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583750 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:05] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4023 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583736 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:09] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1075 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583732 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:13] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1079 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583728 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:17] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1075 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583724 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:17] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1077 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583724 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:17] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1081 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583724 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:19] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1076 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583722 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:19] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1080 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583722 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:37] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1080 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583704 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:45] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir3001 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 580096 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [15:51:45] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1085 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583696 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:47] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1084 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583694 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:49] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1083 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583692 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:51:57] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1085 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583684 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:01] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1086 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583681 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:03] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1081 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583679 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:03] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1083 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583679 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:15] !log reloading nginx on cloudelastic1005 and cloudelastic1006 [15:52:17] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:52:19] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1086 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583663 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:25] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2037 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583656 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:35] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1090 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583646 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:35] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1084 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583646 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:41] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1089 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583640 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:47] RECOVERY - Elasticsearch HTTPS for cloudelastic-chi-eqiad on cloudelastic1006 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:52:47] RECOVERY - Elasticsearch HTTPS for cloudelastic-omega-eqiad-ro on cloudelastic1005 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:52:47] RECOVERY - Elasticsearch HTTPS for cloudelastic-psi-eqiad-ro on cloudelastic1005 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:52:51] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1087 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583632 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:52:53] RECOVERY - Elasticsearch HTTPS for cloudelastic-psi-eqiad on cloudelastic1006 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:52:55] RECOVERY - Elasticsearch HTTPS for cloudelastic-chi-eqiad-ro on cloudelastic1005 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:53:07] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1089 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583614 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:53:11] RECOVERY - Elasticsearch HTTPS for cloudelastic-psi-eqiad on cloudelastic1005 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:53:13] RECOVERY - Elasticsearch HTTPS for cloudelastic-omega-eqiad on cloudelastic1006 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:53:41] RECOVERY - Elasticsearch HTTPS for cloudelastic-omega-eqiad-ro on cloudelastic1006 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:54:31] RECOVERY - Elasticsearch HTTPS for cloudelastic-chi-eqiad-ro on cloudelastic1006 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:54:47] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4022 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583514 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:55:27] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4022 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583474 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:56:41] RECOVERY - Elasticsearch HTTPS for cloudelastic-psi-eqiad-ro on cloudelastic1006 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [15:56:43] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1078 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583398 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:57:03] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir4002 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 579779 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [15:57:03] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir1001 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 579779 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [15:58:15] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir5002 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 579706 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [15:58:27] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4021 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583294 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:58:27] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4025 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583294 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:58:53] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1087 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583269 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:58:53] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2032 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583268 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [15:58:55] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4030 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583266 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:03] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2033 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583139 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:03] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2034 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583139 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:03] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2038 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583138 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:07] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4024 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583134 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:31] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4027 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583110 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:43] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4028 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583099 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:43] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4024 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583099 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:43] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4028 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583099 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:43] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4026 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583098 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:01:51] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir2002 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 579491 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [16:01:53] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4029 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583088 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:02:17] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4026 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583064 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:02:59] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4032 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583023 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:02:59] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4031 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583023 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:03:09] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4031 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583013 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:03:15] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2033 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583006 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:03:15] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2038 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 583006 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:03:47] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir3002 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 579374 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [16:05:27] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2036 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582875 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:05:27] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2040 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582874 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:07:01] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir1002 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 579181 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [16:07:39] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1076 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582742 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:07:39] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1090 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582742 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:07:41] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2036 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582740 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:08:45] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir4001 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 579076 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [16:09:49] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1082 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582613 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:09:51] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2042 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582611 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:09:53] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4029 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582609 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:09:53] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4021 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582609 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:11:59] RECOVERY - Elasticsearch HTTPS for cloudelastic-chi-eqiad on cloudelastic1005 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [16:11:59] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1079 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582482 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:12:03] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4025 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582479 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:12:03] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp4027 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582478 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:14:13] RECOVERY - Elasticsearch HTTPS for cloudelastic-omega-eqiad on cloudelastic1005 is OK: SSL OK - Certificate cloudelastic.wikimedia.org valid until 2021-03-01 19:00:35 +0000 (expires in 62 days) https://wikitech.wikimedia.org/wiki/Search [16:14:13] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp1088 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582348 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:16:25] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1088 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582217 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:16:25] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2035 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582217 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:16:25] RECOVERY - HTTPS non-canonical-redirect-6 on ncredir2001 is OK: SSL OK - OCSP staple validity for wikipedia.fi has 578616 seconds left:Certificate wikipedia.fi valid until 2021-03-20 08:01:44 +0000 (expires in 80 days) https://wikitech.wikimedia.org/wiki/Ncredir [16:16:27] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp4032 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582215 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:16:33] 10Operations, 10Diff-blog, 10Technical blog, 10Traffic, 10HTTPS: Send HSTS header on all Wordpress VIP-hosted domains - https://phabricator.wikimedia.org/T270034 (10Nintendofan885) [16:18:35] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp1077 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582086 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:18:37] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2035 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582085 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:18:37] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2039 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582085 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:18:37] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2041 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 582085 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:20:47] RECOVERY - ats-tls HTTPS en.wikipedia.org ECDSA on cp2037 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 581954 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:20:47] RECOVERY - ats-tls HTTPS en.wikipedia.org RSA on cp2039 is OK: SSL OK - OCSP staple validity for en.wikipedia.org has 581954 seconds left:Certificate *.wikipedia.org contains all required SANs:Certificate *.wikipedia.org (ECDSA) valid until 2021-02-15 09:02:12 +0000 (expires in 47 days) https://wikitech.wikimedia.org/wiki/HTTPS [16:40:10] 10Operations, 10MediaWiki-extensions-Score, 10Security-Team, 10Wikimedia-General-or-Unknown, and 2 others: Extension:Score / Lilypond is disabled on all wikis - https://phabricator.wikimedia.org/T257066 (10Reedy) You'll have to ask the people working on it. You probably won't get an answer till next week.... [20:19:25] (03PS1) 10Andrew Bogott: nfs: include python3-dateutil [puppet] - 10https://gerrit.wikimedia.org/r/652407 [20:21:41] (03CR) 10Andrew Bogott: [C: 03+2] nfs: include python3-dateutil [puppet] - 10https://gerrit.wikimedia.org/r/652407 (owner: 10Andrew Bogott) [21:16:48] 10Operations, 10MediaWiki-extensions-Score, 10Security-Team, 10Wikimedia-General-or-Unknown, and 2 others: Extension:Score / Lilypond is disabled on all wikis - https://phabricator.wikimedia.org/T257066 (10Nemo_bis) >>! In T257066#6713124, @Ipatrol wrote: > Assuming the problem is the CVE, Debian [[ https:...