[05:57:04] <icinga-wm>	 PROBLEM - Blazegraph Port for wdqs-blazegraph on wdqs1009 is CRITICAL: connect to address 127.0.0.1 and port 9999: Connection refused https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook
[05:57:06] <icinga-wm>	 PROBLEM - Check systemd state on wdqs1009 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[05:57:36] <icinga-wm>	 PROBLEM - WDQS HTTP Port on wdqs1009 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 380 bytes in 0.000 second response time https://wikitech.wikimedia.org/wiki/Wikidata_query_service
[05:58:00] <icinga-wm>	 PROBLEM - Blazegraph process -wdqs-blazegraph- on wdqs1009 is CRITICAL: PROCS CRITICAL: 0 processes with UID = 499 (blazegraph), regex args ^java .* --port 9999 .* blazegraph-service-.*war https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook
[06:01:24] <icinga-wm>	 ACKNOWLEDGEMENT - Blazegraph Port for wdqs-blazegraph on wdqs1009 is CRITICAL: connect to address 127.0.0.1 and port 9999: Connection refused Stas Malychev Running some tests for dumplicate terms. - The acknowledgement expires at: 2019-08-14 06:00:46. https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook
[06:01:24] <icinga-wm>	 ACKNOWLEDGEMENT - Blazegraph process -wdqs-blazegraph- on wdqs1009 is CRITICAL: PROCS CRITICAL: 0 processes with UID = 499 (blazegraph), regex args ^java .* --port 9999 .* blazegraph-service-.*war Stas Malychev Running some tests for dumplicate terms. - The acknowledgement expires at: 2019-08-14 06:00:46. https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook
[06:01:24] <icinga-wm>	 ACKNOWLEDGEMENT - Check systemd state on wdqs1009 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. Stas Malychev Running some tests for dumplicate terms. - The acknowledgement expires at: 2019-08-14 06:00:46. https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[06:01:25] <icinga-wm>	 ACKNOWLEDGEMENT - WDQS HTTP Port on wdqs1009 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 380 bytes in 0.001 second response time Stas Malychev Running some tests for dumplicate terms. - The acknowledgement expires at: 2019-08-14 06:00:46. https://wikitech.wikimedia.org/wiki/Wikidata_query_service
[06:09:10] <icinga-wm>	 RECOVERY - Blazegraph process -wdqs-blazegraph- on wdqs1009 is OK: PROCS OK: 1 process with UID = 499 (blazegraph), regex args ^java .* --port 9999 .* blazegraph-service-.*war https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook
[06:09:48] <icinga-wm>	 RECOVERY - Blazegraph Port for wdqs-blazegraph on wdqs1009 is OK: TCP OK - 0.000 second response time on 127.0.0.1 port 9999 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook
[06:09:50] <icinga-wm>	 RECOVERY - Check systemd state on wdqs1009 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[06:14:38] <icinga-wm>	 PROBLEM - Check systemd state on wdqs1009 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[06:16:10] <icinga-wm>	 PROBLEM - Blazegraph Port for wdqs-blazegraph on wdqs1009 is CRITICAL: connect to address 127.0.0.1 and port 9999: Connection refused https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook
[06:39:24] <icinga-wm>	 RECOVERY - Blazegraph Port for wdqs-blazegraph on wdqs1009 is OK: TCP OK - 0.000 second response time on 127.0.0.1 port 9999 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook
[06:39:30] <icinga-wm>	 RECOVERY - Check systemd state on wdqs1009 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[07:40:38] <icinga-wm>	 RECOVERY - WDQS HTTP Port on wdqs1009 is OK: HTTP OK: HTTP/1.1 200 OK - 448 bytes in 0.086 second response time https://wikitech.wikimedia.org/wiki/Wikidata_query_service
[11:29:09] <icinga-wm>	 PROBLEM - High lag on wdqs1007 is CRITICAL: 3615 ge 3600 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:29:43] <icinga-wm>	 PROBLEM - High lag on wdqs1008 is CRITICAL: 3649 ge 3600 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:29:49] <icinga-wm>	 PROBLEM - High lag on wdqs1003 is CRITICAL: 3656 ge 3600 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:29:55] <icinga-wm>	 PROBLEM - High lag on wdqs2004 is CRITICAL: 3660 ge 3600 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:29:57] <icinga-wm>	 PROBLEM - High lag on wdqs1009 is CRITICAL: 3663 ge 3600 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:29:59] <icinga-wm>	 PROBLEM - High lag on wdqs1010 is CRITICAL: 3665 ge 3600 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:30:09] <icinga-wm>	 PROBLEM - High lag on wdqs2006 is CRITICAL: 3675 ge 3600 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:30:27] <icinga-wm>	 PROBLEM - High lag on wdqs2005 is CRITICAL: 3694 ge 3600 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:41:34] <icinga-wm>	 ACKNOWLEDGEMENT - High lag on wdqs1003 is CRITICAL: 4325 ge 3600 Gehel under investigation at https://phabricator.wikimedia.org/T230410 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:41:34] <icinga-wm>	 ACKNOWLEDGEMENT - High lag on wdqs1007 is CRITICAL: 4285 ge 3600 Gehel under investigation at https://phabricator.wikimedia.org/T230410 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:41:35] <icinga-wm>	 ACKNOWLEDGEMENT - High lag on wdqs1008 is CRITICAL: 4319 ge 3600 Gehel under investigation at https://phabricator.wikimedia.org/T230410 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:41:36] <icinga-wm>	 ACKNOWLEDGEMENT - High lag on wdqs1009 is CRITICAL: 4333 ge 3600 Gehel under investigation at https://phabricator.wikimedia.org/T230410 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:41:37] <icinga-wm>	 ACKNOWLEDGEMENT - High lag on wdqs1010 is CRITICAL: 4333 ge 3600 Gehel under investigation at https://phabricator.wikimedia.org/T230410 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:41:38] <icinga-wm>	 ACKNOWLEDGEMENT - High lag on wdqs2004 is CRITICAL: 4330 ge 3600 Gehel under investigation at https://phabricator.wikimedia.org/T230410 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:41:39] <icinga-wm>	 ACKNOWLEDGEMENT - High lag on wdqs2005 is CRITICAL: 4266 ge 3600 Gehel under investigation at https://phabricator.wikimedia.org/T230410 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[11:41:40] <icinga-wm>	 ACKNOWLEDGEMENT - High lag on wdqs2006 is CRITICAL: 4344 ge 3600 Gehel under investigation at https://phabricator.wikimedia.org/T230410 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[12:19:17] <icinga-wm>	 RECOVERY - High lag on wdqs1003 is OK: (C)3600 ge (W)1200 ge 33.41 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[12:19:25] <icinga-wm>	 RECOVERY - High lag on wdqs1010 is OK: (C)3600 ge (W)1200 ge 14.94 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[12:19:37] <icinga-wm>	 RECOVERY - High lag on wdqs2006 is OK: (C)3600 ge (W)1200 ge 46.89 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[12:19:55] <icinga-wm>	 RECOVERY - High lag on wdqs2005 is OK: (C)3600 ge (W)1200 ge 28.11 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[12:20:15] <icinga-wm>	 RECOVERY - High lag on wdqs1007 is OK: (C)3600 ge (W)1200 ge 35.1 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[12:20:47] <icinga-wm>	 RECOVERY - High lag on wdqs1008 is OK: (C)3600 ge (W)1200 ge 28.55 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[12:21:01] <icinga-wm>	 RECOVERY - High lag on wdqs2004 is OK: (C)3600 ge (W)1200 ge 41.03 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[12:21:03] <icinga-wm>	 RECOVERY - High lag on wdqs1009 is OK: (C)3600 ge (W)1200 ge 109.1 https://wikitech.wikimedia.org/wiki/Wikidata_query_service/Runbook%23Update_lag https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen
[21:17:07] <yurik>	 Can wikidata `Lexeme` validations ensure that for a given property, only items that have the same language as the lexeme itself be used?