[10:53:38] Whare the ant [14:56:06] PROBLEM - Response time of WDQS eqiad on einsteinium is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [300000.0] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=13&fullscreen [15:41:37] PROBLEM - Response time of WDQS eqiad on einsteinium is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [300000.0] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=13&fullscreen [15:42:08] PROBLEM - WDQS HTTP Port on wdqs1005 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 380 bytes in 0.001 second response time [15:43:17] RECOVERY - WDQS HTTP Port on wdqs1005 is OK: HTTP OK: HTTP/1.1 200 OK - 434 bytes in 0.077 second response time [15:54:37] RECOVERY - Response time of WDQS eqiad on einsteinium is OK: OK: Less than 5.00% above the threshold [120000.0] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=13&fullscreen [16:09:46] PROBLEM - Response time of WDQS eqiad on einsteinium is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [300000.0] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=13&fullscreen [16:36:17] Help me [16:37:06] With two things [16:37:07] Ok [16:40:57] I need merging help [16:41:02] Most importantly [16:46:42] Notice: we are having trouble with Wikidata Query Service (query.wikidata.org). Investigation in progress, but for the moment you can expect slow response times and timeouts. [16:52:57] PROBLEM - Response time of WDQS eqiad on einsteinium is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [300000.0] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=13&fullscreen [16:55:19] ACKNOWLEDGEMENT - Response time of WDQS eqiad on einsteinium is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [300000.0] Gehel https://phabricator.wikimedia.org/T198042 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=13&fullscreen [16:58:10] Help me [16:58:12] Merge [16:58:22] Know if I should merge these [17:01:06] Hello [17:01:22] Can you help me with knowing if I should merge these [17:58:24] what a mess... [17:58:43] I've already had to restore Q1 since they merged it into a completely unrelated item >_< [18:17:12] PROBLEM - High lag on wdqs1004 is CRITICAL: 8720 ge 3600 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [19:26:19] RECOVERY - Response time of WDQS eqiad on einsteinium is OK: OK: Less than 5.00% above the threshold [120000.0] https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=13&fullscreen [19:40:19] PROBLEM - WDQS HTTP Port on wdqs1004 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 380 bytes in 0.001 second response time [19:42:29] RECOVERY - WDQS HTTP Port on wdqs1004 is OK: HTTP OK: HTTP/1.1 200 OK - 434 bytes in 0.023 second response time [20:56:34] https://query.wikidata.org/ is slow right now [20:56:54] ignore me, seems faster now [21:12:57] edward: Notice: we are having trouble with Wikidata Query Service (query.wikidata.org). Investigation in progress, but for the moment you can expect slow response times and timeouts. [21:14:54] edward: yeah, it looks like something isn't right, but I still don't know what. [21:15:03] Let me now if you have any idea... [21:38:19] PROBLEM - High lag on wdqs1005 is CRITICAL: 6096 ge 3600 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [22:00:39] PROBLEM - WDQS HTTP Port on wdqs1005 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 380 bytes in 0.000 second response time [22:01:48] RECOVERY - WDQS HTTP Port on wdqs1005 is OK: HTTP OK: HTTP/1.1 200 OK - 434 bytes in 0.042 second response time [22:02:39] PROBLEM - High lag on wdqs1005 is CRITICAL: 7556 ge 3600 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [22:27:29] PROBLEM - WDQS HTTP Port on wdqs1004 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 380 bytes in 0.001 second response time [22:28:38] RECOVERY - WDQS HTTP Port on wdqs1004 is OK: HTTP OK: HTTP/1.1 200 OK - 434 bytes in 0.117 second response time