[00:01:49] PROBLEM - High lag on wdqs1007 is CRITICAL: 3885 ge 3600 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [00:05:41] PROBLEM - Check systemd state on wdqs1007 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. [00:08:09] RECOVERY - Check systemd state on wdqs1007 is OK: OK - running: The system is fully operational [00:15:29] PROBLEM - Check systemd state on wdqs1007 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. [04:06:35] PROBLEM - High lag on wdqs1008 is CRITICAL: 3667 ge 3600 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [04:19:45] PROBLEM - Check systemd state on wdqs1008 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. [04:22:13] RECOVERY - Check systemd state on wdqs1008 is OK: OK - running: The system is fully operational [04:25:53] PROBLEM - Check systemd state on wdqs1008 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. [04:27:51] ACKNOWLEDGEMENT - Check systemd state on wdqs1008 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. Stas Malychev https://phabricator.wikimedia.org/T213134 [04:27:51] ACKNOWLEDGEMENT - High lag on wdqs1008 is CRITICAL: 4919 ge 3600 Stas Malychev https://phabricator.wikimedia.org/T213134 https://grafana.wikimedia.org/dashboard/db/wikidata-query-service?orgId=1&panelId=8&fullscreen [05:39:36] !log restarted some Blazegraph servers as precaution against corruption issues [05:39:36] SMalyshev: Not expecting to hear !log here [10:32:49] I wrote a simple query to get all countries and thier ISO code, but some countries e.g. USA or Germany are missing and I can't figure out why. http://tinyurl.com/yaewhtxr [14:43:00] https://chronas.org/ [14:43:48] It would be nice if they indicated on the page that they use Wikidata... because they do it [14:45:16] Mayeb someone from the team could contact them? Or we already know them? [14:45:20] *Maybe [16:41:56] PROBLEM - WDQS HTTP Port on wdqs1006 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Service Temporarily Unavailable - 387 bytes in 0.001 second response time [16:53:12] RECOVERY - WDQS HTTP Port on wdqs1006 is OK: HTTP OK: HTTP/1.1 200 OK - 448 bytes in 0.029 second response time [16:54:12] IRC office hour is starting in #wikimedia-office in a few minutes :) [18:26:20] oops missed it [23:32:18] mee too :(