[00:00:09] something about check_http is weird and this explanation doesn't seem complete [00:00:23] you can use curl to achieve the same thing [00:00:40] -u is equivalent to sending a "Host: foo" [00:00:56] that can't be right, because you will get a 404 if you try to send 'Host: oresweb' [00:01:32] no, you won't [00:01:54] you should [00:01:55] and you do [00:01:56] alex@alex-laptop:~$ curl -H 'Host: oresweb' https://ores.wmflabs.org/v3/scores/fakewiki/$(/bin/date +%s)/ -sI | head -n1 [00:01:57] HTTP/2 404 [00:02:37] the labs proxy isn't gonna recognise oresweb, it'll never be seen by their nginx that knows how to handle that [00:03:34] i already pasted the example, the config snippet from nginx and most of all.. the icinga is working and doing that.. so .. i dunno what is left here [00:03:54] it's clearly not doing this [00:04:04] it's doing something that seems to work, for some reason [00:04:44] you seem to insist that part of a servername somehow needs to be in DNS or be recognized by a proxy, but it's not a host name [00:04:52] but I'm not at all satisfied that we know why it got upset about the redirect, why that triggered an error that looks like some DNS problem, and why providing -S changes things [00:05:10] yeah. [00:05:33] the config change is "we are using https now" and the fix is "tell it to use https" [00:05:52] ores.wmflabs.org is not an ORES server. It's the labs proxy. The labs proxy is only going to know to route your request to ORES if you set 'Host: ores.wmflabs.org'. [00:06:31] yea, and we do that [00:06:37] we are going in circles here [00:06:56] it's serving redirects and -f follow was provided, so why did check_http not do the right thing? [00:07:57] what we've found so far is an odd workaround and I'm not convinced we fully understand what went wrong or what the real root cause of this behaviour from check_http is [00:09:18] the change made on the labs proxy in terms of redirects should not have caused it to do this, and having to specify -S instead of changing http: to https: also seems weird [00:10:34] it doesn't fully add up and I don't like misbehaving HTTP clients like this [00:16:41] ok, feel free to report a bug against check_http. not sure what else to tell you. i think i gave a pretty accurate description of why the check is setup the way it is and why it failed. [00:28:56] i'm out. bye [07:38:24] I'm about to upgrade grafana on cloudmetrics hosts FYI [07:50:07] {{done}} [14:21:20] !log admin rebooting cloudweb2001-dev, labweb1001, labweb1002 to address mediawiki-induced memleak [14:21:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [18:09:03] andrewbogott: are you busy or are you free to check something for me? [18:17:17] nevermind sorry for the ping [18:40:14] !log tools creating 13 new xlarge k8s worker nodes, tools-k8s-worker-67 through tools-k8s-worker-79 [18:40:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [21:14:15] !log shutting down and removing tools-k8s-worker-1 through tools-k8s-worker-19; this load can now be handled by new nodes on ceph hosts [21:14:15] andrewbogott: Unknown project "shutting" [21:15:42] !log tools shutting down and removing tools-k8s-worker-1 through tools-k8s-worker-19; this load can now be handled by new nodes on ceph hosts [21:15:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [21:29:06] !log tools shutting down and removing tools-k8s-worker-20 through tools-k8s-worker-29; this load can now be handled by new nodes on ceph hosts [21:29:11] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [22:26:05] just to double check, if I set up a service (redis in this case) in cloud VPS that listens on 0.0.0.0, as long as my instance doesn't have a public IP or a proxy domain, it won't be accessible from the outside world right (anything not in cloud VPS)? [22:46:23] legoktm: correct [22:46:53] thanks