[02:29:03] <nuria>	 bd808: this was just merhged, please do take a look and see if it needs  more additions: https://gerrit.wikimedia.org/r/#/c/analytics/refinery/source/+/538607/
[15:14:14] <codezee>	 Hi! I want to replicate article histories of select 3-4 articles with their edit contents. I was planning to use quarry to get the data, but found out article text is not maintained there. Is there any other better way than wrangling with dumps?
[15:31:01] <Nikerabbit>	 codezee: you can use the rest or action api
[20:05:04] <Zppix>	 Out of curiosity, do tools.wmflabs.org/ webpages get cached? 
[20:05:12] <Zppix>	 or is that up to the tool maintainer
[20:21:21] <Zppix>	 !log git Upgrade all instances to linux-image-4.19.0-0.bpo.6-amd64
[20:21:24] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL
[20:53:26] <bd808>	 Zppix: there is not a cache in front of tools.wmflabs.org currently. 
[20:54:56] <bd808>	 Someone would need to do some research to figure out if adding one would be likely to be useful or not. 
[21:00:02] <stwalkerster>	 I'm seeing a lot of instability on the nginx web proxy - HTTP requests seem to be getting a random selection of the resource I'm expecting, a 500 Internal Server Error, or even ERR_CONNECTION_CLOSED from my browser.
[21:00:25] <stwalkerster>	 (and this collection of different results is appearing for different resources on the same page load)
[21:00:45] <stwalkerster>	 The specific endpoint I'm trying to reach is https://accounts.wmflabs.org/acc.php
[21:00:59] <Framawiki>	 andrewbogott: hi, I've not dealed recently with sentry project vms nor their configs on horizon
[21:01:06] <mutante>	 i was about to report the same thing stwalkerster mentioned
[21:01:15] <mutante>	 i get it from puppet-compiler.wmflabs.org
[21:01:44] <andrewbogott>	 Framawiki: who should I ask instead then?  Those VMs were created by you and are currently switched off.
[21:02:22] <andrewbogott>	 stwalkerster, mutante I see that too at least some of the time
[21:03:14] <mutante>	 the last couple hours i used that without the issue but maybe an hour ago or so it started for me
[21:03:57] <Framawiki>	 andrewbogott: i'll check them later to see what they contains, if it's not urgent
[21:04:15] <andrewbogott>	 Framawiki: not urgent, just would be good to cleanup unused VMs
[21:05:35] <Framawiki>	 Sure, 'll do, thanks for the ping
[21:06:01] <Framawiki>	 I've received a lot of down then up alerts for quarry.wmflabs.org today, is there some issues with wmcs infrastructure?
[21:06:43] <Framawiki>	 Perhaps same issue mentioned by mutante above?
[21:07:44] <mutante>	 Framawiki: same thing as above?  nginx?
[21:07:56] <mutante>	 SSL error?
[21:08:15] <mutante>	 or The connection to puppet-compiler.wmflabs.org was interrupted while the page was loading.?
[21:08:29] <musikanimal>	 Same, all my VPS instances have had gone down and up over the past day or so
[21:08:48] <Framawiki>	 Perhaps front nginx shared by wmcs, I've not ssh access to debug
[21:08:49] <mutante>	 well.. VPS instance going down is pretty different 
[21:09:16] <mutante>	 Framawiki: yea, it seems to be the domainproxy
[21:09:40] <musikanimal>	 Yeah could be the proxy, I'm going by automated reports that look for 200 OK
[21:09:44] <andrewbogott>	 hm, http://www.koeri.boun.edu.tr is pulling zillions of map tiles
[21:09:53] <andrewbogott>	 I think it's just more traffic than we can handle
[21:11:54] <mutante>	 university in Istanbul with an English page. but when you click email it's 404
[21:12:01] <mutante>	 can we block them?
[21:12:05] <mutante>	 and then mail them?
[21:12:31] <andrewbogott>	 I'm not sure I know how to block them but checking...
[21:13:11] <andrewbogott>	 mutante: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/479041/ maybe
[21:13:24] <mutante>	 ah!
[21:13:34] <mutante>	 maybe he has a special user agent?
[21:14:09] <mutante>	 or instead of nginx config maybe we can just shut down his instance , heh :)
[21:14:16] <mutante>	 if you know where he come sfrom
[21:35:21] <andrewbogott>	 mutante, stwalkerster, getting better behavior now?
[21:36:21] <stwalkerster>	 much, thanks :)
[21:36:37] <mutante>	 andrewbogott: yes!:)
[21:56:32] <wikibugs>	 !log tools.wikibugs Updated channels.yaml to: 304a5e24d3b69bddb38cadc08a3c9a0a024e91de remove ##Zppix-Wikipedia from ZppixBot notifications
[21:56:34] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL