[02:26:57] <bd808>	 !log tools.irc-wmt Moved webservice from Trusty job grid to Stretch job grid
[02:26:58] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.irc-wmt/SAL
[02:28:10] <bd808>	 !log tools.irc-wmt Shutdown webservice because it is redirecting all requests to a non-existent Cloud VPS proxy (https://wmt.wmflabs.org/)
[02:28:11] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.irc-wmt/SAL
[03:19:38] <musikanimal>	 hello! is there anywhere I could find the default uwsgi.ini that is provided with the `uwsgi-python` webserver? I needed to use Python 3.5, so I installed in on Stretch. I think this means I have to use `uwsgi-plain` but I have no idea what to put in there
[03:21:57] <musikanimal>	 I'm just sort of hoping the default uwsgi.ini will work
[03:35:41] <bd808>	 musikanimal: hmmm... we should build a stretch+python3 kubernetes image, but that's a side track to your question.
[03:36:35] <musikanimal>	 I'm probably doing something else wrong, but I managed to get the venv installed without any errors
[03:36:59] <musikanimal>	 so I think a working uwsgi.ini is the last step
[03:37:36] <bd808>	 *nod* the Stretch grid has python 3.5.3 and it should be able to make venvs just fine
[03:37:53] * bd808 is looking to see if uwsgi.ini is documented on wikitech or not
[03:39:38] <bd808>	 looks like not
[03:40:38] <musikanimal>	 darn
[03:41:18] <musikanimal>	 I tried going by the official docs, it's mostly gibberish to me :/
[03:41:21] <bd808>	 musikanimal: so the default actually does not use a uwsgi.ini file. It just has a lot of args that are given to the uwsgi binary
[03:41:40] <bd808>	 https://phabricator.wikimedia.org/diffusion/OSTW/browse/master/toollabs/webservice/services/pythonwebservice.py$19-50
[03:42:15] <bd808>	 and yeah, uwsgi is so super flexible that starting from scratch is intimidating
[03:43:19] <bd808>	 the comparable settings for uwsgi-plain are https://phabricator.wikimedia.org/diffusion/OSTW/browse/master/toollabs/webservice/services/uwsgiwebservice.py$19-27
[03:45:11] <bd808>	 musikanimal: I *think* that you can use the "uwsgi-python" type with --backend=gridengine and everything will otherwise work like the docs for python+kubernetes describe
[03:46:31] <bd808>	 oh... or not? /me reads docs again
[03:47:51] <bd808>	 musikanimal: I see the note on https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web about `webservice --backend=gridengine uwsgi-python ...` only working for python2, but I'm not convinced that is actually true on the Stretch grid
[03:48:19] <musikanimal>	 let me try again, I got some sort of error
[03:48:49] <bd808>	 take a look at https://phabricator.wikimedia.org/T104374#1911373 if that doesn't work for you
[03:50:18] <musikanimal>	 yeah, I get `ImportError: No module named site` and the webserver just dies immediately
[03:50:32] <musikanimal>	 that's with `webservice --backend=gridengine uwsgi-python start`
[03:50:39] <musikanimal>	 on Stretch
[03:51:09] <bd808>	 lame
[03:52:04] <musikanimal>	 that Phab comment looks promising, I'll try to copy what they're doing
[10:45:19] <mutante>	 !log git - gerrit-test3 - re-enabling disabled puppet
[10:45:29] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL
[10:46:20] <hauskatze>	 mutante: working off-hours? must be night in sf? :)
[10:46:54] <mutante>	 no, just changing locations
[10:47:08] <hauskatze>	 :)
[12:41:30] <gtirloni>	 FYI: I've updated one of our LDAP servers to work towards a solution for T217280. If anyone notices issues with LDAP and/or problems logging in, please let me know.
[12:41:36] <stashbot>	 T217280: groups: cannot find name for group ID - https://phabricator.wikimedia.org/T217280
[13:57:38] <gtirloni>	 !log tools fixed SSH warnings in tools-clushmaster-02
[13:57:40] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[14:11:08] <Jayprakash12345>	 \msg
[15:02:08] <wm-bot>	  Technical Advice IRC meeting starting in 60 minutes in channel #wikimedia-tech, hosts: @milimetric & @CFisch_WMDE - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
[15:51:15] <wm-bot>	  Technical Advice IRC meeting starting in 10 minutes in channel #wikimedia-tech, hosts: @milimetric & @CFisch_WMDE - all questions welcome, more infos: https://www.mediawiki.org/wiki/Technical_Advice_IRC_Meeting
[15:54:26] <andrewbogott>	 godog: what can you tell me about deployment-prometheus01?  It's been shut down for nearly a month; is that on purpose?  (I'm asking you because you created it)
[15:57:57] <godog>	 andrewbogott: yeah it can be deleted I think at this point, replaced by deployment-prometheus02
[15:58:07] <andrewbogott>	 godog: great, I will delete.  Thank you
[15:58:27] <andrewbogott>	 !log deployment-prep deleting deployment-prometheus01 on Filippo's advice
[15:58:30] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL
[17:58:37] <notconfusing>	 What do Cloud Services prefer, asking for a quota increase on an existing project or creation of a new project?
[18:02:23] <notconfusing>	 I'm trying to provision a new VPS for a new data-pipeline to feed a tool that does Oauth user-to-user thanking.
[18:05:17] <gtirloni>	 notconfusing: is the new app even remotely related to the existing project? in that, I think we'd prefer to avoid creating new projects 
[18:08:33] <notconfusing>	 gtirloni: the projects could share a database so they are related in that way.  it sounds like more projects are more headache, so quota route is better if i find i need it
[18:08:48] <gtirloni>	 yep, +1
[18:09:04] <notconfusing>	 perfect, thanks
[18:13:54] <notconfusing>	 if i set up a database on a VPS, can the database be seen from within another vPS on the same project?
[18:16:19] <jynus>	 there is 3 things you have to have into account, your vps network settings, your firewall configuration and your database configuration; otherwise yes, it is possible
[18:16:37] <notconfusing>	 wonderful
[18:19:04] <notconfusing>	 thank you for the answers
[18:25:02] <joal>	 Hi cloud-team - I have a question about labswiki (wikitec
[18:25:10] <joal>	 wikitech, right?
[18:25:29] <bd808>	 yup, what's your question joal?
[18:25:35] <joal>	 It seems not replicated to labsdb - Is there any doc around this anywhere?
[18:25:44] <joal>	 Hi bd808 :)
[18:26:12] <jynus>	 https://phabricator.wikimedia.org/T167973
[18:26:21] <bd808>	 you are correct that it is not replicated in the wiki replica database todauy
[18:26:25] <bd808>	 *today
[18:26:58] <joal>	 Awesome - many thanks bd808 :)
[18:27:09] <bd808>	 we are slowly working to make wikitech less "special" but have more work to do on that front
[18:27:28] <joal>	 I'll put a dependency on that task for one of ours (getting edits stats for it)
[18:30:35] <joal>	 Thanks again bd808 and cloud people :)
[18:31:16] <bd808>	 edit stats would be fun for wikitech, but probably not super exciting ;)
[18:32:01] <bd808>	 as someone who does a lot of edit patrolling there I know that the bulk of edits are Foundation staff and bots
[18:53:29] <Kb03>	 Can someone verify if Earwig's copyvio tool is down?
[18:53:37] <Kb03>	 I'm timing out on loading
[18:55:02] <Reedy>	 link?
[18:55:35] <Kb03>	 https://tools.wmflabs.org/copyvios/
[18:55:47] <Kb03>	 Ok, now connections are going through
[18:55:57] <Kb03>	 I timed out for the past 20 mins or so
[18:56:10] <Kb03>	 (while not running a check mind you)
[19:00:15] <bd808>	 the tool thinks it is up and running. Its a python2 uwsgi app on the kubernetes cluster. Nothing spooky looking in its logs and the pod has been running for 17 days
[19:02:50] <Kb03>	 huh
[19:02:59] * Kb03 shrugs
[19:05:57] <Zppix>	 !log git Ran sync; echo 3 > /proc/sys/vm/drop_caches on gerrit-test.git 
[19:05:58] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL
[19:11:33] <wikibugs>	 !log tools.wikibugs Updated channels.yaml to: 8c2212415d8280a5b5d4ca16a48f8a5527a160d2 Bump bs4 to 4.7.1
[19:11:35] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL
[19:16:08] <valhallasw`cloud>	 !log tools.wikibugs Taxonomy crontab deployed on stretch
[19:16:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL
[19:22:19] * valhallasw`cloud waves at wikibugs_stretch
[19:24:26] <valhallasw`cloud>	 !log tools.wikibugs wb2-irc deployed on stretch, seems to work.
[19:24:27] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL
[19:27:44] <bd808>	 valhallasw`cloud: a strong endorsement ;)
[19:28:23] <valhallasw`cloud>	 bd808: JIT conversion! ;-)
[19:28:33] <valhallasw`cloud>	 although I guess there's still four weeks or so
[19:33:42] <valhallasw`cloud>	 !log tools.wikibugs wb2-phab deployed on stretch
[19:33:44] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL
[19:37:11] <valhallasw`cloud>	 !log wb2-grrrrit deployed on stretch, all happy!
[19:37:11] <stashbot>	 valhallasw`cloud: Unknown project "wb2-grrrrit"
[19:37:17] <valhallasw`cloud>	 !log tools.wikibugs wb2-grrrrit deployed on stretch, all happy!
[19:37:18] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL
[19:37:30] <valhallasw`cloud>	 legoktm: ^ :D
[19:42:03] <valhallasw`cloud>	 !log tools.forrestbot ReleaseTaggerBot cron temporarily disabled to allow manual running on Stretch
[19:42:05] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.forrestbot/SAL
[19:54:06] * hauskatze gives Stroopwafels to valhallasw`cloud :D
[19:55:14] <valhallasw`cloud>	 !log tools.forrestbot ReleaseTaggerBot now inserted as crontab on Stretch.
[19:55:22] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.forrestbot/SAL
[19:55:51] <valhallasw`cloud>	 James_F: ^ FYI -- if you see any issues in the coming days, please poke me, as it might be related
[19:56:15] <James_F>	 valhallasw`cloud: Cool, thanks for doing that!
[19:57:05] <valhallasw`cloud>	 np!
[19:57:11] <valhallasw`cloud>	 four tools down, four to go
[20:02:31] <valhallasw`cloud>	 bd808: I seem to be co-maintainer of https://tools.wmflabs.org/gridengine-status/ , but I'm not quite sure what it's used for. Any clue, also as to whether it needs conversion to Stretch?
[20:19:26] <bd808>	 valhallasw`cloud: no, it has to stay on the Trusty grid. Its the way that we monitor it. You are an admin there indirectly via tools.admin
[20:19:44] <valhallasw`cloud>	 ok! One more tool from the list then ;-)
[20:20:07] <bd808>	 the email spam should be ignoring that tool too :)
[20:27:58] <valhallasw`cloud>	 mwoauth.errors.OAuthException: Identity issued 132.5999264717102 seconds in the future!
[20:28:06] <valhallasw`cloud>	 maybe I should fix my NFS >_>
[20:28:14] <valhallasw`cloud>	 NTP*
[21:57:18] <bd808>	 !log tools.trusty-tools Changed nag.py crontab to fire daily rather than weekly
[21:57:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.trusty-tools/SAL
[22:27:22] <AmandaNP>	 anyone having issues with bastion?
[22:27:36] <AmandaNP>	 I can't login to there, network closed connection
[22:30:47] <Krenair>	 AmandaNP, which bastion?
[22:31:09] <Krenair>	 I'm on bastion-eqiad1-01 fine
[22:31:19] <AmandaNP>	 bastion.wmflabs.org
[22:31:23] <AmandaNP>	 sshing in
[22:31:27] <Krenair>	 it seems okay
[22:31:35] <Krenair>	 AmandaNP, can you `ssh -vvv bastion.wmflabs.org` and paste the result?
[22:32:03] <AmandaNP>	 nvrmnd. now it's working. temp issue
[22:36:31] <Krenair>	 odd