[00:20:54] <wikibugs>	 06Labs: Interactive consoles? - https://phabricator.wikimedia.org/T130806#2147065 (10AlexMonk-WMF) >>! In T130806#2147542, @hashar wrote: > Looks like one needs to install the `openstack-nova-serialproxy` service.  There are some details on http://blog.oddbit.com/2014/12/22/accessing-the-serial-console-of-your-n...
[00:42:34] <wikibugs>	 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Flannel is sometimes flaky - https://phabricator.wikimedia.org/T139707#2440354 (10yuvipanda)
[02:23:18] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Dexedor was created, changed by Dexedor link https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/Access_Request/Dexedor edit summary: Created page with "{{Tools Access Request |Justification=Accessing Wikipedia data to find articles likely to be targeted for vandalism. |Completed=false |User Name=Dexedor }}"
[02:29:35] <enterprisey>	 huh
[02:29:43] <enterprisey>	 that tools access req seems a bit sketchy
[02:29:50] <enterprisey>	 Dexedor has exactly zero edits
[02:30:51] <Krenair>	 isn't that the case with most tools access requests?
[02:31:13] <mutante>	 that
[02:31:19] <mutante>	 but also, smart reason if he is a vandal
[02:31:26] <mutante>	 "     Accessing Wikipedia data to find articles likely to be targeted for vandalism."
[02:45:49] <enterprisey>	 Krenair: nah, that's global edits
[02:46:00] <enterprisey>	 so none of his accounts on any wiki whatsoever have edits
[02:46:21] <enterprisey>	 unless I'm bad at using Special:GlobalUsers, which is entirely likely
[02:46:24] <Krenair>	 maybe it's a different name
[02:46:34] <enterprisey>	 but her
[02:46:36] <enterprisey>	 *here
[02:46:36] <enterprisey>	 https://en.wikipedia.org/wiki/Special:CentralAuth/Dexedor
[02:46:44] <Krenair>	 I don't use the same username on wikitech as I do on SUL wikis
[02:46:52] <enterprisey>	 neither do I, actually
[02:46:59] <enterprisey>	 obviously I'm using flawed reasoning here :)
[03:23:13] <zhuyifei1999_>	 the user registered today. can't be a co-incidence
[03:23:37] <zhuyifei1999_>	 enterprisey: 
[03:23:48] <enterprisey>	 yeah, noticed that too
[07:26:15] <wikibugs>	 06Labs, 10Labs-Infrastructure, 06Operations, 13Patch-For-Review: Depleted connection tracking table on labvirt1010 - https://phabricator.wikimedia.org/T139598#2440759 (10MoritzMuehlenhoff) BTW, base::firewall provides an Icinga check for conntrack tables, which has been really useful to notice problems wit...
[08:49:33] <wm-bot>	 Change on 12wikitech.wikimedia.org a page Nova Resource:Tools/Access Request/Dexedor was modified, changed by Tim Landscheidt link https://wikitech.wikimedia.org/w/index.php?diff=729559 edit summary: 
[09:18:07] <wikibugs>	 10Tool-Labs-tools-Pageviews, 10Analytics, 06Community-Tech, 10Pageviews-API, 07I18n: Topviews in the Pageviews labs tool doesn't auto-exclude special pages with localized names - https://phabricator.wikimedia.org/T139725#2440845 (10Amire80)
[09:41:36] <wikibugs>	 06Labs, 10Tool-Labs: uWSGI webservice terminating unexpectedly - https://phabricator.wikimedia.org/T139020#2440894 (10D3r1ck01) Yes @valhallasw, I have but so far now, the uwsgi service is stable, still with 10workers. Technically, I don't think there is a problem with uwsgi and yes it terminates successfully....
[10:23:28] <wikibugs>	 06Labs, 10Tool-Labs, 07Regression: uWSGI webservice terminating unexpectedly - https://phabricator.wikimedia.org/T139020#2440966 (10zhuyifei1999) Since I'm also a user of uwsgi, this is my observations:  * assuming webservice is up and running with id 1 * user issues `webservice restart` * 1234 receives SIGI...
[10:36:11] <wikibugs>	 06Labs, 10Tool-Labs, 07Regression: uWSGI webservice terminating unexpectedly - https://phabricator.wikimedia.org/T139020#2440981 (10D3r1ck01) so @zhuyifei1999, what do you think can be the remedy for this situation???
[12:25:46] <wikibugs>	 06Labs, 10Tool-Labs, 10Phabricator, 07Puppet: Install Arcanist in toollabs::dev_environ - https://phabricator.wikimedia.org/T139738#2441211 (10Dereckson)
[12:27:21] <wikibugs>	 06Labs, 10Tool-Labs, 10Phabricator, 07Puppet: Install Arcanist in toollabs::dev_environ - https://phabricator.wikimedia.org/T139738#2441214 (10Dereckson)
[12:30:00] <wikibugs>	 10PAWS: PAWS terminal doesn't support alt-<char> on Mac OS X keyboard - https://phabricator.wikimedia.org/T139739#2441221 (10Mattias_Ostmar-WMSE)
[12:33:00] <wikibugs>	 10PAWS: PAWS terminal doesn't support alt-<char> on Mac OS X keyboard - https://phabricator.wikimedia.org/T139739#2441221 (10yuvipanda) Upstream just moved to xterm.js from term.js, and I'll attempt to deploy that for us soon - that should hopefully make this better.
[12:38:20] <yuvipanda>	 !log tools starting up tools-web-static-02 again
[12:38:25] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL, Master
[12:40:03] <shinken-wm>	 RECOVERY - Host tools-web-static-02 is UP: PING OK - Packet loss = 0%, RTA = 0.81 ms
[12:47:13] <wikibugs>	 06Labs, 10Tool-Labs, 13Patch-For-Review: Update nginx on tools and labs proxies and static file server - https://phabricator.wikimedia.org/T134383#2441293 (10yuvipanda) This is done for everything except the static nginxes, since those are still running trusty. I need to move them to jessie and then do this.
[12:55:04] <wikibugs>	 06Labs, 10Tool-Labs: Update static tools host to be jessie - https://phabricator.wikimedia.org/T139743#2441298 (10yuvipanda)
[13:11:13] <wikibugs>	 06Labs, 10Tool-Labs, 07Regression: uWSGI webservice terminating unexpectedly - https://phabricator.wikimedia.org/T139020#2441336 (10zhuyifei1999) @D3r1ck01 I don't think the restart cycle happened after `webservice2 uwsgi-python start`, but after `webservice restart`. Would you confirm by running `webservice...
[13:13:17] <zhuyifei1999_>	 yuvipanda: would you look into ^
[13:14:00] <zhuyifei1999_>	 it's reproduceable on video2commons and video2commons-test at least
[13:14:13] <yuvipanda>	 zhuyifei1999_ I think my solution would be to find a way to move out of having to use webservicemonitor asap
[13:14:35] <yuvipanda>	 which would be the k8s migration
[13:14:36] <zhuyifei1999_>	 well, k8s is still in beta
[13:15:55] <wm-bot>	 Change on 12www.mediawiki.org a page OAuth was modified, changed by Yaron Koren link https://www.mediawiki.org/w/index.php?diff=2182506 edit summary: Added OAuthAuthentication extension
[13:16:34] <yuvipanda>	 yeah, but given that I'm the only person who can work on either of these things, and uwsgi is plagued by something far worse atm (the super slow restart times) that are probably leading into this, I don't feel too confident I can provide any actual fixes for this...
[13:16:41] <yuvipanda>	 in any reasonable timeframe.
[13:16:44] <zhuyifei1999_>	 ok
[14:13:54] <paladox>	 !log git generating zuul 2.1.0-151-g30a433b-wmf1jessie1 package for jessie using jenkins-slave-01 instance.
[14:13:58] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master
[14:13:59] <paladox>	 hashar ^^ :)
[14:17:10] <wikibugs>	 06Labs, 10Labs-Infrastructure, 06Operations, 13Patch-For-Review: Depleted connection tracking table on labvirt1010 - https://phabricator.wikimedia.org/T139598#2441481 (10Andrew) 05Open>03Resolved a:03Andrew
[14:57:45] <paladox>	 !log git upgrade zuul to 2.1.0-151-g30a433b-wmf1jessie1 on gerrit-test
[14:57:48] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master
[15:06:48] <wikibugs>	 10Tool-Labs-tools-Pageviews, 06Community-Tech: Topviews in the Pageviews labs tool doesn't auto-exclude special pages with localized names - https://phabricator.wikimedia.org/T139725#2441625 (10MusikAnimal)
[15:09:09] <wikibugs>	 10Tool-Labs-tools-Pageviews, 06Community-Tech, 07I18n: Topviews in the Pageviews labs tool doesn't auto-exclude special pages with localized names - https://phabricator.wikimedia.org/T139725#2441628 (10Amire80) (#i18n is relevant—it's a tag that is applied to all bugs related to internationalization.)
[15:36:01] <wikibugs>	 10Tool-Labs-tools-Pageviews, 06Community-Tech, 07I18n: Topviews in the Pageviews labs tool doesn't auto-exclude special pages with localized names - https://phabricator.wikimedia.org/T139725#2440845 (10MusikAnimal) p:05Triage>03Normal >>! In T139725#2441628, @Amire80 wrote: > (#i18n is relevant—it's a ta...
[16:07:44] <paladox>	 yuvipanda hi, could i have some help please.
[16:07:52] <paladox>	 I carn't ssh into gerrit-test instance
[16:08:10] <paladox>	 and also i pressed shut of instance and carn't seem to get it to reboot now
[16:09:07] <paladox>	 Ive pressed power on now
[16:10:51] <yuvipanda>	 hi paladox
[16:11:01] <yuvipanda>	 were your ssh troubles before or after you turned it off?
[16:11:07] <paladox>	 I had it before
[16:11:26] <paladox>	 It shows
[16:11:27] <paladox>	 channel 0: open failed: connect failed: No route to host
[16:11:27] <paladox>	 stdio forwarding failed
[16:11:27] <paladox>	 ssh_exchange_identification: Connection closed by remote host
[16:12:59] <yuvipanda>	 can you file a bug? I'm in the middle of debugging two other things atm sorry
[16:13:33] <paladox>	 Ok
[16:13:36] <andrewbogott>	 paladox: I can look, what's the name of the instanace?
[16:13:52] <paladox>	 andrewbogott gerrit-test
[16:13:54] <paladox>	 and thanks
[16:14:00] <andrewbogott>	 project name?
[16:14:03] <paladox>	 git
[16:19:32] <andrewbogott>	 paladox: you didn't tinker with the grub settings by chance?
[16:19:44] <paladox>	 I doint think so
[16:19:55] <paladox>	 but maybe, i did apply the labs puppet rules for zuul
[16:20:18] <andrewbogott>	 Hm, wouldn't think that would do it.
[16:20:26] <paladox>	 Oh
[16:20:27] <andrewbogott>	 https://www.irccloud.com/pastebin/68SrJtgp/
[16:20:45] <andrewbogott>	 you were stopping/starting in horizon, right?  So you can see that log too
[16:20:50] <paladox>	 oh yep
[16:21:01] <paladox>	 It says Failed to boot both default and fallback entries.
[16:21:04] <andrewbogott>	 I'm going to run a test to make sure that's not a problem with all of the current instances of that type :/
[16:21:22] <paladox>	 Ok thanks
[16:25:31] <andrewbogott>	 yuvipanda: tool-master-01 and tool-master-02 in 'testlabs' — still good for something?
[16:25:43] <andrewbogott>	 (probably I keep asking you that)
[16:26:22] <yuvipanda>	 andrewbogott I have no idea - I think chasemp created them?
[16:26:44] <andrewbogott>	 oh, you're right, my mistake
[16:27:00] <chasemp>	 if I did remove it
[16:27:14] <chasemp>	 I think it was before I decided to throw in on tools beta at the time
[16:29:31] <andrewbogott>	 chasemp: so shall I delete them?
[16:29:40] <chasemp>	 sounds good
[16:29:49] <paladox>	 andrewbogott but i did do apt-get update and upgrade which may have upgraded linux image
[16:29:50] <paladox>	 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=815125
[16:30:09] <andrewbogott>	 ah, indeed, that's probably the problem
[16:30:36] <paladox>	 Oh
[16:30:47] <andrewbogott>	 paladox: so, the most straightforward course is to kill and rebuild that instance.  Instance creation is disabled right now (for you) but I can create a new one for you.
[16:30:49] <andrewbogott>	 Will that suit?
[16:30:58] <paladox>	 Yes please
[16:31:04] <paladox>	 Thanks.
[16:31:16] <andrewbogott>	 and ok if I delete the existing one?
[16:31:26] <paladox>	 Yep
[16:31:29] <paladox>	 please
[16:31:31] <andrewbogott>	 ok
[16:31:36] <andrewbogott>	 I'll probably name the new one gerrit-test-2 if that's ok
[16:31:44] <paladox>	 Yes
[16:33:52] <paladox>	 andrewbogott seems to work now
[16:33:55] <paladox>	 after rebuilding it
[16:33:59] <andrewbogott>	 !log git deleting gerrit-test and adding a new instance, gerrit-test-2
[16:34:02] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, dummy
[16:34:27] <andrewbogott>	 Yeah.  Sorry about that, dist-upgrade always manages to break something.  It's a thing that would be nice to support but in general if you want an updated system it's best to just start fresh.
[16:34:52] <paladox>	 Oh
[16:34:54] <paladox>	 Ok
[16:36:31] <paladox>	 andrewbogott it seems i carn't view it here https://wikitech.wikimedia.org/w/index.php?title=Special:NovaInstance&action=configure&instanceid=d345d776-56dc-404f-b478-f89010f4446d&project=git&region=eqiad
[16:36:34] <mutante>	 'do-release-upgrade' is an Ubuntu thing and doesn't always work. within Debian it might work better
[16:36:43] <paladox>	 shows instance id as empty
[16:36:48] <paladox>	 Oh
[16:37:06] <mutante>	 like when you just edit the sources.list and apt-get upgrade , dist-upgrade
[16:38:06] <andrewbogott>	 paladox: I'm not clear on how you got to that url
[16:38:06] <paladox>	 never mind
[16:38:15] <paladox>	 I got it from wikitech
[16:38:22] <paladox>	 when clicking configure this instance
[16:41:07] <andrewbogott>	 I'm confused, didn't I create an instance called gerrit-test-2?  I can't find that one now :(
[16:41:28] <andrewbogott>	 oh, you renamed it?
[16:41:49] <paladox>	 Yes and i created gerrit-test from create instance
[16:41:56] <paladox>	 To see if that fixed it and it did
[16:42:50] <wikibugs>	 06Labs, 10Horizon: Disable renaming of instances on Horizon - https://phabricator.wikimedia.org/T139768#2442004 (10Andrew)
[16:43:29] <andrewbogott>	 paladox: what?  'create instance'?
[16:43:44] <paladox>	 andrewbogott yes
[16:43:55] <andrewbogott>	 What do you mean 'from create instance'?
[16:44:00] <paladox>	 Yes
[16:44:06] <paladox>	 I created a new instance
[16:44:21] <andrewbogott>	 how, where?
[16:44:34] <paladox>	 From https://wikitech.wikimedia.org/wiki/Special:NovaInstance
[16:44:41] <paladox>	 https://wikitech.wikimedia.org/w/index.php?title=Special:NovaInstance&action=create&project=git&region=eqiad
[16:45:32] <wikibugs>	 06Labs, 10Labs-Kubernetes, 10Tool-Labs: Set lookupcache=all for k8s worker nodes NFS - https://phabricator.wikimedia.org/T139769#2442026 (10yuvipanda)
[16:45:46] <andrewbogott>	 ok
[16:45:50] <andrewbogott>	 so, I don't understand...
[16:45:56] <andrewbogott>	 I created an instance named gerrit-test-2
[16:46:02] <andrewbogott>	 then you renamed it to gerrit-test
[16:46:07] <andrewbogott>	 and then you created an instance named gerrit-test
[16:46:13] <andrewbogott>	 ?
[16:46:13] <paladox>	 Yes, but it didnt have an instance name so i coulden configure it
[16:46:20] <paladox>	 i deleted it and created a new instance
[16:46:24] <paladox>	 called gerrit-test
[16:46:25] <andrewbogott>	 oh, you deleted it
[16:46:26] <andrewbogott>	 ok then
[16:46:33] <paladox>	 Yes 
[16:46:38] <paladox>	 but you can rename them in horizen
[16:46:46] <paladox>	 horizon
[16:48:49] <andrewbogott>	 yes, but please don't :)
[16:49:01] <andrewbogott>	 because wikitech won't know about the change
[16:49:08] <paladox>	 Oh
[16:49:10] <paladox>	 Sorry
[16:49:35] <andrewbogott>	 np, it's a bug, I just entered it in phab
[16:49:41] <paladox>	 I wonder if this
[16:49:41] <paladox>	 librsvg2-2 librsvg2-common libxslt1.1 linux-image-3.16.0-4-amd64
[16:49:59] <paladox>	 would have broke it, im not doing it without them being ruled out of it breaking
[16:50:04] <paladox>	 the instance after update
[16:50:25] <paladox>	 root@gerrit-test:/etc/apache2# sudo apt-get upgrade
[16:50:25] <paladox>	 Reading package lists... Done
[16:50:26] <paladox>	 Building dependency tree
[16:50:26] <paladox>	 Reading state information... Done
[16:50:26] <paladox>	 Calculating upgrade... Done
[16:50:27] <paladox>	 The following packages will be upgraded:
[16:50:28] <paladox>	   librsvg2-2 librsvg2-common libxslt1.1 linux-image-3.16.0-4-amd64
[16:50:30] <paladox>	 4 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
[16:50:32] <paladox>	 Need to get 34.6 MB of archives.
[16:50:34] <paladox>	 After this operation, 47.1 kB of additional disk space will be used.
[16:50:36] <paladox>	 Do you want to continue? [Y/n]
[16:50:45] <paladox>	 andrewbogott ^^
[16:51:10] <andrewbogott>	 I don't know why that would break things, but history shows that it will :)
[16:51:18] <paladox>	 Oh ok
[16:59:51] <ostriches>	 andrewbogott: What about if someone says "Delete" you go ahead and nuke it rather than wait a month?
[16:59:57] <ostriches>	 Grab every electron you can :D
[17:00:34] <andrewbogott>	 ostriches: I'm not sure.  It's harmless if someone says 'keep' in error, but less harmless if a troll shows up and randomly writes 'delete' under a project.
[17:00:44] <andrewbogott>	 So explicit 'delete' requests will have to be considered carefully :)
[17:01:18] <ostriches>	 The two in conventionextension can go like last year :p
[17:01:47] <andrewbogott>	 ok — is that whole project defunct?
[17:02:03] <ostriches>	 Yep
[17:02:06] <andrewbogott>	 ok
[17:02:08] <ostriches>	 It was a GSoC project from 2012
[17:02:13] <andrewbogott>	 I will consider your 'delete' request carefully :)
[17:02:36] <ostriches>	 pmtpa images lol :p
[17:07:16] <paladox>	 ostriches i am creating https://wikitech.wikimedia.org/wiki/How_to_install_zuul,_gerrit_and_jenkins_on_instances to allow anyone to create zuul and gerrit and jenkins really quickly in labs
[17:07:29] <paladox>	 :)
[17:07:40] <paladox>	 we can translate my steps into a puppet role for labs.
[17:08:14] <ostriches>	 No labs roles ewww
[17:08:36] <paladox>	 ostriches oh why
[17:08:58] <ostriches>	 Well written manifests only need one role with proper hiera :)
[17:09:09] <paladox>	 Oh
[17:09:13] <mutante>	 he means "have a puppet role at all"
[17:09:14] <ostriches>	 That's why I cleaned up the gerrit manifests, so it doesn't need a prod/labs role
[17:09:15] <mutante>	 instead of manual install
[17:09:32] <ostriches>	 Well, I dunno if you should have a role for 3 things if you wanna co-host them :)
[17:09:36] <ostriches>	 I'd rather install 3 roles
[17:10:04] <chasemp>	 ^ yes
[17:10:08] <mutante>	 yea, paladox, roles are cool just not ::labs 
[17:10:13] <paladox>	 Ok
[17:10:19] <paladox>	 We can do roles then
[17:18:07] <Luke081515>	 heh, that purge at labs is a good idea, and I think you choosed a big enough time range ;)
[17:20:20] <wikibugs>	 06Labs, 10wikitech.wikimedia.org: Labs front-page statistics are very wrong - https://phabricator.wikimedia.org/T139773#2442147 (10Andrew)
[17:23:52] <wikibugs>	 06Labs, 10Horizon: Disable renaming of instances on Horizon - https://phabricator.wikimedia.org/T139768#2442004 (10Luke081515) @Andrew: Maybe possible to reactivate this later, if we moved full of wikitech?
[17:26:48] <wikibugs>	 06Labs, 10wikitech.wikimedia.org: Labs front-page statistics are very wrong - https://phabricator.wikimedia.org/T139773#2442147 (10Luke081515) I guess I know at least one reason: If you delete a instance at horizon, the wikitech won't get deleted, labslogbot only adds sometimes the "instance has been deleted"...
[18:25:39] <wikibugs>	 06Labs, 10Tool-Labs: Virtualenvs slow on tool labs NFS - https://phabricator.wikimedia.org/T136712#2442476 (10yuvipanda)
[18:25:41] <wikibugs>	 06Labs, 10Labs-Kubernetes, 10Tool-Labs, 13Patch-For-Review: Set lookupcache=all for k8s worker nodes NFS - https://phabricator.wikimedia.org/T139769#2442473 (10yuvipanda) 05Open>03Resolved a:03yuvipanda I manually added this to hiera pages for all the hosts individually & rolling restarted them, and...
[18:28:57] <yuvipanda>	 zhuyifei1999_ I have uwsgi webservice on k8s figured out I think. what was the tool you told me I could test on?
[18:29:24] <zhuyifei1999_>	 video2commons-test
[18:29:33] <yuvipanda>	 ok!
[18:29:56] <yuvipanda>	 I've also turned on lookupcache in all jessie (k8s) hosts so restarts there should be much faster
[18:30:01] <zhuyifei1999_>	 k
[18:30:08] <yuvipanda>	 and yay you're using a venv! :D
[18:30:20] <yuvipanda>	 this will make this much easier (I think the k8s setup will force everyone to use venvs)
[18:30:54] <zhuyifei1999_>	 I thought uwsgi-python webservice require venv
[18:32:26] <wikibugs>	 06Labs, 10Tool-Labs: Setup running uwsgi webservices on k8s - https://phabricator.wikimedia.org/T139783#2442512 (10yuvipanda)
[18:43:33] <zhuyifei1999_>	 yuvipanda: in case you're testing the oauth login on that test tool, the callback is broken (redirects to the main tool). just add the -test suffix to the url manually
[18:44:09] <yuvipanda>	 ok!
[19:11:27] <chasemp>	 Luke081515: what is the codereview project?
[19:31:20] <halfak>	 o/ 
[19:31:26] <halfak>	 ores-staging-02 is in a weird state. 
[19:31:34] <halfak>	 The wa % is at 99
[19:31:40] <halfak>	 But CPU and memory usage are nominal
[19:32:03] <wikibugs>	 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs: Deploy "Striker" Tool Labs console to WMF production - https://phabricator.wikimedia.org/T136256#2442666 (10bd808) In a discussion on irc, @yuvipanda suggested that this application should live in the labs support vlan and that we could probably either find a...
[19:32:07] <halfak>	 ksoftirqd is using about 12% CPU
[19:32:27] <halfak>	 andrewbogott, is this the kind of thing you want to look into or should I just reboot and hope for the best?
[19:33:37] <wikibugs>	 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs: Deploy "Striker" Tool Labs console to WMF production - https://phabricator.wikimedia.org/T136256#2442668 (10bd808) Deployment with scap3 and a git repo of wheels dependencies seems to be the obvious choice for now. Future improvements for wheels distribution m...
[19:34:37] <andrewbogott>	 halfak: Go ahead and reboot, I think
[19:34:40] <halfak>	 kk
[19:34:55] * halfak logs into the beautiful horizon
[19:35:29] <wikibugs>	 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs: Deploy "Striker" Tool Labs console to WMF production - https://phabricator.wikimedia.org/T136256#2442681 (10bd808) I think `toolsadmin.wikimedia.org` is the right hostname for initial deployment. If and when Striker gains features that are generally useful for...
[19:42:11] <halfak>	 FYI andrewbogott, a reboot seems to have resolved the issue. 
[19:42:12] <Luke081515>	 chasemp: I have to think, if we still need it, it's acutally one vagrant machine, prepared for codereview hours, so that you can easy and fast test code, which was nominated for review. I think we actually have to decide the future of code review hours. Maybe we can abadone that project in a few weeks, but I hope not.
[19:42:28] <halfak>	 -._o_.-
[19:42:55] <chasemp>	 Luke081515: I wasn't second guessing you :) just no clue what it is used for
[19:42:57] <chasemp>	 generic name etc
[19:43:20] <halfak>	 Just deployed models for Czech Wiki and English Wictionary!
[19:43:22] <halfak>	 :D
[19:43:29] <Luke081515>	 ok :)
[19:43:30] <halfak>	 *Wiktionary
[19:44:44] <Luke081515>	 chasemp: because actually not used, I shutdown the instance some days ago. Is there a difference, if we shut down instances (not delete)? So, is it useful to shut down instances, where you can say, that you won't use them for example a week? 
[19:45:55] <chasemp>	 there is a difference in RAM/CPU/Power of course but storage is a big factor and it doesn't help there, it's not a bad plan but it's also not a things we have in mind (shutdown things that are meant to be kept)
[19:47:08] <Luke081515>	 chasemp: There is one thing I'm currently wondering about too: You can pause and resume instances, but if you reasume them, Nagf (graphite) won't let you view data for that instance till you reboot that instance
[19:47:51] <chasemp>	 that would be odd, do you mean it doesnt populate new data in graphite until a reboot?
[19:48:08] <chasemp>	 I assume a pause is a suspend but yeah not sure why that would be at all atm
[19:48:33] <Luke081515>	 IIRC I created a task for that some months ago
[19:48:40] <Luke081515>	 lemme search that one
[19:49:49] <Luke081515>	 T131022
[19:49:49] <stashbot>	 T131022: Graphite is unable to detect, if a paused instance is resumed - https://phabricator.wikimedia.org/T131022
[19:53:13] <chasemp>	 Luke081515: graphite data is populated by the diamond service so...it seems like some how it's not coming back up with it
[19:53:54] <Luke081515>	 I will write an answer to fgiunchedi (I missed his answer), so he can take a look
[19:56:36] <chasemp>	 cool
[20:16:25] <Luke081515>	 chasemp: btw, what do you think of my proposal at -operations? May be useful here too? (I wrote at about 21:48 CEST and later, so balf an hour ago)
[20:20:04] <halfak>	 !log ores deployed ores-wmflabs-deploy:3f116cd
[20:20:08] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL, Master
[20:21:33] <chasemp>	 Luke081515: assumignt ask creation via irc it's a neat idea but I think the outcome woudl be a flourish of low value confusing and conflicting tasks tbh
[20:21:37] <chasemp>	 ask aklapper tho
[20:23:25] <Luke081515>	 yeah, I need a bot approval at least to. Actually this is just analysing that situation, I haven't done code-work yet. The main idea is, that you can create a short, easy understandable task (like via arc todo with arcanist), and add the relevant information later
[20:23:47] <Luke081515>	 later = if you have time, ideally 1-2 hour later. So you don't forget to create that task
[20:23:55] <Luke081515>	 and others who have time at IRC can add info too
[21:13:35] <wikibugs>	 06Labs, 10Labs-Infrastructure, 07Graphite: Graphite is unable to detect, if a paused instance is resumed - https://phabricator.wikimedia.org/T131022#2442953 (10Luke081515) 05Open>03Resolved Seems like the graphite re-setup solved that. I'm not able to reproduce it again.
[21:13:52] <Luke081515>	 chasemp: some bugs solve on their own, I like that kind of bugs :D
[21:14:30] <chasemp>	 heh
[21:14:37] <chasemp>	 probably an unrelated issue from the time
[21:17:47] <legoktm>	 yuvipanda: er, does your comment on the purge page mean you're turning off ircnotifier?
[21:19:04] <yuvipanda>	 I'm going to move it to tools
[21:43:02] <paladox>	 !log git Reinstalled gerrit on gerrit-test instance
[21:43:06] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master
[21:43:16] <paladox>	 !log git configuring gerrit settings on users account on gerrit
[21:43:19] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master
[22:02:22] <wikibugs>	 06Labs, 10Tool-Labs, 06Community-Tech-Tool-Labs: Deploy "Striker" Tool Labs console to WMF production - https://phabricator.wikimedia.org/T136256#2443057 (10mmodell) >>! In T136256#2442668, @bd808 wrote: > Deployment with scap3 and a git repo of wheels dependencies seems to be the obvious choice for now. Fut...
[22:20:23] <wikibugs>	 06Labs, 10Tool-Labs, 13Patch-For-Review: Decommission Tools-exec-wmt.tools.eqiad.wmflabs - https://phabricator.wikimedia.org/T104919#2443127 (10Krinkle)
[22:20:25] <wikibugs>	 06Labs, 10Tool-Labs, 07Documentation: document the need and usage patterns for special exec hosts - https://phabricator.wikimedia.org/T99067#2443126 (10Krinkle)
[22:20:35] <wikibugs>	 06Labs, 10Tool-Labs, 07Documentation: document the need and usage patterns for special exec hosts - https://phabricator.wikimedia.org/T99067#1284594 (10Krinkle)
[22:20:49] <wikibugs>	 06Labs, 10Tool-Labs, 13Patch-For-Review: Decommission tools-exec-wmt.tools.eqiad.wmflabs - https://phabricator.wikimedia.org/T104919#1432226 (10Krinkle)
[22:24:20] <paladox>	 !log git reinstalling jenkins
[22:24:24] <labs-morebots>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL, Master
[22:35:20] <wikibugs>	 06Labs, 10Labs-Other-Projects: video project:  move rendering instances to SSD servers - https://phabricator.wikimedia.org/T139802#2443163 (10Andrew)