[00:53:55] andrewbogott: btw, project creation request queue all done :) (just a FYI) [00:54:14] YuviPanda: thanks! [00:54:20] yw! [00:54:33] andrewbogott: now, new project request tickets would notify all 3 of us (and scfe_de) [00:54:50] and now I'm off for a bit. brb [01:05:12] (03PS1) 10Awight: alert fundraising to mw-core commits on our precious deploy branch [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176855 [01:06:23] 3Wikimedia-Labs-Other: Email not working on phab-01.wmflabs.org - https://phabricator.wikimedia.org/T76427#800116 (10Aklapper) [01:07:49] 3Wikimedia-Labs-Other: Email not working on phab-01.wmflabs.org - https://phabricator.wikimedia.org/T76427#800121 (10TTO) Because I want to test something related to e-mail notifications without spamming the production site. Is there somewhere else I can go to test e-mail in Phabricator? [01:08:29] 3Wikimedia-Labs-Other: Email not working on phab-01.wmflabs.org - https://phabricator.wikimedia.org/T76427#800125 (10TTO) Also, if e-mail is disabled there, it should be stated on the front page. At the moment, e-mail features appear enabled and seem to work, but actually don't do anything. [01:59:00] 3Tool-Labs-tools-Commons-Delinker: Live hacks to git - https://phabricator.wikimedia.org/T70273#800208 (10Liuxinyu970226) [05:22:48] pymysql.err.OperationalError: (1142, "SELECT command denied to user 'u3193'@'10.68.16.7' for table 'page_restrictions'") [05:22:53] Is anyone familiar with cause of this error? [07:03:30] 3Tool-Labs-tools-Other: [tracking] toolserver.org tools that have not been migrated - https://phabricator.wikimedia.org/T60865#800459 (10Nemo_bis) [07:03:31] 3Tool-Labs-tools-Other: Migrate to Tool Labs: https://toolserver.org/~vvv/adminstats.php - https://phabricator.wikimedia.org/T63030#800457 (10Nemo_bis) [10:11:03] Nemo_bis: you probably want to post your comment in T152 instead of T257 :P [10:11:16] T257 is bikeshedding on the exact naming, T152 is getting the export to work [10:16:20] valhallasw`cloud: no, I meant to have it where I put it [10:16:31] I don't care at all about the export working [10:16:44] I just want the hideous button to get out of my life [10:18:45] 3Phabricator, Wikimedia-Labs-Other: Email not working on phab-01.wmflabs.org - https://phabricator.wikimedia.org/T76427#800909 (10Qgil) p:5Triage>3Low I don't recall this being intentional. In fab, email notifications worked. I have also missed them for testing, and I had to test in production. Not a big dea... [10:21:06] Nemo_bis: how about 'make a new bug for that instead of hijacking an existing bug' [10:21:18] valhallasw`cloud: I filed the bug [10:21:49] ...so? [10:22:04] the original bug was about naming [10:22:11] making it about something else is confusing as hell [10:22:30] valhallasw`cloud: so, if you think 'Get rid of "Export to Excel"' is not what you desire, how about 'make a new bug for that instead of hijacking an existing bug' [10:22:58] Nemo_bis: that was /not/ what you desired in the original bug! [10:23:14] a bug is not 'these are Nemo_bis' requests', but 'this is this specific request'... [10:23:36] valhallasw`cloud: if you think that wasn't my desire, I'm sorry that you misunderstood [10:23:51] And I'm glad that the new summary is clearer. [10:24:36] Nemo_bis: well, better be consistent in your trolling, and also adapt the description, then. [10:25:11] valhallasw`cloud: just did so [10:25:17] Perfect. [10:25:28] I agree that some edits to the description made it less clear. [10:25:40] Now it better distinguishes issue from solutions. [10:26:10] I see. [10:26:19] Okay, I now finally understand what you meant [10:27:12] Get rid of "Export to Excel" suggested to me you wanted the button to disappear, rather than have it renamed [10:28:20] I don't care what way, I just want "Excel" to get out of my sight [12:14:10] 3Tool-Labs-tools-Other: Migrate https://toolserver.org/~daniel/WikiSense/* to Tool Labs and provide redirect - https://phabricator.wikimedia.org/T60869#801130 (10Liuxinyu970226) [12:14:21] 3Tool-Labs-tools-Other: Migrate https://toolserver.org/~bawolff/en-wn-editor-stats.php to Tool Labs - https://phabricator.wikimedia.org/T60867#801131 (10Liuxinyu970226) [12:15:11] 3Tool-Labs-tools-Other: Do something with globalusage account on Toolserver - https://phabricator.wikimedia.org/T63827#801133 (10Liuxinyu970226) [12:15:18] 3Tool-Labs-tools-Other: Move contests from toolserver to tool labs - https://phabricator.wikimedia.org/T63826#801135 (10Liuxinyu970226) [12:48:40] 3Tool-Labs-tools-Other: Migrate to Tool Labs: https://toolserver.org/~vvv/adminstats.php - https://phabricator.wikimedia.org/T63030#801201 (10vvv) So, do I actually have to go and reimplement adminstats? I would be happy to do that, but I never wanted to duplicate effort. [12:58:02] Coren: any luck on fixing bigbrother for trusty? [13:35:11] YuviPanda, how do I set the database for quarry? [13:35:20] valhallasw`cloud: 'use ' at start of query [13:35:26] ah! [13:35:34] :) [13:36:38] YuviPanda: and any way to make wikilinks? :> [13:36:58] valhallasw`cloud: sadly none yet, but I'm thinking of having a sandboxed jinja2 template there some time. [13:37:05] mkay [13:43:33] YuviPanda: also, should quarry reports end up here? [13:43:41] valhallasw`cloud: on IRC? [13:43:45] *nod* [13:43:49] quarry bug reports* [13:43:49] valhallasw`cloud: you mean bug reports? [13:43:52] ahehe :) [13:44:01] well, here and -research perhaps [13:44:44] valhallasw`cloud: there's a Quarry project [13:44:52] *nod* [13:45:02] it doesn't end up here because it's not Labs-* :-) [13:45:47] I still like the idea of specifying channels in an extra field for the project [13:45:59] but phab has no way of reading those fields via the API at the moemnt :-( [13:47:16] valhallasw`cloud: yeah, phab API seems... ugh [13:49:58] oh, I forgot the add the project [13:50:01] now I get your comment :-p [13:50:39] a warning 'no project selected' would be useful... [13:51:10] valhallasw`cloud: yup [13:51:24] that would be https://secure.phabricator.com/T5118 and WONTFIX [13:52:09] valhallasw`cloud: I wonder if we should setup a chrome extension / ff extension of sort to fix these things :) [13:52:15] hahaha [13:54:21] * valhallasw`cloud pours YuviPanda a cup of tea [13:55:01] :D [13:55:03] I've cokeee [13:55:04] and cold water [14:25:24] planemad: you should ask here :) [14:26:02] YuviPanda, ok! [14:26:39] planemad: I think valhallasw`cloud has some documentation on running flask stuff on toollabs. [14:26:46] I have written a simple flask api that can query a postgres database and return the results [14:26:53] I don't remember where it is, legoktm used to point me to it usually. [14:27:30] The app works fine locally, but am pretty clueless on how to have it running on toollabs [14:27:55] planemad: https://wikitech.wikimedia.org/wiki/Help:Tool_Labs/Web has documentation [14:28:02] there's an example configuration for flask there as well [14:29:14] YuviPanda, I dont see any mention of flask [14:29:29] planemad: it's collapsed, 'expand section'. plus I suspect you'll have to read a fair bit of that page. [14:29:57] ah ok [14:37:48] YuviPanda, is there a more detailed example? (Spoiler: I have 0 experience working with server stuff) [14:38:05] My api code is here: https://github.com/WikimapsAtlas/wikimapsatlas-server/blob/master/api/api.py [14:38:29] now where does the fcgi come in? [14:38:41] > https://github.com/valhallasw/gerrit-patch-uploader/blob/master/app.fcgi [14:38:55] planemad: fcgi connects the web server (lighttpd) to your code (python) [14:39:10] planemad: I'm in the middle of some slightly time critical work (re-imaging servers). can you email labs-l or wait for 2h? [14:39:22] no problem, thanks [14:43:29] planemad: right, you need the flask fcgi handler [14:43:42] planemad: https://github.com/valhallasw/gerrit-patch-uploader/blob/master/app.fcgi is the fcgi code that runs the actual app [14:43:56] https://github.com/valhallasw/gerrit-patch-uploader/blob/master/lighttpd.conf is the lighttpd config to make lighttpd call that fcgi script [14:44:38] planemad: I'm also busy, but I might be able to answer specific questions [14:55:55] PROBLEM - Puppet failure on tools-exec-wmt is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [15:07:44] why do i use the grid when the jobs aren't executed for ever and ever? grrr [15:08:57] annika_: what is the problem? [15:11:18] andrewbogott: btw, I worked around the havana/icehouse issue by just hacking the local puppet files to make default be havana instead of icehouse. still no login/auth because LDAP is dead. [15:11:44] ok, I'll look at ldap in a bit [15:12:38] andrewbogott: thanks! [15:13:31] YuviPanda: i submitted job dwl6a at 14:40, now it is 15:13, normally this works faster and i don't see the grid overloaded [15:14:25] it is still in the waiting state [15:15:16] let me check [15:19:37] annika_: what user is this? [15:19:37] err [15:19:39] what tool? [15:20:23] giftbot [15:20:54] RECOVERY - Puppet failure on tools-exec-wmt is OK: OK: Less than 1.00% above the threshold [0.0] [15:21:03] annika_: hmm, it does seem stuck. for now, have you tried qdel-ing it and then submitting again? [15:21:18] hm. i'll do that [15:30:44] annika_: did that work? [15:31:28] no, but i had the wrong queue anyway [15:31:50] the queue is full [15:31:50] hmm, ok [15:31:53] aaah [15:31:58] your node's queue is full? [15:32:11] yes [15:33:06] ... what? There are 200 slots on your node. [15:33:20] Or did you mean that the load was too high to start a new job? [15:33:34] i have my usual 200 jobs on it [15:33:40] so, no surprise [15:33:49] but i didn't know about the limit [15:34:15] That can be bumped up trivially if you need it - you're the only one on that node so you can load it as much as you want. [15:34:44] 200 is my default value for 'a lot'. :-) [15:35:16] :D [15:36:35] I just bumped it up to 300. [15:37:08] (Sadly, gridengine doesn't allow 'no limit' for slots) [15:39:35] great :) [15:39:56] coren: can you add me to tools.ggu again? [15:41:32] Done, but that'll only kick in on new sessions. [15:42:06] thank you :) [16:12:16] YuviPanda: can you log in to wikitech-test now? [16:12:28] andrewbogott: sure, moment [16:13:07] (I can't, possibly because I forgot my password) [16:13:32] andrewbogott: oh, that. I thought ssh. let me try [16:13:48] andrewbogott: nope, can't. [16:13:53] hm [16:14:12] [57c94267] [no req] Exception from line 2337 of /srv/vagrant/mediawiki/includes/User.php: There was either an authentication database error or you are not allowed to update your external account. [16:14:13] andrewbogott: ^ [16:14:22] andrewbogott: when I try to set my password manually with the maintainance script [16:14:30] php mediawiki/maintenance/createAndPromote.php --bureaucrat --sysop --force yuvipanda [16:15:30] Ah, vagrant wiped all the internal passwords. [16:15:44] oh?! [16:16:10] andrewbogott: I moved it to the precise-compat branch because we're on precise and it'll just fail hard otherwise. I guess that messed some things up. [16:16:10] looks like. In 10-WikitechPrivateSettings.php [16:19:18] try now? [16:19:22] The cmdline thing, I mean [16:21:03] YuviPanda: I can log in now [16:21:16] andrewbogott: yay, cool. works now [16:21:26] Is that all you needed to get unblocked? [16:28:48] andrewbogott: let me check. [16:55:14] PROBLEM - Free space - all mounts on tools-webproxy is CRITICAL: CRITICAL: tools.tools-webproxy.diskspace._var.byte_percentfree.value (<22.22%) [16:55:41] andrewbogott: hmm, http://wikitech-test.wmflabs.org/wiki/Special:NovaInstance still is empty, but I've to go eat now (plus helping with re-imaging as well) [17:07:20] <_joe_> !log deployment-prep: upgrading HHVM to its latest version [17:07:21] deployment-prep: is not a valid project. [17:07:33] <_joe_> uhm [17:07:40] <_joe_> !log beta: upgrading HHVM to its latest version [17:07:41] beta: is not a valid project. [17:07:56] <_joe_> andrewbogott: the project is deployment-prep [17:08:05] <_joe_> or am I missing something? [17:08:33] !log testlabs this is a test message [17:08:36] Logged the message, dummy [17:08:53] !log deployment-prep this is a test message [17:08:56] Logged the message, dummy [17:09:07] <_joe_> oh, the colons [17:09:13] must be [17:09:17] <_joe_> !log deployment-prep upgrading HHVM to its latest version [17:09:19] Logged the message, Master [17:10:12] RECOVERY - Free space - all mounts on tools-webproxy is OK: OK: All targets OK [17:34:44] (03CR) 10Ejegg: [C: 032] alert fundraising to mw-core commits on our precious deploy branch [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176855 (owner: 10Awight) [18:02:23] (03CR) 10Legoktm: "In the future please don't merge changes unless you plan on deploying them." [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176855 (owner: 10Awight) [18:03:51] !log tools.lolrrit-wm restarting grrrit-wm for https://gerrit.wikimedia.org/r/176855 [18:03:53] Logged the message, Master [18:05:28] hmm, hasn't come back yet. [18:11:06] (03CR) 10Legoktm: "Test" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176855 (owner: 10Awight) [18:11:32] (03CR) 10Legoktm: "I'm not sure why, but this broke the bot." [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176855 (owner: 10Awight) [18:11:42] (03PS1) 10Legoktm: Revert "alert fundraising to mw-core commits on our precious deploy branch" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176967 [18:11:56] (03CR) 10Legoktm: [C: 032 V: 032] Revert "alert fundraising to mw-core commits on our precious deploy branch" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176967 (owner: 10Legoktm) [18:11:59] (03Merged) 10jenkins-bot: Revert "alert fundraising to mw-core commits on our precious deploy branch" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176967 (owner: 10Legoktm) [18:15:30] valhallasw`cloud, need some help with the lighttpd.conf since i've never done it before [18:17:01] valhallasw`cloud, which line do I need to change for the url http://tools.wmflabs.org/wikiatlas2014/api to run the fcgi file? [18:17:42] Is this correct? fastcgi.server += ( "/wikiatlas2014/api" => [18:37:33] (03PS1) 10Awight: alert fundraising to mw-core commits on our precious deploy branch [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176977 [18:38:44] (03CR) 10Legoktm: [C: 032] "woohoo" [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176977 (owner: 10Awight) [18:38:47] (03Merged) 10jenkins-bot: alert fundraising to mw-core commits on our precious deploy branch [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176977 (owner: 10Awight) [18:40:07] !log tools.lolrrit-wm restarting grrrit-wm [18:40:09] Logged the message, Master [18:41:02] YuviPanda: It looks like the keystone DB on wikitech-test-frontend was also wiped. I'm going to set it up again, do you want to follow along? [18:41:05] Or have me walk you through it? [18:41:25] andrewbogott: yes, that would be nice! but I'm in a meeting now, though. [18:41:40] YuviPanda: ok, maybe after the SGE meeting [18:41:44] andrewbogott: yeah, that would be nice. [18:44:30] (03PS1) 10Awight: fix typo, [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176978 [18:44:47] (03CR) 10Legoktm: [C: 032] fix typo, [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176978 (owner: 10Awight) [18:44:50] (03Merged) 10jenkins-bot: fix typo, [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176978 (owner: 10Awight) [19:02:20] andrewbogott: YuviPanda: Give me 2 min to set up and I'm all yours. [19:22:07] (03PS1) 10Awight: Rewrite some flow-style YAML [labs/tools/grrrit] - 10https://gerrit.wikimedia.org/r/176989 [20:07:02] pleclown: yes, I think so [20:07:25] pleclown: maybe another / at the end, but without is probably also fine [20:09:54] YuviPanda / legoktm: not sure where that highlight was... last time the issue with gttttr was it tried to join a nonexistant channel (or it was banned somewhere or so), causing it to give up and retry joining all channels [20:11:11] oh [20:19:23] valhallasw`cloud: this time it was a syntax error :P [20:58:59] legoktm: D: [20:59:15] and somehow the yaml linter didn't catch it :/ [21:50:41] andrewbogott: can i grab a moment of your time ? [21:52:18] matanya: sure, what's up? [21:52:59] hi, can you help cajoel ( cajoel_ ) to get wikimania videos to labstore1001 ? [21:53:24] he has the source, but no access to labstore1001 [21:53:30] need to do a big ugly rsync [21:54:08] hm... [21:54:21] Did you already try doing it via the public IP of the video instance? [21:54:35] video instance? [21:55:12] yes andrewbogott no joy [21:55:19] hm… yesterday someone asked me for a public ip for the 'video' project, I presume for this. [21:55:21] What happened? [21:55:51] andrewbogott: so the file size is large enough that writing to NFS via a labs instance is going to be really, really slow. Need to just write to labstore and then read from labs instance. [21:55:54] andrewbogott: the public IP is useless [21:56:01] andrewbogott: files too big for varnish [21:56:17] Where are the files now? [21:56:27] wikimedia office [21:56:31] SF [21:56:41] Like, on a PC /in/ the office? [21:56:49] on a flash drive? [21:56:50] on a sever in a rack [21:56:55] 2T [21:57:12] ok [21:57:15] we put them on a drive and shipped them to VA, but it was corrupted somehow? [21:57:25] (that was trial #1) [21:58:17] So you need a keypair between that server and labstore… for example [21:58:34] sounds great [21:59:13] 1.4T of actual data [21:59:21] can you just give Daniel a login on the video server? Surely he has a root key for labstore [21:59:26] 10-15G files each [21:59:47] andrewbogott: I'd like to do that -- let me talk to him about that [22:00:10] ok. And, matanya, you ultimately want these files in /data/project/somethingsomething for the 'video' project, right? [22:00:23] yes [22:00:37] 'wmfvids'? [22:00:49] wfm [22:01:01] labstore is like any other prod server [22:01:05] no special key needed, just root [22:01:25] andrewbogott: wikimania2014 is what I call them [22:01:33] mutante: labstore doesn't have user accounts, only root. [22:01:48] andrewbogott: root@labstore1001:/srv/project/video/project/ [22:01:52] So your destdir will be root@labstore1001:/srv/project/video/project/wmfvids [22:02:02] heh, yes :) [22:02:04] want to run an old fashioned rsync port? [22:02:27] no ssh [22:02:29] chroot [22:02:36] good old file drop tech? [22:02:48] bitorrent? [22:02:52] andrewbogott: i don't know why labstore has no personal users, but just use rot [22:02:55] root [22:03:09] mutante: It's because it maintains file permissions for labs. [22:03:20] So it has all the ldap users and none of the normal production accounts. [22:03:26] And there are conflicts so we can't support both [22:03:58] ok, then you know more than i do. does it matter for copying though? [22:04:22] nope, just that you'll only be able to rsync as root [22:04:41] I can sort out the permissions of the copied files after the fact. [22:05:18] mutante: want to come type some rsync cli in to a terminal? [22:05:22] Also, Coren, you're aware that this massive copy is happening? We have plenty of room but this'll take up 10% of remaining free space. [22:05:23] no [22:05:27] and/or I can just make you a login to our backup server [22:05:37] yes he is aware [22:05:41] Coren: approved it [22:05:41] mutante: in the office? [22:05:44] mutante: cool [22:05:45] no [22:05:52] i dont have time for this, please [22:05:54] got vpn? :) [22:05:57] Oh, sorry, mutante, I only nominated you because I figured you were right there. [22:06:00] just run the rsync as root, ok [22:07:35] root password [22:07:43] andrewbogott: you can give me the root key and i can do it :P [22:07:56] * YuviPanda makes a key out of tree roots, gives to matanya [22:08:08] matanya: you don't have access to the SF server. :( [22:08:12] So, without a root in the office I don't think there's a good way to do this. [22:08:25] Erik? [22:08:25] Gage? [22:08:31] Thursday [22:08:33] 3Wikimedia-Labs-General: Community-maintained projects are hard to track - https://phabricator.wikimedia.org/T64837#802563 (10Quiddity) [22:08:41] Erik has root? [22:08:55] andrewbogott: bittorrent [22:09:12] kidding [22:09:35] * YuviPanda sets cajoel_ up with Ubuntu One [22:09:42] you can run rsync in daemon mode [22:09:48] and bypass ssh [22:09:54] I still sort of think that copying via a labs instance with a throttle is the best plan. And just accept that it'll take a couple of days. [22:10:15] andrewbogott: varnish is the blocker [22:10:15] cajoel_: does the server with the files have a public IP? [22:10:25] nope [22:10:27] behidn nat [22:10:36] How is varnish involved? [22:10:48] no idea why we think varnish is involved [22:10:54] some upload thing? [22:10:58] matanya: you mean to upload via a web interface? [22:11:32] no, downloading via http goes through varnish [22:11:33] lock file = /var/run/rsync.lock [22:11:33] log file = /var/log/rsyncd.log [22:11:35] pid file = /var/run/rsyncd.pid [22:11:35] [documents] [22:11:37] path = /home/juan/Documents [22:11:37] comment = The documents folder of Juan [22:11:39] uid = juan [22:11:39] gid = juan [22:11:41] read only = no [22:11:41] list = yes [22:11:43] auth users = rsyncclient [22:11:43] secrets file = /etc/rsyncd.secrets [22:11:45] hosts allow = 192.168.1.0/255.255.255.0 [22:11:54] rsync in daemon mode would take 5m [22:11:55] to setup [22:12:25] cajoel_: I'd prefer not to set up a one-off service on a production machine. [22:12:37] matanya, YuviPanda, tell me again why you don't just rsync to the labs instance? [22:12:51] not enough space [22:13:00] it's mounting the same drive, no? [22:13:02] andrewbogott: mostly because Coren asked matanya not to write to NFS that much. [22:13:10] that too [22:13:22] bwlimit [22:13:31] andrewbogott: but now that I think about it, network IN is going to be a bottleneck enough [22:13:42] and writing through a public IP on the instance to /data/project might be the simplest solution. [22:13:55] we have 200Mbit to production [22:14:02] no i know why that ticket was sitting there since august :P [22:14:20] hehe :) [22:14:28] *now [22:14:38] so... wait for someone with root to be in the office, or just put it through the labs instance. [22:14:47] there are lots of files [22:14:51] we could start some of them [22:14:56] that's true. [22:14:57] how do I get ssh to the labs instance [22:15:02] anything i can do to help ? [22:15:09] So, Coren, what's an acceptable NFS bandwidth for us to hog for days at a stretch? [22:15:18] We'll throttle the copy according to your preference [22:15:22] matanya: add cajoel_ to the project? [22:15:28] doing [22:15:31] Ima be offline for a couple hours while I rip Windows out of my new box and install a real OS now. [22:15:37] andrewbogott: ^ in another channel [22:15:40] hm, ok [22:15:59] cajoel_: what is your wikitech username ? [22:16:02] and there's a public ip on the instance for direct ssh? [22:16:05] jkrauska iirc [22:16:06] YuviPanda: do you know what the bandwidth is between labs and NFS? [22:16:15] i'll do that tto [22:16:38] andrewbogott: nope... looking up email, I remember hearing about it [22:16:42] th [22:16:43] thx [22:16:49] andrewbogott: look at '[Ops] The problem with NFS bandwidth ' [22:18:02] cajoel_: 208.80.155.196 [22:18:40] matanya: direct ssh? [22:18:44] doing [22:19:11] holler back when ready [22:19:13] cajoel_: add your key [22:19:13] andrewbogott: doesn't actually have full numbers tho. [22:19:20] and we are good to go [22:19:27] cajoel_: please rsync with --bwlimit=5000 for now. If that turns out to be unacceptably slow we can appeal to Coren when he appears. [22:19:42] matanya: not seeing open on port 22 [22:20:00] cajoel_: did you upload your key ? [22:20:12] to? [22:20:20] wikitech [22:20:35] yeah [22:20:37] cajoel_: Matanya is asking if you have ever had ssh access to a labs box. [22:20:37] port 22 is notopen [22:20:40] yeah [22:20:42] via bastions [22:21:18] running puppet to verify you are included [22:21:29] yes, you have access [22:22:02] matanya: the IP is not bound to an instance. I will fix... [22:22:04] firm? [22:22:25] ferm? (the iptables thing?) [22:22:34] no [22:22:57] andrewbogott: it seems attached to me :) [22:23:06] 'cause I just assigned it [22:23:13] thanks [22:23:33] cajoel_: can you ssh to encoding02.wmflabs.org ? [22:23:56] encoding02.wmflabs.org: nodename nor servname provided, or not known [22:24:21] do you use proxy command ? [22:24:49] nope [22:24:54] Public IP isn't listed [22:24:56] in the instance [22:25:03] cajoel_: try now [22:25:13] andrewbogott: in [22:25:15] you can use the name video-encoding02.wmflabs.org [22:25:16] 22 responding [22:25:29] matanya: he wasn't using proxy command, hence the public IP. [22:25:29] Host *.eqiad.wmflabs [22:25:29] ProxyCommand ssh -a -W %h:%p bastion1.eqiad.wmflabs [22:25:32] Anyway, I opened everything up :) [22:25:38] in using my key [22:25:45] We can change 22 back to 10.0.0.0/8 after this finishes [22:25:51] matanya: no need for proxy with key [22:25:56] ok [22:25:59] ferm rule then [22:26:06] great thanks for the effort [22:26:13] andrewbogott: what did you change ? [22:26:21] sec group ? [22:26:25] data/scratch? [22:26:30] yeah, security group [22:26:41] or public/dumps? [22:26:47] where to upload? [22:26:56] cajoel_: /data/project/wmfvids [22:27:39] andrewbogott: you could limit 22 to the office IP space [22:28:02] cajoel_: I'm not sure I can do that without blocking traffic to the bastion [22:28:17] Unless matanya strongly objects, I'm ok with leaving it fully open [22:28:28] no, nothing private there [22:28:56]