[00:00:17] let me see if I can figure out changing the uid & dn too [00:00:56] looks like that is "changetype: modrdn" [00:01:21] I've always been paranoid about changing uid [00:01:32] I dunno what random things on random instances might reference it [00:01:56] I would be normally but there seems to be no sign that this account has been used inside Labs [00:04:42] 10Labs, 10Gerrit, 10wikitech.wikimedia.org: Request to rename LegoFan4000 to MacFan4000 on WikiTech - https://phabricator.wikimedia.org/T165624#3353570 (10bd808) I changed the uid as well. ``` $ ldapmodify -v -D 'uid=novaadmin,ou=people,dc=wikimedia,dc=org' -W -f MacFan4000-part2.ldif ldap_initialize( RainbowSprinkles: want to make a note about gerrit and close it? [00:14:01] Oh since you changed uid lemme swap his "username" [00:14:05] Since that's based on shell name [00:15:45] 10Labs, 10Gerrit, 10wikitech.wikimedia.org, 10Release-Engineering-Team (Kanban): Request to rename LegoFan4000 to MacFan4000 on WikiTech - https://phabricator.wikimedia.org/T165624#3353597 (10demon) 05Open>03Resolved a:03demon Updated on Gerrit as well. [00:17:12] RainbowSprinkles: I updated https://wikitech.wikimedia.org/wiki/Renaming_users with basically what I did [00:17:30] we should make a replacement script for this though [00:19:46] Indeed [00:20:16] We never had a "wrap the whole thing" script [00:20:25] Ideally said script could live on terbium [00:21:33] Er, gerrit master [00:21:35] Cobalt [00:21:41] *nod* [00:21:41] After we kill the Wikitech part [00:21:47] Then it's just ldap + gerrit sql [00:22:01] Could probably wrap the rename in a simple gerrit plugin too [00:22:04] Less chance for user error [00:23:43] RainbowSprinkles: want to make a task about it that we might find in 4-5 months when wikitech is just plain SUL? [00:24:04] there is talk about a real web ui for managing LDAP too [00:24:22] that would make all but the gerrit stuff much easier [00:28:31] * RainbowSprinkles can't bring himself to open yet another tab for the day [00:28:41] * bd808 can't either [00:28:50] it will come up again ;) [00:34:32] PROBLEM - Puppet errors on tools-exec-1436 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [00:58:38] 10Labs, 10Operations, 10hardware-requests: Eqiad: (2) hardware access request for labnet1003/1004 - https://phabricator.wikimedia.org/T158204#3353676 (10faidon) [00:58:41] 10Labs, 10Datasets-General-or-Unknown, 10Dumps-Generation, 10Operations, 10hardware-requests: Eqiad: Hardware request for labstore1006/7, dataset1002/3 - https://phabricator.wikimedia.org/T161311#3353679 (10faidon) [00:58:47] 10Labs, 10Operations, 10hardware-requests: Codfw: (1) hardware access request for labtestvirt2003 [region 2] - https://phabricator.wikimedia.org/T161765#3353685 (10faidon) [01:09:32] RECOVERY - Puppet errors on tools-exec-1436 is OK: OK: Less than 1.00% above the threshold [0.0] [01:33:14] 10Labs, 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10Release-Engineering-Team (Next): Fix ci puppet role to support stretch - https://phabricator.wikimedia.org/T166611#3353704 (10Dzahn) added stretch support for jenkins: https://gerrit.wikimedia.org/r/3... [01:34:33] 10Labs, 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10Release-Engineering-Team (Next): Fix ci puppet role to support stretch - https://phabricator.wikimedia.org/T166611#3353705 (10Dzahn) @paladox is the original "E: Package 'libmysqlclient-dev' has no ins... [03:40:29] PROBLEM - Puppet errors on tools-exec-1436 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [04:10:30] RECOVERY - Puppet errors on tools-exec-1436 is OK: OK: Less than 1.00% above the threshold [0.0] [05:10:34] 10Labs, 10DBA, 10User-bd808, 10cloud-services-team (Kanban): Prepare and check storage layer for atjwiki - https://phabricator.wikimedia.org/T167715#3353839 (10Marostegui) Similar to: T168021#3353382 I did: ``` GRANT SELECT, SHOW VIEW ON `atjwiki\_p`.* to labsdbuser; ``` And then ran the script: `sudo /us... [05:11:15] 10Labs, 10DBA, 10User-bd808, 10cloud-services-team (Kanban): setup dewiki and wikidatawiki on the labsdb1009, 1010 and 1011 - https://phabricator.wikimedia.org/T168021#3353312 (10Marostegui) Adding the grant did work on: T167715#3353839 [05:24:32] 10Labs, 10DBA, 10User-bd808, 10cloud-services-team (Kanban): Prepare and check storage layer for atjwiki - https://phabricator.wikimedia.org/T167715#3353848 (10Marostegui) 05Open>03Resolved a:05bd808>03Marostegui And views created on labsdb1010 and 1011 after adding the grant. Also created on labsd... [07:06:40] 10Labs, 10Tracking: Existing Labs project quota increase requests (Tracking) - https://phabricator.wikimedia.org/T140904#3353902 (10yuvipanda) [07:06:42] 10Labs: Request increased quota for paws labs project - https://phabricator.wikimedia.org/T168039#3353890 (10yuvipanda) [07:09:19] 10Labs, 10PAWS, 10Tool-Labs, 10Tools-Kubernetes: Consider moving PAWS to its own k8s cluster, rather than using Tools' k8s cluster - https://phabricator.wikimedia.org/T167086#3353905 (10yuvipanda) https://phabricator.wikimedia.org/T168039 for an IP quota increase :) I now have a fully functional (includin... [07:50:34] 10Labs, 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10Release-Engineering-Team (Next): Fix ci puppet role to support stretch - https://phabricator.wikimedia.org/T166611#3353944 (10Paladox) This ticket can be for master too. But was manly for contint. [08:21:09] 10MediaWiki-extensions-OpenStackManager, 10Librarization, 10MediaWiki-extensions-Translate, 10MW-1.30-release-notes (WMF-deploy-2017-05-23_(1.30.0-wmf.2)), 10Patch-For-Review: Bring in spyc for OpenStackManager and Translate via composer - https://phabricator.wikimedia.org/T75945#3353992 (10Umherirrender... [08:36:48] 10Tool-Labs-tools-Xtools, 10Community-Tech: Create an XTools logo - https://phabricator.wikimedia.org/T167345#3330053 (10Ricordisamoa) Here's a shitty logo design for $0 {F8466813} [09:14:04] 10Tool-Labs-tools-Xtools, 10Community-Tech: Create an XTools logo - https://phabricator.wikimedia.org/T167345#3330053 (10Malyacko) Have there been attempts to reach out to potential folks in the Wikimedia community, e.g. dropping a line on the [[ https://www.mediawiki.org/wiki/Design#Process | design mailing l... [10:40:31] 10Labs-project-Phabricator: Date problem on phab-01.wmflabs.org, comments timestamp was reset - https://phabricator.wikimedia.org/T165679#3354166 (10Aklapper) p:05High>03Triage >>! In T165679#3289315, @Paladox wrote: > Yep, seems to be working now I assume that means "not high priority" [10:40:56] 10Labs-project-Phabricator: Date problem on phab-01.wmflabs.org, comments timestamp was reset - https://phabricator.wikimedia.org/T165679#3354170 (10Paladox) yep, not high priority. [10:45:08] PROBLEM - Puppet errors on tools-exec-1410 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [10:47:30] 10Labs, 10Labs-Infrastructure, 10Continuous-Integration-Infrastructure, 10Nodepool, and 2 others: Lower rate of Nodepool requests to OpenStack API - https://phabricator.wikimedia.org/T167803#3354185 (10hashar) p:05Triage>03Normal [11:14:56] 10Labs-project-Extdist, 10VisualEditor: extdist tarball generator is erroring on VisualEditor REL1_23 - https://phabricator.wikimedia.org/T121748#3354283 (10hashar) 05Open>03declined REL1_23 is end of life. The extension distributor no more ships REL1_23 releases. [11:20:07] RECOVERY - Puppet errors on tools-exec-1410 is OK: OK: Less than 1.00% above the threshold [0.0] [11:41:07] PROBLEM - Puppet errors on tools-exec-1410 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [12:15:16] Hello Wikipedians [12:15:40] !HElp [12:15:47] !Help [12:16:01] what do you need help with? [12:17:13] There was an article that I wanted to create it had been deleted previously for poor language and COI by some user. But this time I made it with non-promotional language and proved notibility. But it was flagged for speedy deletion and deleted. [12:17:21] without any discussion [12:17:42] en.wikipedia.org/wiki/UrbanClap [12:18:26] did you try talking to the admin who deleted it? [12:19:04] not yet. But I am not not sure how to talk to the admin. [12:20:56] hm... [12:21:07] RECOVERY - Puppet errors on tools-exec-1410 is OK: OK: Less than 1.00% above the threshold [0.0] [12:21:08] It looks like it has been deleted more than once [12:21:36] also you might want to go to #wikipedia-en-help as this channel is about wikitech [12:29:20] 10Labs-project-Phabricator, 10Phabricator: Details should be hidden if empty - https://phabricator.wikimedia.org/T168058#3354562 (10Dvorapa) [12:36:59] 10Labs-project-Phabricator, 10Phabricator: Add descriptions to Create Task (Simple) - https://phabricator.wikimedia.org/T168060#3354607 (10Dvorapa) [12:40:44] 10Labs-project-Phabricator, 10Phabricator: Advanced search should support any() and not() functions - https://phabricator.wikimedia.org/T168061#3354622 (10Dvorapa) [12:49:36] 10Labs-project-Phabricator, 10Phabricator: Add descriptions to Create Task (Simple) - https://phabricator.wikimedia.org/T168060#3354661 (10Aklapper) 05Open>03Invalid This already exists: You have to click the {icon search} icon on the right of the "Tags" field on https://phabricator.wikimedia.org/maniphest... [12:50:25] 10Labs-project-Phabricator, 10Phabricator: Add descriptions to Create Task (Simple) - https://phabricator.wikimedia.org/T168060#3354664 (10Aklapper) (Duplicate of {T234}, basically) [13:11:42] 10Labs, 10Tracking: Existing Labs project quota increase requests (Tracking) - https://phabricator.wikimedia.org/T140904#3354720 (10Andrew) [13:11:43] 10Labs: Request increased quota for paws labs project - https://phabricator.wikimedia.org/T168039#3354717 (10Andrew) 05Open>03Resolved a:03Andrew done [13:16:44] 10Labs, 10Labs-Infrastructure, 10Operations, 10ops-codfw: rack/setup/install labtestpuppetmaster2001 - https://phabricator.wikimedia.org/T167157#3354726 (10Papaul) I troubleshoot this with Daniel by replacing replacing eth0 with eth1 MAC address in the DHCP file but same problem can not boot also from eth... [13:53:18] !log wikispeech Deploy latest from Git master: a58b224 (T164822) [13:53:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikispeech/SAL [13:53:21] T164822: mw-parser-output class breaks Wikipeech - https://phabricator.wikimedia.org/T164822 [14:28:02] 10Tool-Labs-tools-Other: multidesc: cannot connect to database due to reading wrong replica.my.cnf file - https://phabricator.wikimedia.org/T166949#3312597 (10Krinkle) I ran into the same problem with tools.orphantalk. was consistently failing for over a week (according to... [14:29:09] 10Labs, 10Labs-Infrastructure, 10Tools-Kubernetes: Homedir/UID info breaks after a while in Tools Kubernetes (can't read replica.my.cnf) - https://phabricator.wikimedia.org/T166949#3354977 (10Krinkle) [14:29:22] 10Labs, 10Labs-Infrastructure, 10Tools-Kubernetes: Homedir/UID info breaks after a while in Tools Kubernetes (can't read replica.my.cnf) - https://phabricator.wikimedia.org/T166949#3354993 (10Krinkle) p:05Triage>03High [15:18:31] 10Tool-Labs-tools-Xtools, 10Community-Tech: Create an XTools logo - https://phabricator.wikimedia.org/T167345#3355176 (10MusikAnimal) >>! In T167345#3354015, @Ricordisamoa wrote: > Here's a shitty logo design for $0 > {F8466813} Not bad in my opinion! This doubles as a text logo and the same X design could be... [15:18:57] 10Labs, 10Labs-Infrastructure, 10Tools-Kubernetes: Homedir/UID info breaks after a while in Tools Kubernetes (can't read replica.my.cnf) - https://phabricator.wikimedia.org/T166949#3355177 (10Magnus) Yes, I have been restarting at least a dozen tools because of this over the last week or two (?). [15:42:29] halfak: it looks like you are stopping tools-login again… are you doing this stuff by hand or is it some automated thing running there? [15:47:49] andrewbogott, ahh sorry I didn't realize I was causing a problem. [15:47:53] * halfak kills his process. [15:48:14] thanks [15:48:19] I just want to run split on a few files and thought it'd be OK to do that on login. [15:48:27] I guess it take more CPU than I thought. [15:49:53] * halfak moves over to dev [15:50:08] it's mostly IO that we've been short on. I don't know if there's a good way to throttle that, but you can always submit these things to the grid [15:50:45] is there a | in the command? [15:50:54] like 'cat | split | foo' [15:50:59] chasemp, there isn't. [15:51:01] if so teh pv utility can ratelimit [15:51:04] ah [15:51:04] split filename [15:51:12] I could do that though [15:51:15] I could also nice [15:51:19] But that won't help IO [15:52:15] halfak: generally if you are putting pv like "pv -L 3k" in between reading teh file and operating on it teh end IO usage is similarly constrained [15:52:36] idk if 'cat foo | pv -L 3k | split ' works but it would help most likely [15:53:09] chasemp, would it be OK if I ran the same thing unconstrained on dev or should I do a still limit there? [15:53:32] it's probably going to cause the same havok but it's depends on how /long/ that runs [15:53:47] 10 minutes of big job there isn't perceived the same as 1 hour etc [15:54:00] better to farm it out to the grid at that point [15:54:10] it will find a less populated host and not clutter up the bastion [15:54:26] every /instance/ has its own IO quota to NFS (assuming you are reading or writing from NFS shares) [15:55:01] Hmm... I think I'll just try to do this job on another server and then move it to tools :) [15:55:07] heh ok :) [15:55:28] It's fine I swear but it is more complicated [15:55:29] * halfak frees up 30 GB for the work [15:55:51] Right. I need to start this job in 5 minutes or it will wait until next week :S [15:56:01] Damn time constraints! [15:56:13] andrewbogott, thanks for the ping. [15:56:59] no problem, sorry to interrupt your workflow [15:57:19] na. s'cool. Cost of working in shared spaces and all :) [15:57:36] I just realized I've spelled havok wrong for 10 years because of the x-men character [15:57:42] for adults it's havoc [15:58:26] lol [16:03:36] Also https://en.wikipedia.org/wiki/Havok_(software) [16:36:47] 10Tool-Labs-tools-Xtools, 10Community-Tech: Create an XTools logo - https://phabricator.wikimedia.org/T167345#3355311 (10kaldari) I like it! [17:56:21] !log git running apt-get dist-upgrade on gerrit-mysql to ensure running on latest packages and updates [17:56:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL [18:34:32] PROBLEM - Puppet errors on tools-exec-1403 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [18:54:40] (03Draft2) 10Zppix: change old project names for projects maintained by Zppix and add an additional project [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/359497 [19:09:31] RECOVERY - Puppet errors on tools-exec-1403 is OK: OK: Less than 1.00% above the threshold [0.0] [19:13:46] 10Labs, 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10Release-Engineering-Team (Next): Fix ci puppet role to support stretch - https://phabricator.wikimedia.org/T166611#3355739 (10hashar) p:05Triage>03Normal [19:14:03] 10Labs, 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10Release-Engineering-Team (Kanban): Fix ci puppet role to support stretch - https://phabricator.wikimedia.org/T166611#3301885 (10hashar) [19:20:38] 10Labs, 10Labs-Infrastructure, 10WikiApiary: Wikiapiary project instance needs more local disk - https://phabricator.wikimedia.org/T162534#3355789 (10bd808) [19:20:51] 10Labs, 10Tracking: Existing Labs project quota increase requests (Tracking) - https://phabricator.wikimedia.org/T140904#3355791 (10bd808) [19:20:53] 10Labs, 10Labs-Infrastructure, 10WikiApiary: Wikiapiary project instance needs more local disk - https://phabricator.wikimedia.org/T162534#3166400 (10bd808) [19:22:42] 10Labs, 10Labs-Infrastructure, 10WikiApiary: Wikiapiary project instance needs more local disk - https://phabricator.wikimedia.org/T162534#3166400 (10bd808) I removed the quota tracking bug as a parent here. It sounds like you need to apply `role::labs::lvm::srv` to allocate currently allowed disk quota on t... [19:25:53] PROBLEM - Puppet errors on tools-exec-1407 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [19:26:23] bd808: any luck with https://phabricator.wikimedia.org/T167985? [19:27:41] bearloga: I stalled out on it and asked andrew if he could look but haven't heard back. Let me try to figure it out again [19:28:43] bearloga: do you mind if I add myself as a project admin in shiny-r for a bit to try and debug live? [19:29:56] * bd808 is WP:BOLD [19:30:14] !log shiny-r Added self (BryanDavis) as project admin to debug T167985 [19:30:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Shiny-r/SAL [19:30:17] T167985: Horizon bug: hidden web proxy after deleting instance - https://phabricator.wikimedia.org/T167985 [19:34:49] Urbanecm: https://phabricator.wikimedia.org/T60784 [19:35:42] that's the real cause of the issue you sent to labs-l [19:35:51] not python2 vs 3 [19:37:38] bd808: don't mind at all :) [19:37:52] I figured ;) [19:38:30] I got no better error trace from the horizon server's logs, but I have one more place to look (the dynamicproxy server) [19:57:11] (03Draft2) 10Zppix: Make readme look pretty [labs/icinga2] - 10https://gerrit.wikimedia.org/r/359506 [19:58:32] paladox ^ [19:59:30] 10Labs, 10Horizon, 10User-bd808, 10cloud-services-team (Kanban): Horizon bug: hidden web proxy after deleting instance - https://phabricator.wikimedia.org/T167985#3355870 (10bd808) The only log message on the server is: ``` [Fri Jun 16 19:32:16.106226 2017] [:error] [pid 1715] Recoverable error: Duplicate... [20:05:52] RECOVERY - Puppet errors on tools-exec-1407 is OK: OK: Less than 1.00% above the threshold [0.0] [20:08:07] 10Labs, 10Horizon, 10User-bd808, 10cloud-services-team (Kanban): Horizon bug: hidden web proxy after deleting instance - https://phabricator.wikimedia.org/T167985#3355910 (10bd808) Our dns server has records for both discovery-beta and discovery that point to the Labs proxy ip: ``` $ dig discovery-beta.wm... [20:12:07] 10Labs, 10Horizon, 10User-bd808, 10cloud-services-team (Kanban): Horizon bug: hidden web proxy after deleting instance - https://phabricator.wikimedia.org/T167985#3355942 (10bd808) a:05bd808>03Andrew [20:13:04] bearloga: you have uncovered a new and exciting DNS leak! andrewbogott wants to take a look and try to find the root cause [20:20:32] (03CR) 10Paladox: [V: 032 C: 032] "Thanks." [labs/icinga2] - 10https://gerrit.wikimedia.org/r/359506 (owner: 10Zppix) [20:26:48] !log ores staging ores-wmflabs-deploy:7529113 [20:26:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL [20:26:57] <3 stashbot [20:30:23] All looks good. [20:37:11] 10Labs, 10Tool-Labs: Cron Spam: Package 'sudo-ldap' has conffile prompt and needs to be upgraded manually - https://phabricator.wikimedia.org/T168094#3355995 (10Reedy) [20:38:14] 10Labs, 10Tool-Labs: Cron Spam: Package 'sudo-ldap' has conffile prompt and needs to be upgraded manually - https://phabricator.wikimedia.org/T168094#3356008 (10Andrew) a:03Andrew [20:44:33] bd808: yay! neat :) [20:47:19] I want some fast temp storage on dev tools that won't bother nfs. Where should I dump some files? [20:47:27] Only need about 4GB max [20:52:30] !log ores deploying ores-wmflabs-deploy:7529113 [20:52:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL [21:05:59] halfak: /tmp is usually good [21:06:09] bd808, cool I'll use that. thanks [21:06:11] that's local to each host [21:06:21] !log ores deploying ores-wmflabs-deploy:c952f57 (reverting due to issues) [21:06:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL [21:06:26] the IOPS problems are a pain :/ [21:19:53] PROBLEM - Puppet errors on tools-worker-1027 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [21:19:55] 10Labs, 10Tool-Labs: Shell LOCALE neither consistent nor sane across grid engine nodes - https://phabricator.wikimedia.org/T60784#3356109 (10bd808) [21:34:52] !log ores staging ores-wmflabs-deploy:e8f6932 [21:34:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL [21:37:21] !log ores deploying ores-wmflabs-deploy:e8f6932 [21:37:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Ores/SAL [21:54:53] RECOVERY - Puppet errors on tools-worker-1027 is OK: OK: Less than 1.00% above the threshold [0.0] [22:15:05] 10Labs, 10PAWS, 10Tool-Labs, 10Tools-Kubernetes: Consider moving PAWS to its own k8s cluster, rather than using Tools' k8s cluster - https://phabricator.wikimedia.org/T167086#3356203 (10yuvipanda) I've a working cluster on the PAWS project now! https://paws.deis.youarenotevena.wiki/hub/login :D I think n... [22:20:43] madhuvishy: is it possible to use mailx in SWAP? i tried from within a notebook and from within a terminal. it gets stuck permanently without error message until one does interrupt the kernel / hit ^C [22:20:59] would be useful to send out a notification once a long-running query has completed [22:23:40] madhuvishy: ... gets stuck when trying to send an email, i mean (mailx -s "test" ..@wikimedia.org) [22:24:58] 10Labs, 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10Release-Engineering-Team (Watching / External): Fix ci puppet role to support stretch - https://phabricator.wikimedia.org/T166611#3356212 (10greg) (meta: moving to #releng-external as this is not being... [22:32:11] HaeB: madhu is off doing fun stuff today [22:33:15] bd808: ok thanks [22:33:42] bd808: wait, are you saying getting mailx to work in jupyter isn't fun? ;) [22:33:42] probably worth a phab task to at least track the problem/idea [22:33:52] heh [22:34:35] I would guess that there is no outbound MX route or that its trying to talk to a gateway that is blackholeing the traffic [22:35:46] 10PAWS: Ability to view other users' notebooks - https://phabricator.wikimedia.org/T156980#3356240 (10Tbayer) [22:36:14] 10PAWS: Functionality to share & view SWAP notebooks - https://phabricator.wikimedia.org/T156934#3356241 (10Tbayer) [22:39:44] 10PAWS: mailx gets stuck trying to send out email from SWAP - https://phabricator.wikimedia.org/T168103#3356269 (10Tbayer) [22:40:33] 10PAWS: mailx gets stuck trying to send out email from SWAP - https://phabricator.wikimedia.org/T168103#3356269 (10Tbayer) From @bd808 on IRC: "I would guess that there is no outbound MX route or that its trying to talk to a gateway that is blackholeing the traffic" [22:42:39] HaeB: are you literally jsut running `mailx -s "test" ..@wikimedia.org` or did you pipe something to stdin? [22:42:58] literally that [22:43:05] mailx reads the message body from stdin by default [22:43:05] i mean, with my actual email address [22:43:18] so it's waiting for you to give it something to send [22:43:27] oh! see that's why i ask on irc first ;) [22:43:44] try `echo foo|mailx ...` [22:44:12] although it does normally prompt you for text, no? [22:44:16] (when running locally) [22:44:17] it still may not work, but it will probably fail differently [22:44:24] it just sits there [22:44:47] then when you hit ctrl-D to end stdin it will prompt you for a cc: [22:47:41] 10PAWS: mailx fails trying to send out email from SWAP - https://phabricator.wikimedia.org/T168103#3356319 (10Tbayer) [22:47:58] bd808: ok, it's failing better now ;) [22:48:24] :) [22:48:50] yeah, missing exim config seems more "normal" [22:53:09] 10PAWS: mailx fails trying to send out email from SWAP - https://phabricator.wikimedia.org/T168103#3356322 (10bd808) >>! In T168103#3356269, @Tbayer wrote: > From @bd808 on IRC: "I would guess that there is no outbound MX route or that its trying to talk to a gateway that is blackholeing the traffic" This was b...