[10:31:44] <Tanvir>	 Hello, I can't login to tools lab. I tried and got "Permission denied (publickey,hostbased)."
[10:31:45] <Tanvir>	  Not sure why. Btw, I am trying logging in after a long time.
[10:35:46] <arturo>	 Tanvir: how long is long time? more or less
[10:36:23] <arturo>	 years? or months?
[10:36:51] <arturo>	 Tanvir: what is your username?
[10:38:36] <Tanvir>	 arturo, Tanvir Rahman
[10:38:58] <Tanvir>	 is my username. And I was inactive for more than a year I guess.
[10:39:31] <Tanvir>	 I just update my ssh public keys on both wikitech preference and gerrit.
[10:45:07] <arturo>	 I mean, shell username
[10:45:31] <arturo>	 Tanvir: could you try `ssh tools-bastion-03.eqiad.wmflabs` ? let's see if I can see a line in the logs
[10:46:01] <arturo>	 I see you in the member list of the `tools` project in CloudVPS
[10:47:09] <Tanvir>	 arturo, "tanvir"
[10:49:34] <Tanvir>	 arturo, tried and output https://pastebin.com/raw/LZHrCU3d
[10:58:50] <arturo>	 are you in bastion-01? which host is that?
[10:59:43] <arturo>	 that host may not have your priv/pub ssh keys
[10:59:58] <Tanvir>	 bastion-01
[10:59:59] <Tanvir>	 yes
[11:00:11] <Tanvir>	 tanvir@bastion-01:~$
[11:00:12] <Tanvir>	  <-- here.
[11:00:24] <arturo>	 that is a WMF host, right? 
[11:00:45] <Tanvir>	 As far as I know, yes.
[11:01:05] <arturo>	 try `ssh tools-bastion-03.eqiad.wmflabs` from your personal host
[11:01:56] <Tanvir>	 arturo, same thing. Permission denied.
[11:02:53] <Tanvir>	 Why don't they have my public keys? Should I upload my public keys anywhere else other than wikitech and gerrit?
[11:03:26] <arturo>	 as far as I know, adding it to wikitech will put your ssh key in ldap, which is all we need
[11:04:12] <arturo>	 try `ssh -v tools-bastion-03.eqiad.wmflabs` and see what ssh pubkey are you using
[11:05:59] <Tanvir>	 arturo, got a lot of lines. https://pastebin.com/raw/s9W6YV3s
[11:06:42] <chicocvenancio>	 Tanvir: why are you trying from bastion-01?
[11:07:03] <arturo>	 bastion-01 likely won't have your pubkey
[11:07:09] <chicocvenancio>	 What operating system are you using? 
[11:07:15] <Tanvir>	 Windows.
[11:07:21] <arturo>	 oh
[11:07:34] <Tanvir>	 And I tried "tanvir@bastion.wmflabs.org" from PuTTY. That brought me to bastion-01
[11:07:56] <chicocvenancio>	 Set up putty to connect to login.wmflabs.org
[11:08:08] <chicocvenancio>	 Scratch that
[11:08:14] <chicocvenancio>	 Login.tools.wmflabs.org 
[11:08:47] <chicocvenancio>	 Same settings as for bastion should work
[11:09:58] <Tanvir>	 I thing I am in the right place now. Just saw the big red TOOLFORGE.
[11:10:04] <chicocvenancio>	 But you can see more detailed instructions here Help:Access_to_Toolforge_instances_with_PuTTY_and_WinSCP
[11:10:24] <chicocvenancio>	 https://wikitech.wikimedia.org/wiki/Help:Access_to_Toolforge_instances_with_PuTTY_and_WinSCP
[11:10:41] <chicocvenancio>	 Tanvir:cool! 
[11:11:38] <Tanvir>	 chicocvenancio, now just "become <toolname>" right?
[11:13:50] <chicocvenancio>	 Yep
[11:14:20] <Tanvir>	 Jo. Worked.
[14:14:12] <Hauskatze>	 !help mail sent to tools.toolname@tools.wmflabs.org not arriving, even with a .forward file
[14:14:12] <wm-bot>	 Hauskatze: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team
[14:38:36] <zhuyifei1999_>	 Hauskatze: do you have a test case?
[14:38:51] <Hauskatze>	 yes
[14:39:06] <Hauskatze>	 mail to tools.maurelio@tools.wmflabs.org
[14:39:20] <Hauskatze>	 I can send from that address, I cannot receive anything
[14:39:45] <Hauskatze>	 https://wikitech.wikimedia.org/wiki/Help:Toolforge#Mail_to_a_Tool
[14:40:00] <Hauskatze>	 tried also to tools.mabot@... same result
[14:40:19] <Hauskatze>	 on .maurelio I set later a .forward file with an address on it
[14:40:28] <Hauskatze>	 same result: mail never arrives
[14:40:59] <chicocvenancio>	 From where are you sending these? 
[14:41:48] <zhuyifei1999_>	 !log tools `chown tools.pywikibot:tools.pywikibot /shared/pywikipedia/` Prior owner: tools.russbot:project-tools T192732
[14:41:50] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[14:41:50] <stashbot>	 T192732: Rename pywikipedia to pywikibot on Toolforge - https://phabricator.wikimedia.org/T192732
[14:43:13] <Hauskatze>	 chicocvenancio: I tried a self-send via my gmail account to tools.{maurelio|mabot}
[14:43:23] <Hauskatze>	 or does it work only for internal mail?
[14:43:42] <zhuyifei1999_>	 it should work for external afaik
[14:44:26] <Hauskatze>	 hmm, toolname.anything@tools.wmflabs.org -- isn't tools.{toolname} ?
[14:46:35] * zhuyifei1999_ brb 15 mins. will check the logs after that
[14:46:45] <Hauskatze>	 well, mail to maurelio.{random}@tools.wmflabs.org not delivered either
[14:46:53] <chicocvenancio>	 No, but an email to Merge2pdf.test@tools.wmflabs.org did fail for me with a `550 Unrouteable address` as well
[15:02:12] <zhuyifei1999_>	 Hauskatze: 2018-04-23 14:54:26 H=REDACTED.com [REDACTED] F=<REDACTED@REDACTED.com> temporarily rejected RCPT <maurelio.wmf@tools.wmflabs.org>: require_files: error for /data/project/maurelio/.forward.wmf: Permission denied
[15:02:48] <zhuyifei1999_>	 it can't read your forward files
[15:02:58] <zhuyifei1999_>	 the mail servers don't run as root
[15:03:05] * chicocvenancio wonders why it failed for me
[15:04:16] <zhuyifei1999_>	 2018-04-23 14:43:52 H=REDACTED.com [REDACTED] F=<REDACTED@REDACTED.com> rejected RCPT <Merge2pdf.test@tools.wmflabs.org>: Unrouteable address
[15:04:28] <zhuyifei1999_>	 maybe because of capitalization?
[15:04:56] <Hauskatze>	 zhuyifei1999_: and the tools.maurelio one?
[15:05:09] <chicocvenancio>	 Maybe, but a second one to merge2pdf.test@tools.wmflabs.org failed as well
[15:05:22] <chicocvenancio>	 This time without a message 
[15:05:46] <chicocvenancio>	 And that tool does not have a .forward file
[15:06:32] <Hauskatze>	 According to the docs, if the tool does not have a .forward file it does deliver the mail to the individual maintainers' mail addresses
[15:06:33] <zhuyifei1999_>	 Hauskatze: all of the lines that contain 'maurelio' and 'mabot' ends with 'Permission denied' afaict
[15:07:06] <Hauskatze>	 ~/maurelio/.forward is rw-r--r--
[15:07:17] <Hauskatze>	 so it should be readable by third parties as well
[15:07:47] <Hauskatze>	 resending one to tools.maurelio@tools.wmflabs.org
[15:09:18] <zhuyifei1999_>	 chicocvenancio: that has an id of 1fAcji-0000mf-Bf it seems successful
[15:09:27] <zhuyifei1999_>	 Hauskatze: the directory is not readable
[15:10:11] <zhuyifei1999_>	 https://phabricator.wikimedia.org/T164272
[15:11:33] <Hauskatze>	 zhuyifei1999_: maurelio should be
[15:11:48] <Hauskatze>	 mabot not
[15:11:54] <Hauskatze>	 but maurelio is empty
[15:13:12] <zhuyifei1999_>	 03:12:46 0 ✓ zhuyifei1999@tools-bastion-02: ~$ stat /data/project/maurelio/.forward.wmfstat: cannot stat ‘/data/project/maurelio/.forward.wmf’: Permission denied
[15:13:12] <zhuyifei1999_>	 03:12:50 1 ✗ zhuyifei1999@tools-bastion-02: ~$ ls -ld /data/project/maurelio
[15:13:12] <zhuyifei1999_>	 drwxrws--- 4 tools.maurelio tools.maurelio 4096 Apr 23 14:10 /data/project/maurelio
[15:13:12] <zhuyifei1999_>	 03:12:53 0 ✓ zhuyifei1999@tools-bastion-02: ~$ 
[15:13:35] <Hauskatze>	 chmod 775 maurelio
[15:14:25] <zhuyifei1999_>	 and is mail working now?
[15:14:25] <chicocvenancio>	 Try the email again
[15:14:42] <Hauskatze>	 drwxrwsr-x 4 tools.maurelio tools.maurelio 4096 Apr 23 14:10 /data/project/maurelio
[15:15:49] <Hauskatze>	 zhuyifei1999_: I'm starting to receive them now
[15:15:57] <zhuyifei1999_>	 k
[15:15:58] <andrewbogott>	 ragesoss: sorry I missed your response before.  I'd like to move that VM to a different host that's not so CPU bound — it'll cause 20 minutes or so of downtime.  Would that be OK?  and, if so, when is a good time for that to happen?
[15:16:35] <Hauskatze>	 maybe the mailer should run as root
[15:16:55] <chicocvenancio>	 Doesn't seem like a safe idea 
[15:18:02] <chicocvenancio>	 Any exploits in the mailing system would give root access 
[15:19:42] <Hauskatze>	 zhuyifei1999_: can you check the logs for 'strigiwm' as sender?
[15:19:58] <Hauskatze>	 they ain't arriving
[15:21:05] <zhuyifei1999_>	 2018-04-23 15:19:47 1fAdFq-0001gT-Kr => strigiwm@gmail.com <tools.maurelio@tools.wmflabs.org> R=dnslookup T=remote_smtp H=gmail-smtp-in.l.googl
[15:21:05] <zhuyifei1999_>	 e.com [209.85.144.27] X=TLS1.2:RSA_AES_128_CBC_SHA1:128 C="250 2.0.0 OK 1524496787 50-v6si6802046qtn.237 - gsmtp"
[15:21:05] <zhuyifei1999_>	 2018-04-23 15:19:47 1fAdFq-0001gT-Kr Completed
[15:21:42] <Hauskatze>	 weird, I've not got it
[15:23:33] <zhuyifei1999_>	 chicocvenancio Hauskatze: just making sure, the mails are not sent from an 'inbox' that is supposed to receive the mails, right? there are certain mail providers that de-duplicates mail so you don't receive a mail that you sent
[15:23:43] <zhuyifei1999_>	 afaik
[15:25:55] <Hauskatze>	 zhuyifei1999_: sending though the address that's supposed to receive the mail, so that might be the cause
[15:25:58] <Hauskatze>	 Gmail
[15:26:11] <Hauskatze>	 strigiwm -> tools.maurelio -> strigiwm
[15:26:19] <zhuyifei1999_>	 I can try mailing you if you want
[15:26:43] <Hauskatze>	 well, if you'd like to email tools.maurelio@tools.wmflabs.org
[15:29:13] <zhuyifei1999_>	 k
[15:31:35] <zhuyifei1999_>	 Hauskatze: I sent one with 'This is a exim4 test mail' as the title to that address
[15:31:45] <Hauskatze>	 zhuyifei1999_: just received it, thanks!
[15:31:50] <zhuyifei1999_>	 np
[15:41:27] <wikibugs_>	 (03Draft1) 10Paladox: Remove unused vars from shell script [labs/icinga2] - 10https://gerrit.wikimedia.org/r/428353
[15:41:29] <wikibugs_>	 (03Draft2) 10Paladox: Remove unused vars from shell script [labs/icinga2] - 10https://gerrit.wikimedia.org/r/428353
[15:41:32] <wikibugs_>	 (03CR) 10Paladox: [V: 032 C: 032] Remove unused vars from shell script [labs/icinga2] - 10https://gerrit.wikimedia.org/r/428353 (owner: 10Paladox)
[15:53:23] <wikibugs_>	 (03Draft1) 10Paladox: Fix scripts [labs/icinga2] - 10https://gerrit.wikimedia.org/r/428359
[15:53:25] <wikibugs_>	 (03Draft2) 10Paladox: Fix scripts [labs/icinga2] - 10https://gerrit.wikimedia.org/r/428359
[15:53:28] <wikibugs_>	 (03CR) 10Paladox: [V: 032 C: 032] Fix scripts [labs/icinga2] - 10https://gerrit.wikimedia.org/r/428359 (owner: 10Paladox)
[17:59:10] <Tanvir>	 Why after setting up the BotPasswords, the script still asks for "password for all accounts"?
[17:59:43] <chicocvenancio>	 what script?
[18:00:31] <Tanvir>	 I have sh script that runs the clean_sandbox.py
[18:01:34] <Tanvir>	 At the top the script it has the pwb.py login -all -pass so the bot will login first, but it asks for password when I run it.
[18:02:39] <Tanvir>	 But I did config BotPasswords in the user-config.py so shouldn't it take the password from user-password.py?
[18:35:03] <zhuyifei1999_>	 Tanvir: what tool is it?
[18:36:05] <Tanvir>	 zhuyifei1999_, you mean the script?
[18:36:16] <Tanvir>	 clean_sandbox.py
[18:36:41] <zhuyifei1999_>	 no, the tool account you are running the script under
[18:36:52] <zhuyifei1999_>	 it's running under toolforge, right?
[18:38:40] <zhuyifei1999_>	 Tanvir: 
[18:38:42] <Tanvir>	 zhuyifei1999_, yes
[18:39:08] <Tanvir>	 wikitanvirbot is the tool account
[18:39:15] <zhuyifei1999_>	 k
[18:40:27] <zhuyifei1999_>	 how did you setup pywikibot and/or BotPasswords? I don't see a user-config.py under either /data/project/wikitanvirbot/.pywikibot/ or /data/project/wikitanvirbot/
[18:42:41] <Tanvir>	 I downloaded my own copy of pywikibot. It should be under /data/project/wikitanvirbot/pywikibot
[18:42:50] <zhuyifei1999_>	 oh
[18:44:12] * zhuyifei1999_ is looking into how I did BotPasswords for my bot
[18:46:12] <Tanvir>	 I created BotPasswords in my home wiki (bnwiki). Setup the BotPasswords when I ran python generate_user_files.py.
[18:46:27] <zhuyifei1999_>	 Tanvir: can you try specifying an absolute path for password_file in user-config.py?
[18:46:34] <zhuyifei1999_>	 hmm
[18:46:40] <Tanvir>	 Trying to do that.
[18:49:02] <Tanvir>	 zhuyifei1999_, gotta go, be back in about half an hour.
[18:49:06] <zhuyifei1999_>	 ok
[19:22:38] <Tanvir>	 zhuyifei1999_, back
[19:22:44] <zhuyifei1999_>	 k
[19:23:14] <Tanvir>	 so my user-password.py and user-config.py are in the same directory. Should I still specify the path?
[19:25:00] <Tanvir>	 Btw, zhuyifei1999_, when I specified family and lang in my user-password.py it didn't ask to login to bnwiki but asked for password when it tried login to bnwiktionary.
[19:25:57] <Tanvir>	 I don't know why, but then I tried with login.py -all -pass, then it asked for password for all projects.
[19:26:37] <zhuyifei1999_>	 hmm
[19:29:37] <zhuyifei1999_>	 Tanvir: can I debug this on Thursday? I gtg in 10 mins
[19:29:53] <zhuyifei1999_>	 and will be busy on tomorrow & wednesday
[19:35:13] <Tanvir>	 zhuyifei1999_, sure thing. I will to solve myself in the mean time. Thanks for your help!
[19:35:22] <Tanvir>	 try to solve*
[19:35:40] <zhuyifei1999_>	 np
[20:51:31] <subbu>	 hello .. .i had a qn. about using php vs. hhvm in mediawiki via vagrant? is that managed manually via some config/puppet file on the VM or is there a vagrant role available for using php?
[20:55:54] <subbu>	 paladox, andrewbogott either of you know? ^
[20:56:27] <subbu>	 as far as we can tell, hhvm is not enabled as a role on the vm.
[21:11:41] <paladox>	 subbu php was made the default we we updated to stretch
[21:11:48] <paladox>	 i think hhvm can be installed
[21:12:13] <paladox>	 through the https://github.com/wikimedia/mediawiki-vagrant/blob/master/puppet/modules/role/manifests/hhvm.pp
[21:12:19] <subbu>	 i see .. well, that is another problem for us then .. this is a vm from 2 years back and it is still running ubuntu 14 trusty
[21:12:34] <subbu>	 so, as arlolra guessed, hhvm was probably the default back then.
[21:13:38] <subbu>	 is there a way to upgrade our vm oses without having to running vagrant destroy .. which i presume would destroy our mediawiki install (and the db)?
[21:13:52] <paladox>	 Nope i doint think so
[21:14:38] <subbu>	 meh .. ok.
[22:46:12] <zhuyifei1999_>	 !log quarry behaving abnormally. https://quarry.wmflabs.org/query/26629 has been queued for 16 mins... (hopefully) have some time to investigate
[22:46:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL
[22:50:37] <zhuyifei1999_>	 !log quarry Does quarry only have an effective concurrency limit of 3, despite having a few dozen celery worker processes?!
[22:50:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL