[00:33:24] <Ursula>	 Trusty is not doing, Allah is doing.
[00:33:45] <Ursula>	 I replied on https://phabricator.wikimedia.org/T204496
[01:23:05] <joehillen20>	 Αlⅼɑh is doiᥒg
[01:32:09] <prtksxna>	 bd808: Hey o/
[01:33:35] <prtksxna>	 Any clues about https://phabricator.wikimedia.org/T204191
[01:46:23] <zhuyifei1999_>	 prtksxna: tools.db.svc.eqiad.wmflabs is correct
[01:46:57] <prtksxna>	 zhuyifei1999_: When I try that I get - https://tools.wmflabs.org/wptestblog/wp-admin/install.php
[01:47:20] * zhuyifei1999_ is looking
[01:47:44] <zhuyifei1999_>	 is it on k8s?
[01:48:04] <prtksxna>	 zhuyifei1999_: How can I check that?
[01:48:49] <zhuyifei1999_>	 do you see anything in `kubectl get pods` or `qstat`?
[01:48:57] <zhuyifei1999_>	 the former is k8s, the latter is grid
[01:49:49] <prtksxna>	 zhuyifei1999_: qstat is empty, kubectl returns the name and stuff
[01:50:11] <prtksxna>	 Thanks for trying to help with this zhuyifei1999_ I really appreciate it :) 
[01:52:35] <zhuyifei1999_>	 I just strace'ed and can confirm it is connecting to 10.64.37.9 (the IP of tools.db.svc.eqiad.wmflabs)
[01:54:14] <zhuyifei1999_>	 [pid 14083] recvfrom(6, "M\0\0\4\377\25\4#28000Access denied for user 's53850'@'10.", 49, MSG_DONTWAIT, NULL, NULL) = 49
[01:54:14] <zhuyifei1999_>	 [pid 14083] poll([{fd=6, events=POLLIN|POLLERR|POLLHUP}], 1, 1471228928) = 1 ([{fd=6, revents=POLLIN}])
[01:54:15] <zhuyifei1999_>	 [pid 14083] recvfrom(6, "68.17.223' (using password: YES)", 93, MSG_DONTWAIT, NULL, NULL) = 32
[01:54:51] <zhuyifei1999_>	 are you sure you are using the correct username/password pair?
[01:55:58] <prtksxna>	 zhuyifei1999_: I am using user - s53850 and the password from replica.my.cnf
[01:56:19] <prtksxna>	 zhuyifei1999_: I have pasted the config file in the task too, could show you the password too if that helps :/ 
[01:56:47] <zhuyifei1999_>	 yes, I mean does php have the correct password? (I just confirmed with `sql local` that the one in replica.my.cnf works)
[01:57:02] <zhuyifei1999_>	 oh I'm root on toolforge. I can debug :)
[01:57:18] <prtksxna>	 PHP should be using wp-config, in which I think I have put everything correctly
[01:57:24] <prtksxna>	 zhuyifei1999_: You can see the file on the task too
[01:57:36] <prtksxna>	 :) 
[01:57:48] <zhuyifei1999_>	 public_html/wp-config.php? looking
[01:57:57] <prtksxna>	 zhuyifei1999_: Yep, that is the one
[01:59:16] <zhuyifei1999_>	 ah I see
[01:59:24] <zhuyifei1999_>	 check the last character :P
[01:59:42] <zhuyifei1999_>	 there's a typo
[02:00:56] <prtksxna>	 :| 
[02:01:38] * zhuyifei1999_ clears my console and pretend never seeing the password :)
[02:03:46] <prtksxna>	 Thanks zhuyifei1999_ :) 
[02:03:52] <zhuyifei1999_>	 np
[02:04:04] <zhuyifei1999_>	 you might want to close that ticket
[02:04:11] <prtksxna>	 BTW -  Did you make any other changes too? I fixed the password, but https://tools.wmflabs.org/wptestblog/wp-admin/install.php doesn't work any more
[02:04:32] <zhuyifei1999_>	 no, I only read the file
[02:05:00] <zhuyifei1999_>	 [pid 14081] write(5, "\1\7\0\1\0x\0\0PHP Parse error:  syntax error, unexpected end of file in /data/project/wptestblog/public_html/wp-config.php on line 95\n\1\6\0\1\0M\3\0Status: 500 Internal Server Error\r\nContent-type: text/html; charset=UTF-8\r\n\r\n\0\0\0\1\3\0\1\0\10\0\0\0\0\0\0\0/xh", 232) = 232
[02:05:05] <prtksxna>	 Typo again 
[02:05:07] <prtksxna>	 sorry
[02:05:11] <zhuyifei1999_>	 lol np
[02:06:09] <zhuyifei1999_>	 prtksxna: umm, you might want to chmod go-r /data/project/wptestblog/public_html/wp-config.php
[02:07:12] <prtksxna>	 Thanks zhuyifei1999_  - https://tools.wmflabs.org/wptestblog/ runs a wordpress now :) 
[02:07:35] <zhuyifei1999_>	 ok but, I just found I can read that file without being root, please chmod
[02:07:59] <prtksxna>	 zhuyifei1999_: Sorry I don't understand these things well. This would make it non-readable for everyone but me and root?
[02:08:19] <zhuyifei1999_>	 yes
[02:08:30] <zhuyifei1999_>	 or anyone else you add to the tool
[02:08:37] <zhuyifei1999_>	 (I just did it for you)
[02:08:43] <zhuyifei1999_>	 I mean the chmod
[02:09:33] <prtksxna>	 Oh, I was about to… Thanks zhuyifei1999_ 
[02:09:38] * zhuyifei1999_ should really PM next time I see something similar :(
[02:09:43] <zhuyifei1999_>	 np
[02:11:52] <zhuyifei1999_>	 prtksxna: oh btw for the manual of chmod, see https://linux.die.net/man/1/chmod (or `$ man chmod`)
[03:41:18] <Nineain12>	 Αlⅼɑh iѕ dоіᥒɡ
[03:45:52] <bd808>	 zhuyifei1999_: thanks for helping prtksxna out. :)
[03:47:26] <zhuyifei1999_>	 np :)
[04:55:31] <prtksxna>	 :) 
[06:42:54] <Iguil>	 Alⅼah ⅰs ԁoing
[06:42:54] <Iguil>	 s∪n is not ⅾoinɡ Allɑh iѕ doiᥒɡ
[06:56:57] <wikibugs>	 (03CR) 10Lokal Profil: "recheck" [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460716 (https://phabricator.wikimedia.org/T204351) (owner: 10Lokal Profil)
[06:57:03] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Make template addition dependent on starting parameter [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460716 (https://phabricator.wikimedia.org/T204351) (owner: 10Lokal Profil)
[07:14:37] <wikibugs>	 (03CR) 10Jean-Frédéric: Include empty sources in Monuments database statistics (031 comment) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459783 (https://phabricator.wikimedia.org/T203349) (owner: 10Jean-Frédéric)
[07:18:10] <wikibugs>	 (03PS4) 10Lokal Profil: Make template addition dependent on starting parameter [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/460716 (https://phabricator.wikimedia.org/T204351)
[07:55:53] <wikibugs>	 (03PS1) 10Lokal Profil: Add mechanism for hiding column from summation [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/461081
[07:58:01] <wikibugs>	 (03PS6) 10Lokal Profil: Use StatisticsTable [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459562
[08:21:06] <wikibugs>	 (03CR) 10Jean-Frédéric: [C: 032] Add mechanism for hiding column from summation (031 comment) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/461081 (owner: 10Lokal Profil)
[08:23:22] <wikibugs>	 (03Merged) 10jenkins-bot: Add mechanism for hiding column from summation [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/461081 (owner: 10Lokal Profil)
[08:25:03] <wikibugs>	 (03CR) 10jenkins-bot: Add mechanism for hiding column from summation [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/461081 (owner: 10Lokal Profil)
[08:46:20] <Frojoe>	 Aⅼlɑh is dоiᥒg
[09:03:47] <sgallagh10>	 Аⅼlah iѕ ԁoiᥒg
[09:20:09] <arturo>	 !log grantreview migrating project to eqiad1
[09:20:10] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Grantreview/SAL
[09:52:33] <Celmor>	 Αⅼlah iѕ ԁoiᥒg
[09:53:29] <avdi23>	 Aⅼlaһ іs doinɡ
[09:57:43] <suds12>	 Ꭺⅼⅼaһ iѕ ԁoіng
[09:57:43] <suds12>	 sᥙn iѕ nοt doіng Аⅼlaһ iѕ doіng
[09:57:43] <suds12>	 ⅿoഠn is not doing Αⅼlah is ԁoіᥒg
[10:03:20] <chronic>	 Allaһ is dοiᥒɡ
[10:38:48] <arturo>	 !log grantreview migration to eqiad1 completed, all seems working, but the webproxy http://grantreview-dev.wmflabs.org/ seems broken (not sure if was working previous to the migration)
[10:38:50] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Grantreview/SAL
[11:37:31] <paladox>	 arturo: delete and recreate the proxy 
[11:38:51] <arturo>	 paladox: trying
[11:42:38] <paladox>	 Ok ;)
[11:42:53] <paladox>	 It happened to me with one of the instances
[11:42:59] <paladox>	 Not all (I doint think)
[11:43:41] <arturo>	 anyway the VM is not listening on tcp/8080 and it wasn't before the migration
[11:43:50] <arturo>	 a that's where the proxy points to
[11:44:21] <arturo>	 so I won't do anything right now and will wait for others to be awake and clarify what's going on here
[11:44:28] <arturo>	 !log grantreview make myself projectadmin
[11:44:31] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Grantreview/SAL
[11:44:53] <arturo>	 paladox: are you in the US?
[11:45:32] <paladox>	 arturo: nope I am in the U.K.
[11:45:40] <arturo>	 ok
[12:02:59] <leelu>	 Аlⅼаh iѕ doing
[12:02:59] <leelu>	 ѕun ⅰs not doiᥒg Aⅼlaһ іѕ doiᥒɡ
[12:03:00] <leelu>	 ⅿооᥒ is ᥒοt dοiᥒg Αllah is ԁoіᥒɡ
[12:03:40] <TML24>	 Alⅼah iѕ doing
[12:38:02] <StepS>	 Ꭺⅼlaһ ⅰѕ ԁoiᥒg
[13:19:14] <gtirloni>	 !log wmcs-nfs created project
[13:19:15] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wmcs-nfs/SAL
[13:33:05] <gtirloni>	 !log wmcs-nfs pointed instances to nfs-puppetmaster-01.wmcs-nfs.eqiad.wmflabs
[13:33:06] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wmcs-nfs/SAL
[14:05:16] <riton22>	 Alⅼɑh іѕ dοiᥒg
[14:09:17] <teadrop___18>	 Αlⅼɑһ іs ⅾοіᥒg
[14:11:08] <wikibugs>	 (03CR) 10Lokal Profil: Add mechanism for hiding column from summation (031 comment) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/461081 (owner: 10Lokal Profil)
[14:11:55] <wikibugs>	 (03CR) 10Lokal Profil: Use StatisticsTable (032 comments) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459562 (owner: 10Lokal Profil)
[15:09:31] <bd808>	 arturo: I'm trying to see what is going on with the grantreview-03.grantreview.eqiad.wmflabs instance that you migrated to eqiad1. I can ping it from bastion, but my ssh session is hanging.
[15:09:50] <bd808>	 It gets to the key exchange but never gives me a shell.
[15:10:14] <arturo>	 weird
[15:10:29] <arturo>	 puppet ran well
[15:10:37] <bd808>	 I'll poke at it more after the morning meetings
[15:10:58] <arturo>	 BTW, there is a webproxy for 8080
[15:11:10] <arturo>	 but apparently nothing listens in that port
[15:11:30] <bd808>	 mediawiki-vagrant should if the LXC container is tarted
[15:11:35] <bd808>	 *started
[15:12:23] <bd808>	 https://phabricator.wikimedia.org/P7563 -- looks like the hang is actually getting from the bastion to the instance -- "debug1: getpeername failed: Bad file descriptor"
[15:16:02] <arturo>	 not sure what that means
[15:31:29] <wikibugs>	 (03PS1) 10Paladox: Up icinga2 time to reporting to every 1hr and 30mins [labs/icinga2] - 10https://gerrit.wikimedia.org/r/461144
[15:33:26] <wikibugs>	 (03PS2) 10Paladox: Up icinga2 time to reporting to every 1hr and 30mins [labs/icinga2] - 10https://gerrit.wikimedia.org/r/461144
[15:33:29] <wikibugs>	 (03CR) 10Paladox: [V: 032 C: 032] Up icinga2 time to reporting to every 1hr and 30mins [labs/icinga2] - 10https://gerrit.wikimedia.org/r/461144 (owner: 10Paladox)
[15:41:47] <wikibugs>	 (03CR) 10Lokal Profil: Include empty sources in Monuments database statistics (031 comment) [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/459783 (https://phabricator.wikimedia.org/T203349) (owner: 10Jean-Frédéric)
[15:49:51] <awight>	 Trying to find the iginga2-wm horizon project or server name...
[15:52:37] <awight>	 There it is, https://gerrit-icinga.wmflabs.org/authentication/login
[15:53:34] <awight>	 paladox: Can I ask for ao password reset on icinga2?
[15:53:40] <awight>	 s/ao/a/
[15:53:42] <paladox>	 awight yup
[15:54:26] <awight>	 ty!  We have some hosts acting up or decommissioned, I might need to disable in config.
[15:54:42] <paladox>	 awight i belive it's acting up due to neturon
[15:54:49] <paladox>	 gerrit-mysql is in neutron
[15:55:10] <paladox>	 awight you may need to ssh into one of the hosts and do "sudo service nagios-nrpe-server" and then tell me the ip it's rejecting.
[15:55:25] <awight>	 kk will do
[15:55:40] <paladox>	 *sudo service nagios-nrpe-server status
[15:56:18] <awight>	 paladox: > Server listening on 10.68.19.132 port 5666
[15:56:24] <awight>	 > Allowing connections from: 127.0.0.1,172.16.1.180
[15:56:34] <awight>	 This is for ores-worker-01.ores.eqiad.wmflabs
[15:56:36] <paladox>	 yep, though does it say host not allowed?
[15:56:53] <awight>	 not immediately obvious, I'll -v the logs
[15:57:03] <paladox>	 ok
[15:57:32] <paladox>	 telnet ores-web-01.ores.eqiad.wmflabs 5666
[15:57:33] <paladox>	 Trying 10.68.17.182...
[15:57:35] <paladox>	 fails on ^^
[15:58:48] <paladox>	 im wondering if the firewall is failing here.....
[15:59:19] <paladox>	 awight if you do "sudo iptables -A INPUT -p tcp --dport 5666 -j ACCEPT"
[15:59:22] <paladox>	 does that work?
[16:00:09] <awight>	 I don't see log messages about rejecting IPs, but do see this recent puppet diff,
[16:00:13] <awight>	 /Nrpe/File[/etc/nagios/nrpe_local.cfg]/content) -allowed_hosts=127.0.0.1,10.68.23.211
[16:00:16] <awight>	 /Nrpe/File[/etc/nagios/nrpe_local.cfg]/content) +allowed_hosts=127.0.0.1,172.16.1.180
[16:00:25] <paladox>	 yup that was today as the ip changed
[16:00:28] <paladox>	 for gerrit-mysql
[16:00:33] <paladox>	 (as it's in neutron now)
[16:00:37] <awight>	 sorry gtg for an hour...
[16:00:39] <paladox>	 ok
[16:00:47] <awight>	 Your guess makes sense, fwiw
[16:00:47] <paladox>	 awight try "sudo iptables -A INPUT -p tcp --dport 5666 -j ACCEPT" :)
[16:01:43] <paladox>	 yep
[16:03:26] <awight>	 there are no iptables
[16:03:47] <awight>	 Maybe it's the virtualization firewall
[16:04:57] <paladox>	 yup
[16:05:05] <paladox>	 is it in horizion for the ores project
[16:05:12] <paladox>	 ie in "security groups"?
[16:08:23] <awight>	 We have a nagios security group rule, accept port 5666 tcp from any.
[16:08:28] <awight>	 Muysterious!
[16:08:34] * awight lets the typo ride
[16:08:38] <paladox>	 hmm
[16:16:35] <harej>	 awight: this is an awesome coincidence because we just discussed this in our cloud services meeting
[16:16:40] <harej>	 port 5666 is a monitoring thing
[16:18:29] <awight>	 nagios :)
[16:18:53] <awight>	 harej: Apparently there may be a network thing responsible for all the icinga failures in #wikimedia-ai
[16:19:04] <awight>	 related to the neutron migration.
[16:19:34] <arturo>	 !log git make myself projectadmin
[16:19:37] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Git/SAL
[17:58:49] <quiddity>	 Urbanecm, re: "[Cloud] Python Flask app - localization" - Along with Intuition (https://tools.wmflabs.org/intuition/) there is also ToolTranslate (https://tools.wmflabs.org/tooltranslate/) but I'm not sure if that helps (IANAD) hence I'm replying here instead of the mailing list. :)
[17:59:16] <harej>	 There's also Worldly, which I created. I stopped short of recommending it because I'm self conscious about my work, but it's a very simple system that uses YAML files.
[17:59:21] <harej>	 https://github.com/harej/worldly
[17:59:39] <harej>	 Don't follow those "how to install" instructions. I have no idea what I was thinking.
[18:01:55] <harej>	 You're probably best off incorporating it as a library
[18:02:24] <harej>	 Worldly is used by Wikipedia Requests, https://tools.wmflabs.org/wprequests
[18:47:50] <zhuyifei1999_>	 There is also my simple https://github.com/toolforge/video2commons/blob/master/video2commons/frontend/i18n.py
[19:44:12] <gtirloni>	 !log wmcs-nfs recreated all servers from scratch with standalone puppetmaster
[19:44:13] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wmcs-nfs/SAL
[19:51:33] <Alforz29>	 Allɑһ is doіᥒɡ
[19:51:33] <Alforz29>	 ѕun is not dഠiᥒɡ Αllah іs doinɡ
[20:28:24] <Urbanecm>	 quiddity, thanks, but maybe I didn't explain my question enough. I'm thinking about a library that takes a directory from local disk and uses it in Flask's Jinja2 templates (I mean, I, as the developer, add message keys and English localization, translators (no matter if via tool you linked or via Translatewiki.net or via direct commits) adds localization and Flask selects the best locale and the tool can be truly international).
[20:29:12] <Urbanecm>	 To try with other words, I'm trying to use translation data I already have [https://phabricator.wikimedia.org/source/tool-wikinity/browse/master/messages/ is one example, but this is consumed via Intuition] in a Flask app
[20:30:11] <Urbanecm>	 In another words, my plan is to rewrite Wikinity to be a Flask app to stop me from having several Python script serving as CGI script consumed by JavaScript which is itself in a PHP file all to be able to use Intuition, which is PHP only
[20:30:24] <quiddity>	 Urbanecm, my fault entirely:  I'm not a dev, so I was just wanting to point out another tool related to Intuition that I was aware of. Everything else makes very little sense to me. >.<
[20:30:50] <Urbanecm>	 I'm just trying to explain what I'm looking for :D
[20:35:08] <Urbanecm>	 harej, looks like interesting thing to inspire from, looking into it
[20:51:35] <harej>	 Urbanecm: i don't blame you at all for not wanting to use gettext, for what it's worth
[20:51:38] <harej>	 it's oooold
[20:51:41] <harej>	 and really weird
[20:51:51] <harej>	 basically you have to pre-compile your message files to binary because that's how gettext rolls
[20:52:11] <Urbanecm>	 yup, as I said, on list, https://tools.wmflabs.org/commons-mass-description is running via gettext
[20:52:14] <harej>	 i'm pretty sure every computer these days is capable of it, but the workflow leaves much to be desired
[20:53:01] <bd808>	 gettext is a C library and not really planned for a rapid iteration web workflow
[20:53:24] <harej>	 Couldn't have put it better myself
[20:53:30] <Urbanecm>	 that's why I'm looking for something more prettier...
[20:53:37] <Urbanecm>	 ...and easily usable...
[20:53:43] <Urbanecm>	 ...and I couldn't find anything...
[20:53:52] <Urbanecm>	 ...so I wrote to the list and hoping somebody would have a good advice :D
[20:54:17] <harej>	 If you try out Worldly let me know how it works
[20:54:48] <bd808>	 I wrote a small PHP library for i18n that could be ported to Python pretty easily, but I'm not sure if I or anyone else would keep it maintained -- https://github.com/wikimedia/simplei18n
[20:55:18] <Urbanecm>	 looking
[20:55:44] * bd808 is amazed that the code there is 4 years old
[20:58:13] <Urbanecm>	 I feel like that's "you can inspire and write something own"... I think there _must_ be an existing solution, localization is something every WM tool should have...
[21:00:01] <bd808>	 *nod* interestingly I'm reviewing a position paper for the TechCon meeting next month that pretty much says the same thing. And that we (the movement/Foundation) should build out template solutions to make this less of a reinvention process for Tools devs
[21:00:02] <harej>	 Unfortunately, socializing and exchanging code is not one of our strong suits. A lot of wheel re-invention. In open source software in general, I'm actually surprised there isn't some popular Python alternative to gettext. Which is why I created one :P
[21:00:07] <harej>	 (I looked, really!)
[21:01:01] <bd808>	 Django wraps gettext. It is probably the python project that could have supported something easier for web dev work
[21:02:02] <harej>	 Our use case of supporting web apps in multiple languages is a surprisingly uncommon one.
[21:07:01] <Urbanecm>	 bd808, Tracker (WMCZ's expenses tracking app) is in Django and it works fine. https://phabricator.wikimedia.org/diffusion/WCTR/ has the source code. But Django's not everytime the right thing to use...
[23:32:11] <legoktm>	 gettext is a well recognized standard, so Python projects adopting it are showing their maturity in not reinventing a wheel :-)