[00:59:21] <Sveta>	 bd808: dupdet tool died again, takes forever to load and it does not automatically restart
[11:26:37] <arturo>	 !log tools merged change to the sudo module to allow sssd transition
[11:26:46] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[12:16:36] <arturo>	 !log tools T221225 set sssd/sudo in the hiera config for the tools-sgebastion prefix, and reboot tools-sgebastion-07/08
[12:16:39] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[12:16:39] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[12:27:06] <arturo>	 !log tools T221225 set sssd/sudo in the hiera config for the tools-docker-registry prefix, and reboot tools-docker-registry-[03-04]
[12:27:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[12:27:09] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[12:31:31] <arturo>	 !log tools T221225 set sssd/sudo in the hiera config for the tools-checker prefix, and reboot tools-checker-03
[12:31:33] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[12:32:33] <arturo>	 !log tools drop the tools-bastion puppet prefix, unused
[12:32:35] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[12:39:50] <arturo>	 !log tools T221225 disable puppet in all tools-worker nodes in preparation for sssd
[12:39:52] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[12:39:53] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[13:04:48] <arturo>	 !log tools T221225 depool and rebooted tools-worker-1001 in preparation for sssd migration
[13:04:52] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[13:04:52] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[15:37:57] <bd808>	 Sveta: we need logs and tracking of what is going wrong to help. This is exactly the sort of thing we try to track with Phabricator tasks. I don't think things are going to work out in trying to get real time help from me here on irc.
[17:07:59] <arturo>	 !log tools T221225 switch to sssd/sudo in puppet prefix for tools-worker
[17:08:02] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[17:08:04] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[17:09:51] <arturo>	 !log tools T221225 depool & switch to sssd/sudo & reboot & repool tools-worker-1001
[17:09:54] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[17:12:55] <arturo>	 !log tools T221225 depool & switch to sssd/sudo & reboot & repool tools-worker-1002
[17:12:57] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[17:26:55] <arturo>	 !log tools T221225 hard reboot tools-worker-1001
[17:27:07] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[17:27:07] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[17:35:12] <arturo>	 !log tools T221225 hard reboot tools-worker-1001 again
[17:35:14] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[17:35:15] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[17:36:03] <Lucas_WMDE>	 reboots will continue until functionality improves?
[17:38:18] <arturo>	 :-/ eventually it will work after a reboot
[17:44:21] <arturo>	 !log tools T221225 back to classic/ldap hiera config in the tools-worker puppet prefix
[17:44:25] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[17:44:25] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[17:49:12] <arturo>	 !log tools T221225 repool tools-worker-1002 (using nscd/nslcd and sudoldap)
[17:49:14] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[18:13:20] <arturo>	 !log tools T221225 created tools-worker-1029 to test sssd/sudo stuff
[18:13:23] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[18:13:23] <stashbot>	 T221225: sssd integration needs to be updated to include sudo config from LDAP support - https://phabricator.wikimedia.org/T221225
[18:14:18] <bstorm_>	 !log admin T209527 switched mounts from labstore1003 to cloudstore1008 for scratch
[18:14:20] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[18:14:20] <stashbot>	 T209527: Set up scratch and maps NFS services on cloudstore1008/9 - https://phabricator.wikimedia.org/T209527
[18:15:06] <arturo>	 !log tools T221225 for the record, tools-worker-1001 is not working after trying with sssd
[18:15:08] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[18:17:40] <valhallasw`cloud>	 James_F: do you want to change the ret['message'] = event['change']['subject'] or just try it as-is first?
[18:17:55] <valhallasw`cloud>	 (in the first case, I'll wait, in the second case, I'll merge and deploy)
[18:20:26] <James_F>	 valhallasw`cloud: I was going to leave it for now and consider doing it as a separate patch.
[18:20:33] <valhallasw`cloud>	 James_F: sounds good to me
[18:20:46] <James_F>	 Cool.
[18:20:55] <valhallasw`cloud>	 Thanks for looking into this!
[18:21:20] <James_F>	 Thanks for helping our developers remaingproductive. :-)
[18:21:51] <valhallasw`cloud>	 Now let's see if I can still remember how to deploy wikibugs :-p
[18:23:11] <James_F>	 C+2 and auto-restart?
[18:23:22] <James_F>	 Oh, it's not just a config change. Hmm
[18:23:49] <valhallasw`cloud>	 I think it's a python setup.py deploy kind of thing
[18:24:21] <valhallasw`cloud>	 but then I first have to make sure it's actually merged
[18:24:23] * valhallasw`cloud eyes jenkins
[18:28:06] <valhallasw`cloud>	 %$%$#@%$#@
[18:29:17] <valhallasw`cloud>	 ok, that's one
[18:30:13] <valhallasw`cloud>	 ok, that seems functional
[18:30:48] <valhallasw`cloud>	 James_F: I think everything is up and running agai
[18:30:48] <James_F>	 Should it be back in other channels?
[18:30:48] <valhallasw`cloud>	 n
[18:30:58] <valhallasw`cloud>	 no, it only joins when it needs to report the first message
[18:31:14] <valhallasw`cloud>	 Freenode doesn't like it when bots try to join hundreds of channels :-)
[18:31:18] <James_F>	 Oh, right.
[20:50:53] <tomthirtee>	 !Help Hi, can someone help me with my directory? I can't get any script or command to work in mnt/nfs/labstore-secondary-tools-home/tomthirteen
[20:51:48] <tomthirtee>	 ! I get "Unexpected directory listing line 'sql frwiki_p select'. Invalid rights description 'ql frwiki'" It screwed up a command script I ran
[20:54:35] <bstorm_>	 tomthirteen: you seem to have some interesting things happening with your scripts?  I don't see anything wrong with your directory
[20:54:54] <bstorm_>	 There is a file named `Miss_Peregrines_Home_for_Peculiar_Children_fr.txt;??sql frwiki_p select`
[20:56:01] <bstorm_>	 I was able to touch a test file and remove it as you in the directory.
[20:57:26] <tomthirtee>	 bstorm I get this https://imgur.com/a/YQ8upLj
[20:57:39] <tomthirtee>	 It's all cocked-eyed
[20:57:52] <tomthirtee>	 This has happened before and they were able to help
[21:00:46] <bd808>	 tomthirtee: I'll poke around a bit. That error message makes me think that your scp client doesn't like some file names in your $HOME
[21:02:08] <bd808>	 tomthirtee: You ended up with a filename that has an embedded newline from some shell rediection gone horribly wrong. It's an empty file, so I'm going to remove it.
[21:02:25] <tomthirtee>	 > bd808 Thanks! This is the result from ls https://imgur.com/a/dttNNoF
[21:03:14] <tomthirtee>	 > bd808 I am trying to run a bash file with commands like sql frwiki_p 'select * from logging where log_title = "Albanie" and log_timestamp < 20170101000' > Albania_fr.txt;
[21:03:46] <bd808>	 tomthirtee: yeah, it was that Miss_Peregrines_Home... file. The ?? you see in ls was actually a \r\n pair in the file name.
[21:04:01] <tomthirtee>	 Can I do this with parentheses? It seems to trip on sql frwiki_p 'select * from logging where log_title = "Ouragan_Matthew_(2016)" and log_timestamp < 20170101000' > Hurricane_Matthew_fr.txt;
[21:05:55] <bd808>	 tomthirtee: SQL strings should be quoted with single quotes rather than double quotes. You probably need single quotes for the SQL string to keep the '*' from expanding, so I guess you will need to escape the single quotes around string literals with a backslash do make that kind of query using shell arguments
[21:06:08] <bd808>	 *to make that
[21:06:13] <tomthirtee>	 How come I can't see the r/n/
[21:06:46] <bd808>	 tomthirtee: because the shell didn't know how to display it I guess. I could see the \r\n in the name when I tried to do tab completion
[21:07:30] <tomthirtee>	 I'm sorry, bd808. I didn't understand you.
[21:07:43] <bd808>	 tomthirtee: you have some other 'interesting' file names in there: ".भारत .txt" and ".भारत .txt?"
[21:08:13] <tomthirtee>	 Where are you seeing that?
[21:08:59] <bd808>	 tomthirtee: I can see them when doing an `ls -al` directory listing in your $HOME on a Toolforge bastion
[21:09:29] <bd808>	 the -a is for "all files" (include things that start with a literal .)
[21:09:54] <bd808>	 and -l for "long" listings that give permissions and size information
[21:10:34] <tomthirtee>	 Ok. I am sorry. Could you write out what you meant in the explanation?
[21:11:05] <bd808>	 tomthirtee: which explanation was unclear?
[21:11:40] <tomthirtee>	 SQL strings should be quoted with single quotes rather than double quotes. You probably need single quotes for the SQL string to keep the '*' from expanding, so I guess you will need to escape the single quotes around string literals with a backslash do make that kind of query using shell arguments
[21:12:16] <tomthirtee>	 I am using python to create sql commands from a list of titles that I need to search in 270 language wikis
[21:12:51] <tomthirtee>	 it's like sql frwiki_p 'select * from logging where log_title = "Ouragan_Matthew_(2016)" and log_timestamp < 20170101000' > Hurricane_Matthew_fr.txt;
[21:13:04] <tomthirtee>	 with the log_title and name of txt file as variables
[21:16:38] <bd808>	 tomthirtee: the 'log_title = "Ouragan_Matthew_(2016)"' part is where you are using double quotes (") rather than single quotes ('). This might be what is causing your query to fail when the string literal contains parentheses.
[21:17:25] <bd808>	 if you are using python to generate the command, you could try switching to actually using a python library to query the database instead of shelling out to the `sql` wrapper script to do the query
[21:20:06] <tomthirtee>	 Ok, should I switch it then to sql frwiki_p "select * from logging where log_title = 'Albanie' and log_timestamp < 20170101000" > Albania_fr.txt;
[21:21:12] <tomthirtee>	 and why doesn't it not handle the log_timestamp correclty?
[21:22:15] <bd808>	 tomthirtee: I don't know. I think this is the first I've heard you mention the log_timestamp?
[21:22:44] <tomthirtee>	 it keeps returning empty set when I manually type it in, but without the log_timestamp it returns over 70 records
[21:24:34] <bd808>	 tomthirtee: and you are looking for records from before 2017-01-01T00:00Z? Or did you mean to use > for the date comparison?
[21:25:00] <tomthirtee>	 yes
[21:25:25] <bd808>	 there were two different questions there. :)
[21:25:37] <tomthirtee>	 yes, yes
[21:25:50] <tomthirtee>	 well, < before
[21:26:05] <tomthirtee>	 and then > after for outfile
[21:26:57] <tomthirtee>	 select * from logging where log_title = "Albanie" and log_timestamp < 20170101000 returns an empty set when there are records
[21:31:13] <bd808>	 tomthirtee: ah. I see the problem now. You need more digits in your timestamp. The format in the database is YYYYMMDDHHMMSS. 20170101000 would be 0002-01-70T10:10:00
[21:31:56] <tomthirtee>	 ugh, I'm SORRY!
[21:31:59] <bd808>	 So you need 3 more 0's at the end to get to what you intended: 2017-01-01T00:00:00
[21:32:00] <tomthirtee>	 dumbass mistake
[21:32:15] <bd808>	 no worries. its easy to miss that sort of thing
[21:38:54] <tomthirtee>	 bd808 so this should be no prob: sql frwiki_p "select * from logging where log_title = 'Ouragan_Matthew_(2016)' and log_timestamp < 20170101000000" > Hurricane_Matthew_fr.txt;
[21:42:02] <bstorm_>	 !log admin unmounting labstore1003-scratch on all cloud clients
[21:42:03] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[21:42:59] <bd808>	 tomthirtee: I get 13 results when I run that interactively
[21:45:15] <tomthirtee>	 yup me too :)
[21:45:47] <tomthirtee>	 but can I script that for all the titles and wikis I have to search? 
[21:58:52] <bd808>	 tomthirtee: give it a shot. :) YOu can always come back and ask more questions if it continues to fail for you
[22:00:57] <tomthirtee>	 sorry for the million questions
[22:01:11] <tomthirtee>	 how are we handling apostrophes in the names?
[22:01:50] <tomthirtee>	 This raises an exception: sql frwiki_p "select * from logging where log_title = 'Miss_Peregrine_et_les_Enfants_particuliers' and log_timestamp < 20170101000000" > Miss_Peregrine's_Home_for_Peculiar_Children_fr.txt;
[22:08:56] <bd808>	 tomthirtee: In mysql/mariadb you escape a single quote in a string literal by doubling it. You really might get better results if you switch to using pymysql to make your queries. It knows how to handle all the string escaping. There is a handy wrapper library for it too that takes care of most of the details of connecting -- https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Python
[22:15:53] <tomthirtee>	 bd808 >  wow, I didn't know an apostrophe need to be doubled! I will investigate pymysql
[22:16:21] <tomthirtee>	 bd808 > Again, thank you SO much!
[22:16:37] <bd808>	 tomthirtee: you are welcome :)
[22:19:42] <tomthirtee>	 You're the best. Have a good da.
[22:19:45] <tomthirtee>	 day*