[00:01:08] Zppix: if the tmux session is running on the remote side, then no. [00:01:39] being able to detach and reattach is one of the main features of tmux and screen [00:02:08] So i ran tmux on toolforge it wouldnt be killed if i was disconnected? [00:02:27] BUT... if you are running long running things on a toolforge bastion then they can and will be killed when an admin notices them [00:02:57] I was planning on running something on tmux for like an hour and then kill it [00:23:57] Also which dbs are more up to date analytics or labsdb? [00:26:13] Zppix: use the *.analytics.db.svc.eqiad.wmflabs dbs. They are faster [00:26:28] newer hardware [00:26:33] They seem to be out of date though [00:26:55] zero replag -- https://tools.wmflabs.org/replag/ [00:27:10] so, no. Not "out of date" [00:29:16] Hmm [00:29:47] do you have a specific thing that you are querying for that make you think there is data drift from production? [00:36:00] I was doing the query for userpages without a registered user account attached, and i noticed that some of the results didnt even have pages that existed [00:36:39] Like https://en.wikipedia.org/wiki/User:Article_created_during_a_Wikipedia_training_program,_we_request_a_second_reviewer_to_check_and_comment [00:38:58] What's your sql query? [00:41:27] https://www.irccloud.com/pastebin/7u6o2RNc [00:43:43] Reedy ^ [00:45:36] Something looks fishy about that query. It's not an outer join on user so user_name IS NULL should not match anything... I think [00:45:48] it's also going to have horrible runtime [00:46:28] bd808: it finished in like 10 mins [00:46:31] due to the double wildcard LIKE and RLIKE combo. That's a table scan of all things in NS 2&3 [00:46:41] And i took it from the example query page on wikitech [00:46:44] and returned 0 results? [00:47:01] sql on the wikitech page can be horrible :) [00:47:23] It returned a decent amnt [00:47:26] https://wikitech.wikimedia.org/wiki/Help:MySQL_queries#Ownerless_pages_in_the_user_space [00:48:28] anyway... how did that lead you to https://en.wikipedia.org/wiki/User:Article_created_during_a_Wikipedia_training_program,_we_request_a_second_reviewer_to_check_and_comment ? [00:48:56] that's a redlink page, but it showed in your results? [00:51:23] Yes [00:52:00] select * from page where page_title = 'Article_created_during_a_Wikipedia_training_program,_we_request_a_second_reviewer_to_check_and_comment' and page_namespace IN (2,3); -- Empty set (0.00 sec) [00:52:24] Why does it show in my query? [00:53:06] select * from user where user_name = REPLACE('Article_created_during_a_Wikipedia_training_program,_we_request_a_second_reviewer_to_check_and_comment', '_', ' '); -- Empty set (0.00 sec) [00:53:49] Zppix: good question, but one I can't answer I guess [00:54:14] since that title isn't in the enwiki db that I'm looking at (enwiki.analytics) [00:57:11] Let me run my query on quarry and ill see if you can see it [01:40:21] (03PS1) 10MacFan4000: Add documentation [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) [01:42:34] bd808: would regex work on binary data? [01:43:11] I mean for that crontab thing [01:43:46] the only thing really holding us back is it parsing the crontab and adding jsub [01:43:53] iirc [02:39:46] (03CR) 10Zppix: [C: 04-1] "It appears you documented every sopel module known tl man, please only document modules i use as well as defaults..." [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) (owner: 10MacFan4000) [02:40:14] bd808: O [02:40:17] ERM [02:40:22] I'm not horrible :( [02:40:44] SQL: then why you be giving me bad results xD [02:40:58] Zppix: Garbage in, garbage out? [02:41:19] SQL: thats not the relationship agreed upon :P [02:57:19] That should be the welcoming message when running the mysql command ;) [02:57:51] eddiegp_: what? [02:58:11] "Mysql is not a recycling facility. If you put garbage in, you'll get garbage out." [02:58:37] TBH i know very little about sql [02:59:45] Zppix: well, I'm 40, I have 3 kids, My favorite hobby is woodworking [02:59:51] I'm no better, I usually just manage to find some library hiding if from me xD [03:04:01] SQL: SELECT * FROM ssn [03:04:04] xD [03:09:20] Zppix: https://xkcd.com/1409/ [03:36:40] * Cyberpower678 shoves SQL in Zppix 's face [03:44:04] Cyberpower678: how rood [03:45:43] DELETE FROM people WHERE `irc_nick` = 'Zppix'; [03:46:01] * Cyberpower678 deleted Zppix [03:46:42] rip [03:47:24] * Zppix is now know as deleted [03:53:47] lol [06:33:25] bd808: I wonder if we can use surrogates [07:42:14] (03PS1) 10Zhuyifei1999: crontab: convert to mostly binary processing, and use surrogates for text [labs/toollabs] - 10https://gerrit.wikimedia.org/r/394923 (https://phabricator.wikimedia.org/T181948) [08:43:49] (03CR) 10jenkins-bot: Localisation updates from https://translatewiki.net. [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/394933 (owner: 10L10n-bot) [09:20:23] labsdb1005 got rebooted, check that your toolsdb connections work as intended [09:45:07] !log tools.stewardbots Taking down SULWatcher/WatcherSUL for maintenance [09:45:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [10:13:22] !log tools.stewardbots SULWatcher: Ran DELETE FROM logging WHERE l_timestamp<20171201000000; - [10:13:22] Query OK, 233730 rows affected (34.70 sec) [10:13:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [10:15:17] !log tools.stewardbots Query OK, 233730 rows affected (34.70 sec). Table now has 487 rows instead. Old data cleaned. [10:15:19] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [10:24:04] !log tools.stewardbots Fixed T151113 by cleaning data older than 20171201000000. [10:24:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [10:24:07] T151113: Delete old data and/or stop logging to stewardbots' SULWatcher SQL DB - https://phabricator.wikimedia.org/T151113 [10:29:02] !log tools.stewardbots Updated stewardbots to 2693b97f6 [10:29:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [10:29:23] !log tools.stewardbots And now restart SULWatcher, maintenance finished [10:29:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [17:09:53] (03PS2) 10MacFan4000: Add documentation [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) [17:20:22] Question: Is there a way to log client-side events (like EventLogging) on Toolforge? [17:33:18] something like Sentry? https://github.com/getsentry/sentry [17:38:14] oooo https://phabricator.wikimedia.org/project/profile/976/ [17:44:32] !log deployment-prep ORES: Try enwiki models on simplewiki, T181848 (6baed71) [17:44:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [17:44:41] T181848: Experiment with using English Wikipedia models on Simple English - https://phabricator.wikimedia.org/T181848 [18:09:22] I will let you know when I see ato_ around here [18:09:22] @notify ato_ [18:15:51] zhuyifei1999: I am afraid I have f*cked up something on my Locale and I obiously can not set it back as it was. [18:16:58] what;s your output of $ locale [18:17:02] *what's [18:18:05] I can't see ant relevant configuration in your tool directory that sets locale [18:18:56] Do you need all of 15 lines? [18:19:25] Almost all of it like this:LANG=en_US.UTF-8 [18:19:32] hmm [18:19:42] then it should be utf8 [18:19:51] only "LANGUAGE=" empty [18:20:10] That is good news [18:20:22] that is on the same shell you are executing crontab right? [18:20:37] yes [18:21:17] that's weird [18:21:48] your exception message shows it attempting to decode with ascii, and that should only happen with non-utf8 locales [18:22:29] * zhuyifei1999_ tests stuffs [18:26:45] ato_01: what about $ python3 -c 'print(__import__("locale").getpreferredencoding())' [18:28:30] and $ python3 -c 'print(__import__("locale").getlocale())' [18:29:18] UTF-8 and ('en_US', 'UTF-8') [18:30:31] then that means thelocale is correct [18:30:46] Sounds good [18:30:57] idk why crontab was trying ascii [18:31:05] * zhuyifei1999_ reads some python source [18:32:06] zhuyifei1999_: Once I have tried things like "sudo apt-get install language-pack-hu" and "sudo locale-gen hu_HU.UTF-8" but hopefully nothing happened because it asked for a password what I dod not know... [18:32:29] uh, don't do that [18:32:45] you can't sudo, and that won't fix your problem [18:32:53] even if you can [18:32:56] I dont do it again [18:34:09] actually, what's in $ python3 -c 'print(__import__("locale").getpreferredencoding(False))'? [18:34:39] UTF-8 [18:35:29] subprocess uses https://docs.python.org/3/library/io.html#io.TextIOWrapper, which has `locale.getpreferredencoding(False)` as encoding [18:36:13] if it returns utf-8, there would be no reason for python to decode subprocess data with ascii... [18:37:17] if you add some non-ascii data into your crontab, will it error out again? [18:39:43] if not, then I have no idea what's going on, since everything else says your locale is right [18:40:51] with the old crontab I receive an error message again [18:40:57] "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf3 in position 10: invalid continuation byte" [18:41:12] but I am happy for now [18:41:41] * zhuyifei1999_ tests with latin-1 crontab [18:43:58] oh well, at least it says utf8 not ascii [18:44:01] More happier could me make only a log with original hungarian letters like "ó" "á" etc. [18:45:27] ? [18:45:48] could you elaborate? [18:46:55] For example a log like https://tools.wmflabs.org/ato/log/frissdump.txt [18:47:11] It was nice up to ~2015-08-10 [18:47:31] And then something happened [18:48:16] And my hungarian letters are changed to "krix-krax" [18:49:00] it looks okay to me o.O [18:49:11] * zhuyifei1999_ needs to go to lunch, brb in ~30 min [18:49:27] ok, no prob. ;) [18:59:35] (03CR) 10Reception123: [C: 031] "Looks good to me, though since Zppix is the owner he should +2" [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) (owner: 10MacFan4000) [19:05:10] (03CR) 10Zppix: [C: 04-1] ".settimeforma is not a valid command." [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) (owner: 10MacFan4000) [19:16:43] ato_01: back [19:18:43] the first part is encoded in latin1 [19:20:20] later parts are utf8 [19:20:53] after the last line of A legfrissebb dump dátuma: 2015. augusztus 10. [19:21:51] ok [19:22:16] and there was a server change [19:22:29] or something like this [19:22:38] I can't find the relevant server change though [19:24:19] I have one more log. https://tools.wmflabs.org/ato/log/sign-of-life.txt [19:25:18] According that the change was at 2015-08-15 [19:33:06] zhuyifei1999_: I should go to sleep. Thank you for all. [19:33:15] k [19:53:43] (03PS3) 10MacFan4000: Add documentation [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) [19:53:59] (03CR) 10MacFan4000: "Fixed" [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) (owner: 10MacFan4000) [20:45:02] (03CR) 10Zppix: [C: 032] "Thank you very much!" [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) (owner: 10MacFan4000) [20:45:08] (03Merged) 10jenkins-bot: Add documentation [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/394906 (https://phabricator.wikimedia.org/T149500) (owner: 10MacFan4000) [20:53:27] !log tools.zppixbot Pulling [[gerrit:394906]] into public_html/ T149500 [20:53:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.zppixbot/SAL [20:53:30] T149500: Write documentation for ZppixBot - https://phabricator.wikimedia.org/T149500 [21:39:23] (03Draft1) 10Zppix: Fix syntax in documentation.html [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/395101 [21:39:28] (03Abandoned) 10Zppix: Fix syntax in documentation.html [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/395101 (owner: 10Zppix) [21:44:39] (03Draft2) 10MacFan4000: Fix [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/395112 [21:45:37] (03CR) 10Zppix: [V: 032 C: 032] Fix [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/395112 (owner: 10MacFan4000) [21:45:44] (03Merged) 10jenkins-bot: Fix [labs/tools/ZppixBot] - 10https://gerrit.wikimedia.org/r/395112 (owner: 10MacFan4000) [22:22:24] (03PS2) 10Krinkle: Add tests for App::getDB() [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/394705 [22:22:26] (03CR) 10Krinkle: [C: 032] Add tests for App::getDB() [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/394705 (owner: 10Krinkle) [22:23:20] (03Merged) 10jenkins-bot: Add tests for App::getDB() [labs/tools/guc] - 10https://gerrit.wikimedia.org/r/394705 (owner: 10Krinkle) [23:44:25] !help hello all! Is anyone good at using the logging table? I need to find when a page was locked against edits performed by IP addresses in 2016. I can't get a handle of how this works [23:44:25] tomthirteen: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [23:45:26] !help I see that under log_action if there is "protect" then IP addresses can't edit the page [23:45:26] tomthirteen: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [23:46:12] !help but I see that people have also closed the page to IP addresses under "modify." Is there any structure to this? [23:46:12] tomthirteen: If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-team [23:46:34] Please stop putting !help in all your msgs its only needed once [23:47:51] k sorry