[00:36:06] zhuyifei1999_: Did that work for you? [00:36:44] I went to switch the hack script on the tool to use this instead of take and seeing chown complain with missing operand [00:38:33] userhomes-hack.bash in tools.paws [00:38:40] feel free to fix and resubmit [00:38:46] to the grid with jsub [00:39:00] madhuvishy: that was running as root [00:39:32] aaah [00:39:46] I don't think tools.paws itself can run chown as root [00:43:31] zhuyifei1999_: yeah makes sense. I decided to just submit it as a job from root [00:43:38] so we can stop doing this manually [00:43:53] k [00:44:01] I think that'll probably take care of it [00:57:33] it would be better if paws created the directory with the right permission itself [00:57:57] otherwise we'd always have 500s the first time one launch the server [01:20:59] (03PS1) 10Legoktm: Save logs right away, don't let containers accummulate [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411962 [01:21:13] (03CR) 10Legoktm: [C: 032] Save logs right away, don't let containers accummulate [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411962 (owner: 10Legoktm) [01:21:29] (03Merged) 10jenkins-bot: Save logs right away, don't let containers accummulate [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411962 (owner: 10Legoktm) [01:21:58] zhuyifei1999_: the first attempt takes a while anyway (5min timeout) [01:22:12] ? [01:22:22] what first attempt? [01:22:36] For PAWS first access [01:23:05] The dirs are only created for the first time a user visits PAWS [01:23:31] And the user pod creation will take a while [01:51:19] chicocvenancio: ok [03:21:02] (03PS1) 10Legoktm: Don't continually try and remove containers [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411967 [03:21:15] (03CR) 10Legoktm: [C: 032] Don't continually try and remove containers [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411967 (owner: 10Legoktm) [03:21:44] (03Merged) 10jenkins-bot: Don't continually try and remove containers [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411967 (owner: 10Legoktm) [03:23:37] (03PS1) 10Legoktm: Accidentally deleted a line [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411968 [03:23:45] (03CR) 10Legoktm: [C: 032] Accidentally deleted a line [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411968 (owner: 10Legoktm) [03:24:08] (03Merged) 10jenkins-bot: Accidentally deleted a line [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/411968 (owner: 10Legoktm) [12:59:32] (03Draft1) 10MarcoAurelio: add 'minus-x' check/fix to composer.json [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/412442 [12:59:35] (03PS2) 10MarcoAurelio: add 'minus-x' check/fix to composer.json [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/412442 [13:00:02] (03CR) 10jerkins-bot: [V: 04-1] add 'minus-x' check/fix to composer.json [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/412442 (owner: 10MarcoAurelio) [13:06:39] (03PS3) 10MarcoAurelio: add 'minus-x' check/fix to composer.json [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/412442 [13:07:09] (03CR) 10jerkins-bot: [V: 04-1] add 'minus-x' check/fix to composer.json [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/412442 (owner: 10MarcoAurelio) [13:08:27] (03Abandoned) 10MarcoAurelio: add 'minus-x' check/fix to composer.json [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/412442 (owner: 10MarcoAurelio) [13:18:37] (03PS1) 10MarcoAurelio: add minus-x check/fix [labs/tools/stewardbots] - 10https://gerrit.wikimedia.org/r/412469 [13:54:56] (03PS1) 10Umherirrender: Add mediawiki/oauthclient-php to upgrade list [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/412488 [19:21:28] Amir1: there's a few dexbot greps that are bringing tools-bastion-03 to a grinding halt [19:21:47] valhallasw`cloud: shoot [19:21:49] Amir1: would it be possible to run those on the grid, or alternatively on one of the -dev hosts? [19:21:51] let me stop it [19:22:48] thanks! [19:23:00] thanks valhallasw`cloud [19:23:48] valhallasw`cloud: unfortunately, I'm just looking for an old code :( it's not a "real" thing [19:23:56] let me narrow my grep [19:24:01] that might help [19:24:19] -r greps easily consume too much io and halts the bastion [19:24:33] * chicocvenancio has been guilty of doing similar things in the past [19:24:44] grep -i "resultexcelc.txt" */*/*.py [19:24:58] I hope this is fine, let me know if it's making issues [19:25:09] Amir1: tools-dev.wmflabs.org is a better place to run it (although the grid would still be preferrable) [19:25:28] Amir1: but at least with tools-dev it's the secondary bastion, which affects much less people [19:26:05] I see, thanks :) [19:26:54] bd808_: thanks for the merge request, deployed. [19:36:33] hi, how many long lagged in toolforge replicated databases? [19:37:31] rxy: https://tools.wmflabs.org/replag/ [19:37:45] thanks! [19:38:07] 13 minutes... [19:39:05] yes, that should be fine for analytics usage. Real-time tools should use the web.db cluster which has <1s replag [19:39:27] * chicocvenancio remembers days long replag in toolserver [19:40:03] toolserver lol [19:46:17] !log rcm Xenon: Running update [19:46:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Rcm/SAL [20:05:26] (03CR) 10Legoktm: [C: 032] Add mediawiki/oauthclient-php to upgrade list [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/412488 (owner: 10Umherirrender) [20:05:41] (03Merged) 10jenkins-bot: Add mediawiki/oauthclient-php to upgrade list [labs/libraryupgrader] - 10https://gerrit.wikimedia.org/r/412488 (owner: 10Umherirrender)