[00:00:05] 10PAWS: Paws display 502 - Bad gateway error - https://phabricator.wikimedia.org/T172080#3484841 (10SamanthaNguyen) [00:15:26] 10Quarry, 10Patch-For-Review: Excel does not recognize Quarry CSV output as UTF-8 - https://phabricator.wikimedia.org/T76126#3484849 (10IKhitron) @zhuyifei1999, thank you very much! I have not excel today, so I asked somebody to check. He is absolutely happy. [00:21:10] 10Quarry: Quarry's indentation function is not completely functional - https://phabricator.wikimedia.org/T101424#1338373 (10zhuyifei1999) https://github.com/codemirror/CodeMirror/issues/1942 suggests to use `CodeMirror.extendMode("sql", {electricChars: ")"});` [00:31:07] PROBLEM - Puppet errors on tools-exec-1409 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [00:31:23] 10Quarry: Add an option to export result in Wikilist - https://phabricator.wikimedia.org/T137268#3484860 (10zhuyifei1999) >>! In T137268#2808805, @Dvorapa wrote: > I know this method, but it is super complicated to do it this way, then export results e.g. in excel format, then get everything from excel format in... [00:31:36] 10Quarry: Add an option to export result in Wikilist - https://phabricator.wikimedia.org/T137268#3484861 (10zhuyifei1999) p:05Triage>03Low [00:37:50] 10Quarry: Quarry runs thousands times slower in last months - https://phabricator.wikimedia.org/T160188#3091603 (10zhuyifei1999) This task is essentially "Invalid" unless some clear steps-to-reproduce are provided. [00:40:06] 10Quarry: Investigate redash.io (open source query and report system) - https://phabricator.wikimedia.org/T131651#3484867 (10zhuyifei1999) [00:40:09] 10Cloud-Services, 10Quarry: Consider moving Quarry to be an installation of Redash - https://phabricator.wikimedia.org/T169452#3484870 (10zhuyifei1999) [00:48:53] 10Quarry: Quarry runs thousands times slower in last months - https://phabricator.wikimedia.org/T160188#3484872 (10IKhitron) How can I do this? I know that it happens, but can't bring you the same run before and after without a time machine. [00:49:22] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [00:56:21] 10Quarry: Quarry's indentation function is not completely functional - https://phabricator.wikimedia.org/T101424#3484877 (10Huji) 05Open>03Resolved a:03Huji [01:00:58] 10Quarry: Quarry should store the query runtime along with the results - https://phabricator.wikimedia.org/T172082#3484883 (10Huji) [01:01:35] !log quarry `zhuyifei1999@quarry-main-01:/srv/quarry$ sudo git fetch; sudo git checkout 7dd8c60; sudo service uwsgi restart` T101424 [01:01:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [01:01:38] T101424: Quarry's indentation function is not completely functional - https://phabricator.wikimedia.org/T101424 [01:04:02] 10Quarry: Quarry should store the query runtime along with the results - https://phabricator.wikimedia.org/T172082#3484904 (10zhuyifei1999) [01:04:05] 10Quarry: Include query execution time - https://phabricator.wikimedia.org/T126888#3484907 (10zhuyifei1999) [01:06:09] RECOVERY - Puppet errors on tools-exec-1409 is OK: OK: Less than 1.00% above the threshold [0.0] [01:07:27] 10Quarry: Include query execution time - https://phabricator.wikimedia.org/T126888#3484909 (10Huji) [01:09:19] 10Quarry: Quarry's indentation function is not completely functional - https://phabricator.wikimedia.org/T101424#3484910 (10Huji) 05Resolved>03Open @zhuyifei1999 unsure if it should take effect in minutes or not, but I just checked now and it didn't solve the problem on https://quarry.wmflabs.org/ yet. Re-op... [01:22:53] PROBLEM - Puppet errors on tools-exec-1429 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [01:24:20] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [01:24:34] 10Quarry: Some querries cannot be 'unstarred' - https://phabricator.wikimedia.org/T165169#3258930 (10zhuyifei1999) Issues: * The table `star` allows duplicates * On duplicate, [[https://github.com/wikimedia/analytics-quarry-web/blob/7dd8c60973fd03692491877fea2bec8f9acb2987/quarry/web/app.py#L159|the button will... [01:51:06] 10Quarry: Quarry's indentation function is not completely functional - https://phabricator.wikimedia.org/T101424#3484937 (10zhuyifei1999) Works for me. Have you cleared your browser cache? [01:58:32] 10Quarry: Quarry's indentation function is not completely functional - https://phabricator.wikimedia.org/T101424#3484938 (10zhuyifei1999) My test is: press `(`, enter, `)` [02:02:56] RECOVERY - Puppet errors on tools-exec-1429 is OK: OK: Less than 1.00% above the threshold [0.0] [03:07:04] 10Quarry: Some long queries give no results - https://phabricator.wikimedia.org/T109016#1537760 (10zhuyifei1999) Probably because of result loading takes time. [03:50:12] PROBLEM - Puppet errors on tools-exec-1432 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [03:51:24] 10Quarry: Quarry query in unknown state - https://phabricator.wikimedia.org/T170464#3432141 (10zhuyifei1999) The logs are unfortunately lost :( [04:16:50] 10PAWS: HTTP 404 error (or sometimes 502 error) when trying to access the Control Panel of PAWS - https://phabricator.wikimedia.org/T140525#2467657 (10R96340) This bug happens when I access my bot account with a running server in another account at the **same browser.** I think it might be related to that. //I t... [04:20:14] RECOVERY - Puppet errors on tools-exec-1432 is OK: OK: Less than 1.00% above the threshold [0.0] [05:14:20] 10Cloud-Services, 10DBA, 10User-MarcoAurelio: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3485010 (10Marostegui) [05:15:20] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [05:55:19] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [09:39:41] (03PS1) 10Hashar: Define npm peer dependency explicitly [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/368627 (https://phabricator.wikimedia.org/T172094) [09:44:35] 10Quarry: Gigantic query results cause a SIGKILL and the query status do not update - https://phabricator.wikimedia.org/T172086#3485418 (10zhuyifei1999) Oops, sorry forgot to add the tags [09:46:22] 10Cloud-Services, 10Toolforge, 10DBA: labsdb1001 and labsdb1003 short on available space - https://phabricator.wikimedia.org/T132431#3485437 (10Marostegui) [09:46:26] 10Cloud-Services, 10Tool-Erwin's-tools, 10Toolforge, 10Tools, 10DBA: s51362 has been rate limited to 2 concurrent connections for creating hundreds of 1400-second queries to labsdb1001 and labsdb1003 every 10 seconds - https://phabricator.wikimedia.org/T162519#3485434 (10Marostegui) 05Open>03Resolved... [09:47:52] 10Data-Services, 10cloud-services-team, 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3485449 (10Marostegui) Any update about this? [10:12:29] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1416 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [10:24:05] 10Quarry: Gigantic query results cause a SIGKILL and the query status do not update - https://phabricator.wikimedia.org/T172086#3485554 (10zhuyifei1999) Recent OOMs: ``` zhuyifei1999@quarry-runner-01:~$ zcat /var/log/messages*.gz | cat /var/log/messages* - | grep oom | grep python2.7 Jul 21 05:11:05 quarry-runne... [10:32:20] 10Quarry: Gigantic query results cause a SIGKILL and the query status do not update - https://phabricator.wikimedia.org/T172086#3485563 (10zhuyifei1999) Uh, ``` zhuyifei1999@quarry-runner-01:~$ zcat /var/log/messages*.gz | cat /var/log/messages* - | grep oom | grep invoked Jul 27 12:34:04 quarry-runner-01 kernel... [10:36:17] 10Quarry: Quarry query in unknown state - https://phabricator.wikimedia.org/T170464#3485566 (10zhuyifei1999) Probably not an OOM as in T172086. ``` MariaDB [quarry]> select * from query where id = 18832; +-------+---------+-------------------+---------------+---------------------+-----------+-----------+--------... [10:41:03] 10Quarry: Quarry query in unknown state - https://phabricator.wikimedia.org/T170464#3485572 (10zhuyifei1999) Ok, I forked the query to https://quarry.wmflabs.org/query/20623 and reproduced the issue: ``` zhuyifei1999@quarry-runner-01:~$ grep 33b3c878-4f00-46a1-ad8c-a39ef615872f /var/log/syslog -B 1 -A 11 Jul 31... [10:43:23] 10Quarry: Quarry cannot store results with `table_a.column_name` and `table_b.column_name` in the same result - https://phabricator.wikimedia.org/T170464#3485577 (10zhuyifei1999) [10:47:29] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1416 is OK: OK: Less than 1.00% above the threshold [0.0] [10:51:14] 10Quarry: Quarry cannot store results with `table_a.column_name` and `table_b.column_name` in the same result - https://phabricator.wikimedia.org/T170464#3485627 (10zhuyifei1999) a:05awight>03zhuyifei1999 Reproduced on vagrant with: ```lang=sql SELECT query.id, query_revision.id FROM query, query_revision WH... [11:08:34] 10Quarry: Quarry cannot store results with `table_a.column_name` and `table_b.column_name` in the same result - https://phabricator.wikimedia.org/T170464#3485676 (10zhuyifei1999) ``` MariaDB [quarry]> SELECT query.id, query_revision.id -> FROM query, query_revision -> WHERE query.id = 1 -> AND query_... [11:26:13] 10Quarry: Quarry cannot store results with identical column names - https://phabricator.wikimedia.org/T170464#3485732 (10zhuyifei1999) [11:44:46] 10Quarry: Quarry cannot store results with identical column names - https://phabricator.wikimedia.org/T170464#3485792 (10zhuyifei1999) a:05zhuyifei1999>03None Some background for anyone who can cleanly resolve this: Results are stored in SQLite tables. [11:50:50] 10PAWS, 10MediaWiki-extensions-OAuth: Oauth for PAWS fails - presumably because of username change - https://phabricator.wikimedia.org/T161696#3485819 (10FaFlo) any news on this? [11:51:05] (03CR) 10Addshore: [C: 032] Add 3 more tags to design channel [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/367338 (owner: 10Quiddity) [11:51:28] (03CR) 10Addshore: [C: 032] Add IRC channel feed for Anti-Harassment board [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/368537 (owner: 10Kaldari) [11:52:19] (03CR) 10Addshore: [C: 032] Due to rebranding of cloud i must again change my projects in this config. [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/368590 (owner: 10Zppix) [11:54:28] (03Merged) 10jenkins-bot: Add 3 more tags to design channel [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/367338 (owner: 10Quiddity) [11:54:35] (03CR) 10jenkins-bot: Add 3 more tags to design channel [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/367338 (owner: 10Quiddity) [11:55:57] (03Merged) 10jenkins-bot: Add IRC channel feed for Anti-Harassment board [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/368537 (owner: 10Kaldari) [11:55:59] (03Merged) 10jenkins-bot: Due to rebranding of cloud i must again change my projects in this config. [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/368590 (owner: 10Zppix) [11:56:06] (03CR) 10jenkins-bot: Add IRC channel feed for Anti-Harassment board [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/368537 (owner: 10Kaldari) [11:56:06] (03CR) 10jenkins-bot: Due to rebranding of cloud i must again change my projects in this config. [labs/tools/wikibugs2] - 10https://gerrit.wikimedia.org/r/368590 (owner: 10Zppix) [11:58:00] kaldari: anti-harassment board? Nice to see some progress there :) [12:16:22] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [12:40:33] 10Quarry: Quarry's indentation function is not completely functional - https://phabricator.wikimedia.org/T101424#3485978 (10Huji) 05Open>03Resolved It was a cache issue. Good catch! (No pun intended). [12:41:57] 10Quarry: Make a Quarry automatically refresh on a set time interval - https://phabricator.wikimedia.org/T141698#3485985 (10zhuyifei1999) [12:42:01] 10Quarry: Recurring queries - https://phabricator.wikimedia.org/T101835#3485988 (10zhuyifei1999) [12:56:22] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [13:11:14] 10PAWS: Paws display "502 - Bad gateway error" for specific user - https://phabricator.wikimedia.org/T172080#3486067 (10Aklapper) [13:18:16] (03CR) 10Hashar: [C: 032] Define npm peer dependency explicitly [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/368627 (https://phabricator.wikimedia.org/T172094) (owner: 10Hashar) [13:19:58] (03Merged) 10jenkins-bot: Define npm peer dependency explicitly [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/368627 (https://phabricator.wikimedia.org/T172094) (owner: 10Hashar) [13:20:07] (03PS1) 10Hashar: build: bump grunt and drop grunt-cli [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/368765 [13:35:32] 10Quarry: Include query execution time - https://phabricator.wikimedia.org/T126888#3486103 (10zhuyifei1999) See also T77941 [13:37:31] 10Cloud-VPS, 10cloud-services-team (Kanban), 10Operations: Fix fqdn for promethium - https://phabricator.wikimedia.org/T172111#3486106 (10Andrew) [13:43:56] 10Quarry: Add SHOW EXPLAIN support to Quarry - https://phabricator.wikimedia.org/T146483#3486149 (10zhuyifei1999) a:03zhuyifei1999 I'll try the button implementation, and show the button when the status is "running", and not store the results. Sometimes when you show explain too early the query plans of subque... [13:47:25] (03CR) 10jenkins-bot: Define npm peer dependency explicitly [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/368627 (https://phabricator.wikimedia.org/T172094) (owner: 10Hashar) [13:47:53] 10Cloud-VPS, 10cloud-services-team (Kanban), 10Operations: Switch to new labs puppetmasters - https://phabricator.wikimedia.org/T171786#3486182 (10Andrew) [13:47:57] 10Cloud-VPS, 10cloud-services-team (Kanban), 10Operations: Fix fqdn for promethium - https://phabricator.wikimedia.org/T172111#3486180 (10Andrew) 05Open>03Resolved that was easy [13:52:24] in case I mount extra space to srv, will existing content there moved then? [13:53:44] Sagan you have to move the content first [13:53:52] otherwise you will loose it [13:54:07] so shutdown the service, move it to temp, mount, and then move it back? [13:54:28] yep [13:54:45] Sagan: and to check whether files are opened on srv: sudo lsof /srv [13:54:52] should list file descriptors opened that match /srv [13:55:09] ok :) thx for your help you two :) [13:55:33] your welcome :) [14:06:16] 10cloud-services-team (FY2017-18), 10Goal, 10Patch-For-Review: Refactor OpenStack Puppet to account for Neutron - https://phabricator.wikimedia.org/T171494#3486248 (10chasemp) Comments from gerrit @andrew and I talked about on IRC: ```If the puppet compiler is happy then I'm happy. One question is regarding... [14:13:29] PROBLEM - Puppet errors on tools-webgrid-lighttpd-1416 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [14:44:53] 10Data-Services, 10cloud-services-team, 10DBA, 10Security-Team, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3486381 (10bd808) ping for @Bawolff, @dpatrick, @Reedy. If #security-team can confirm that this is already public data on wiki I t... [14:48:28] RECOVERY - Puppet errors on tools-webgrid-lighttpd-1416 is OK: OK: Less than 1.00% above the threshold [0.0] [15:02:26] o/ [15:02:47] I'm trying to create an index in labsDB. It's on a pretty big user table that the DBAs approved. [15:02:55] (03PS1) 10BryanDavis: Remove legacy www content [labs/toollabs] - 10https://gerrit.wikimedia.org/r/368798 (https://phabricator.wikimedia.org/T140254) [15:02:59] Regretfully, I keep losing my connection when the process is mostly complete. [15:03:23] Any way I can create this index without losing my database connection and interrupting it? [15:04:12] Losing your connection like some server side connection killer is stopping it? [15:04:22] mornin halfak, sounds like query killer stuff? [15:04:23] ah yeah [15:04:29] yeah, I guess so. I'm losing my SSH connection [15:04:34] ah [15:04:47] ssh to Cloud VPS vs mysql loss of connection? [15:05:00] run in a screen session? [15:05:05] "ERROR 2013 (HY000): Lost connection to MySQL server during query" [15:05:06] if it's a long running thing like that I would use tmux or dtach or something to untangle from your session life [15:05:08] ah [15:05:18] Sorry. I'm *not* losing my ssh connection [15:05:18] lol [15:05:20] typing is hard [15:06:41] that sounds like query killer stuff [15:06:50] halfak: the only thing I can think of is one of us running it for you but I'm unsure if the query killer controls and if that's excempt [15:07:33] halfak: is there a task where the DBA folks +1'd this we can circle back to to persist talking it out? [15:07:35] It's almost done when it dies :( I've been considering just running it until it accidentally finishes a bit sooner but that would probably cause a waste of resources. [15:08:00] heh, I'm not sure if that would ever work out [15:08:36] chasemp, https://phabricator.wikimedia.org/T146718 [15:08:52] ah gotcha, that one :) [15:09:13] halfak: side note I'm loving the new profile image [15:09:18] :DDD [15:10:35] halfak: this component of the labsdb self protection is the perview of the DBAs really, I'm hesistant to side step, I know that's been a long haul but can you reopen and ping manuel there and I'll make a note for us (cloud ppls) to poke and follow along? I imagine it's a 10m task for them [15:12:05] side note 2 halfak if you are going to be in montreal would love to pick your brain a bit on next labsdb cluster iteration ideas we have [15:14:04] (03CR) 10BryanDavis: [C: 032] Remove legacy www content [labs/toollabs] - 10https://gerrit.wikimedia.org/r/368798 (https://phabricator.wikimedia.org/T140254) (owner: 10BryanDavis) [15:15:11] (03Merged) 10jenkins-bot: Remove legacy www content [labs/toollabs] - 10https://gerrit.wikimedia.org/r/368798 (https://phabricator.wikimedia.org/T140254) (owner: 10BryanDavis) [15:15:57] chasemp, https://phabricator.wikimedia.org/T146718#3486484 [15:16:01] Was that what you meant? [15:16:13] yup tx [15:16:24] I'll make it a note on our meeting too [15:22:44] Hi. I was following http://botbot.readthedocs.io/en/latest/install.html#install so I did createdb but I got errors [15:22:48] https://www.irccloud.com/pastebin/X0r004Cx/ [15:23:10] !lot tools.admin moved toollabs git clone to toollabs.orphan.rm-me-soon [15:23:57] bd808 i think you meant !log? [15:24:05] Reception123: our local databases are not postgres [15:24:11] !log tools.admin moved toollabs git clone to toollabs.orphan.rm-me-soon [15:24:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.admin/SAL [15:24:18] thanks Zppix [15:24:21] np [15:24:25] bd808: Ok, so what would I have to do then? [15:26:06] Reception123: "ToolsDB" is the MySQL database server where you can make per-tool databases and tables -- https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#User_databases [15:26:28] Ok, thanks [15:26:36] But Botbot's "Postgresql with hStore extension" requirement isn't something that Toolforge supports generally [15:27:43] ERROR 1044 (42000): Access denied for user 's53452'@'%' to database 'botbot' [15:27:49] I tried "CREATE DATABASE" [15:28:21] !log tools remove python-keystoneclient from bastion-03 [15:28:22] bd808: I'm wondering if you can give a review of https://gerrit.wikimedia.org/r/#/c/368805/ when you got time, in case I screwed up somewhere [15:28:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [15:28:25] your database name needs to start with "s53452__" as described on the wikitech page [15:29:29] oh sorry [15:30:59] bd808 thanks for helping him with dbs i've not made a true db with toolforge so i couldnt help him myself :) [15:31:36] Zppix: the upstream botbot python code isn't going to work with mysql though [15:32:35] ugh that was what i was worried about if anything we can look for alts [15:33:01] Is there a way to request new services such as "Postgresql with hStore extension" be installed? [15:33:15] Or yes as Zppix said an alternative that you can suggest? [15:33:42] You can request it in a phabricator ticket, but honestly we are not likely to add a shared postgres server [15:34:08] we don't have the bandwidth to support yet another shared database server [15:34:19] human bandwidth that is [15:35:01] bd808 yeah i dont expect you guys to add that support since you already have mysql support [15:35:09] there are lots and lots of irc bot libraries to work from. I'd need to know more about why you chose botbot to suggest an alternative [15:35:39] bd808: I chose botbot because it has a "search" feature as well as you can search stuff by dates [15:37:50] if you are just wanting better search for channel logs you might try contributing patches to mw-bot -- https://wm-bot.wmflabs.org/browser/ -- https://github.com/benapetr/wikimedia-bot/tree/master/public_html/logs [15:40:37] bd808 i suggested that too but i think we'll look for others just to have options [15:41:11] maning mw-bot better helps lots and lots of channels ;) [15:42:14] (03CR) 10Jforrester: [C: 031] build: bump grunt and drop grunt-cli [labs/tools/heritage] - 10https://gerrit.wikimedia.org/r/368765 (owner: 10Hashar) [15:47:53] bd808: we found a new bot, pierc. I'm not sure what to put for hostname though "mysql> GRANT ALL PRIVILEGES ON pierc.* TO "pierc_user"@"hostname" " [15:49:18] Reception123: you don't have the ability to create new users. The credentials in $HOME/replica.mycnf are what you have to use [15:49:59] this is a shared service, so many tutorials you find that assume you have root level control will be subtly wrong. [15:50:27] I know, I did use them, but I'm not sure what I'd put for hostname [15:52:26] Reception123: tools.labsdb is the server you will connect to. That's what `sql local` attaches you to [15:52:33] Ok, thanks [15:53:27] ERROR 1044 (42000): Access denied for user 's53452'@'%' to database 's53452__pierc' [15:53:42] I did use the credentials given but it still doesn't seem to be working [15:58:18] Reception123: I see that database and can connect to it, so I guess you should double check your configuration. [16:07:18] hey bd808 my creds dont work for somereason i can even connect to labsdb [16:07:38] what account/tool? [16:08:09] irclogsbot after using become with my zppix1 shell user [16:08:15] irclogbot* [16:08:39] bd808 i got it nevermind [16:09:05] `sql local` worked fine from that tool [16:09:35] bd808: uh, not sure if you saw my earlier message [16:10:24] zhuyifei1999_: I did. I looked a bit and didn't see anything horrible, but I haven't "really" reviewed [16:10:43] um I guess I'll wait then [16:10:43] you probably need some test env for hacking on quarry [16:11:18] yeah I got a vagrant setup [16:11:30] zhuyifei1999_: I'm not sure if or when I'll have time for deep looks. If you think its right then go for it. you can always revert if things go wrong. [16:11:50] it's difficult to execute a real query. the internal database have little data [16:11:51] k [16:13:28] https://www.irccloud.com/pastebin/yWzHH95L/ [16:14:09] ah sorry I'm stupid I forgot use [16:15:37] but now I'm getting GRANT command denied to user 's53452'@'10.68.23.58' for table 's53452__pierc' [16:17:23] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [16:17:32] Reception123: you need to "use s53452__pierc;" to change to that db [16:17:50] I did "use" [16:18:20] the "MariaDB [(none)]>" prompt says that you have no default database selected. [16:18:21] https://www.irccloud.com/pastebin/dUzMI886/ [16:18:34] Yeah, then I realized I forgot USE and corrected it [16:19:03] Reception123: your use has no admin rights. you cannot create new users or permissions. [16:19:26] the credentials in your $HOME/replica.my.cnf are the ones you can use [16:20:28] Ok [16:20:44] so you need to skip past instructions for setting up new users and just add those credentials to the config for whatever you are trying to use. [16:22:08] !log quarry `zhuyifei1999@quarry-runner-01:/srv/quarry$ sudo git fetch; sudo git checkout 6447943` T146483 [16:22:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [16:22:13] T146483: Add SHOW EXPLAIN support to Quarry - https://phabricator.wikimedia.org/T146483 [16:28:54] !log quarry Restarted celery-quarry-worker on quarry-runner-01 T146483 [16:28:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [16:28:56] T146483: Add SHOW EXPLAIN support to Quarry - https://phabricator.wikimedia.org/T146483 [16:29:02] trying to run a python script but getting "mysql_config.txt missing." even though ls clearly indicates that the file does exist [16:30:04] running locally or on the job grid or on kubernetes or ? [16:30:08] !log quarry Repeated for quarry-runner-02 [16:30:09] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [16:30:34] locally [16:30:38] what is the current working directory of the command and is the config file in that directory? [16:31:00] ~/pierc/bot both the command and the config file are under it [16:31:34] !log quarry Repeated for quarry-main-01, but restarted uwsgi T146483 [16:31:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [16:33:11] Reception123: is your shell's working directory ~/pierc/bot when you run it? Meaning did you cd ~/pierc/bot first? [16:33:19] yes [16:33:24] weird [16:33:55] any suggestions? [16:36:00] 10Quarry, 10Patch-For-Review: Add SHOW EXPLAIN support to Quarry - https://phabricator.wikimedia.org/T146483#3486857 (10zhuyifei1999) 05Open>03Resolved Should be fixed. After cleaning up browser cache there should be an "Explain" button just after "This query is currently executing...". [16:37:26] Reception123: I think that the file is getting read but config.py is breaking because of an extra blank line at the end of your config file. The error reporting there is pretty bad [16:37:50] yup, removed the line and it works. [16:37:51] Thanks [16:38:01] "value = temp[1]" is going to throw an error when it hits a blank line [16:45:19] bd808: the bot works now, but I'm not sure where the "webserver" is defined and where I could consult the logs [16:47:07] Reception123: it sounds like you need to read some more stuff in https://wikitech.wikimedia.org/wiki/Help:Toolforge . If you are running it locally on tools-login that will end badly for you. [16:47:19] ok [16:47:59] broadly you need to run the web UI using or `webservice` command and run the data collecting bot as a grid job or a kubernetes pod [16:48:27] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web [16:48:35] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid [16:48:42] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes [16:57:21] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [18:04:21] 10Cloud-VPS, 10cloud-services-team (Kanban), 10Operations: Switch to new labs puppetmasters - https://phabricator.wikimedia.org/T171786#3487120 (10Andrew) [18:04:25] 10Cloud-VPS, 10cloud-services-team (Kanban), 10Operations, 10Patch-For-Review: instance root passwords vs. multiple puppetmasters - https://phabricator.wikimedia.org/T171959#3487118 (10Andrew) 05Open>03Resolved Due to various concerns we're going to just disable these passwords for now. [18:18:20] PROBLEM - Puppet errors on tools-webgrid-generic-1401 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [18:22:05] !log striker Testing T172077 fix by removing Yuvi from project [18:22:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Striker/SAL [18:22:08] T172077: removing a user from projectadmin on wikitech produces a blank page - https://phabricator.wikimedia.org/T172077 [18:24:28] 10cloud-services-team (Kanban), 10wikitech.wikimedia.org, 10MW-1.30-release-notes (WMF-deploy-2017-08-01_(1.30.0-wmf.12)), 10Patch-For-Review, 10User-bd808: removing a user from projectadmin on wikitech produces a blank page - https://phabricator.wikimedia.org/T172077#3487159 (10bd808) 05Open>03Resolv... [18:43:19] 10Toolforge, 10cloud-services-team (Kanban), 10Patch-For-Review, 10User-bd808: Modernize the admin tool's codebase - https://phabricator.wikimedia.org/T140254#3487221 (10bd808) 05Open>03Resolved [18:50:39] 10Quarry: Weird race condition makes query stuck in queued forever - https://phabricator.wikimedia.org/T172143#3487252 (10zhuyifei1999) [18:51:53] bd808: fyi, your query ^. I don't know why this happens [18:53:22] RECOVERY - Puppet errors on tools-webgrid-generic-1401 is OK: OK: Less than 1.00% above the threshold [0.0] [19:06:28] 10Data-Services, 10Quarry: Long-running Quarry query (querry?) produces strangely incorrect results - https://phabricator.wikimedia.org/T135087#3487373 (10zhuyifei1999) [20:55:20] 10Cloud-Services, 10DBA: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3477438 (10MarcoAurelio) [22:39:42] 10Cloud-VPS, 10cloud-services-team (Kanban): New Trusty base images hang on boot - https://phabricator.wikimedia.org/T172064#3487960 (10Andrew) - removing all custom-installed packages - building on a fresh build instance [23:33:18] PROBLEM - Puppet errors on tools-exec-1414 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0]