[10:32:15] !log toolsbeta live-hacking toolsbeta-test-k8s-haproxy-1 to point to just the k8s apiserver in control-1 Turn on --v=10 in control-1 for extended debug [10:32:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [11:09:35] Repeat: [16:39:33] Does anyone know about an archive of old dumps (de-WP). Maybe beginning of 2019 & 2018? [11:25:12] sorry Wurgl I have no clues [11:30:41] arturo :-( [15:07:18] how many queries can I run simultaneously on the replication DB? [15:07:38] Or how can I programmatically find this information out? [15:08:01] If at all applicable. [15:13:21] mmm I dont know from the top of my head Cyberpower678 [15:13:34] I wonder if there are docs in wikitech [15:15:27] I need to do some heavy work with the DB, and unfortunately, ever since the actor table was introduced, queries have been slow as shit. (Sorry) [15:15:56] I have a list generating script that did its job in 2.5 hours last year. Now it takes 14. [15:16:17] So I'm trying to get around that by introducing async queries [15:16:46] there is a fairly low limit of concurrent connections, I believe [15:16:48] 15 or so IIRC [15:17:01] I assume that also limits the number of queries you can run simultaneously [15:17:02] Not connections! Queries [15:17:39] If I'm reading the mysqli docs correctly, it's one link multiple queries. [15:18:16] Nvm. Read it wrong. It is one query per connection [15:19:08] arturo: Lucas_WMDE: Is there any chance I can have it temporarily hiked to 100? [15:20:29] 20 is the limit, apparently [15:20:56] Cyberpower678: feel free to open a phab task if you want. I can't make that decision [15:21:13] SHOW VARIABLES LIKE "max_connections"; returns 1024 [15:21:27] max_user_connections is the relevant variable, I think [15:21:35] derp [15:21:48] well, max_connections probably too, but you’ll hit the other one first :) [15:22:04] Lucas_WMDE: I get 10? [15:22:04] T232210 is the only request I know of [15:22:04] T232210: OAbot runs out of database connections - https://phabricator.wikimedia.org/T232210 [15:22:11] not sure if there are others [15:22:21] Why do you get 20 and I get 10? [15:22:22] that you could copy for your phab task [15:22:27] I’m looking in operations/puppet [15:22:43] I need the change kind of now. I'm on a timer. [15:23:03] https://gerrit.wikimedia.org/g/operations/puppet/+/21c64883e3988ac30f43dee73529c135a8c29656/modules/role/templates/mariadb/mysqld_config/tools.my.cnf.erb#71 [15:23:17] I can’t help you with that [15:23:29] I’m just another user, who’s heard things related to this discussion before… [15:24:05] What happens if you actually run the query SHOW VARIABLES LIKE "max_user_connections"; [15:24:08] Lucas_WMDE: ^ [15:24:41] it shows 20 for me [15:25:00] WTF? [15:25:08] It shows 10 for me [15:25:17] andrewbogott: bd808 ^ [15:25:20] huh [15:25:27] Why am I being limited? [15:27:52] bryan is traveling this week Cyberpower678 [15:27:59] :-( [15:28:45] As is brooke, and I don't know much. Best to create a phab task I think. [15:29:07] Like I said, I'm on a short timer. [15:31:35] Cyberpower678: our databases are always under heavy load. Many people using very few resources. We don't usually make exceptions to the limits with such short notices (if we do at all) [15:32:19] arturo: but it still seems unfair that I'm being constrained to 10 connection while everyone else gets 20 [15:33:12] I don't know why you have 10. I can speculate that your tool was sending special load to the DB, enough for it you be rewarded a special low limit [15:33:43] but I don't usually work with our DBs [15:33:58] so I lack any context in this particular case [15:36:29] IABot has it's own VPS DB, so it was throttled at one point, but that throttle should have been reversed. So I was told [15:37:10] Before that the DB lived on Toolforge and hurt the shared DB. [15:37:22] do you have any links? [15:37:26] No [15:38:07] again, I'm just trying to better understand the context [15:38:41] so let me back to my original suggestion: open a phab task [15:39:03] I will make sure it is evaluated by the right people [15:39:23] I opened the task [15:39:30] https://phabricator.wikimedia.org/T238337 [15:40:10] thanks! [15:40:41] may I suggest you include extensive details in the phab task, like what Database, why do you think you have a 10 conn limit, why do you think everyone is using 20, etc [15:40:58] and for what do you need more, etc [16:30:15] Cyberpower678: I'm also at 10. no idea why. [16:32:38] 20 seems to be for the tools db. 10 seems for the replica [16:36:15] Oh [16:36:19] Wierd [18:54:35] Yes the difference is toolsdb vs replicas [18:54:48] Everyone has 10 connection limit on the replicas [20:17:09] I want to be able to start a task I'd normally run on the grid from a webservice, but k8s webservices can't talk to the grid. Should I run the webservice on the grid or is there a better option? [21:59:43] !log quarry `zhuyifei1999@quarry-web-01:/tmp$ find /tmp/* -mtime +360 -user www-data -exec sudo rm -v {} \;` T238375 [21:59:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Quarry/SAL [21:59:47] T238375: quarry-web-01 leaks files in /tmp - https://phabricator.wikimedia.org/T238375