[13:58:33] !log tools.quickcategories deployed 58a6f3e625 (batches list) [13:58:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL [14:02:00] !log tools.quickcategories deployed 5e8877a1f4 (link to batches list from index) [14:02:00] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL [15:30:44] what are the storage limits on Toolforge? like could I store a ~30 GB SQLite database [15:30:45] ? [15:32:14] I don't think there are any hard limits [15:32:22] ooo [15:32:25] good to know [15:32:28] But if you're using more than your fair share... You might be asked to clean up [15:32:44] IIRC, it's usually 100s of GB, or TBs before you're likely to get flagged [15:32:55] ah, that's good to know [15:33:18] I'm looking to make a bot that would download an SQLite database (from an external source) and make edits to wikidata from that database [15:33:32] Depending on what you're doing, you might get shouted at for IO usage than disk space [15:33:39] ooo [15:34:38] davidwbarratt, yeah storage on toolforge is over NFS [15:34:45] For example https://phabricator.wikimedia.org/T217993 [15:34:57] https://phabricator.wikimedia.org/T206239 [15:35:37] ooo! [15:36:14] they only update the SQLite database every few weeks so it's probably not a big deal [15:36:19] "T183953: tools.iabot is using 1.3T of 8T available tools nfs storage" [15:36:19] T183953: tools.iabot is using 1.3T of 8T available tools nfs storage - https://phabricator.wikimedia.org/T183953 [15:36:23] stuff like htat :) [15:36:30] sheesh [15:36:39] ok, well it looks like I'll be well under that. :) [15:37:03] bare in mind reads of the DB could be slow too [15:37:42] yeah that's fine, but good to know. :) [15:39:36] there is a mysql instance provided for tools somewhere [15:40:18] yeah I've seen that, the external org provides an SQLite database, so downloading and querieing is probably faster than trying to convert it, but I suppose it depends on the format [15:45:33] perhaps a Cloud VPS project would be better for this? (they have non-NFS storage) [15:45:34] an SQLite database would match several of the criteria listed at https://wikitech.wikimedia.org/wiki/Help:Shared_storage#Disadvantages_of_Shared_NFS_directories, I think [16:56:23] !log tools.quickcategories briefly deployed and then undeployed again some experimental code to track slow queries (now in a slowqueries branch) [16:56:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL [17:42:40] !log tools.awmd-stats Deploy changes e12c2a1 and 9f52343 and restarting tool to pickup changes [17:42:42] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.awmd-stats/SAL [17:44:34] !log tools.awmd-stats Changes deployed and working as expected. Tool up and running again. [17:44:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.awmd-stats/SAL [20:05:15] !log tools.quickcategories deployed ec15439d88 (add missing CSRF checks 😱) [20:05:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL [20:05:59] (yay, the emoji wasn’t mangled ^^) [20:52:23] !log tools.quickcategories installed beautifulsoup4 [20:52:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL [20:53:01] !log tools.quickcategories deployed 8a3ff068e2 (HTML improvements) [20:53:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL [21:13:48] hello wonderful Cloud Services! Could this get some attention? https://phabricator.wikimedia.org/T221339 [21:25:12] dunno if you'll have much luck over the weekend musikanimal [21:46:54] yes indeed