[00:20:25] <Dispenser>	 Is there a good spot to put dumps?  (In the unlikely that people want to use them?
[00:21:43] <Dispenser>	 I've got json of all disambiguation pages (monthly, 2 years) and now the first 4 KB SVG from Commons
[00:29:31] <harej>	 Dispenser: about how big are these dumps?
[00:30:22] <zhuyifei1999_>	  /data/project/shared & /data/scratch?
[00:32:41] <Dispenser>	 200 - 244 MB for the DAB dumps (tar.gz).  For SVG still deciding between compression methods: .tar.gz - 615 MB; .tar.bz2 - 519 MB; .tar.xz - 377 MB 
[00:34:47] <harej>	 What zhuyifei said sounds logical; if that doesn't somehow work you could just create a small webservice through toolforge
[00:35:03] <harej>	 I'd like for us to one day have a figshare-type service for datasets that our community comes up wiht
[00:36:52] <Dispenser>	 27x DAB dump files (JSON API call, Dec 2015 - Feb 2018, 5.8 GB total)
[00:37:29] <Dispenser>	 /data/scratch isn't backed up
[00:42:00] <Dispenser>	 The way I process them is by streaming it.  In Python:
[00:42:02] <Dispenser>	 with tarfile.open("wikimedia_commons_SVG_4KB_head.tar.gz", "r|gz") as tar: for tarinfo in enumerate(tar): tar.extractfile(tarinfo).read()
[00:43:00] <Dispenser>	 Takes 10 minutes for 250,000 DAB pages, if the files were written out it take 8 hours
[01:05:14] <wikibugs>	 (03CR) 10Bstorm: "For the template, yes I can see that.  It might be tidier to import it from a file.  I can modify it so that it has an output path option " (033 comments) [labs/tools/cdnjs-index] (beta) - 10https://gerrit.wikimedia.org/r/409448 (https://phabricator.wikimedia.org/T182604) (owner: 10Bstorm)
[01:21:52] <zhuyifei1999_>	 !log tools `# find /data/project/ -maxdepth 1 -perm -o+w \! -uid 0 -exec chmod -v o-w {} \;` Affected tools: wikisource-tweets, gsociftttdev, dow, ifttt-testing, elobot. All mode 2777 -> 2775
[01:21:57] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[01:25:02] <wikibugs>	 (03CR) 10Zhuyifei1999: tools-cdnjs-beta: switching to a reverse proxy solution (031 comment) [labs/tools/cdnjs-index] (beta) - 10https://gerrit.wikimedia.org/r/409448 (https://phabricator.wikimedia.org/T182604) (owner: 10Bstorm)
[01:28:42] <zhuyifei1999_>	 !log tools `# find /home/ -maxdepth 1 -perm -o+w \! -uid 0 -exec chmod -v o-w {} \;` Affected: only /home/tr8dr, mode 0777 -> 0775
[01:28:47] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL