Talk:Toolserver:User-store

Update
It's not clear whether one can create new directories or not here. I thought not, and that you needed to ask permission (and document it here), but I see that the page is outdated. This is the current situation, and some things seem unneeded or misplaced. Nemobis 08:31, 5 February 2012 (UTC)

Stats compression
A huge portion of the space is taken by visitors stats, although now they have two mirrors (WMF and IA). The oldest ones are compressed in LZMA (xz). Compressing gz or xz is useless, can only increase size. I made some tests of compression of a whole month uncompressed,  (184G): --Nemo 10:23, 22 March 2012 (UTC)
 * takes ~27h (6 cores, less than 100M memory) and gives 41G
 * takes ~56h (2 cores, 800M memory) and gives 37G
 * takes about 3 days (2 cores, 2700M memory) and gives 35G
 * tar with xz uses LZMA with standard settings and can only give worse results (I tried it but it got killed by mistake, wasn't going anywhere though)
 * individual gz are 51.2G
 * individual xz of this month are not yet available for comparison
 * Maybe it would be worth trying xz with higher compression, but I can't locate the stats files on user-store right now. --Nemo 14:02, 20 October 2013 (UTC)