From kde-bugs-dist Tue Jun 30 20:19:27 2015 From: =?UTF-8?Q?Daniel=20Vr=C3=A1til=20?= Date: Tue, 30 Jun 2015 20:19:27 +0000 To: kde-bugs-dist Subject: [Akonadi] [Bug 338402] File system cache is inneficient : too many file per directory Message-Id: X-MARC-Message: https://marc.info/?l=kde-bugs-dist&m=143569558609364 https://bugs.kde.org/show_bug.cgi?id=3D338402 --- Comment #13 from Daniel Vr=C3=A1til --- Hi all, so the problem with files just endlessly piling up in file_db_data should finally be fixed. Now for the original bug report: the file_db_data contain= ing too many files. So I was thinking about how to decrease the file count - obviously the right solution is the levelled cache as Bastien pointed out. I think in our case = one level should be enough. The filenames of the external payload parts consist= of incremental database unique ID, so I am planning to use the last two digits= of the ID for the folder name to ensure even distribution of files into the ca= che folders. To have some numbers here, using modulo 100 for folder name means 100 folde= rs in file_db_data (file_db_data/00 - 99/). With 1 million emails in Akonadi (which is a performance baseline for me) and with average ratio of external= vs internal cache being cca 1:2 we get cca 500 000 external files in file_db_d= ata, so that means cca 5 000 files per folder. That sounds like a reasonable num= ber to me.=20 With 2 levels of indirection (using last 3rd and 2nd digit for L1 and last digit for L2 (so file_db_data/00-99/0-9/) we would have 100 folders with 10 folders in each so 1000 folders in total. That would give us cca 500 emails= per folder with 1 000 000 emails in Akonadi. For the baseline of 1 000 000 emai= ls two levels of indirection seem to be unnecessary. Regarding migration, we cannot do automatic migration on start, that would = take too much time and resources to perform during start, so only newly created files would be moved to the cache folders. It would be possible to implement the full migration as part of akonadictl fsck though, so users could run it manually. What are your opinions? --=20 You are receiving this mail because: You are watching all bug changes.=