I just noticed that KFileItemModel::removeExpandedItems() fails to remove expanded items that are currently filtered, which can make expanded items re-appear incorrectly after switching from Details to Icons View.
The fix is quite straightforward - just iterate over the filtered items and remove those which have a parent.
Testing
Fixes the bug for me. Old and new unit tests pass.