[prev in list] [next in list] [prev in thread] [next in thread] 

List:       linux-xfs
Subject:    Question on the xfs inode slab memory
From:       Jianan Wang <wangjianan.zju () gmail ! com>
Date:       2023-05-31 21:29:52
Message-ID: CAMj1M42L6hH9weqroQNaWu_SG+Yg8NrAuzgNO1b8jiWPJ2M-5A () mail ! gmail ! com
[Download RAW message or body]

Hi all,

I have a question regarding the xfs slab memory usage when operating a
filesystem with 1-2 billion inodes (raid 0 with 6 disks, totally
18TB). On this partition, whenever there is a high disk io operation,
like removing millions of small files, the slab kernel memory usage
will increase a lot, leading to many OOM issues happening for the
services running on this node. You could check some of the stats as
the following (only includes the xfs related):

#########################################################################
Active / Total Objects (% used):  281803052 / 317485764 (88.8%)
Active / Total Slabs (% used): 13033144 / 13033144 (100.0%)
Active / Total Caches (% used): 126 / 180 (70.0%)
Active / Total Size (% used): 114671057.99K / 127265108.19K (90.1%)
Minium / Average / Maximum Object : 0.01K / 0.40K / 16.75K

OBJS               ACTIVE      USE     OBJ SIZE     SLABS
OBJ/SLAB    CACHE SIZE    NAME
78207920      70947541      0%       1.00K           7731010
 32            247392320K     xfs_inode
59945928      46548798      0%       0.19K           1433102
 42              11464816K     dentry
25051296      25051282      0%       0.38K           599680
  42            9594880K         xfs_buf
#########################################################################

The peak slab memory usage could spike all the way to 100GB+.

We are using Ubuntu 18.04 and the xfs version is 4.9, kernel version is 5.4

#########################################################################
Linux# cat /etc/*-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Linux# sudo apt list | grep xfs
libguestfs-xfs/bionic-updates 1:1.36.13-1ubuntu3.3 amd64
nfs-ganesha-xfs/bionic 2.6.0-2 amd64
obexfs/bionic 0.11-2build1 amd64
x11-xfs-utils/bionic 7.7+2build1 amd64
xfsdump/bionic 3.1.6+nmu2 amd64
xfslibs-dev/bionic 4.9.0+nmu1ubuntu2 amd64
xfsprogs/bionic,now 4.9.0+nmu1ubuntu2 amd64 [installed]
xfstt/bionic 1.9.3-3 amd64
xfswitch-plugin/bionic 0.0.1-5ubuntu5 amd64

Linux# uname -a
Linux linux-host 5.4.0-45-generic #49~18.04.2-Ubuntu SMP Wed Aug 26
16:29:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
#########################################################################

Is there any potential way to limit the slab memory increase for a
node as a whole, or the only thing we could do is to reduce the
filesystem inode or iops usage?

Thanks in advance!
Jianan Wang
[prev in list] [next in list] [prev in thread] [next in thread]