[prev in list] [next in list] [prev in thread] [next in thread]
List: libvirt-users
Subject: Performance Discrepancies and Limitations in Local Storage IOPS Testing
From: Jan Wasilewski <finarffin () gmail ! com>
Date: 2023-08-24 10:29:43
Message-ID: CAN4DDNiO2oTaLGLBLrodnm8hcJyEEeB4PFUPR7oYuLn0DXaB1Q () mail ! gmail ! com
[Download RAW message or body]
Hi,
In the past few weeks, I conducted performance tests to evaluate IOPS
(Input/Output Operations Per Second) performance for locally attached
disks. The original discussion started on the openstack-discuss mailing
list since all the tests were conducted within an OpenStack cloud
environment. However, I decided to initiate a discussion here, as it
appears that the performance differences might be rooted in lower-level
factors rather than the OpenStack platform itself. It seems that these
differences are closely related to variations in libvirt/qemu versions and
kernel configurations.
Ultimately, I discovered that the performance is significantly enhanced
when the hypervisor is deployed on top of Ubuntu 22.04LTS. Under this
setup, I was able to achieve around 100,000 IOPS during my fio tests
[1][2]. In contrast, conducting a similar test with the hypervisor deployed
on Ubuntu 20.04LTS yielded significantly lower results, averaging around
20,000 IOPS [3][4]. An intriguing observation is that attaching an nvme
disk to my Ubuntu 22.04LTS system and utilizing it as local storage led to
slightly diminished performance, hovering around 90,000 IOPS [5]. This
outcome is somewhat unexpected, as I initially anticipated higher figures.
It's particularly noteworthy that when I conduct the same test directly on
top of the hypervisor, the numbers align more closely with expectations
[6][7]. This pattern suggests that there might be a limitation imposed
either by libvirt/qemu or the kernel itself.
Having meticulously reviewed all the release notes, I failed to come across
any information pertaining to noteworthy performance enhancements
concerning local storage and IOPS. Given this, I'd like to reach out to you
directly to inquire if you possess any insights into such limitations. Any
guidance or suggestions that could help optimize my local storage results
would be greatly appreciated.
Looking forward to your input.
Best regards
/Jan Wasilewski
*References:[1] Configuration details(libvirt/qemu/kernel version) for
Ubuntu 22.04: https://paste.openstack.org/show/b8svl0bOfX0WHTGvgI1h/
<https://paste.openstack.org/show/b8svl0bOfX0WHTGvgI1h/>[2] fio results for
VM test with Ubuntu 22.04:
https://paste.openstack.org/show/bUpiq1y0o58S1ThNgqLd/
<https://paste.openstack.org/show/bUpiq1y0o58S1ThNgqLd/>[3] Configuration
details(libvirt/qemu/kernel version) for Ubuntu 20.04:
https://paste.openstack.org/show/b005K2L6ZutGxDrCOZbL/
<https://paste.openstack.org/show/b005K2L6ZutGxDrCOZbL/>[4] fio results for
VM test with Ubuntu 20.04:
https://paste.openstack.org/show/b8JeVOn4YCPSX7uaqR0N/
<https://paste.openstack.org/show/b8JeVOn4YCPSX7uaqR0N/>[5] fio results for
VM test with Ubuntu 22.04 and nvme as a local storage disk:
https://paste.openstack.org/show/b75M4bI00LQePTmYSaUI/
<https://paste.openstack.org/show/b75M4bI00LQePTmYSaUI/>[6] fio results for
hypervisor test with Ubuntu 22.04 and nvme storage:
https://paste.openstack.org/show/bj7NeSiwwiWS6MbJROts/
<https://paste.openstack.org/show/bj7NeSiwwiWS6MbJROts/>[7] fio results for
hypervisor test with Ubuntu 22.04 and ssd storage:
https://paste.openstack.org/show/bSeEHXcbrY9YlYWQWGKS/
<https://paste.openstack.org/show/bSeEHXcbrY9YlYWQWGKS/>*
[Attachment #3 (text/html)]
<div dir="ltr">Hi,<br><br>In the past few weeks, I conducted performance tests to \
evaluate IOPS (Input/Output Operations Per Second) performance for locally attached \
disks. The original discussion started on the openstack-discuss mailing list since \
all the tests were conducted within an OpenStack cloud environment. However, I \
decided to initiate a discussion here, as it appears that the performance differences \
might be rooted in lower-level factors rather than the OpenStack platform itself. It \
seems that these differences are closely related to variations in libvirt/qemu \
versions and kernel configurations.<br><br>Ultimately, I discovered that the \
performance is significantly enhanced when the hypervisor is deployed on top of \
Ubuntu 22.04LTS. Under this setup, I was able to achieve around 100,000 IOPS during \
my fio tests [1][2]. In contrast, conducting a similar test with the hypervisor \
deployed on Ubuntu 20.04LTS yielded significantly lower results, averaging around \
20,000 IOPS [3][4]. An intriguing observation is that attaching an nvme disk to my \
Ubuntu 22.04LTS system and utilizing it as local storage led to slightly diminished \
performance, hovering around 90,000 IOPS [5]. This outcome is somewhat unexpected, as \
I initially anticipated higher figures. It's particularly noteworthy that when I \
conduct the same test directly on top of the hypervisor, the numbers align more \
closely with expectations [6][7]. This pattern suggests that there might be a \
limitation imposed either by libvirt/qemu or the kernel itself.<br><br>Having \
meticulously reviewed all the release notes, I failed to come across any information \
pertaining to noteworthy performance enhancements concerning local storage and IOPS. \
Given this, I'd like to reach out to you directly to inquire if you possess any \
insights into such limitations. Any guidance or suggestions that could help optimize \
my local storage results would be greatly appreciated.<br><br>Looking forward to your \
input.<br><br>Best regards<br>/Jan Wasilewski<br><br><i>References:<br>[1] \
Configuration details(libvirt/qemu/kernel version) for Ubuntu 22.04: <a \
href="https://paste.openstack.org/show/b8svl0bOfX0WHTGvgI1h/">https://paste.openstack.org/show/b8svl0bOfX0WHTGvgI1h/</a><br>[2] \
fio results for VM test with Ubuntu 22.04: <a \
href="https://paste.openstack.org/show/bUpiq1y0o58S1ThNgqLd/">https://paste.openstack.org/show/bUpiq1y0o58S1ThNgqLd/</a><br>[3] \
Configuration details(libvirt/qemu/kernel version) for Ubuntu 20.04: <a \
href="https://paste.openstack.org/show/b005K2L6ZutGxDrCOZbL/">https://paste.openstack.org/show/b005K2L6ZutGxDrCOZbL/</a><br>[4] \
fio results for VM test with Ubuntu 20.04: <a \
href="https://paste.openstack.org/show/b8JeVOn4YCPSX7uaqR0N/">https://paste.openstack.org/show/b8JeVOn4YCPSX7uaqR0N/</a><br>[5] \
fio results for VM test with Ubuntu 22.04 and nvme as a local storage disk: <a \
href="https://paste.openstack.org/show/b75M4bI00LQePTmYSaUI/">https://paste.openstack.org/show/b75M4bI00LQePTmYSaUI/</a><br>[6] \
fio results for hypervisor test with Ubuntu 22.04 and nvme storage: <a \
href="https://paste.openstack.org/show/bj7NeSiwwiWS6MbJROts/">https://paste.openstack.org/show/bj7NeSiwwiWS6MbJROts/</a><br>[7] \
fio results for hypervisor test with Ubuntu 22.04 and ssd storage: <a \
href="https://paste.openstack.org/show/bSeEHXcbrY9YlYWQWGKS/">https://paste.openstack.org/show/bSeEHXcbrY9YlYWQWGKS/</a></i></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic