[prev in list] [next in list] [prev in thread] [next in thread] 

List:       vdsm-devel
Subject:    =?utf-8?q?=5Bovirt-devel=5D?= Backup: how to download only used extents from imageio backend
From:       Michael Ablassmeier <abi () grinser ! de>
Date:       2020-06-30 6:13:22
Message-ID: 20200630061322.3cbkfpy2lbyrx5jy () lrrr ! dead-link ! org
[Download RAW message or body]

hi,

im currently looking at the new incremental backup api that has been
part of the 4.4 and RHV 4.4-beta release. So far i was able to create
full/incremental backups and restore without any problem.

Now, using the backup_vm.py example from the ovirt-engine-sdk i get
the following is happening during a full backup:

 1) imageio client api requests transfer
 2) starts qemu-img to create a local qemu image with same size
 3) starts qemu-nbd to serve this image
 4) reads used extents from provided imageio source, passes data to
 qemu-nbd process 
 5) resulting file is a thin provisioned qcow image with the actual
 data of the VM's used space.

while this works great, it has one downside: if i backup a virtual
machine with lots of used extents, or multiple virtual machines at the
same time, i may run out of space, if my primary backup target is
not a regular disk.

Imagine i want to stream the FULL backup to tape directly like

 backup_vm.py full [..] <vm_uuid> /dev/nst0

thats currently not possible, because qemu-img is not able to open
a tape device directly, given its nature of the qcow2 format.

So what iam basically looking for, is a way to download only the extents
from the imageio server that are really in use, not depending on qemu-*
tools, to be able to pipe the data somehwere else.

Standard tools, like for example curl, will allways download the full
provisioned image from the imageio backend (of course).

I noticed is that it is possible to query the extents via:

 https://tranfer_node:54322/images/d471c659-889f-4e7f-b55a-a475649c48a6/extents

As i failed to find them, are there any existing functions/api calls
that could be used to download only the used extents to a file/fifo
pipe?

So far, i played around with the _internal.io.copy function, beeing able
to at least read the data into a in memory BytesIO stream, but thats not
the solution to my "problem" :)

bye,
    - michael
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ATIBK6AWZHHGVMKXUWYQU2CAVC74TDUJ/



[prev in list] [next in list] [prev in thread] [next in thread]