[prev in list] [next in list] [prev in thread] [next in thread]
List: libvirt-users
Subject: Re: [libvirt-users] [libvirt] vm live storage migration with snapshots
From: Edward Young <edward.and.young () gmail ! com>
Date: 2015-02-13 1:47:38
Message-ID: CALHzvtQnyxYz4bg6ctsPxe1JMWcH4Ny-0NNcVXbMJ6+aYfCEpw () mail ! gmail ! com
[Download RAW message or body]
On Wed, Feb 11, 2015 at 5:09 PM, Eric Blake <eblake@redhat.com> wrote:
> [dropping multiple lists; let's just use libvirt-users]
>
> On 02/11/2015 02:45 PM, Edward Young wrote:
>
> > I perform a simple test, but failed.
> >
> > In the source, I create: base <- mid <- active (2 snapshots, the active
> > one is the current one)
> > In order to migrate this vm to the destination, I manually copy both base
> > and mid to the destination, and put them in the sam e directory
> > /var/lib/libvirt/images/
> > Then I call the following commands:
> > 'sudo virsh migrate --live --unsafe --verbose --copy-storage-inc vm1
> > qemu+ssh://192.168.1.3/system tcp://192.168.1.3'
> >
> > The error I got is:
> > error: Requested operation is not valid: cannot migrate domain with 2
> > snapshots.
> >
> > I do not know the reason for this.
>
> That's because libvirt is not (yet) able to migrate snapshots in a way
> that the destination can revert to them. It is possible to work around
> this in a couple of ways:
>
> 1. Manually dump all the snapshot XML configurations on the source side
> (virsh snapshot-list, then virsh snapshot-dumpxml in a loop), then
> delete the snapshot tracking (virsh snapshot-delete --metadata in a
> loop), then do the migration, then manually recreate the snapshot
> configuration on the destination (virsh snapshot-create --redefine in a
> loop for each of the xml files you saved earlier).
>
> If you don't plan to revert to snapshots on the destination, then you
> can skip the snapshot-dumpxml loop, as well as the snapshot-create
> --redefine loop. But if that's the case, then you can take a shortcut:
>
> 2. Don't track external snapshot data in libvirt in the first place
> (virsh snapshot-create --no-metadata when taking external snapshots).
> Instead, the backing chain is sufficient to know that external snapshot
> data was saved.
>
It works for me. Thanks!
>
> >
> >
> >> In fact, modern
> >> libvirt/qemu does this for you by setting up an NBD server on the
> >> destination, doing a data mirror from the source into the destination
> >> (so that you DO have shared storage, at least for the duration of the
> >> migration), then doing the live migration, then tearing down the NBD
> >> mirroring link.
> >>
> >
> > Sounds interesting! Can you provide me an example or point me to a link
> > about how to use this function?
>
> It's all done under the hood as part of the implementation of 'virsh
> migrate --copy-source-*', if done as part of migration. But if you want
> to manually recreate the steps, the source code in
> src/qemu/qemu_migration.c shows the QMP commands used to set up the NBD
> server on the destination, then the mirroring is done with equivalent of
> the 'virsh blockcopy' command pointing to the NBD server as the
> destination.
>
I will take a look at the code.
>
> >
> > Sorry to take you so much time to answer my questions. I appreciate it
> > deeply. After I solve this issue, I will write a blog and share these
> > knowledge with other people.
>
> The libvirt-users list archives has a lot of questions and posts from
> other users on the same topic; it may also help if you peruse the archives.
>
I tried previously, but I can't find them. I search them directly by
google? Is there anyway to find the related posts in the archive?
Thanks a lot!
Ed
>
> --
> Eric Blake eblake redhat com +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
>
>
[Attachment #3 (text/html)]
<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Feb \
11, 2015 at 5:09 PM, Eric Blake <span dir="ltr"><<a \
href="mailto:eblake@redhat.com" target="_blank">eblake@redhat.com</a>></span> \
wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px \
#ccc solid;padding-left:1ex">[dropping multiple lists; let's just use \
libvirt-users]<br> <span class=""><br>
On 02/11/2015 02:45 PM, Edward Young wrote:<br>
<br>
> I perform a simple test, but failed.<br>
><br>
> In the source, I create: base <- mid <- active (2 snapshots, the \
active<br> > one is the current one)<br>
> In order to migrate this vm to the destination, I manually copy both base<br>
> and mid to the destination, and put them in the sam e directory<br>
> /var/lib/libvirt/images/<br>
> Then I call the following commands:<br>
> 'sudo virsh migrate --live --unsafe --verbose --copy-storage-inc vm1<br>
> qemu+ssh://<a href="http://192.168.1.3/system" \
target="_blank">192.168.1.3/system</a> tcp://<a href="http://192.168.1.3" \
target="_blank">192.168.1.3</a>'<br> ><br>
> The error I got is:<br>
> error: Requested operation is not valid: cannot migrate domain with 2<br>
> snapshots.<br>
><br>
> I do not know the reason for this.<br>
<br>
</span>That's because libvirt is not (yet) able to migrate snapshots in a way<br>
that the destination can revert to them. It is possible to work around<br>
this in a couple of ways:<br>
<br>
1. Manually dump all the snapshot XML configurations on the source side<br>
(virsh snapshot-list, then virsh snapshot-dumpxml in a loop), then<br>
delete the snapshot tracking (virsh snapshot-delete --metadata in a<br>
loop), then do the migration, then manually recreate the snapshot<br>
configuration on the destination (virsh snapshot-create --redefine in a<br>
loop for each of the xml files you saved earlier).<br>
<br>
If you don't plan to revert to snapshots on the destination, then you<br>
can skip the snapshot-dumpxml loop, as well as the snapshot-create<br>
--redefine loop. But if that's the case, then you can take a shortcut:<br>
<br>
2. Don't track external snapshot data in libvirt in the first place<br>
(virsh snapshot-create --no-metadata when taking external snapshots).<br>
Instead, the backing chain is sufficient to know that external snapshot<br>
data was saved.<br></blockquote><div>It works for me. Thanks! </div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> <span class=""><br>
><br>
><br>
>> In fact, modern<br>
>> libvirt/qemu does this for you by setting up an NBD server on the<br>
>> destination, doing a data mirror from the source into the destination<br>
>> (so that you DO have shared storage, at least for the duration of the<br>
>> migration), then doing the live migration, then tearing down the NBD<br>
>> mirroring link.<br>
>><br>
><br>
> Sounds interesting! Can you provide me an example or point me to a link<br>
> about how to use this function?<br>
<br>
</span>It's all done under the hood as part of the implementation of \
'virsh<br> migrate --copy-source-*', if done as part of migration. But if \
you want<br> to manually recreate the steps, the source code in<br>
src/qemu/qemu_migration.c shows the QMP commands used to set up the NBD<br>
server on the destination, then the mirroring is done with equivalent of<br>
the 'virsh blockcopy' command pointing to the NBD server as the \
destination.<br></blockquote><div>I will take a look at the code. </div><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> <span class=""><br>
><br>
> Sorry to take you so much time to answer my questions. I appreciate it<br>
> deeply. After I solve this issue, I will write a blog and share these<br>
> knowledge with other people.<br>
<br>
</span>The libvirt-users list archives has a lot of questions and posts from<br>
other users on the same topic; it may also help if you peruse the \
archives.<br></blockquote><div>I tried previously, but I can't find them. I \
search them directly by google? Is there anyway to find the related posts in the \
archive?</div><div><br></div><div><br></div><div>Thanks a \
lot!</div><div>Ed</div><div><br></div><div> </div><blockquote class="gmail_quote" \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div \
class="HOEnZb"><div class="h5"><br>
--<br>
Eric Blake eblake redhat com <a href="tel:%2B1-919-301-3266" \
value="+19193013266">+1-919-301-3266</a><br> Libvirt virtualization library <a \
href="http://libvirt.org" target="_blank">http://libvirt.org</a><br> <br>
</div></div></blockquote></div><br></div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic