Switch livemedia-creator to qemu instead of virt-install

[Brian C. Lane]

I'm considering switching lmc to use qemu directly instead of
virt-install+libvirtd.

Currently lmc can use virt-install to create the full range of supported
images. But this doesn't work when running inside mock, which is used by
Fedora releng's koji system.

It also supports the --no-virt mode which runs anaconda directly in the
same environment as lmc. The benefit is less things to setup
and it works inside mock. But the drawback is that it uses device-mapper
for installing to a partitioned disk image, and device-mapper doesn't
work inside mock. So when running in mock you can only create a live iso
or a filesystem image, not anything that needs a partitioned disk.

I've been doing some experimenting and it ends up that qemu works inside
mock.

Switching to qemu (with optional auto-detection of when it could use
qemu-kvm to speed things up) would simplify the code and allow it to
create all the image types no matter where it is run from. It is also
possible (I haven't tried this yet) that it could be used to create
cross-arch images as well, since qemu has a wide range of supported
arches.

[Dennis Gilmore]

Post by Brian C. Lane
I'm considering switching lmc to use qemu directly instead of
virt-install+libvirtd.
Currently lmc can use virt-install to create the full range of supported
images. But this doesn't work when running inside mock, which is used by
Fedora releng's koji system.
It also supports the --no-virt mode which runs anaconda directly in the
same environment as lmc. The benefit is less things to setup
and it works inside mock. But the drawback is that it uses device-mapper
for installing to a partitioned disk image, and device-mapper doesn't
work inside mock. So when running in mock you can only create a live iso
or a filesystem image, not anything that needs a partitioned disk.
I've been doing some experimenting and it ends up that qemu works inside
mock.
Switching to qemu (with optional auto-detection of when it could use
qemu-kvm to speed things up) would simplify the code and allow it to
create all the image types no matter where it is run from. It is also
possible (I haven't tried this yet) that it could be used to create
cross-arch images as well, since qemu has a wide range of supported
arches.

Would not be opposed to the change, it should work on platforms where we have
no hardware virt capability. We (releng) are reaching out to the lvm team to
see what can be done to make device-mapper work in a chroot, if it is
possible.

Switching to using qemu would require lmc to learn a lot of details about
running vms for different arches. But it would enable making images for non
native arches as the commands are mostly the same.

To use kvm would we need to bind mount /dev/kvm into the chroot?

Dennis

[Brian C. Lane]

Post by Dennis Gilmore

Post by Brian C. Lane
I'm considering switching lmc to use qemu directly instead of
virt-install+libvirtd.
Currently lmc can use virt-install to create the full range of supported
images. But this doesn't work when running inside mock, which is used by
Fedora releng's koji system.
It also supports the --no-virt mode which runs anaconda directly in the
same environment as lmc. The benefit is less things to setup
and it works inside mock. But the drawback is that it uses device-mapper
for installing to a partitioned disk image, and device-mapper doesn't
work inside mock. So when running in mock you can only create a live iso
or a filesystem image, not anything that needs a partitioned disk.
I've been doing some experimenting and it ends up that qemu works inside
mock.
Switching to qemu (with optional auto-detection of when it could use
qemu-kvm to speed things up) would simplify the code and allow it to
create all the image types no matter where it is run from. It is also
possible (I haven't tried this yet) that it could be used to create
cross-arch images as well, since qemu has a wide range of supported
arches.

Would not be opposed to the change, it should work on platforms where we have
no hardware virt capability. We (releng) are reaching out to the lvm team to
see what can be done to make device-mapper work in a chroot, if it is
possible.
Switching to using qemu would require lmc to learn a lot of details about
running vms for different arches. But it would enable making images for non
native arches as the commands are mostly the same.

True, I'd encapsulate all that in a way that would make it easy to
extend or modify. I think the ability to make images for other arches
would be a big plus, even if it is a bit slow.

Post by Dennis Gilmore
To use kvm would we need to bind mount /dev/kvm into the chroot?

I haven't tried that yet, but at a minimum, yes. I'm not sure what else
is needed to make qemu-kvm work.

[Fabian Deutsch]

Post by Brian C. Lane
I'm considering switching lmc to use qemu directly instead of
virt-install+libvirtd.
Currently lmc can use virt-install to create the full range of supported
images. But this doesn't work when running inside mock, which is used by
Fedora releng's koji system.

Happy silent nights.

We are now also using lmc to build oVirt Node. And we actually do use
it in mock (on Jenkins):

http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-fc23-x86_64/11/console
http://jenkins.ovirt.org/job/ovirt-node-ng_master_build-artifacts-fc23-x86_64/

Post by Brian C. Lane
It also supports the --no-virt mode which runs anaconda directly in the
same environment as lmc. The benefit is less things to setup
and it works inside mock. But the drawback is that it uses device-mapper
for installing to a partitioned disk image, and device-mapper doesn't
work inside mock. So when running in mock you can only create a live iso
or a filesystem image, not anything that needs a partitioned disk.

Yes. Also a big -1 on the no-virt approach.

The nice isolation we have when using the VM lmc approach is gone, and
this can lead to all sorts of issues 8we have seen this over the years
with livecd-creator).
SELinux is a big candidate for causing trouble in this setup.

Post by Brian C. Lane
I've been doing some experimenting and it ends up that qemu works inside
mock.
Switching to qemu (with optional auto-detection of when it could use
qemu-kvm to speed things up) would simplify the code and allow it to
create all the image types no matter where it is run from. It is also

That's funny.
I actually created this set of scripts to fake lmc while lmc was not
running in our Jenkins environment:
https://github.com/fabiand/image-tools

Those scripts are also using qemu directly, and I made the same
observations as you. It's all working quite well.
The drawback is that you probably end up doing a bit of the stuff
which virt-install is doing, doing yourself. i.e. --inject-initrd.
And I suppose there are more features from virt-install which you require.

Post by Brian C. Lane
possible (I haven't tried this yet) that it could be used to create
cross-arch images as well, since qemu has a wide range of supported
arches.

Doesn't libvirtd also support to run different qemu arches (or however
it's called)? And if so, couldn't virt-install support it aswell?

In the end my take would rather be to stick to virt-install and see
how it can be run in mock.

My 2ct.

- fabian

[Brian C. Lane]

Here's my working branch for using qemu with lmc:

https://github.com/bcl/lorax/tree/master-lmc-qemu

and here's a script that can be run from the git repo that will copy
things into the repo and run it with mock:

https://github.com/bcl/lorax/tree/master-lmc-qemu

eg. lmc-mock ~/isos/Fedora-Server-netinst-x86_64-23.iso ./fedora-livemini.ks

This branch works fine for me creating live iso's on my dev system
directly, and using the above script to run it inside mock. Note the
creation of /dev/kvm, without that it takes about 90 minutes to run.
With kvm it finishes in about 10 minutes for my minimal (no-gui) test
kickstart.

I haven't tried doing any cross-arch installations yet, at the least
they are going to be *very* slow compared to using kvm on a matching
arch.

I'm leaning towards changing this for F25 to give people time to give it
a try and shake out any bugs. So, give it a try and let me know what you
think.