livemedia-creator --live-iso deliverables

Forgot to cc acarter on the original email

Post by Dennis Gilmore
Hi,
Today I was running through runs of livemedia-creator in mock to figure out
what exactly we need to get koji to run in order to produce the liveiso's I
got things running and was able to produce a live iso. aside from things
like the iso being ~1G bigger than what we get from livecd-creator I had
one big surprise. The surprise was the output that we got. In Release
engineering we want the tooling to make the artifact as we are going to
ship it. This means that someone running the tool at home can make the same
thing as what we make. I was expecting for output something along the lines
of Fedora-Live- Workstation-x86_64-rawhide-20151215.iso which is what we
get from livecd- creator. Instead of just an iso what I got was a tree
EFI
EFI/BOOT
EFI/BOOT/BOOTX64.EFI
EFI/BOOT/MokManager.efi
EFI/BOOT/grubx64.efi
EFI/BOOT/grub.cfg
EFI/BOOT/fonts
EFI/BOOT/fonts/unicode.pf2
images
images/pxeboot
images/pxeboot/vmlinuz
images/pxeboot/initrd.img
images/install.img
images/efiboot.img
images/macboot.img
images/product.img
images/boot.iso
isolinux
isolinux/vmlinuz
isolinux/initrd.img
isolinux/isolinux.bin
isolinux/vesamenu.c32
isolinux/ldlinux.c32
isolinux/libcom32.c32
isolinux/libutil.c32
isolinux/isolinux.cfg
isolinux/grub.conf
isolinux/boot.msg
isolinux/splash.png
isolinux/memtest
LiveOS
LiveOS/squashfs.img
where the only thing that we will ship is images/boot.iso but we will have
to do manipulation of the tree to keep what we want and name it correctly
and discard all the extra bits. this means that every use that makes
livecds will need to write tooling to extract the boot.iso, rename it and
discard the extra bits.
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/
as an example of how we ship live media it is not a great thing to have to
do. I strongly feel that a small piece of additional code in
livemedia-creator to correctly name the iso given that there is a
--image-name flag is reasonable and will save every person that makes
livemedia from having to rewrite basically the same function, or us just
writing a wrapper around livemedia- creator to do so.
I am happy to write the code but want to make sure that it will be welcome
before I go off and do so. This is a feature we are trying to get into the
compose tools and process for Fedora 24 so that we can get rid of livecd-
creator and appliance-creator once and for all
Regards
Dennis

Alexander Todorov

2015-12-16 08:34:27 UTC

Post by Dennis Gilmore
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/
as an example of how we ship live media it is not a great thing to have to
do. I strongly feel that a small piece of additional code in
livemedia-creator to correctly name the iso given that there is a
--image-name flag is reasonable and will save every person that makes
livemedia from having to rewrite basically the same function, or us just
writing a wrapper around livemedia- creator to do so.

Hi Dennis,
FYI if you add this, please make it an option so that existing livemedia-creator
behavior isn't affected (where possible). Just recently QE has developed some
tests which use not only the boot.iso but other bits from the generated tree.

--
Alex

Adam Williamson

2015-12-16 17:54:32 UTC

Hmm, well that answers one of my questions. ;) What tests are these?

--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Alexander Todorov

2015-12-17 14:42:11 UTC

Hmm, well that answers one of my questions. ;) What tests are these?

Using lmc create a Live ISO or a disk image. Then use virt-install to boot the
ISO/image. Once the VM is running ssh into it and make sure it's operational.

I can provide you with more details in private if you need them.

--
Alex

Adam Williamson

2015-12-17 17:16:36 UTC

Hmm, well that answers one of my questions. ;) What tests are these?

Using lmc create a Live ISO or a disk image. Then use virt-install to boot the
ISO/image. Once the VM is running ssh into it and make sure it's operational.

But what other bits from the generated tree would that use?

--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Alexander Todorov

2015-12-18 15:39:57 UTC

Hmm, well that answers one of my questions. ;) What tests are these?

Using lmc create a Live ISO or a disk image. Then use virt-install to boot the
ISO/image. Once the VM is running ssh into it and make sure it's operational.

But what other bits from the generated tree would that use?

install.img

Brian C. Lane

2015-12-16 17:23:17 UTC

You can probably thank DNF for that. It's idea of the right packages is
different from yums.

Post by Dennis Gilmore
surprise. The surprise was the output that we got. In Release engineering we
want the tooling to make the artifact as we are going to ship it. This means
that someone running the tool at home can make the same thing as what we make.
I was expecting for output something along the lines of Fedora-Live-
Workstation-x86_64-rawhide-20151215.iso which is what we get from livecd-
creator. Instead of just an iso what I got was a tree

Remember that this is not livecd-creator, and it was not designed to
give you the same output.

Post by Dennis Gilmore
where the only thing that we will ship is images/boot.iso but we will have to
do manipulation of the tree to keep what we want and name it correctly and
discard all the extra bits. this means that every use that makes livecds will
need to write tooling to extract the boot.iso, rename it and discard the extra
bits.

Don't you already have to do that with the new lmc koji code to grab the
output from the run inside mock? I really don't see the need to add yet
this to lmc when your code can easily move the boot.iso to the final
location/name and remove the intermediate directory.

Post by Dennis Gilmore
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as
an example of how we ship live media it is not a great thing to have to do. I
strongly feel that a small piece of additional code in livemedia-creator to
correctly name the iso given that there is a --image-name flag is reasonable
and will save every person that makes livemedia from having to rewrite
basically the same function, or us just writing a wrapper around livemedia-
creator to do so.

lmc uses lorax to create the iso output, which it why it looks very much
like a boot.iso tree. I want to minimize differences from lorax as much
as possible.

Post by Dennis Gilmore
I am happy to write the code but want to make sure that it will be welcome
before I go off and do so. This is a feature we are trying to get into the
compose tools and process for Fedora 24 so that we can get rid of livecd-
creator and appliance-creator once and for all

Thanks for doing this, in this case I think it belongs in the new koji
code, not lmc.

--
Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT)

Adam Williamson

2015-12-16 17:53:25 UTC

Remember that this is not livecd-creator, and it was not designed to
give you the same output.

Well, yes, but also, it's supposed to be livecd-creator's replacement,
and it's supposed to perform the same role. Surely one of its #1 goals
should be "to produce what release engineering wants it to produce",
since that's probably its most critical role. If releng is telling you
its output is not what they want, that seems like something to
consider.

You didn't provide any response to dgilmore's objection to this design:
that anyone else using lmc would have to recreate that work. Obviously
not everyone who wants to build a live image for testing has a Koji
instance they can run it through.

Post by Dennis Gilmore
given http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/ as
an example of how we ship live media it is not a great thing to have to do. I
strongly feel that a small piece of additional code in livemedia-creator to
correctly name the iso given that there is a --image-name flag is reasonable
and will save every person that makes livemedia from having to rewrite
basically the same function, or us just writing a wrapper around livemedia-
creator to do so.

lmc uses lorax to create the iso output, which it why it looks very much
like a boot.iso tree. I want to minimize differences from lorax as much
as possible.

But presumably you also want a tool that produces what its users
actually want. As someone who uses livecd-creator regularly, I want
what dgilmore wants: a live ISO file. I don't want all that other
stuff. Naming is a bit trickier, but it certainly seems reasonable to
at least provide the ability to choose the output file name with a
parameter (as livecd-creator did).

Perhaps a useful question would be: is there anyone who *would* want
all that stuff, when building a live image? And if not, what reason is
there to provide it?

--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Brian C. Lane

2015-12-16 18:32:32 UTC

Post by Adam Williamson
Perhaps a useful question would be: is there anyone who *would* want
all that stuff, when building a live image? And if not, what reason is
there to provide it?

There are, and they do. Also remember that lmc has been in use by others
for quite a while now and that you and Fedora releng are not the only
users.

--
Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT)

Dennis Gilmore

2015-12-16 19:05:03 UTC

Post by Adam Williamson
Perhaps a useful question would be: is there anyone who *would* want
all that stuff, when building a live image? And if not, what reason is
there to provide it?

There are, and they do. Also remember that lmc has been in use by others
for quite a while now and that you and Fedora releng are not the only
users.

What are the use cases for the extra things? and are they something we want to
support?

Dennis

Dennis Gilmore

2015-12-16 19:13:02 UTC

Post by Dennis Gilmore
surprise. The surprise was the output that we got. In Release
engineering we want the tooling to make the artifact as we are going to
ship it. This means that someone running the tool at home can make the
same thing as what we make. I was expecting for output something along
the lines of Fedora-Live-
Workstation-x86_64-rawhide-20151215.iso which is what we get from
livecd- creator. Instead of just an iso what I got was a tree

Remember that this is not livecd-creator, and it was not designed to
give you the same output.

anyone at home making livecds's or people doing remixes will be expecting the
same output as livecd-creator

Post by Dennis Gilmore
where the only thing that we will ship is images/boot.iso but we will
have to do manipulation of the tree to keep what we want and name it
correctly and discard all the extra bits. this means that every use
that makes livecds will need to write tooling to extract the boot.iso,
rename it and discard the extra bits.

There are many cases where people make livecds without koji. each one of them
needs to write something?

Post by Adam Williamson
that anyone else using lmc would have to recreate that work. Obviously
not everyone who wants to build a live image for testing has a Koji
instance they can run it through.

Post by Dennis Gilmore
given
http://dl.fedoraproject.org/pub/fedora/linux/releases/23/Live/x86_64/
as an example of how we ship live media it is not a great thing to have
to do. I strongly feel that a small piece of additional code in
livemedia-creator to correctly name the iso given that there is a
--image-name flag is reasonable and will save every person that makes
livemedia from having to rewrite basically the same function, or us
just writing a wrapper around livemedia- creator to do so.

lmc uses lorax to create the iso output, which it why it looks very much
like a boot.iso tree. I want to minimize differences from lorax as much
as possible.

anyone that respins fedora lives's either adding updates, or making a remix
adding things we can not ship is going to want what I am asking for and will
not have koji and will have to re-implement something to get the output
needed. There are many use cases outside of releng that wants to make a livecd
and just a livecd.

There is nothing wrong with code reuse. I do not see how we can ship what is
produced even if we decide that some of the extra stuff is useful. we can not
ship 20 isos called boot.iso. people regularly grab isos from different parts
of the tree and sync them to a common directory.

Dennis

Colin Walters

2015-12-16 20:02:39 UTC

Post by Dennis Gilmore
anyone at home making livecds's or people doing remixes will be expecting the
same output as livecd-creator

I'd say anyone using all of this tooling is going to have to be ready to adapt to some
degree.

Post by Dennis Gilmore
There are many cases where people make livecds without koji. each one of them
needs to write something?

FWIW rpm-ostree-toolbox presently also wraps livemedia-creator and does
do output postprocessing - I didn't find it really arduous to maintain.

But it'd also seem sane to me to have a --output-exactly-like-livecd-creator
option in lmc.

Brian C. Lane

2016-01-07 18:25:37 UTC

This post might be inappropriate. Click to display it.

Dennis Gilmore

2016-01-08 16:35:38 UTC