Blocker status of RHBZ #1033778 (shrinking unknown volumes)

[Adam Williamson]

Hi folks! So as promised on IRC here's a mail to discuss this issue.

https://bugzilla.redhat.com/show_bug.cgi?id=1033778 is proposed as a
release blocker. As I understand it, dlehman is strongly of the opinion
that it shouldn't be accepted, on technical grounds. However, it does
constitute a fairly clear violation of the release criteria.

The bug - as I understand it - is that a particular type of partition
("Apple Core Storage" volumes, whatever those are) appears as "Unknown"
in anaconda, and the guided install workflow (the "Reclaim Space"
screen) allows you to try and resize it. If you do this, it fails to
work at all and causes complete loss of all data on the partition.

The Final release criteria state:

"Any installer mechanism for resizing storage volumes must correctly
attempt the requested operation."

with a footnote:

"This means that if the installer offers mechanisms for resizing
storage volumes, then it must run the appropriate resizing tool with
the appropriate parameters for the resize the user chooses. The reason
it's worded this way is we specifically don't want to cover cases where
the requested resize operation then fails for some reason - dirtily
unmounted or over-fragmented partition, for instance. We only want to
cover the case that the installer's resize code itself is badly
broken."

It seems pretty inarguable that as the criteria currently stand, this
bug violates them, and thus it should be a Final release blocker.
However, we delayed the decision at this week's blocker review meeting
since we were aware you folks might want this not to be a blocker.

For clarity, if it were to be accepted as a blocker, this would not
mean that anaconda would be required to resize such partitions
successfully, nor does it necessarily imply that anaconda must be able
to *recognize* such partitions. So far as the blocker process is
concerned it would be a sufficient resolution if *either* anaconda
attempted to resize such partitions 'appropriately' (I'm not sure if
that is even possible) *or* refused to attempt to resize them. The
criteria do not, I think, place any effective restrictions on exactly
*how* the latter resolution could be effected.

So, I guess the question is: do we think the criteria should stand as
written but this bug be rejected, and if so on what grounds? Do we
think the criteria should be adjusted to reflect some kind of technical
restriction? Or do we in fact think the bug should be accepted (and
presumably the 'fix' should be to disallow resizing of this kind of
partition somehow)?

I asked on IRC whether it would be possible to disallow resizing of
'unknown' partitions. As I understand it the problem with this is it's
difficult or impossible to distinguish between an empty partition and
one that contains an unknown filesystem (and, presumably, one that is
not empty, but contains garbage).

I wonder, though, is that actually a problem? Do we need to allow
'resizing' of such partitions, or can we simply say that you should
delete and recreate such partitions, and we will only allow the
operation we call 'resizing' for partitions we definitely know the
filesystem of and that we can safely resize that filesystem?

Thanks!

[David Lehman]

Post by Adam Williamson
Hi folks! So as promised on IRC here's a mail to discuss this issue.
https://bugzilla.redhat.com/show_bug.cgi?id=1033778 is proposed as a
release blocker. As I understand it, dlehman is strongly of the opinion
that it shouldn't be accepted, on technical grounds. However, it does
constitute a fairly clear violation of the release criteria.
The bug - as I understand it - is that a particular type of partition
("Apple Core Storage" volumes, whatever those are) appears as
"Unknown"
in anaconda, and the guided install workflow (the "Reclaim Space"
screen) allows you to try and resize it. If you do this, it fails to
work at all and causes complete loss of all data on the partition.
"Any installer mechanism for resizing storage volumes must correctly
attempt the requested operation."
"This means that if the installer offers mechanisms for resizing
storage volumes, then it must run the appropriate resizing tool with
the appropriate parameters for the resize the user chooses. The reason

What is the appropriate tool (and parameters) for resizing the
formatting on a device with unrecognized/unknown formatting?

Post by Adam Williamson
it's worded this way is we specifically don't want to cover cases where
the requested resize operation then fails for some reason - dirtily
unmounted or over-fragmented partition, for instance. We only want to
cover the case that the installer's resize code itself is badly
broken."
It seems pretty inarguable that as the criteria currently stand, this
bug violates them, and thus it should be a Final release blocker.
However, we delayed the decision at this week's blocker review
meeting
since we were aware you folks might want this not to be a blocker.
For clarity, if it were to be accepted as a blocker, this would not
mean that anaconda would be required to resize such partitions
successfully, nor does it necessarily imply that anaconda must be able
to *recognize* such partitions. So far as the blocker process is
concerned it would be a sufficient resolution if *either* anaconda
attempted to resize such partitions 'appropriately' (I'm not sure if
that is even possible) *or* refused to attempt to resize them. The
criteria do not, I think, place any effective restrictions on exactly
*how* the latter resolution could be effected.
So, I guess the question is: do we think the criteria should stand as
written but this bug be rejected, and if so on what grounds? Do we
think the criteria should be adjusted to reflect some kind of
technical
restriction? Or do we in fact think the bug should be accepted (and
presumably the 'fix' should be to disallow resizing of this kind of
partition somehow)?
I asked on IRC whether it would be possible to disallow resizing of
'unknown' partitions. As I understand it the problem with this is it's
difficult or impossible to distinguish between an empty partition and
one that contains an unknown filesystem (and, presumably, one that is
not empty, but contains garbage).
I wonder, though, is that actually a problem? Do we need to allow
'resizing' of such partitions, or can we simply say that you should
delete and recreate such partitions, and we will only allow the
operation we call 'resizing' for partitions we definitely know the
filesystem of and that we can safely resize that filesystem?
Thanks!

[Adam Williamson]

Post by David Lehman

Post by Adam Williamson
"This means that if the installer offers mechanisms for resizing
storage volumes, then it must run the appropriate resizing tool with
the appropriate parameters for the resize the user chooses. The reason

What is the appropriate tool (and parameters) for resizing the
formatting on a device with unrecognized/unknown formatting?

I don't believe we explicitly considered that question at the time of
writing the criterion. However, my interpretation as the person who
drafted it (IIRC) would be that the "appropriate tool" for any
partition with data on it is a tool which is at least *intended* to
preserve the data.

In an ideal world, with no specific technical concerns, it would be my
expectation that we would not offer an operation named "resize" in the
case where we have no idea how to preserve any contents of the volume.
We would only offer an operation named "resize" in cases where we do
actually have some idea how to perform a non-destructive resize. I'm
entirely open to there being technical reasons why this is not
possible, though.

[David Lehman]

Post by Adam Williamson

Post by David Lehman

Post by Adam Williamson
"This means that if the installer offers mechanisms for resizing
storage volumes, then it must run the appropriate resizing tool with
the appropriate parameters for the resize the user chooses. The reason

What is the appropriate tool (and parameters) for resizing the
formatting on a device with unrecognized/unknown formatting?

I don't believe we explicitly considered that question at the time of
writing the criterion. However, my interpretation as the person who
drafted it (IIRC) would be that the "appropriate tool" for any
partition with data on it is a tool which is at least *intended* to
preserve the data.
In an ideal world, with no specific technical concerns, it would be my
expectation that we would not offer an operation named "resize" in the
case where we have no idea how to preserve any contents of the
volume.
We would only offer an operation named "resize" in cases where we do
actually have some idea how to perform a non-destructive resize. I'm
entirely open to there being technical reasons why this is not
possible, though.

It is not possible to distinguish between a lack of meaningful data and
meaningful data that the OS has no means of recognizing. A partition
type flag or GUID is not generally sufficient for this purpose.

We do have the option to prevent resize of devices with no formatting
or unrecognized formatting. You could certainly argue that it makes
more sense to remove such a device than to resize it if you need more
space. I'll just explain that it's in the name of protecting careless
users when the bugs start coming in.

David

[Adam Williamson]

Post by David Lehman

Post by Adam Williamson

Post by David Lehman

Post by Adam Williamson
"This means that if the installer offers mechanisms for resizing
storage volumes, then it must run the appropriate resizing tool with
the appropriate parameters for the resize the user chooses. The reason

What is the appropriate tool (and parameters) for resizing the
formatting on a device with unrecognized/unknown formatting?

I don't believe we explicitly considered that question at the time of
writing the criterion. However, my interpretation as the person who
drafted it (IIRC) would be that the "appropriate tool" for any
partition with data on it is a tool which is at least *intended* to
preserve the data.
In an ideal world, with no specific technical concerns, it would be my
expectation that we would not offer an operation named "resize" in the
case where we have no idea how to preserve any contents of the volume.
We would only offer an operation named "resize" in cases where we do
actually have some idea how to perform a non-destructive resize. I'm
entirely open to there being technical reasons why this is not
possible, though.

It is not possible to distinguish between a lack of meaningful data and
meaningful data that the OS has no means of recognizing. A partition
type flag or GUID is not generally sufficient for this purpose.
We do have the option to prevent resize of devices with no formatting
or unrecognized formatting. You could certainly argue that it makes
more sense to remove such a device than to resize it if you need more
space. I'll just explain that it's in the name of protecting careless
users when the bugs start coming in.

So basically it's another shoot-yourself-in-the-foot conundrum, with
the standard choices? Leave the safety off, put a dumbass "Are you sure
you want to do that?" message in front of it, or disallow it and annoy
some partition pokemon with an implausible use case for it?

Well hey, life sucks. I *think* my preference, if you asked my opinion,
would be to disallow it and just say "if it's an empty partition you
really want to make smaller, just do it in another tool or delete it
and recreate it smaller". But if you think the balance here should be
in favour of letting people shoot themselves in the foot, I'm not gonna
argue it, and we'll just have to rewrite the criterion somehow.

[David Lehman]

Post by Adam Williamson

Post by David Lehman

Post by Adam Williamson

Post by David Lehman

Post by Adam Williamson
"This means that if the installer offers mechanisms for resizing
storage volumes, then it must run the appropriate resizing
tool
with
the appropriate parameters for the resize the user chooses.
The
reason

What is the appropriate tool (and parameters) for resizing the
formatting on a device with unrecognized/unknown formatting?

I don't believe we explicitly considered that question at the time of
writing the criterion. However, my interpretation as the person who
drafted it (IIRC) would be that the "appropriate tool" for any
partition with data on it is a tool which is at least *intended* to
preserve the data.
In an ideal world, with no specific technical concerns, it would
be
my
expectation that we would not offer an operation named "resize"
in
the
case where we have no idea how to preserve any contents of the volume.
We would only offer an operation named "resize" in cases where we do
actually have some idea how to perform a non-destructive resize. I'm
entirely open to there being technical reasons why this is not
possible, though.

It is not possible to distinguish between a lack of meaningful data and
meaningful data that the OS has no means of recognizing. A
partition
type flag or GUID is not generally sufficient for this purpose.
We do have the option to prevent resize of devices with no
formatting
or unrecognized formatting. You could certainly argue that it makes
more sense to remove such a device than to resize it if you need more
space. I'll just explain that it's in the name of protecting
careless
users when the bugs start coming in.

So basically it's another shoot-yourself-in-the-foot conundrum, with
the standard choices? Leave the safety off, put a dumbass "Are you sure
you want to do that?" message in front of it, or disallow it and annoy
some partition pokemon with an implausible use case for it?
Well hey, life sucks. I *think* my preference, if you asked my
opinion,
would be to disallow it and just say "if it's an empty partition you
really want to make smaller, just do it in another tool or delete it
and recreate it smaller". But if you think the balance here should be
in favour of letting people shoot themselves in the foot, I'm not gonna
argue it, and we'll just have to rewrite the criterion somehow.

I don't have a strong preference as to which corner-case we turn our
back on here. I do think that if this qualifies as a blocker under the
current criteria a rewrite/edit is definitely in order.

David

[Adam Williamson]

Post by David Lehman

Post by Adam Williamson
So basically it's another shoot-yourself-in-the-foot conundrum, with
the standard choices? Leave the safety off, put a dumbass "Are you sure
you want to do that?" message in front of it, or disallow it and annoy
some partition pokemon with an implausible use case for it?
Well hey, life sucks. I *think* my preference, if you asked my opinion,
would be to disallow it and just say "if it's an empty partition you
really want to make smaller, just do it in another tool or delete it
and recreate it smaller". But if you think the balance here should be
in favour of letting people shoot themselves in the foot, I'm not gonna
argue it, and we'll just have to rewrite the criterion somehow.

I don't have a strong preference as to which corner-case we turn our
back on here. I do think that if this qualifies as a blocker under the
current criteria a rewrite/edit is definitely in order.

OK. We have a blocker review meeting coming up in 90 minutes or so; I
will propose that we adjust the criterion wording to apply only to
known filesystems.

[Adam Williamson]

Post by Adam Williamson

Post by David Lehman

Post by Adam Williamson
So basically it's another shoot-yourself-in-the-foot conundrum, with
the standard choices? Leave the safety off, put a dumbass "Are you sure
you want to do that?" message in front of it, or disallow it and annoy
some partition pokemon with an implausible use case for it?
Well hey, life sucks. I *think* my preference, if you asked my opinion,
would be to disallow it and just say "if it's an empty partition you
really want to make smaller, just do it in another tool or delete it
and recreate it smaller". But if you think the balance here should be
in favour of letting people shoot themselves in the foot, I'm not gonna
argue it, and we'll just have to rewrite the criterion somehow.

I don't have a strong preference as to which corner-case we turn our
back on here. I do think that if this qualifies as a blocker under the
current criteria a rewrite/edit is definitely in order.

OK. We have a blocker review meeting coming up in 90 minutes or so; I
will propose that we adjust the criterion wording to apply only to
known filesystems.

We agreed to reword the criterion, and the bug is rejected as a blocker
(though for the record, it seems like general sentiment at the meeting
was that it would be a good idea to either disallow resize of unknown
volumes or have a specific pop-up warning for them that any data on the
volume will be lost).

[Chris Murphy]

Post by David Lehman
It is not possible to distinguish between a lack of meaningful data and
meaningful data that the OS has no means of recognizing. A partition
type flag or GUID is not generally sufficient for this purpose.

You're effectively asserting the validity of ignoring partition type
GUIDs as if this is free space, or an invalid/stale partition type,
whenever you don't recognize the contents or GUID. This is folly. e.g.
Chromium OS puts binary blobs in partitions with no filesystems at
all, depending only on partition type GUID for identification.
Anaconda sees 21 of 28 Chromium OS partitions as resizable, doing so
will of course break them.

OS X and Windows tools don't let users shrink partitions with
unrecognized contents. Ubuntu and openSUSE installer-partitioners also
don't permit it. Anaconda is manifestly more dangerous, in the same
situation. This situation disproportionately affects OS X users
because the OS X default on disk layout is affected by this bug now
for just over a year so I expect more users will be affected.

I've evaluated five installer-partitioners for comparison with respect
to this bug. How many have you evaluated? Can you point to any others
that offer resize UI for partitions with unrecognized contents? If so
and it's free software I will promptly go file a bug for that product
too, because it's a flawed design that sets the user up for failure.

Post by David Lehman
We do have the option to prevent resize of devices with no formatting
or unrecognized formatting. You could certainly argue that it makes
more sense to remove such a device than to resize it if you need more
space.

David, that's what I've been saying for two years in the bug report,
while you've been consistently blaming the user for running into it.

Post by David Lehman
I'll just explain that it's in the name of protecting careless
users when the bugs start coming in.

Unconvincing. You know more about storage esoterics, and you know
about this data loss bug, the user doesn't. You've been using the
appeal to ridicule and scapegoating logical fallacies from day 1,
directed at the user and in the jab at Karel Zak. Fallacies might work
for some people until you get caught and they see that you have no
actual good arguments, so consider yourself caught. Blaming and
mocking the user and another developer, both of whom have had no say
in this design or consequential behavior, are not compelling reasons
to invalidate bug or its status as a blocker bug.

For 10+ years, resize in a GUI partitioner has been considered by
users to be safe, and not only because resize UI isn't offered when it
can't be done safely. Ask a file system developer (I've asked a few)
and they expect filesystem resize to be a fail safe operation. It
might fail, but there should be no data loss. If there is, it's a
major bug, they'd freak out and fix it.

On the one hand you admit things could be handled better, but then you
resort to yet more backhanded blame and mockery in the very same bug
post. That's a big part of what gets my goat about this bug, it should
have long ago been an ordinary "oops, not good, we should probably fix
that in the next release or two" kind of bug. Bugs happen. Bad bugs
happen. Blaming others for this bug isn't indicated.

As for whether it's a blocker bug: In three blocker meetings everyone
was prepared to make it a blocker on the merits of the bug. My
educated guess is no one wanted to fight with you about it when you
dole out derision like Kleenex, and also added further duress by
saying:

18:38:23 <adamw> <dlehman> lol
18:38:23 <adamw> <dlehman> I might go so far as to get myself kicked
out of the fedora project before "fixing" that
https://meetbot.fedoraproject.org/fedora-blocker-review/2016-02-22/f24-blocker-review.2016-02-22-17.00.log.html

This is untenable for QA to make the bug a blocker when the upstream
announces in advance they won't fix it. How does that play out? Your
(circular) logic here is, it's not a blocker because you say it's not
a blocker. You've simply bugged out of participating in a neutral 3rd
party process established to arrive at a minimum quality releasable
product. QA folks went with the pragmatic path of least resistance,
didn't they? I see no where that they agreed with your reasoning,
because you didn't provide any reasoning.

How many more users need to hit this bug before you'd fix it inside of
two minutes? I'd be willing to bet one or maybe two more. I seriously
doubt you really believe this is never a blocker even if 10 OS X users
get impaled by it. And yes it's completely reasonable that everyone
has more important and interesting things to do than fix this bug,
that applies to a lot of bugs and even sometimes blockers, but that's
the process.

This sounds pretty close to perfect as a conditional blocker, a.k.a.
it's a "local configuration dependent issue" since it mainly affects
OS X users with the default partitioning layout now in use on that OS,
and then the user proceeds with resizing this unknown partition for
whatever reason. Fix it whenever you want. Maybe someone beats you to
it. And consider trusting QA to make it a blocker at some point if
more users hit it. I don't really see a problem with that method of
making it not an immediate blocker for this release.
https://fedoraproject.org/wiki/Blocker_Bug_FAQ

[Adam Williamson]

As for whether it's a blocker bug:  In three blocker meetings everyone
was prepared to make it a blocker on the merits of the bug. My
educated guess is no one wanted to fight with you about it when you
dole out derision like Kleenex, and also added further duress by

This tone is really not warranted. We agreed the bug would have
violated the criterion as written, but we also observed that this isn't
a situation that was actually envisaged when writing the criterion; the
criterion was written thinking about the case where anaconda
incorrectly tried to resize a partition it understood, it was not
intended to prescribe anaconda's behaviour with regard to partitions it
does not understand. I can tell you that, as I wrote it.

18:38:23 <adamw> <dlehman> lol
18:38:23 <adamw> <dlehman> I might go so far as to get myself kicked
out of the fedora project before "fixing" that
https://meetbot.fedoraproject.org/fedora-blocker-review/2016-02-22/f24-blocker-review.2016-02-22-17.00.log.html

The quote out of context is maybe misleading, and I think quoting it at
two removes (from #anaconda IRC to the blocker meeting to here) is a
little unfair. I believe that, in context, dlehman was referring to the
idea of "fixing" detection of Apple Core partitions, not the
possibility of tightening down on resize of unknown partitions.

This is untenable for QA to make the bug a blocker when the upstream
announces in advance they won't fix it. How does that play out? Your
(circular) logic here is, it's not a blocker because you say it's not
a blocker. You've simply bugged out of participating in a neutral 3rd
party process established to arrive at a minimum quality releasable
product. QA folks went with the pragmatic path of least resistance,
didn't they? I see no where that they agreed with your reasoning,
because you didn't provide any reasoning.

We discussed it in this very thread, the reasoning is available in
dlehman's earlier posts to it. Personally I think his position is
entirely reasonable.

The blocker process is not one where "QA make[s] the bug a blocker".
The blocker process, as designed, is a collaboration between QA, devel,
releng, and product management (FPL and FPM). Note the inclusion of
devel there. The criteria were initially written in close collaboration
with the development teams, and are frequently adjusted to reflect
their conception of what's practical and reasonable, and development
teams absolutely are expected and intended to have input into the
blocker process.

This sounds pretty close to perfect as a conditional blocker, a.k.a.
it's a "local configuration dependent issue" since it mainly affects
OS X users with the default partitioning layout now in use on that OS,
and then the user proceeds with resizing this unknown partition for
whatever reason. Fix it whenever you want. Maybe someone beats you to
it. And consider trusting QA to make it a blocker at some point if
more users hit it. I don't really see a problem with that method of
making it not an immediate blocker for this release.
https://fedoraproject.org/wiki/Blocker_Bug_FAQ

I'm not quite sure what you're suggesting here; are you suggesting that
we could have rejected it as a blocker on the basis that it's a
conditional violation of the criterion and we declare that the impact
is not broad enough? We could have done that, sure, but in the end we
went a different way as we wanted to clarify the question of whether
the criterion was intended to dictate anaconda's behaviour with regard
to unknown partitions.

[Chris Murphy]

Post by Adam Williamson

Post by Chris Murphy
This is untenable for QA to make the bug a blocker when the upstream
announces in advance they won't fix it. How does that play out? Your
(circular) logic here is, it's not a blocker because you say it's not
a blocker. You've simply bugged out of participating in a neutral 3rd
party process established to arrive at a minimum quality releasable
product. QA folks went with the pragmatic path of least resistance,
didn't they? I see no where that they agreed with your reasoning,
because you didn't provide any reasoning.

We discussed it in this very thread, the reasoning is available in
dlehman's earlier posts to it. Personally I think his position is
entirely reasonable.

What/where? I've addressed the ridicule and scapegoating already, the
only other one I'm seeing is a strawman argument:

On Wed, Feb 24, 2016 at 7:40 AM, David Lehman <***@redhat.com> wrote:
"What is the appropriate tool (and parameters) for resizing the
formatting on a device with unrecognized/unknown formatting?"

Comment 33c. "The tool doesn't exist." No one is asking
anaconda-blivet to perform magic by successfully resizing devices with
unknown contents, so the question David's asking is an obvious
fallacy.
Expected results:
Should be listed as "not resizeable"

Post by Adam Williamson
The blocker process is not one where "QA make[s] the bug a blocker".
The blocker process, as designed, is a collaboration between QA, devel,
releng, and product management (FPL and FPM). Note the inclusion of
devel there. The criteria were initially written in close collaboration
with the development teams, and are frequently adjusted to reflect
their conception of what's practical and reasonable, and development
teams absolutely are expected and intended to have input into the
blocker process.

Post by Chris Murphy
This sounds pretty close to perfect as a conditional blocker, a.k.a.
it's a "local configuration dependent issue" since it mainly affects
OS X users with the default partitioning layout now in use on that OS,
and then the user proceeds with resizing this unknown partition for
whatever reason. Fix it whenever you want. Maybe someone beats you to
it. And consider trusting QA to make it a blocker at some point if
more users hit it. I don't really see a problem with that method of
making it not an immediate blocker for this release.
https://fedoraproject.org/wiki/Blocker_Bug_FAQ

I'm not quite sure what you're suggesting here; are you suggesting that
we could have rejected it as a blocker on the basis that it's a
conditional violation of the criterion and we declare that the impact
is not broad enough?

Yes.

Post by Adam Williamson
We could have done that, sure, but in the end we
went a different way as we wanted to clarify the question of whether
the criterion was intended to dictate anaconda's behaviour with regard
to unknown partitions.

And you did clarify it: Unintended data loss of user valued data in
guided partitioning, where many prognostications where made by
Anaconda folks how data safe it is designed to be, is possible. And
yet no matter how many more people hit it, it would not be considered
a blocker.

It's a bad bug in custom partitioning, it's an egregious bug in guided
partitioning. In my opinion you've pretty much only listened to
David's circular argumentation, and haven't incorporated anything that
takes the user into account. You really think it's OK for this to
never be a blocker no matter how many more get hit by it?