odd package conflict behavior

[Dave Peterson]

I just saw some surprising behavior with package conflicts, and would
like to know if anyone can shed some light on this.  On an x86_64
system running RHEL4, I observe the following:

    # rpm -qa --qf '%{name}-%{version}-%{release}.%{arch}\n' | grep e2fs
    e2fsprogs-1.35-11.6.EL4.x86_64
    e2fsprogs-devel-1.35-11.6.EL4.x86_64
    #
    # rpm -i e2fsprogs-1.35-12.17.el4.i386.rpm
    warning: e2fsprogs-1.35-12.17.el4.i386.rpm: V3 DSA signature: NOKEY,
key ID db42a60e
            file /usr/share/man/man8/ext2online.8.gz from install of
e2fsprogs-1.35-12.17.el4 conflicts with file from package
e2fsprogs-1.35-11.6.EL4
            file /usr/share/man/man8/mke2fs.8.gz from install of
e2fsprogs-1.35-12.17.el4 conflicts with file from package
e2fsprogs-1.35-11.6.EL4
            file /usr/share/man/man8/mkfs.ext2.8.gz from install of
e2fsprogs-1.35-12.17.el4 conflicts with file from package
e2fsprogs-1.35-11.6.EL4
            file /usr/share/man/man8/mkfs.ext3.8.gz from install of
e2fsprogs-1.35-12.17.el4 conflicts with file from package
e2fsprogs-1.35-11.6.EL4
    #
    # rpm -e --nodeps e2fsprogs-1.35-11.6.EL4.x86_64
    #
    # rpm -i e2fsprogs-1.35-11.6.EL4.x86_64.rpm
e2fsprogs-1.35-12.17.el4.i386.rpm
    warning: e2fsprogs-1.35-11.6.EL4.x86_64.rpm: V3 DSA signature:
NOKEY, key ID db42a60e
    #
    # rpm -qa --qf '%{name}-%{version}-%{release}.%{arch}\n' | grep e2fs
    e2fsprogs-1.35-12.17.el4.i386
    e2fsprogs-devel-1.35-11.6.EL4.x86_64
    e2fsprogs-1.35-11.6.EL4.x86_64
    #

I start out with package A (e2fsprogs-1.35-11.6.EL4.x86_64) installed,
and attempt to install package B (e2fsprogs-1.35-12.17.el4.i386).
This fails due to file conflicts.  Next I forcibly remove A.  Finally,
I attempt to install both A and B using a single RPM command and this
succeeds (no conflicts).  In other words, RPM's notion of whether A
and B conflict appears to depend on whether A is initially installed.
Is this the intended behavior, or have I found a bug?

I did some investigation using gdb, and observed the following: When A
is initially installed and I attempt to install B, the detection of
file conflicts is done inside handleInstInstalledFiles().  Below is a
simplification of the relevant conflict detection logic (from
rpm-4.3.3-7_nonptl, the version of RPM that ships with RHEL4):

    if (rpmfiCompare(otherFi, fi)) {
        int rConflicts = reportConflicts;
        /* Resolve file conflicts to prefer Elf64 (if not forced). */
        if (tscolor != 0 && FColor != 0 && FColor != oFColor)
        {
            if (oFColor & 0x2) {
                ...
                rConflicts = 0;
            } else if (FColor & 0x2) {
                ...
                rConflicts = 0;
            }
        }

        if (rConflicts) {
            // report a conflict ...
        }
        ...
    }

In the case where neither package is initially installed and we
attempt to install both packages with a single RPM command, the
detection of file conflicts is done inside handleOverlappedFiles().
Below is a simplification of the relevant conflict detection logic:

    if (rpmfiCompare(otherFi, fi)) {
        int rConflicts = reportConflicts;
        /* Resolve file conflicts to prefer Elf64 (if not forced) ... */
        if (tscolor != 0) {
            if (FColor & 0x2) {
                ...
                rConflicts = 0;
            } else if (oFColor & 0x2) {
                ...
                rConflicts = 0;
            } else if (FColor == 0 && oFColor == 0) {
                ...
                rConflicts = 0;
            }
            ...
        }

        if (rConflicts) {
            // report a conflict ...
        }
    }

Notice that these pieces of logic are not equivalent.  The logic in
handleInstInstalledFiles() will report some file conflicts that the
corresponding logic in handleOverlappedFiles() ignores.  Looking at
the RPM 4.6.0 source code, I see that the
"else if (FColor == 0 && oFColor == 0)" disappears from
handleOverlappedFiles().  This brings the two pieces of logic closer
together, but handleInstInstalledFiles() will still report some
conflicts that handleOverlappedFiles() ignores.  Looking at the RPM
5.1.6 source code, I see that the
"else if (FColor == 0 && oFColor == 0)" again appears.

Perhaps it would make sense to fix things so that the conflict
detection behavior is uniform between handleInstInstalledFiles() and
handleOverlappedFiles().  One way of achieving this might be to
somehow refactor the code so that all of the conflict detection logic
resides in a single function that is called by both
handleInstInstalledFiles() and handleOverlappedFiles().

Any comments, other ideas, or additional insights would be greatly
appreciated.

Thanks,
Dave