[Rpm-maint] [rpm-software-management/rpm] API improvement to accommodate for RPM CoW (PR#1470) (Discussion #2057)

[David Disseldorp]

> Thing is, since the payload is typically compressed, offsets are useless because they'd just point to somewhere in the middle of a compression stream, you can't jump to it without reading the whole thing anyhow.

zstd frame headers can carry both compressed and uncompressed sizes, which makes it a little easier to seek around within the compressed payload.

> If the files in the payload were individually compressed, it'd seem quite reasonable to have offsets stored. Of course that would likely loose something in the compression ratio.

That's mostly what I did in my rpm-rs based poc - files aren't individually compressed, but the start and end of individual file data segments correspond to zstd frame boundaries, so individual files can be extracted or written via `BTRFS_IOC_ENCODED_WRITE`/reflinked without much effort.


> > The rpm payload format isn't modified, although there's a slight "bending" of the cpio/newc spec to use the filename field for padding. zstandard frames making up the compressed rpm payload explicitly carry both compressed and uncompressed lengths, to allow detection of filesystem-supported I/O sizes.
> 
> Note that the cpio/newc format is on its way out, you don't want to design too much around that. In rpm v4 it's used for packages where all contained files are below 4GB, but v6 will always use the newer rpm-specific format which doesn't carry any file metadata in the payload: https://rpm-software-management.github.io/rpm/manual/format_v6.html#payload

Returning to this, I suppose I could still pad `07070X` payloads relatively easily by injecting individual "padding" files into the archive. This would still be compatible with V4, although it might be nice to reserve a special *padding* path in the V6 spec which can be skipped over (where V4 would still extract / install it). Is the V6 draft still open for these kinds of proposals?
cc: @lnussel

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/discussions/2057#discussioncomment-9281616
You are receiving this because you are subscribed to this thread.

Message ID: <rpm-software-management/rpm/repo-discussions/2057/comments/9281616 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20240430/c5559153/attachment.html>