[Rpm-maint] [RFC][PATCH] Large file support

[Florian Festi]

Hi!

I had a look at the 4GB per packaged file limit. The current cpio format 
[1] uses 8 bytes to encode 32 bit integers as hexadecimal ASCII strings. 
So there is no way of fixing this problem while staying compatible with 
the cpio format (and keep rpm2cpio working).

Having a look at the tar formats I do not belief that switching to tar 
is a real option. The format is just horrible (GNU tar needs over 200 
lines to read an integer out of a header field) and full of hacks to 
remain backward compatible (header in header + extentions). This would 
be all not that bad if there where a nice little library we could link 
against...

My favorite solution would be to use no payload format at all and just 
rely on the meta data we ship in the header anyway. While this would 
surely be possible it requires redoing the hard link handling (as hard 
links are treated specially in the payload - like shipping the content 
just once) and modifying the next upper layer within rpm (fsm.c) which 
is probably the most horrible place in the whole code base. Volunteers 
welcome!

A much simpler alternative would be to use a slightly modified cpio 
format. With a new magic for large files we could just put an binary 
integer into the c_filesize field (or all integer fields). Another 
solution could be to keep the hexadecimal encoding and just double the 
c_filesize or even some more integer fields.
This will both render the payload incompatible with cpio if there are 
large files (and only then).

I did not yet ask cpio upstream or our cpio package maintainer about 
accepting patches to at least read such archives...

Attached patch uses a binary integer for large file sizes. Patch is 
untested and assumes that everything else that deals with file sizes 
already is 64 bit save.

Comments? Ideas? Panic?

Florian

[1] http://people.freebsd.org/~kientzle/libarchive/man/cpio.5.txt (see 
"New ASCII Format")

-- 
________________________________________________________________________
Reg. Adresse: Red Hat GmbH, Hauptstätter Str. 58, 70178 Stuttgart
Handelsregister: Amtsgericht Muenchen HRB 153243
Geschaeftsfuehrer: Brendan Lane, Charlie Peters, Michael Cunningham,
Charles Cachera

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: 0001-Add-support-for-64-bit-file-sizes-to-cpio.patch
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20090909/fe5fba73/attachment.diff>