[Rpm-maint] [rpm-software-management/rpm] Add macro to force fsync() on close() (#187)

[Jeff Johnson]

FYI: several notes (after looking at the patch)

1) The measurements I cited above are using mmap(2) to decompress directly into pages (avoiding a buffer copy) marked with MADV_DONTNEED (which permits delayed/larger writes) to avoid a buffer copy. In addition, there is a check on file size of <= 128Mb to avoid consuming too many pages on extremely large files.

However, I have no reason to believe that mmap(2) instead of a loop over buffered writes will change the basic observation(s) that package management is a huge I/O load, and that fsync(2) will schedule large amounts of I/O that can starve/stall other running processes..

I suggest testing on a moderately large package with 3000+ files (like kernel  source) to see the affect on package management. Slowing down package management (by perhaps a factor of 50x) will most certainly use fewer resources, and not compete with other running processes, as a side effect of calling fsync(2), but is perhaps not the best implementation. YMMV.

2) Instead of adding Fsync to walk the FD_t stack, fsync'ing every fdno, there is already Fileno() to return the topmost fdno of a FD_t, which is almost certainly all that is needed.

Using fsync(Fileno(fd)) (or equivalent: Fileno() has negative error returns) either at the end of the loop in lib/rpmfi.c rpmfiArchiveReadToFilePsm(), or (equivalently) in the parent caller, achieves the same fsync without pretending that Fsync() "works" with all possible fdno's.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/187#issuecomment-291768318
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20170404/8d47b1b5/attachment.html>