[Rpm-maint] join wiki edit group; intro; cross development

[Mark Hatle]

seth vidal wrote:
> On Thu, 2006-12-28 at 10:34 -0600, Mark Hatle wrote:
>> I just wanted to add a clarification to my Berkley DB statement below (I
>> have already changed the wiki page to match.)
>>
>> * A Berkley DB database HATES being created on one arch and used on
>> another. The contents of the database are intact and read-able
>> cross-endian. The problem is specific to locking. Different
>> architectures do locking in different ways which may cause problems.
>> (Clarified from original email.) It may be possible to build Berkley DB
>> on different architectures with the same locking formats, however there
>> may be endian specific items in the locking. (Also different
>> architectures may not support the same type of locking. Read the BDB
>> documentation when configuring the locking mechanisms.)
>>
>>     * Note: sqlite was added to attempt to work around this issue. At a
>> tradeoff of slower performance, and no concurrent transactions. Sqlite
>> automatically addresses the locking issues in my experience.
> 
> If the database were restructured a bit would the sqlite implementation
> gain much in speed? Especially if the data were broken out a bit more
> and some indexes implemented?

I doubt there would be any speed gain.  It's hard to beat the key/value
pair (binary data) storage format of Berkley DB for speed.

For an SQL back end, you'd need to store the data in as individual
elements instead of a packed binary structure.  The advantage of this
would be the ability to query the database outside of RPM/rpmlib.
However, the rewrite to librpm would be very extreme at best, and simply
not worth it, IMHO.

The current SQLlite implementation just stores the key/value pairs (the
same way that BDB does).  The speed issues are completely in the
different database designs and not the data processing.

The other nice thing with the current design, if you need the
flexibility (and don't need concurrent access) you can create the
database as an SQLite database.  Then when you switch over to the target
system you can run an rpm --rebuilddb and switch it to BDB, gaining both
speed and concurrent access back.  The key in my experience is to create
and write to the BDB database on the system type it will be used on.
(This is completely a locking issue, and not even really a "bug" in BDB.
 It's a configuration issue because each system has differently
optimized locking mechanisms, not of which are compatible with each
other!  It "should" be possible to use something like futex on all
platforms and BDB will just work... of course then all platforms need to
support futex.)

--Mark