[Rpm-maint] [rpm-software-management/rpm] Create binary rpm packages in parallel? (better multiple CPU core utilization) (#211)

[Jeff Johnson]

Again ... OPENMP is the least intrusive (but perhaps the most abstract) implementation. OPENMP has queues and pools and is viable even in 2017.

I parallelized the rpm --verify loop over files a few years back using OPENMP.

Here are the top level annotations that were needed:

`#if defined(_OPENMP)
    #pragma omp parallel for private(i) reduction(+:ec)
#endif
    for (i = 0; i < (int)fc; i++) {
        int fflags = fi->fflags[i];
        rpmvf vf;
        int rc;

        /* If not querying %config, skip config files. */
        if (FF_ISSET(qva->qva_fflags, CONFIG) && FF_ISSET(fflags, CONFIG))
            continue;

        /* If not querying %doc, skip doc files. */
        if (FF_ISSET(qva->qva_fflags, DOC) && FF_ISSET(fflags, DOC))
            continue;

        /* If not verifying %ghost, skip ghost files. */
        if (!FF_ISSET(qva->qva_fflags, GHOST) && FF_ISSET(fflags, GHOST))
            continue;

        /* Gather per-file data into a carrier. */
        vf = rpmvfNew(ts, fi, i, omitMask);

        /* Verify per-file metadata. */
        rc = rpmvfVerify(vf, spew);
        if (rc)
            ec += rc;

        (void) rpmvfFree(vf);
        vf = NULL;
    }
`

OTOH, I had to rewrite lib/verify.c to add a per-file rpmvf ("verify file") structure to avoid thread races within rpmfi iterators (which are rather deficient). You may encounter similar problems traversing the packages linked list.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/211#issuecomment-303720772
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20170524/b8c41cc0/attachment.html>