[Rpm-maint] [rpm-software-management/rpm] Run binary package generation via thread pools (#226)

Thu Sep 6 08:27:46 UTC 2018

@ignatenkobrain testing your patches in the Yocto project (with some extra debugging statements) on openSUSE's open build service instance its clear that the packages are still being built sequentially I'm not sure why yet, but here's some debug output below.

Some of us at SUSE are quite keen to get this working and I have time allocated to it primarily because it should speed up our kernel builds significantly but also because it will reduce the load on our build service significantly.

`START:packageBinaries` is printed at the start of the packageBinaries function, `START:writerpm` and `END:writerpm` are called either side of the `writeRPM` function the number being printed is just the address of the `pkg` pointer so I could keep track if the write's were actually being done in parallel `[ 457s]` is the timestamp of the current build in seconds

```
[  457s] ~~~~ START:packageSources ~~~~
[  457s] ~~~## START:writesourcerpm:37583280 ##~~~
[  457s] Wrote: /home/abuild/rpmbuild/SRPMS/efl-1.20.7-260.7.src.rpm
[  458s] ~~~## END:writesourcerpm:37583280 ##~~~
[  458s] ~~~~ END:packageSources ~~~~
[  458s] ~~~~ START:packageBinaries ~~~~
[  458s] ~~~## START:writerpm:37582688 ##~~~
[  458s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/efl-1.20.7-260.7.x86_64.rpm
[  470s] ~~~## END:writerpm:37582688 ##~~~
[  470s] ~~~## START:writerpm:37627248 ##~~~
[  470s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/efl-debugsource-1.20.7-260.7.x86_64.rpm
[  470s] ~~~## END:writerpm:37627248 ##~~~
[  470s] ~~~## START:writerpm:37630240 ##~~~
[  470s] Wrote: /home/abuild/rpmbuild/RPMS/noarch/efl-lang-1.20.7-260.7.noarch.rpm
[  475s] ~~~## END:writerpm:37630240 ##~~~
[  475s] ~~~## START:writerpm:37631968 ##~~~
[  475s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/efl-devel-1.20.7-260.7.x86_64.rpm
[  475s] ~~~## END:writerpm:37631968 ##~~~
[  475s] ~~~## START:writerpm:37642944 ##~~~
[  475s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libecore1-1.20.7-260.7.x86_64.rpm
[  475s] ~~~## END:writerpm:37642944 ##~~~
[  475s] ~~~## START:writerpm:37608864 ##~~~
[  475s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libector1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37608864 ##~~~
[  476s] ~~~## START:writerpm:37610224 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libedje1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37610224 ##~~~
[  476s] ~~~## START:writerpm:37612560 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeldbus1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37612560 ##~~~
[  476s] ~~~## START:writerpm:37614288 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeet1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37614288 ##~~~
[  476s] ~~~## START:writerpm:37665200 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeeze1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37665200 ##~~~
[  476s] ~~~## START:writerpm:37666688 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libefl1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37666688 ##~~~
[  476s] ~~~## START:writerpm:37669168 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libefreet1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37669168 ##~~~
[  476s] ~~~## START:writerpm:37671472 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libefreet_mime1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37671472 ##~~~
[  476s] ~~~## START:writerpm:37774416 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libefreet_trash1-1.20.7-260.7.x86_64.rpm
[  476s] ~~~## END:writerpm:37774416 ##~~~
[  476s] ~~~## START:writerpm:37774000 ##~~~
[  476s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeina1-1.20.7-260.7.x86_64.rpm
[  477s] ~~~## END:writerpm:37774000 ##~~~
[  477s] ~~~## START:writerpm:37775984 ##~~~
[  477s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeio1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37775984 ##~~~
[  478s] ~~~## START:writerpm:37777648 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libelementary1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37777648 ##~~~
[  478s] ~~~## START:writerpm:37779280 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libelocation1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37779280 ##~~~
[  478s] ~~~## START:writerpm:37781248 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libelput1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37781248 ##~~~
[  478s] ~~~## START:writerpm:37782784 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libelua1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37782784 ##~~~
[  478s] ~~~## START:writerpm:37876144 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libembryo1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37876144 ##~~~
[  478s] ~~~## START:writerpm:37877696 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libemotion1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37877696 ##~~~
[  478s] ~~~## START:writerpm:37879984 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libemile1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37879984 ##~~~
[  478s] ~~~## START:writerpm:37881376 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeo1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37881376 ##~~~
[  478s] ~~~## START:writerpm:37833856 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeolian1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37833856 ##~~~
[  478s] ~~~## START:writerpm:37835344 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libethumb1-1.20.7-260.7.x86_64.rpm
[  478s] ~~~## END:writerpm:37835344 ##~~~
[  478s] ~~~## START:writerpm:37837008 ##~~~
[  478s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libephysics1-1.20.7-260.7.x86_64.rpm
[  479s] ~~~## END:writerpm:37837008 ##~~~
[  479s] ~~~## START:writerpm:37838960 ##~~~
[  479s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libevas1-1.20.7-260.7.x86_64.rpm
[  479s] ~~~## END:writerpm:37838960 ##~~~
[  479s] ~~~## START:writerpm:37840496 ##~~~
[  479s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libethumb_client1-1.20.7-260.7.x86_64.rpm
[  480s] ~~~## END:writerpm:37840496 ##~~~
[  480s] ~~~## START:writerpm:37801952 ##~~~
[  480s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/edje-1.20.7-260.7.x86_64.rpm
[  484s] ~~~## END:writerpm:37801952 ##~~~
[  484s] ~~~## START:writerpm:37804000 ##~~~
[  484s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/elementary-1.20.7-260.7.x86_64.rpm
[  484s] ~~~## END:writerpm:37804000 ##~~~
[  484s] ~~~## START:writerpm:37807440 ##~~~
[  484s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/elementary-examples-1.20.7-260.7.x86_64.rpm
[  484s] ~~~## END:writerpm:37807440 ##~~~
[  484s] ~~~## START:writerpm:37924816 ##~~~
[  484s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/elua-1.20.7-260.7.x86_64.rpm
[  484s] ~~~## END:writerpm:37924816 ##~~~
[  484s] ~~~## START:writerpm:37926784 ##~~~
[  484s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/embryo-1.20.7-260.7.x86_64.rpm
[  484s] ~~~## END:writerpm:37926784 ##~~~
[  484s] ~~~## START:writerpm:37928752 ##~~~
[  484s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/evas-generic-loaders-1.20.7-260.7.x86_64.rpm
[  487s] ~~~## END:writerpm:37928752 ##~~~
[  487s] ~~~## START:writerpm:37930992 ##~~~
[  487s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/efl-examples-1.20.7-260.7.x86_64.rpm
[  487s] ~~~## END:writerpm:37930992 ##~~~
[  487s] ~~~## START:writerpm:37932608 ##~~~
[  487s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/efl-testsuite-1.20.7-260.7.x86_64.rpm
[  490s] ~~~## END:writerpm:37932608 ##~~~
[  490s] ~~~## START:writerpm:37809472 ##~~~
[  490s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/enlightenment-theme-upstream-0.21.0-260.7.x86_64.rpm
[  493s] ~~~## END:writerpm:37809472 ##~~~
[  493s] ~~~## START:writerpm:37811968 ##~~~
[  493s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/enlightenment-theme-dark-0.21.0-260.7.x86_64.rpm
[  493s] ~~~## END:writerpm:37811968 ##~~~
[  493s] ~~~## START:writerpm:37814384 ##~~~
[  493s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/enlightenment-x-dark-icon-theme-0.21.0-260.7.x86_64.rpm
[  499s] ~~~## END:writerpm:37814384 ##~~~
[  499s] ~~~## START:writerpm:37625936 ##~~~
[  499s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/efl-debuginfo-1.20.7-260.7.x86_64.rpm
[  501s] ~~~## END:writerpm:37625936 ##~~~
[  501s] ~~~## START:writerpm:43870128 ##~~~
[  501s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/efl-devel-debuginfo-1.20.7-260.7.x86_64.rpm
[  503s] ~~~## END:writerpm:43870128 ##~~~
[  503s] ~~~## START:writerpm:41441408 ##~~~
[  503s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libecore1-debuginfo-1.20.7-260.7.x86_64.rpm
[  503s] ~~~## END:writerpm:41441408 ##~~~
[  503s] ~~~## START:writerpm:42232032 ##~~~
[  503s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libector1-debuginfo-1.20.7-260.7.x86_64.rpm
[  505s] ~~~## END:writerpm:42232032 ##~~~
[  505s] ~~~## START:writerpm:44036368 ##~~~
[  505s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libedje1-debuginfo-1.20.7-260.7.x86_64.rpm
[  505s] ~~~## END:writerpm:44036368 ##~~~
[  505s] ~~~## START:writerpm:42311248 ##~~~
[  505s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeldbus1-debuginfo-1.20.7-260.7.x86_64.rpm
[  505s] ~~~## END:writerpm:42311248 ##~~~
[  505s] ~~~## START:writerpm:42311712 ##~~~
[  505s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeet1-debuginfo-1.20.7-260.7.x86_64.rpm
[  505s] ~~~## END:writerpm:42311712 ##~~~
[  505s] ~~~## START:writerpm:44183296 ##~~~
[  505s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeeze1-debuginfo-1.20.7-260.7.x86_64.rpm
[  506s] ~~~## END:writerpm:44183296 ##~~~
[  506s] ~~~## START:writerpm:41396416 ##~~~
[  506s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libefl1-debuginfo-1.20.7-260.7.x86_64.rpm
[  506s] ~~~## END:writerpm:41396416 ##~~~
[  506s] ~~~## START:writerpm:40907472 ##~~~
[  506s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libefreet1-debuginfo-1.20.7-260.7.x86_64.rpm
[  506s] ~~~## END:writerpm:40907472 ##~~~
[  506s] ~~~## START:writerpm:41391808 ##~~~
[  506s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libefreet_mime1-debuginfo-1.20.7-260.7.x86_64.rpm
[  506s] ~~~## END:writerpm:41391808 ##~~~
[  506s] ~~~## START:writerpm:43229904 ##~~~
[  506s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libefreet_trash1-debuginfo-1.20.7-260.7.x86_64.rpm
[  507s] ~~~## END:writerpm:43229904 ##~~~
[  507s] ~~~## START:writerpm:42711120 ##~~~
[  507s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeina1-debuginfo-1.20.7-260.7.x86_64.rpm
[  507s] ~~~## END:writerpm:42711120 ##~~~
[  507s] ~~~## START:writerpm:41072128 ##~~~
[  507s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeio1-debuginfo-1.20.7-260.7.x86_64.rpm
[  512s] ~~~## END:writerpm:41072128 ##~~~
[  512s] ~~~## START:writerpm:37989728 ##~~~
[  512s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libelementary1-debuginfo-1.20.7-260.7.x86_64.rpm
[  512s] ~~~## END:writerpm:37989728 ##~~~
[  512s] ~~~## START:writerpm:41288480 ##~~~
[  512s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libelocation1-debuginfo-1.20.7-260.7.x86_64.rpm
[  512s] ~~~## END:writerpm:41288480 ##~~~
[  512s] ~~~## START:writerpm:43342752 ##~~~
[  512s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libelput1-debuginfo-1.20.7-260.7.x86_64.rpm
[  512s] ~~~## END:writerpm:43342752 ##~~~
[  512s] ~~~## START:writerpm:43124208 ##~~~
[  512s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libelua1-debuginfo-1.20.7-260.7.x86_64.rpm
[  512s] ~~~## END:writerpm:43124208 ##~~~
[  512s] ~~~## START:writerpm:37924192 ##~~~
[  512s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libembryo1-debuginfo-1.20.7-260.7.x86_64.rpm
[  512s] ~~~## END:writerpm:37924192 ##~~~
[  512s] ~~~## START:writerpm:44740400 ##~~~
[  512s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libemotion1-debuginfo-1.20.7-260.7.x86_64.rpm
[  513s] ~~~## END:writerpm:44740400 ##~~~
[  513s] ~~~## START:writerpm:41465376 ##~~~
[  513s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libemile1-debuginfo-1.20.7-260.7.x86_64.rpm
[  513s] ~~~## END:writerpm:41465376 ##~~~
[  513s] ~~~## START:writerpm:38012256 ##~~~
[  513s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeo1-debuginfo-1.20.7-260.7.x86_64.rpm
[  513s] ~~~## END:writerpm:38012256 ##~~~
[  513s] ~~~## START:writerpm:40414736 ##~~~
[  513s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libeolian1-debuginfo-1.20.7-260.7.x86_64.rpm
[  513s] ~~~## END:writerpm:40414736 ##~~~
[  513s] ~~~## START:writerpm:42748160 ##~~~
[  513s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libethumb1-debuginfo-1.20.7-260.7.x86_64.rpm
[  514s] ~~~## END:writerpm:42748160 ##~~~
[  514s] ~~~## START:writerpm:43754176 ##~~~
[  514s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libephysics1-debuginfo-1.20.7-260.7.x86_64.rpm
[  518s] ~~~## END:writerpm:43754176 ##~~~
[  518s] ~~~## START:writerpm:43852032 ##~~~
[  518s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libevas1-debuginfo-1.20.7-260.7.x86_64.rpm
[  518s] ~~~## END:writerpm:43852032 ##~~~
[  518s] ~~~## START:writerpm:47427392 ##~~~
[  518s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/libethumb_client1-debuginfo-1.20.7-260.7.x86_64.rpm
[  519s] ~~~## END:writerpm:47427392 ##~~~
[  519s] ~~~## START:writerpm:43164560 ##~~~
[  519s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/edje-debuginfo-1.20.7-260.7.x86_64.rpm
[  519s] ~~~## END:writerpm:43164560 ##~~~
[  519s] ~~~## START:writerpm:42696096 ##~~~
[  519s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/elementary-debuginfo-1.20.7-260.7.x86_64.rpm
[  519s] ~~~## END:writerpm:42696096 ##~~~
[  519s] ~~~## START:writerpm:47993152 ##~~~
[  519s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/embryo-debuginfo-1.20.7-260.7.x86_64.rpm
[  519s] ~~~## END:writerpm:47993152 ##~~~
[  519s] ~~~## START:writerpm:46933104 ##~~~
[  519s] Wrote: /home/abuild/rpmbuild/RPMS/x86_64/evas-generic-loaders-debuginfo-1.20.7-260.7.x86_64.rpm
[  519s] ~~~## END:writerpm:46933104 ##~~~
[  519s] ~~~~ END:packageBinaries ~~~~
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/226#issuecomment-419009761
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20180906/6b9fe396/attachment-0001.html>