[Rpm-maint] [rpm-software-management/rpm] Run binary package generation via thread pools (#226)

[Igor Gnatenko]

It would be nice to run dependency generators in parallel as well (although it might disallow us in future to do some advanced generators which would take subpackages into account, but AFAIK no one is planning to work on something like this in foreseeable future so doesn't seem to be a problem). This is what taking quite a lot of time for some of subpackages, so if they would run in parallel, this would speedup build.

# Results

## Before

```
real    56m45.083s                                  
user    39m34.072s                                  
sys     29m33.089s                                  
```

## After

```
real    48m16.123s
user    51m54.226s
sys     31m9.800s
```

## Hardware

```
$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              144
On-line CPU(s) list: 0-143
Thread(s) per core:  2
Core(s) per socket:  18
Socket(s):           4
NUMA node(s):        4
Vendor ID:           GenuineIntel
CPU family:          6
Model:               79
Model name:          Intel(R) Xeon(R) CPU E7-8860 v4 @ 2.20GHz
Stepping:            1
CPU MHz:             1200.305
CPU max MHz:         3200.0000
CPU min MHz:         1200.0000
BogoMIPS:            4389.14
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            46080K
NUMA node0 CPU(s):   0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,64,68,72,76,80,84,88,92,96,100,104,108,112,116,120,124,128,132,136,140
NUMA node1 CPU(s):   1,5,9,13,17,21,25,29,33,37,41,45,49,53,57,61,65,69,73,77,81,85,89,93,97,101,105,109,113,117,121,125,129,133,137,141
NUMA node2 CPU(s):   2,6,10,14,18,22,26,30,34,38,42,46,50,54,58,62,66,70,74,78,82,86,90,94,98,102,106,110,114,118,122,126,130,134,138,142
NUMA node3 CPU(s):   3,7,11,15,19,23,27,31,35,39,43,47,51,55,59,63,67,71,75,79,83,87,91,95,99,103,107,111,115,119,123,127,131,135,139,143
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb cat_l3 cdp_l3 intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
```

## Comments

It didn't really seem to generate binary packages in parallel, but probably it worked, not sure ;)

```
Wrote: /home/brain/rpmbuild/RPMS/noarch/texlive-xespotcolor-svn40118-36.fc26.5.noarch.rpm                                                                                                                          
Wrote: /home/brain/rpmbuild/RPMS/noarch/texlive-xesearch-svn16041.0-36.fc26.5.noarch.rpm                                                          
Wrote: /home/brain/rpmbuild/RPMS/noarch/texlive-uothesis-doc-svn25355.2.5.6-36.fc26.5.noarch.rpm                                                                                                                   
Wrote: /home/brain/rpmbuild/RPMS/noarch/texlive-kdgdocs-doc-svn24498.1.0-36.fc26.5.noarch.rpm                                                     
Wrote: /home/brain/rpmbuild/RPMS/noarch/texlive-polyglossia-svn40138-36.fc26.5.noarch.rpm                                                                                                                          
Finished binary package job, result 0, filename /home/brain/rpmbuild/RPMS/noarch/texlive-xespotcolor-svn40118-36.fc26.5.noarch.rpm              
Finished binary package job, result 0, filename /home/brain/rpmbuild/RPMS/noarch/texlive-xesearch-svn16041.0-36.fc26.5.noarch.rpm                                                                                  
Finished binary package job, result 0, filename /home/brain/rpmbuild/RPMS/noarch/texlive-uothesis-doc-svn25355.2.5.6-36.fc26.5.noarch.rpm   
Wrote: /home/brain/rpmbuild/RPMS/noarch/texlive-unicode-math-doc-svn38462-36.fc26.5.noarch.rpm                                                                                                                     
Finished binary package job, result 0, filename /home/brain/rpmbuild/RPMS/noarch/texlive-unicode-math-doc-svn38462-36.fc26.5.noarch.rpm    
Finished binary package job, result 0, filename /home/brain/rpmbuild/RPMS/noarch/texlive-kdgdocs-doc-svn24498.1.0-36.fc26.5.noarch.rpm                                                                             
Finished binary package job, result 0, filename /home/brain/rpmbuild/RPMS/noarch/texlive-polyglossia-svn40138-36.fc26.5.noarch.rpm              
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/226#issuecomment-325885808
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20170829/dc342764/attachment-0001.html>