[Rpm-maint] speeding up internal deps generator

[Panu Matilainen]

On 09/15/2014 07:38 PM, Michael Schroeder wrote:
> On Mon, Sep 15, 2014 at 05:34:48PM +0200, Thierry Vignaud wrote:
>> We've recently switched to internal deps generator.
>>
>> However one of the deps scripts we use [1] is very slow b/c of this[2]:
>>
>>   +for path in \
>> +    $(for tlpath in \
>> +    $(find ${RPM_BUILD_ROOT}/usr/lib64 ${RPM_BUILD_ROOT}/usr/lib
>> /usr/lib64 /usr/lib -name '*.typelib'); do
>> +            dirname $tlpath; done | uniq ); do
>> +    export GI_TYPELIB_PATH=$GI_TYPELIB_PATH:$path
>> +done
>>
>> Previously, it was run once (with so called "external" deps generator).
>> With internal deps generator, rpm forks it for every matching file.
>> When lots of files match, this is very slow as it runs tons of find
>> for every matching file.
>>
>> This could be workarounded if internal deps generator could offer a
>> new flag (instead of just "exeonly" & ""magic_and_path" ) that, if
>> found in the .attr file, would make rpm to call the script just once
>> with the whole list of files instead of calling it once per file.
>> It could be named "all_in_one", "fork_once_only", or whatever.
>>
>> This would be helpfull for most packages builds as most of standard
>> prov/req scripts already handle more than on file name and thus could
>> use this new flag.
>>
>> WDYT?
>
> In that case the script needs to "annotate" the output in some way.
> It has to transport the information, which files generated the dependency.
> Otherwise things like 'rpm -q --filerequire' would no longer work.

Yup.

While the per-file dependency tracking is kinda neat and makes sense in 
several cases, there are also numerous cases where it does NOT make 
sense: for example perl/python modules usually consist of several files 
which together form the module, and thinking of individual files as 
providing something is not meaningful at all.

https://bugzilla.redhat.com/show_bug.cgi?id=678605 is an example of 
similar problems on requires-side.

Clearly the current 1:1 file<->dependency relation model does not 
entirely match up with reality, so there's room for improvement. Perhaps 
we could indeed add a per fileattr (or even per fileattr-dependency 
level) flag which makes dependencies generated on package, not file level.

	- Panu -