[Rpm-maint] [rpm-software-management/rpm] RFC: What approach to improving performance through threads or non-blocking I/O is acceptable in RPM? (#517)

[Jeff Johnson]

Recent issues -- the pending PR for speeding up rpm kernel builds and the addition of fsync come to mind -- lead me to ask up front:

What approaches to speeding up rpm installs are deemed acceptable?

The techniques (event loop and/or threads and/or coroutines etc etc) are very well known.

But so far the RFE's are being dealt with on a case-by-case, time permitting, show me testable working code, basis. Exactly as it should be with rpm code, where reliability and compatibility and maintai ability are at least as important as performance.

One of the most difficult decisions is architectural: what problems need to be solved and what techniques should be used?

This RFC attempts to ascertain what types of approaches are considered acceptable, and what bottleneck problems are deemed important.

Possible answers, based on examining code and reading issue resolutions, include

* rpm is fine exactly as it is, highly tuned for a specific task which it does sufficiently well.

* rpm should attempt to be thread-safe and leave all the performance issues to higher level tools

* rpm patches need to be done through PRs with test cases in order to decide what to do.

Possible implementations for threading were discussed ad hoc in the PR for speeding up rpmbuild kernel builds, included both OPENMP markup as well as POSIX threads and a thread pool, each of which has both pro/con arguments.

Avoiding blocking system calls using an event loop does not depend on threads whatsoever, and nodejs is positive proof that performance does not necessarily depend on using multiple cpu's through a thread scheduler.

So this RFC attempts to ask two question before generating patches:

1) What bottlenecks exist in rpm code need solving?

2) What implementation approaches are deemed acceptable to solving existing bottlenecks?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/517
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20180816/b06308e0/attachment.html>