[Rpm-maint] [rpm-software-management/rpm] Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite … (PR #1936)

[Elia Pinto]

…for detecting heap corruption for glibc <2.34. For glibc >=2.34 use the new heap consistency checking environment variables

    Recent versions of Linux libc (later than 5.4.23) and glibc (2.x)
    include a malloc() implementation which is tunable via environment
    variables. When MALLOC_CHECK_ is set, a special (less efficient)
    implementation is used which is designed to be tolerant against
    simple errors, such as double calls of free() with the same argument,
    or overruns of a single byte (off-by-one bugs). When MALLOC_CHECK_
    is set to 3, a diagnostic message is printed on stderr
    and the program is aborted.

    Setting the MALLOC_PERTURB_ environment variable causes the malloc
    functions in libc to return memory which has been wiped and clear
    memory when it is returned.
    Of course this does not affect calloc which always does clear the memory.

    The reason for this exercise is, of course, to find code which uses
    memory returned by malloc without initializing it and code which uses
    code after it is freed. valgrind can do this but it's costly to run.
    The MALLOC_PERTURB_ exchanges the ability to detect problems in 100%
    of the cases with speed.

    The byte value used to initialize values returned by malloc is the byte
    value of the environment value. The value used to clear memory is the
    bitwise inverse. Setting MALLOC_PERTURB_ to zero disables the feature.

    This technique can find hard to detect bugs.
    It is therefore suggested to always use this flag (at least temporarily)
    when testing out code or a new distribution (http://udrepper.livejournal.com/11429.html).

    We have to be careful if you want to use valgrind with the test suite however.

    Memcheck wraps client calls to malloc(), and puts a "red zone" on
    each end of each block in order to detect access overruns.
    Memcheck already detects double free() (up to the limit of the buffer
    which remembers pending free()). Thus memcheck subsumes all the
    documented coverage of MALLOC_CHECK_.

    If MALLOC_CHECK_ is set non-zero when running memcheck, then the
    overruns that might be detected by MALLOC_CHECK_ would be overruns
    on the wrapped blocks which include the red zones.  Thus MALLOC_CHECK_
    would be checking memcheck, and not the client.  This is not useful,
    and actually is wasteful.  The only possible [documented] advantage
    of using MALLOC_CHECK_ and memcheck together, would be if MALLOC_CHECK_
    detected duplicate free() in more cases than memcheck because memcheck's
    buffer is too small.

    Therefore we shouldn't use use MALLOC_CHECK_ and valgrind at the
    same time.

    MALLOC_CHECK_ and MALLOC_PERTURB_ were the environment variables used until glibc 2.34.
    In glibc >= 2.34 MALLOC_CHECK_ and MALLOC_PERTURB_ environment
    variables have been replaced by GLIBC_TUNABLES.  Also the new
    glibc requires that you preload a library called libc_malloc_debug.so
    to get these features.

    This patch was inspired by a Richard W.M. Jones ndbkit patch
    (https://gitlab.com/nbdkit/nbdkit/-/commit/362e0fdcae37db876e13b944102a5c152e6bc563)
You can view, comment on, or merge this pull request online at:

  https://github.com/rpm-software-management/rpm/pull/1936

-- Commit Summary --

  * Add MALLOC_CHECK_ and MALLOC_PERTURB_ libc env to the test suite for detecting heap corruption for glibc <2.34. For glibc >=2.34 use the new heap consistency checking environment variables

-- File Changes --

    M configure.ac (17)
    M tests/Makefile.am (1)
    M tests/atlocal.in (22)

-- Patch Links --

https://github.com/rpm-software-management/rpm/pull/1936.patch
https://github.com/rpm-software-management/rpm/pull/1936.diff

-- 
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/pull/1936
You are receiving this because you are subscribed to this thread.

Message ID: <rpm-software-management/rpm/pull/1936 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rpm.org/pipermail/rpm-maint/attachments/20220224/df6894fb/attachment-0001.html>