[yocto] liblzma: memory allocation failed

[Andrea Adami]

On Mon, Sep 17, 2018 at 10:28 AM Burton, Ross <ross.burton at intel.com> wrote:
>
> On Mon, 17 Sep 2018 at 08:13, Peter Bergin <peter at berginkonsult.se> wrote:
> > I'm pretty sure I have narrowed down the root cause to the restriction
> > of virtual memory and that liblzma base its memory calculations on
> > physical RAM.


Hello,

well, not only.
You can set the memory footprint for compression/decompression.

In OE for legacy kernels we use in our BSP:
# sane defaults for devices with only 32Mb RAM (see man xz)
XZ_COMPRESSION_LEVEL = "-2e"

Default is -3, the -2 uses right half the RAM for compressing,.
Pls check man xz.

Cheers
Andrea


> >
> > To prove this I added a printout in rpm-native/rpmio/rpmio.c and the
> > function lzopen_internal.
> >
> >          uint64_t memory_usage = lzma_stream_encoder_mt_memusage(&mt_options);
> >         rpmlog(RPMLOG_NOTICE, "DBG: memory_usage %lu\n", memory_usage);
> >
> >
> > The value of memory_usage is the same regardless of which 'ulimit -v'
> > value I set. On the host with 256GB of physical RAM and 32GB of virtual
> > memory, memory_usage is ~5.1GB. On another host with 16GB of physical
> > RAM I get memory_usage of ~660MB.
> >
> > I guess you have not seen this kind of failure if you not have
> > restricted virutal memory on your host. If you want to try to reproduce
> > this set 'ulimit -v 8388608' (8GB) in your shell and then 'bitbake
> > glibc-locale -c package_write_rpm -f'.
>
> Wouldn't a solution be to change lzma to look at free memory, not
> total physical memory?
>
> Ross
> --
> _______________________________________________
> yocto mailing list
> yocto at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto