Can't start Paraview any more: malloc(): invalid size (unsorted)

trilbytim · December 29, 2023, 1:03pm

I’m on Ubuntu 22.04. I have Paraview 5.6 installed via OpenFOAM, and that still works, however I’ve mostly been using version 5.12.0-RC1-MPI-Linux-Python3.10-x86_64 that I just run from the pre-compiled binary. This has been working fine until suddenly it didn’t! I now get an error like this:

(base) timbo@timbo-laptop:~/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin$ ./paraview
malloc(): invalid size (unsorted)
error: exception occurred: Subprocess aborted

I had been tinkering with settings recently (although I think I have opened it successfully since that), so I deleted the home/.config/paraview folder, but it has made no difference. I have also deleted and re-downloaded the binary, but it still doesn’t work. I have also tried an earlier binary (5.9) and this also doesn’t work any more. Luckily I can still use my version 5.6 that I had from OpenFOAM, but I would like to get 5.12 working again.

ben.boeckel · December 29, 2023, 3:05pm

Is it possible to get a backtrace from there? Perhaps these settings will help:

$ export MALLOC_CHECK_=3
$ ./paraview
$ echo "or…"
$ gdb ./paraview
(gdb) set follow-fork-mode child
(gdb) run

trilbytim · December 30, 2023, 1:18pm

I get this output from that:

(base) timbo@timbo-laptop:~/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin$ export MALLOC_CHECK_=3
(base) timbo@timbo-laptop:~/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin$ ./paraview
malloc(): invalid size (unsorted)
error: exception occurred: Subprocess aborted
(base) timbo@timbo-laptop:~/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin$ echo "or_"
or_
(base) timbo@timbo-laptop:~/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin$ gdb ./paraview
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./paraview...
(No debugging symbols found in ./paraview)
(gdb) set follow-fork-mode child
(gdb) run
Starting program: /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/paraview 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Attaching after Thread 0x7ffff7e46000 (LWP 12613) fork to child process 12617]
[New inferior 2 (process 12617)]
[Detaching after fork from parent process 12613]
[Inferior 1 (process 12613) detached]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
process 12617 is executing new program: /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/paraview-real
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffd28ba640 (LWP 12627)]
[New Thread 0x7fffd1bf7640 (LWP 12628)]
malloc(): invalid size (unsorted)

Thread 2.1 "paraview-real" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffd6cd7cc0 (LWP 12617)]
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140736797179072) at ./nptl/pthread_kill.c:44
44	./nptl/pthread_kill.c: No such file or directory.
(gdb)

ben.boeckel · December 30, 2023, 7:22pm

Ah, I forgot to ask for backtrace from gdb once it gets to that point.

trilbytim · December 31, 2023, 12:56pm

Ah, OK, here it is:

(gdb) backtrace
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140736797179072)
    at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140736797179072)
    at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140736797179072, signo=signo@entry=6)
    at ./nptl/pthread_kill.c:89
#3  0x00007ffff7c42476 in __GI_raise (sig=sig@entry=6)
    at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7c287f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff7c89676 in __libc_message (action=action@entry=do_abort, 
    fmt=fmt@entry=0x7ffff7ddbb77 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#6  0x00007ffff7ca0cfc in malloc_printerr (
    str=str@entry=0x7ffff7ddebc0 "malloc(): invalid size (unsorted)")
    at ./malloc/malloc.c:5664
#7  0x00007ffff7ca40dc in _int_malloc (
    av=av@entry=0x7ffff7e19c80 <main_arena>, bytes=bytes@entry=41)
    at ./malloc/malloc.c:4002
#8  0x00007ffff7ca5139 in __GI___libc_malloc (bytes=41)
    at ./malloc/malloc.c:3329
#9  0x00007fffef2ae98c in operator new(unsigned long) ()
   from /lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007fffef2f4979 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /lib/x86_64-linux-gnu/libstdc++.so.6
--Type <RET> for more, q to quit, c to continue without paging--c
#11 0x00007ffff08e6a46 in vtkInformationKey::vtkInformationKey(char const*, char const*) () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libvtkCommonCore-pv5.12.so.1
#12 0x00007ffff08e3f8f in vtkInformationIntegerKey::vtkInformationIntegerKey(char const*, char const*) () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libvtkCommonCore-pv5.12.so.1
#13 0x00007fffc94da006 in _GLOBAL__sub_I_vtkAbstractArray.cxx () from /opt/paraviewopenfoam56/lib/libvtkCommonCore-pv5.6.so.1
#14 0x00007ffff7fc947e in call_init (l=<optimised out>, argc=argc@entry=1, argv=argv@entry=0x7fffffffcfb8, env=env@entry=0x4cc2c0) at ./elf/dl-init.c:70
#15 0x00007ffff7fc9568 in call_init (env=0x4cc2c0, argv=0x7fffffffcfb8, argc=1, l=<optimised out>) at ./elf/dl-init.c:33
#16 _dl_init (main_map=0x8b58a0, argc=1, argv=0x7fffffffcfb8, env=0x4cc2c0) at ./elf/dl-init.c:117
#17 0x00007ffff7d74905 in __GI__dl_catch_exception (exception=<optimised out>, operate=<optimised out>, args=<optimised out>) at ./elf/dl-error-skeleton.c:182
#18 0x00007ffff7fd0ff6 in dl_open_worker (a=0x7fffffffc3e0) at ./elf/dl-open.c:808
#19 dl_open_worker (a=a@entry=0x7fffffffc3e0) at ./elf/dl-open.c:771
#20 0x00007ffff7d748a8 in __GI__dl_catch_exception (exception=<optimised out>, operate=<optimised out>, args=<optimised out>) at ./elf/dl-error-skeleton.c:208
#21 0x00007ffff7fd134e in _dl_open (file=<optimised out>, mode=-2147483391, caller_dlopen=0x7ffff08b87d5 <vtkDynamicLoader::OpenLibrary(char const*, int)+117>, nsid=-2, argc=1, argv=<optimised out>, env=0x4cc2c0) at ./elf/dl-open.c:883
#22 0x00007ffff7c9063c in dlopen_doit (a=a@entry=0x7fffffffc650) at ./dlfcn/dlopen.c:56
#23 0x00007ffff7d748a8 in __GI__dl_catch_exception (exception=exception@entry=0x7fffffffc5b0, operate=<optimised out>, args=<optimised out>) at ./elf/dl-error-skeleton.c:208
#24 0x00007ffff7d74973 in __GI__dl_catch_error (objname=0x7fffffffc608, errstring=0x7fffffffc610, mallocedp=0x7fffffffc607, operate=<optimised out>, args=<optimised out>) at ./elf/dl-error-skeleton.c:227
#25 0x00007ffff7c9012e in _dlerror_run (operate=operate@entry=0x7ffff7c905e0 <dlopen_doit>, args=args@entry=0x7fffffffc650) at ./dlfcn/dlerror.c:138
#26 0x00007ffff7c906c8 in dlopen_implementation (dl_caller=<optimised out>, mode=<optimised out>, file=<optimised out>) at ./dlfcn/dlopen.c:71
#27 ___dlopen (file=<optimised out>, mode=<optimised out>) at ./dlfcn/dlopen.c:81
#28 0x00007ffff08b87d5 in vtkDynamicLoader::OpenLibrary(char const*, int) () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libvtkCommonCore-pv5.12.so.1
#29 0x00007ffff6f9a6fa in vtkPVPluginLoader::LoadPluginInternal(char const*, bool) () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libvtkRemotingCore-pv5.12.so.1
#30 0x00007ffff6f9c322 in vtkPVPluginLoader::LoadPluginsFromPath(char const*) () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libvtkRemotingCore-pv5.12.so.1
#31 0x00007ffff6f9c894 in vtkPVPluginLoader::LoadPluginsFromPluginSearchPath() () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libvtkRemotingCore-pv5.12.so.1
#32 0x00007fffedd2c236 in vtkInitializationHelper::InitializeMiscellaneous(int) () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libvtkRemotingApplication-pv5.12.so.1
#33 0x00007ffff66b3708 in pqApplicationCore::pqApplicationCore(int&, char**, vtkCLIOptions*, bool, QObject*) () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libpqCore-pv5.12.so.1
#34 0x00007ffff79d917a in pqPVApplicationCore::pqPVApplicationCore(int&, char**, vtkCLIOptions*, bool, QObject*) () from /home/timbo/software/ParaView-5.12.0-RC1-MPI-Linux-Python3.10-x86_64/bin/../lib/libpqApplicationComponents-pv5.12.so.1
#35 0x000000000040eacb in pqparaviewInitializer::Initialize(int, char**) ()
#36 0x00000000004093c8 in main ()

ben.boeckel · December 31, 2023, 7:02pm

Your paraviewopenfoam56 install is getting loaded in a ParaView 5.12 environment and ends up loading two vtkCommonCore libraries. Things are doomed from the moment that happens because VTK doesn’t have ABI compatibility. You’re lucky things went bad with a fatal error rather than a silent error .

I would recommend anything applicable from:

remove /opt/paraviewopenfoam56/lib from LD_LIBRARY_PATH
rebuild the OpenFOAM plugin against a 5.12 SDK (@mwestphal has containers to do this I believe)

mwestphal · January 2, 2024, 7:48am

See this but please note I never tried with the OpenFOAM stack:
https://gitlab.kitware.com/paraview/paraview-easy-plugin-builder

trilbytim · January 2, 2024, 5:37pm

Thank you. Removing /opt/paraviewopenfoam56/lib from LD_LIBRARY_PATH does allow version 5.12 to run now. I’ve added a line into my .bashrc file to do this. However, do you know where this folder is getting added to LD_LIBRARY_PATH in the first place, so I can prevent it being added?

The binary of version 5.12 seems to open my OpenFOAM data fine without needing any plugin, so I don’t think I even need this version 5.6 cluttering up my system. But it got installed with OpenFOAM, so I don’t want to mess up my OpenFOAM installation if I try to remove it.

ben.boeckel · January 2, 2024, 5:55pm

I’d search these places. paraviewopenfoam56 seems like an easy target to look for.

/etc/profile
~/.bash* (if you’re using bash; other similar paths for other shells)
~/.profile

trilbytim · January 2, 2024, 10:19pm

I couldn’t find it in any of those places, however my home/.bashrc file included source /opt/openfoam9/etc/bashrc So I looked at this file, but it also didn’t list the add /opt/paraviewopenfoam56/lib to LD_LIBRARY_PATH

However, it did do run a foamClean function on LD_LIBRARY_PATH where it removes some old directories from the path and removes duplicates, so I added /opt/paraviewopenfoam56/lib to the list of directories to be cleaned from LD_LIBRARY_PATH, and this did work.