valgrind

[joske]

@aristocratos When you run btop under valgrind, it screams so many errors, it's not even funny. When running htop on same machine, it doesn't emit any error. So there seems to be fishy stuff going on. No idea what though. Most things seem to come from the c/c++ library.

[aristocratos]

For OSX you mean? No errors reported under linux.

Haven't setup a VM for OSX on x86 hardware yet so can't run valgrind. But do you wanna paste a dump of it just so I get the gist of it.

[joske]

when I run on my manjaro linux at home, indeed no specific problems reported by valgrind. When I ran some time ago on RHEL 7.6 at work, 1000s of errors. Will capture some output with a recent build tomorrow. Now I'll try on hackintosh (valgrind does not compile/install on m1, but you probably knew already).

…

On Mon, 25 Oct 2021 at 16:34, aristocratos ***@***.***> wrote: For OSX you mean? No errors reported under linux. Haven't setup a VM for OSX on x86 hardware yet so can't run valgrind. But do you wanna paste a dump of it just so I get the gist of it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEB7R5EQ5THH4SFZ3PCL6TUIVTHJANCNFSM5GVNV2IQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[joske]

on macOS it does not even run in valgrind. It seems to crash, no gui was visible. See attached log. Tell me if you want me to rerun with other options.

…

On Mon, 25 Oct 2021 at 16:34, aristocratos ***@***.***> wrote: For OSX you mean? No errors reported under linux. Haven't setup a VM for OSX on x86 hardware yet so can't run valgrind. But do you wanna paste a dump of it just so I get the gist of it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEB7R5EQ5THH4SFZ3PCL6TUIVTHJANCNFSM5GVNV2IQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[aristocratos]

See attached log

Um, where? :)

[joske]

when running nvim under valgrind I get similar errors

…

On Mon, 25 Oct 2021 at 16:34, aristocratos ***@***.***> wrote: For OSX you mean? No errors reported under linux. Haven't setup a VM for OSX on x86 hardware yet so can't run valgrind. But do you wanna paste a dump of it just so I get the gist of it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEB7R5EQ5THH4SFZ3PCL6TUIVTHJANCNFSM5GVNV2IQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[joske]

Hmm, I replied via email, it seems it didn't attach in github.
valgrind.log
nvim.log

[aristocratos]

Yeah, I don't think valgrind is working properly, alot of the issues seems to be in the regex functions from the standard library. And the errors in the log seems to stem only from old code that valgrind doesn't complain about on linux.

So don't know if valgrind is gonna be that useful in finding the leaks on macos unless all that noise can be suppressed.

[joske]

valgrind on macos big sur seems to be completely unsupported...

[aristocratos]

Btw, RHEL 7.6 is really outdated, were you running a native compiled build? In that case it's possible there are issues in the old glibc in RHEL 7.6 that valgrind picks up on.

[joske]

Yes, I know. But can't upgrade. It had similar issues as what we see in macos: use of uninitialized data mostly somewhere in the system libraries.

[aristocratos]

Did you compile with gcc10 or 11 on that system? Was thinking it could be related to the gcc10 libraries not being as mature for C++20 as the gcc11 ones or something like that maybe.

[joske]

can not compile on that system, this is a release. Newest gcc on that system is 8.3.

[aristocratos]

You remember if it was the musl build or the older glibc based static build?

[joske]

older build. I think it's 1.0.15.

[aristocratos]

You should try again with a musl build when you get a chance, and see if you get the same errors.

[joske]

ok, will do that tomorrow.

[aristocratos]

Found the issue with process cpu usage in macos btw https://developer.apple.com/documentation/driverkit/3433733-mach_timebase_info

Of course mac has it's own special conversion for cpu clock ticks :)

Thinking it could be something related with the issue in freebsd, some old legacy bsd stuff possibly.

[joske]

valgrind.log

This is with 1.0.19 musl. This runs, and seems to only complain about btop code itself. It's not in debug so stacktraces are not very useful I guess. Can you provide a static debug build (if needed)?

[aristocratos]

Here's a debug static musl build: btop.zip

Run valgrind with --track-origins=yes

[joske]

I did. I used this command:

valgrind --log-file=/tmp/valgrind.log --tool=memcheck --track-origins=yes --leak-check=yes --verbose --show-leak-kinds=all --error-limit=no btop

[aristocratos]

Having run valgrind (with the flags you posted above) on both the static musl binary and a dynamically linked binary I'm pretty sure the problem is with regex in the static builds.

No errors in the dynamic build and no leaks in either of the builds.
So I don't think this is anything to worry about since it's not leaking.

Not sure why regex behaves differently when compiled statically though.

[aristocratos]

Can maybe experiment with at compile-time regex compilation, to see if that makes a difference...

[aristocratos]

Removed all the use of regex which apparently didn't offer better performance than brute force string operations anyway :)

And now only get 3 errors from the musl static build.

All of them from robin_hood.h and with: Conditional jump or move depends on uninitialised value.

Which I'm not sure if they really matter that much, might be some minuscule slowdown from holding back the branch predictor but would be heavily outweighed by the speed of the robin hood maps compared to the standard maps.

Or am I interpreting these errors completely wrong?

[joske]

I don't know, not familiar enough with valgrind. I know it exists, but have never really used it ;-).

[joske]

BTW, regex can never be faster than a simple well written indexOf ;-).

[joske]

valgrind on freebsd:

==2121== HEAP SUMMARY:
==2121==     in use at exit: 800,119 bytes in 103 blocks
==2121==   total heap usage: 114,935 allocs, 114,832 frees, 69,712,965 bytes allocated
==2121== 
==2121== 32 bytes in 1 blocks are definitely lost in loss record 6 of 58
==2121==    at 0x48503D1: calloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==2121==    by 0x4FBF73: Mem::collect_disk(robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Mem::disk_info, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (btop_collect.cpp:476)
==2121==    by 0x4FE519: Mem::collect(bool) (btop_collect.cpp:646)
==2121==    by 0x4F8C70: Shared::init() (btop_collect.cpp:165)
==2121==    by 0x4ECA38: main (btop.cpp:773)
==2121== 
==2121== 32 bytes in 1 blocks are definitely lost in loss record 7 of 58
==2121==    at 0x48503D1: calloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==2121==    by 0x4FBF89: Mem::collect_disk(robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Mem::disk_info, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (btop_collect.cpp:477)
==2121==    by 0x4FE519: Mem::collect(bool) (btop_collect.cpp:646)
==2121==    by 0x4F8C70: Shared::init() (btop_collect.cpp:165)
==2121==    by 0x4ECA38: main (btop.cpp:773)
==2121== 
==2121== 512 bytes in 16 blocks are definitely lost in loss record 21 of 58
==2121==    at 0x48503D1: calloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==2121==    by 0x4FBF89: Mem::collect_disk(robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Mem::disk_info, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (btop_collect.cpp:477)
==2121==    by 0x4FE519: Mem::collect(bool) (btop_collect.cpp:646)
==2121==    by 0x4E7D20: Runner::_runner(void*) (btop.cpp:460)
==2121==    by 0x4EF6769: ??? (in /lib/libthr.so.3)
==2121== 
==2121== 2,312 bytes in 1 blocks are possibly lost in loss record 50 of 58
==2121==    at 0x484B8F4: malloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==2121==    by 0x4C90439: devstat_getdevs (in /lib/libdevstat.so.7)
==2121==    by 0x4FBF9F: Mem::collect_disk(robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Mem::disk_info, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (btop_collect.cpp:478)
==2121==    by 0x4FE519: Mem::collect(bool) (btop_collect.cpp:646)
==2121==    by 0x4F8C70: Shared::init() (btop_collect.cpp:165)
==2121==    by 0x4ECA38: main (btop.cpp:773)
==2121== 
==2121== 37,504 (512 direct, 36,992 indirect) bytes in 16 blocks are definitely lost in loss record 55 of 58
==2121==    at 0x48503D1: calloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==2121==    by 0x4FBF73: Mem::collect_disk(robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Mem::disk_info, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (btop_collect.cpp:476)
==2121==    by 0x4FE519: Mem::collect(bool) (btop_collect.cpp:646)
==2121==    by 0x4E7D20: Runner::_runner(void*) (btop.cpp:460)
==2121==    by 0x4EF6769: ??? (in /lib/libthr.so.3)
==2121== 
==2121== LEAK SUMMARY:
==2121==    definitely lost: 1,088 bytes in 34 blocks
==2121==    indirectly lost: 36,992 bytes in 16 blocks
==2121==      possibly lost: 2,312 bytes in 1 blocks
==2121==    still reachable: 759,727 bytes in 52 blocks
==2121==         suppressed: 0 bytes in 0 blocks
==2121== Reachable blocks (those to which a pointer was found) are not shown.
==2121== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==2121== 
==2121== For lists of detected and suppressed errors, rerun with: -s
==2121== ERROR SUMMARY: 230 errors from 21 contexts (suppressed: 0 from 0)

[aristocratos]

@joske
Yeah, 2 uses of calloc without freeing in Mem::collect_disk().
Those two structs should be created with unique_ptr for proper RAII.

[joske]

@aristocratos Can you do that? Trying but failing to find the correct way ;-).

[aristocratos]

@joske
No problem :)
Just haven't had time to properly look through the freebsd code (and fix up the disk IO stuff) yet.

But looks like there shouldn't be any other leak issues looking at the output from valgrind.

[joske]

think I figured it out. Now it only complains about a malloc in devstat_getdevs(). The man page doesn't say you should free anything, so not sure if this is a problem or not.

[jos@bsd ~/Projects/btop]$ valgrind --leak-check=full --track-origins=yes bin/btop 
==5355== Memcheck, a memory error detector
==5355== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==5355== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==5355== Command: bin/btop
==5355== 
==5355== Syscall param ioctl(generic) points to uninitialised byte(s)
==5355==    at 0x50475AA: ioctl (in /lib/libc.so.7)
==5355==    by 0x4FA55C6: fdevname_r (in /lib/libc.so.7)
==5355==    by 0x4FC433F: ttyname (in /lib/libc.so.7)
==5355==    by 0x4C2BDD: Term::init() (btop_tools.cpp:116)
==5355==    by 0x4EC6F2: main (btop.cpp:741)
==5355==  Address 0x7fc000474 is on thread 1's stack
==5355==  in frame #1, created by fdevname_r (???:)
==5355==  Uninitialised value was created by a stack allocation
==5355==    at 0x4FA5590: fdevname_r (in /lib/libc.so.7)
==5355== 
==5355== Syscall param ioctl(generic) points to uninitialised byte(s)
==5355==    at 0x50475AA: ioctl (in /lib/libc.so.7)
==5355==    by 0x4FA55C6: fdevname_r (in /lib/libc.so.7)
==5355==    by 0x4FC433F: ttyname (in /lib/libc.so.7)
==5355==    by 0x4C2C08: Term::init() (btop_tools.cpp:116)
==5355==    by 0x4EC6F2: main (btop.cpp:741)
==5355==  Address 0x7fc000474 is on thread 1's stack
==5355==  in frame #1, created by fdevname_r (???:)
==5355==  Uninitialised value was created by a stack allocation
==5355==    at 0x50E1590: ??? (in /lib/libc.so.7)
==5355== 
==5355== 
==5355== HEAP SUMMARY:
==5355==     in use at exit: 774,303 bytes in 50 blocks
==5355==   total heap usage: 136,145 allocs, 136,095 frees, 97,162,817 bytes allocated
==5355== 
==5355== 2,888 bytes in 1 blocks are definitely lost in loss record 44 of 50
==5355==    at 0x484B8F4: malloc (in /usr/local/libexec/valgrind/vgpreload_memcheck-amd64-freebsd.so)
==5355==    by 0x4C90439: devstat_getdevs (in /lib/libdevstat.so.7)
==5355==    by 0x4FC07F: Mem::collect_disk(robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Mem::disk_info, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, robin_hood::detail::Table<true, 80ul, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, robin_hood::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void>, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (btop_collect.cpp:482)
==5355==    by 0x4FE5F9: Mem::collect(bool) (btop_collect.cpp:650)
==5355==    by 0x4F8C70: Shared::init() (btop_collect.cpp:167)
==5355==    by 0x4ECA38: main (btop.cpp:773)
==5355== 
==5355== LEAK SUMMARY:
==5355==    definitely lost: 2,888 bytes in 1 blocks
==5355==    indirectly lost: 0 bytes in 0 blocks
==5355==      possibly lost: 0 bytes in 0 blocks
==5355==    still reachable: 771,415 bytes in 49 blocks
==5355==         suppressed: 0 bytes in 0 blocks
==5355== Reachable blocks (those to which a pointer was found) are not shown.
==5355== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==5355== 
==5355== For lists of detected and suppressed errors, rerun with: -s
==5355== ERROR SUMMARY: 264 errors from 17 contexts (suppressed: 0 from 0)

[aristocratos]

Well it's just 1 block so I would guess it's allocated and reused until end of program.