Bug: Crashes with SIGABRT after about 2 hours

[bjornbm]

Contact Details

[email protected]

What happened?

Llamafile crashes with SGIABRT. Excerpt from terminal:

libc++abi: terminating due to uncaught exception of type std::__1::system_error: condition_variable wait failed: Connection timed out

error: Uncaught SIGABRT (SI_0) on Narcissus pid 21086 tid 262146

Steps to reproduce:

1. Start gemma-2-2b-it.Q6_K.llamafile
2. Wait

Full transcript pasted in log field.

On macos 15.1.1, on M1 chip.

Version

llamafile v0.8.16

What operating system are you seeing the problem on?

Mac

Relevant log output

$ uname -a                           
Darwin Narcissus 24.1.0 Darwin Kernel Version 24.1.0: Thu Oct 10 21:03:15 PDT 2024; root:xnu-11215.41.3~2/RELEASE_ARM64_T6000 arm64
$ time ./gemma-2-2b-it.Q6_K.llamafile

██╗     ██╗      █████╗ ███╗   ███╗ █████╗ ███████╗██╗██╗     ███████╗
██║     ██║     ██╔══██╗████╗ ████║██╔══██╗██╔════╝██║██║     ██╔════╝
██║     ██║     ███████║██╔████╔██║███████║█████╗  ██║██║     █████╗
██║     ██║     ██╔══██║██║╚██╔╝██║██╔══██║██╔══╝  ██║██║     ██╔══╝
███████╗███████╗██║  ██║██║ ╚═╝ ██║██║  ██║██║     ██║███████╗███████╗
╚══════╝╚══════╝╚═╝  ╚═╝╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚══════╝╚══════╝
software: llamafile 0.8.16
model:    gemma-2-2b-it.Q6_K.gguf
compute:  Apple Metal GPU
server:   http://127.0.0.1:8080/

A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.
>>> libc++abi: terminating due to uncaught exception of type std::__1::system_error: condition_variable wait failed: Connection timed out

error: Uncaught SIGABRT (SI_0) on Narcissus pid 21086 tid 262146
 /Users/bjorn/Downloads/gemma-2-2b-it.Q6_K.llamafile
 Darwin Cosmopolitan 3.9.6 MODE=aarch64; Darwin Kernel Version 24.1.0: Thu Oct 10 21:03:15 PDT 2024; root:xnu-11215.41.3~2/RELEASE_ARM64_T6000 Narcissus 24.1.0
 cosmoaddr2line /Users/bjorn/Downloads/gemma-2-2b-it.Q6_K.llamafile 19e546600 19e453c18 1045fab6c 8002f5c08 8000007dc 80030bfac 80030c104 8002cfa88 8002cfdc4 8002d019c 8002d05a0 8002d36d8 8002d3c94 8002cf4f4 80000041c 800282440
 0000000000000000 x0 ec15c575966c7d52 x8  0000000000000148 x16 000000010465c440 x24
 0000000000000000 x1 ec15c574e0c84d52 x9  000000020b8f1850 x17 0000000000000001 x25
 0000000000000008 x2 0000000000000001 x10 0000000000000000 x18 0000000000000000 x26
 0000000802b43000 x3 0000000000000002 x11 0000000000000006 x19 00000000000000ff x27
 0000000000000000 x4 00000000000013b0 x12 0000000000003503 x20 0000000105aab080 x28
 000000000000525e x5 000000010848dd68 x13 0000000176a430e0 x21 000000010848ecc0 x29
 0000000000000008 x6 0000000000000001 x14 00000008004510f4 x22 000000019e57ef70 x30
 0000000000000021 x7 0000000000000000 x15 000000010848efd0 x23 000000010848eca0 x31
 000000010848eca0 sp 19e546600 pc NULL-1638867592
 000000010848ecc0 fp 19e453c18 lr NULL-1639861360
 000000010848ece0 fp 1045fab6c lr NULL+73147108
 000000010848ecf0 fp 8002f5c08 lr raise+48
 000000010848ed00 fp 8000007dc lr abort+48
 000000010848ed20 fp 80030bfac lr NULL+2963236
 000000010848ed40 fp 80030c104 lr demangling_terminate_handler()+236
 000000010848ee60 fp 8002cfa88 lr std::__terminate(void (*)())+16
 000000010848eea0 fp 8002cfdc4 lr call_terminate+44
 000000010848eeb0 fp 8002d019c lr scan_eh_tab+452
 000000010848eed0 fp 8002d05a0 lr __gxx_personality_v0+104
 000000010848ef80 fp 8002d36d8 lr unwind_phase1.constprop.0+128
 000000010848f000 fp 8002d3c94 lr _Unwind_RaiseException+60
 000000010848f290 fp 8002cf4f4 lr __cxa_throw+92
 000000010848fe50 fp 80000041c lr std::__1::__throw_system_error(int, char const*)+112
 000000010848fe80 fp 800282440 lr NULL+2399160
 000000010848feb0 fp 80003dc44 lr httplib::ThreadPool::worker::operator()()+124
 000000010848ff00 fp 80003db94 lr _ZNSt3__114__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEN7httplib10ThreadPool6workerEEEEEEPvSB_+48
 000000010848ff50 fp 8002daa6c lr PosixThread+116
 000000010848ff70 fp 8002fe358 lr __stack_call+24
zsh: abort      ./gemma-2-2b-it.Q6_K.llamafile
./gemma-2-2b-it.Q6_K.llamafile  0.47s user 0.97s system 0% cpu 2:00:21.60 total
$ ./gemma-2-2b-it.Q6_K.llamafile --version
llamafile v0.8.16

[beartell]

Same happened to me..

llamafile libc++abi: terminating due to uncaught exception of type std::__1::system_error: condition_variable wait failed: Connection time out