CRASH *** SIGABRT (@0x1) received by PID 1 (TID 0x7f063645e700) from PID 1; stack trace: ***

[umatomba]

Hi Robert,

I am trying to test the 5 node cluster.

Space ycsb configuration:

space usertable
key k
attributes field0, field1, field2, field3, field4,
           field5, field6, field7, field8, field9
tolerate 2 failure

After the cluster setup the config looks like:

server 1998750160347642936 192.168.100.3:2012 AVAILABLE
server 3667604389049738774 192.168.100.5:2012 AVAILABLE
server 6903549772667245983 192.168.100.2:2012 AVAILABLE
server 12441010121126734978 192.168.100.1:2012 AVAILABLE
server 12691374307851476699 192.168.100.4:2012 AVAILABLE

So, all servers have AVAILABLE state.

On the workload configuration i have recordcount=1000000.

So, lets start data loading:

java -Djava.library.path=/usr/local/lib/ com.yahoo.ycsb.Client -load -db org.hyperdex.ycsb.HyperDex -P /hyperdex_build/ycsb-0.1.4/workloads/workloada -p "hyperdex.host=192.168.100.1" -p "hyperdex.port=1982" -p theads=16

After this operation, we see only 999990 elements in usertable:

>>> c = hyperdex.client.Client('192.168.100.1', 1982)
>>> c.count('usertable', {'k': hyperdex.client.Regex('.*')})
999990L

And coordinator2 is down with crash:

I0129 12:53:48.107548 1 daemon.cc:2439] the connection with the prev node is 100% healed
*** SIGABRT (@0x1) received by PID 1 (TID 0x7f063645e700) from PID 1; stack trace: ***
@ 0x7f063d8f8340 (unknown)
@ 0x7f063cb2ccc9 (unknown)
@ 0x7f063cb300d8 (unknown)
@ 0x7f063cb25b86 (unknown)
@ 0x7f063cb25c32 (unknown)
@ 0x40de68 (unknown)
@ 0x411cfd (unknown)
@ 0x412008 (unknown)
@ 0x442d00 (unknown)
@ 0x443563 (unknown)
@ 0x4437f4 (unknown)
@ 0x444585 (unknown)
@ 0x425511 (unknown)
@ 0x7f063d8f0182 start_thread
@ 0x7f063cbf100d (unknown)
@ 0x0 (unknown)

All logs available hier:
https://github.com/umatomba/hyperdex-test/tree/master/5nodes/test2/1kk/out_a_45split/logs_on_db_dataload

The tools used in this test:

Docker image: https://registry.hub.docker.com/u/umatomba/docker-hyperdex/dockerfile/
Vagrant setup: https://github.com/umatomba/vagrant-docker-hyperdex