nil calls causing agents to become stranded

[system123]

As no callback was fired when a nil object was passed to the connect method, agents where not aware that their status had become :on_call and thus could not take the correct action.

Two changes were made.

1. Ensure the callback is sent on connection failure.
2. Remove all nil objects from the queue before trying to connect agents and callees.

[bklang]

Is there any theory as to how objects in the queue became nil? This seems like a bandaid, and probably incomplete?

[system123]

I cannot reproduce the issue on a small scale, only when we have more than 50 agents. The outbound call is valid and active when it is queued but it seems that by the time the loop checking for connections pops that call off the queue the call is then nil. My only thought is that the call dies and becomes nil somewhere between it being queued and getting connection, yet the connection is happening before the call is removed.

My thinking for this is that call_waiting? returns true still meaning the queue has a nil object on it, rather than it being empty. I have been battling to find the exact cause. This bandaid does solve the issue, but I agree it is not an ideal solution.

[bklang]

My only thought is that the call dies and becomes nil somewhere between it being queued and getting connection, yet the connection is happening before the call is removed.

The thing is that values in an array can't become nil. The call can end, the actor can die, the value can become useless, but the value itself can't magically change from Adhearsion::Call to nil.

I can only think of two ways that a nil object would be enqueued:

1. It enters the queue that way from the application - perhaps adding logging to the code that adds things to the queue may reveal the source?
2. There is a yet-to-be-discovered bug in ElectricSlide that puts a nil into the queue - perhaps this can happen when a call connection fails? Maybe a call variable gets cleared and a nil value is re-added to the queue for the next agent? Adding trace logging to the code that puts a failed-attempt call back into the queue array could reveal this

[bklang]

In addition to this, it appears that the :connection_failed callback is also skipped in the case that the agent call is hung up. Is that a condition you want to handle too?

[system123]

I noticed this too, however, if the connection fails as the agent is dead I don't think we need to worry about a callback as it can be assumed that the agent lifecycle should be over. However, this might not be valid in the call_connection case. Perhaps for completeness I'll add a callback there too.