Change Stream support

[goliatone]

I'm looking at starting a new project and want to use elixir. I was wondering if this library has support for mongo's Change Streams

[ankhers]

Currently no. It is something that I would like added to this package. Is it something that you would be interested in adding? What is the timeline on starting your project?

[goliatone]

I can take a stab at adding the feature but currently have no idea where to start, so don't count on it. I have to start pretty soon, might use a different language to tackle the task but eventually it would be nice to align with the rest of the service which consists of different Elixir apps.

[ankhers]

I have started support for Change Streams in #205. It is not complete yet, but would you mind testing it as it grows?

[Tyler8oliver]

I am wondering what progress has been made with this issue? If it hasn't been completed I may be able to find time to complete it.

[ankhers]

It is incomplete, but working. I need to double check the logic in the aggregation cursor to see if it follows the expected reconnection logic in the change stream specification.

If you find some time, I would appreciate the help.

[klmurphy72]

I am currently playing with the change_stream branch. I am an Elixir noob but this looks good.

If a change does not occur within a short period of the time say :max_time 3000 (3 seconds) the stream ends.

If I change :max_time to a number higher than 5000 and a change does not occur in 5 seconds I get the following:

[17:19:55.500][error] pid=<0.210.0> Mongo.Protocol (#PID<0.210.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.181
.0> timed out because it checked out the connection for longer than 5000ms
[17:19:55.510][error] pid=<0.181.0> Task #PID<0.181.0> started from #PID<0.176.0> terminating
** (Mongo.Error) tcp recv: unknown POSIX error - :timeout
(mongodb) lib/mongo/cursor.ex:171: anonymous fn/2 in Enumerable.Mongo.AggregationCursor.next_fun/1
(elixir) lib/stream.ex:1377: Stream.do_resource/5
(elixir) lib/enum.ex:2979: Enum.reduce/3
(ersc_comm) lib/ersc_comm.ex:178: anonymous fn/0 in ERSCCommApp.start/2

Looking at the spec it would seem that the ability to pass in more setup information to the $change_stream information would be necessary for restarts.

HTH

[ankhers]

There is quite a bit of work that still needs to be done here in order to ensure proper compliance with the spec.

[massivefermion]

hi
any progress on this?

[Douvi]

Hey @ankhers,

Sorry to bugging you, do you know when Change Stream can be ready?
I will start very soon a project to use Change Stream and this feature is mandatory to have it to start using Elixir :(...
If you want I can help you, but I would like to know from @klmurphy72 feedback what we should do to avoid timeout connection?

Edouard.

[zookzook]

Hi Edouard,

maybe you would like to try out this.

[joeapearson]

@zookzook has provided support for this in his alternate MongoDB driver at https://github.com/zookzook/elixir-mongodb-driver . When we come to looking at this I suggest looking there for hints.