Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/implement different actions #74

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

nilskuhn
Copy link

@nilskuhn nilskuhn commented Sep 1, 2020

This is an implementation for issue 16.

…lugins#65 and logstash-plugins#66

Makes plugin work with mongodb driver, version 2.6 (again)
in addition to inserting new ones
fixing mongo driver version to 2.6
probably doesn't make sense in most use cases and causes trouble if size or order of array elements changes
@nilskuhn
Copy link
Author

nilskuhn commented Sep 3, 2020

Locally tests are green. So as usual ;) Works on my machine...
I am using jruby-9.1.12.0 via rvm:

rvm use jruby-9.1.12.0

Can anybody please tell me which are the correct ruby versions for which elastic stack versions and for which ruby versions the tests have to run?

@nilskuhn
Copy link
Author

Could anybody please take a look and let me know whether my changes make sense for you, too?
It's working for us so far.

@arichiardi
Copy link

@nilskuhn I am considering this patch for implementing a query that updates documents using the $inc operator from mongo:

https://docs.mongodb.com/manual/reference/operator/update/inc/

I wonder if I can do that - I would need to spike that probably but some hint from you would be greatly appreciated 😄

@arichiardi
Copy link

arichiardi commented Oct 7, 2020

Ok if I understand this right we don't have that option at the moment, it would mean adding a custom :update clause with $inc instead of the hardcoded $set

@arichiardi
Copy link

@nilskuhn I am trying your PR - I am planning to add the $inc support to this branch (if possible) and also maybe add support of composite query_value.

Do you have any input on this?

query_value is used like described https://docs.mongodb.com/ruby-driver/v2.6/tutorials/ruby-driver-bulk-operations[here]
for `update` and `replace` examples:

:filter => {query_key => query_value}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The usage of this is not really clear from this example, would you mind improving it a bit?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was a week on vacation. Will look into this asap.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem take your time as you saw I went a bit beyond the original intent 😃

Hope you like the result

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It gets clear if you follow the link to mongodb documentation I added to ascii-documentation and take a look at update and replace examples.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So in general you don't want to update a document in mongodb with whole logstash event but instead just increment some fields in mongodb? And do you want to read size to increment fields with from logstash event?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I am building projections from the events and I am doing now something like:

    if [@metadata][event_type] == "stock.updated" and [quantity] > 0 {
        mongodb {
            id => "orders.projection.mongodb.stock-update"
            uri => "${MONGO_DNS}"
            collection => "product-aggregates"
            database => "carts"
            isodate => true
            action => "update"
            filter => {
                "_id" => "[_id]"
                "store_id" => "[store_id]"
            }
            update_expressions => {
                "$inc" => {"stock" => "[stock_delta]"}
            }
        }
    }

@cavanwang
Copy link

why not update? I need a feature for updating MongoDB documents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants