Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify getting limit values and checking them #237

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

jgoizueta
Copy link
Contributor

@jgoizueta jgoizueta commented Nov 29, 2016

Now a function Checks.check handles both tasks allowing for
multiple checks (checking functions) and multiple limits per check.

See #220

Now, in node classes we'll typically need something like this to check limits:

var LIMITS = {
    maximumNumberOfRows: {
        default: 1e6,
        name: 'maximumNumberOfRows',
        message: 'too many source rows'
    }
};

MyNodeClass.prototype.checkLimits = function(context, callback) {
     Checks.check(this, context, [{
         checker: function(node, context, limitValues, params, done) {
            // this function receives limit values and should check them for the node,
            // calling done(error) if exceeded;
            // The Checks module has some functions like `Checks.limitNumberOfRows`
            // that can be used directly here
         },
         params: { input: 'source' }, // parameters passed to the checker
         limits: LIMITS                      // limits definitions
     }], callback);
 };

The Checks.check functions helps with the process of getting limit values from the configuration and calling checker functions. An array of objects that define each checker is passed.

Using one of the predefined checker functions (limitNumberOfRows, which doesn't require additional parameters) the above example would be:

MyNodeClass.prototype.checkLimits = function(context, callback) {
    Checks.check(this, context, [{
        checker: Checks.limitNumberOfRows,
        limits: LIMITS
    }], callback);
};

Now a function Checks.check handles both tasks allowing for
multiple checks (checking functions) and multiple limits per check.
@jgoizueta jgoizueta self-assigned this Nov 29, 2016
return { value: limit || defaultValue, message: errorMessage };
if (!limit) {
// apply default value; a sync. function (with a node parameter) can be used
if (typeof defaultValue === 'function') {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a remnant of a previsous iteration. I think I'll remove it, since it's not needed now

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If possible, I would remove it now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do

@jgoizueta
Copy link
Contributor Author

@rochoa care to take a look? Current functionality isn't change but adding or change limit checks should be easier now.

// ...
function limitValues(node, context, limits, callback) {
var values = {};
async.forEachOf(limits, function(limitParams, limit, done) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we use async here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need really, at some point in writing here I was expecting that getting limit values could eventually need some async operation, but now I see no reason for this. I'll remove it. My only doubt is whether to maintain this limitValues function as async (with callback) or not...

@@ -471,9 +478,8 @@ function validate(validator, params, expectedParamName) {
}

Node.prototype.checkLimits = function(context, callback) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this method must be overridden, can we document what is the signature for the callback?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

var defaultLimit = null;
var limit = context.getLimit(this.getType(), 'maximumNumberOfRows', defaultLimit, 'too many result rows');
limits.limitNumberOfRows(this, context, limit, callback);
Checks.check(this, context, [{
Copy link
Contributor

@rochoa rochoa Nov 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to have that default limit? Or should it be implicit?

I mean: could we just return callback(null);?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having this here allow us to limit the number of result rows for any analysis type by setting a limit value in the configuration.

If there's no configuration value, since where have defaultLimit here the behaviour is equivalent to callback(null) (because limitNumberOfRows does exactly that for null/Infinite limit values).

limit = context.getLimit(TYPE, limitName, defaultLimit, 'too many source rows');
limits.limitInputRows(this, 'source', context, limit, callback);
Checks.check(this, context, [{
checker: Checks.limitInputRows,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it a bit weird that the Checks.check function receives a reference to a function of the same module? I have the feeling this is still very _DSL_ish approach.

Why not something like:

this.limits.isWithinRowsLimit(this.source, limits.maximumNumberOfRows.default, callback)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where limits has already configuration limits, that could be injected from outside, to override the default limit provided here.

Copy link
Contributor Author

@jgoizueta jgoizueta Nov 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for the duality (Checks.check & Checks.limitInputRows) and the DSLish approach here is that Checks.check simplifies the processing if we want to combine multiple separate checks (e.g. check input rows and output rows), and handling for each separate check its limits values.

Then, for each check passed to Checks.check, the user can supply a checker function or use a predefined one such as Checks.limitInputRowshere.

I see your point with the more object-oriented approach you propose, but I think we would miss the ability to easily have multiple checks. (which may not be important really, I'm not sure... so far we're not needing it).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I like about this.limits.checkingOp(...) vs Checks.check(..., [{...definitions...}]) is how it simplifies some things (we don't need a check for input rows, and passing the input parameter since we have a check for output rows).

@jgoizueta jgoizueta force-pushed the 220-limits-declaration branch from b1c808f to 3d3e2de Compare December 2, 2016 11:12
var values = {};
Object.keys(limits).forEach(function(limit) {
var limitParams = limits[limit];
var limitName = limit || limitParams.name;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be var limitName = limitParams.name || limit; shouldn't it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants