Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Feat/natlog and open ie #52

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/browser/corenlp.min.js

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@
"build": "gulp build",
"build:browser": "gulp build:browser",
"prepublish": "npm run lint && npm run test && npm run compile",
"lint": "eslint 'src/**/*.js' 'test/*.js'",
"lint": "eslint \"src/**/*.js\" \"test/*.js\"",
"lint:fix": "npm run lint -- --fix",
"test": "nyc --reporter=html --reporter=text mocha test/setup.js --sort 'src/**/*.spec.js' --compilers js:babel-core/register --timeout 30000",
"test": "nyc --reporter=html --reporter=text mocha test/setup.js --sort \"src/**/*.spec.js\" --compilers js:babel-core/register --timeout 30000",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please, make sure this doesn't come as new change, since it was introduced by #51 ... rebasing your branch with the latest master HEAD should do the trick

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, thats great. I never had a concrete use-case for git rebase and therefore probably never really understood it. Thanks for changing that. 😄

"test:watch": "npm run test -- --watch",
"doc": "DEBUG=gulp-jsdoc3 gulp doc"
},
Expand Down
4 changes: 4 additions & 0 deletions src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ import Expression from './simple/expression';
import Annotable from './simple/annotable';
import Annotator from './simple/annotator';
import TokenizerAnnotator from './simple/annotator/tokenize';
import NaturalLogicAnnotator from './simple/annotator/natlog';
import OpenIEAnnotator from './simple/annotator/openie';
import WordsToSentenceAnnotator from './simple/annotator/ssplit';
import POSTaggerAnnotator from './simple/annotator/pos';
import MorphaAnnotator from './simple/annotator/lemma';
Expand Down Expand Up @@ -58,6 +60,8 @@ export default {
RelationExtractorAnnotator,
RegexNERAnnotator,
CorefAnnotator,
NaturalLogicAnnotator,
OpenIEAnnotator,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: please, use the same order as for the import statements (that's OCD 😆I know).

},
},
/**
Expand Down
4 changes: 4 additions & 0 deletions src/index.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import Token from './simple/token';
import Annotable from './simple/annotable';
import Annotator from './simple/annotator';
import TokenizerAnnotator from './simple/annotator/tokenize';
import OpenIEAnnotator from './simple/annotator/openie';
import NaturalLogicAnnotator from './simple/annotator/natlog';
import WordsToSentenceAnnotator from './simple/annotator/ssplit';
import POSTaggerAnnotator from './simple/annotator/pos';
import MorphaAnnotator from './simple/annotator/lemma';
Expand Down Expand Up @@ -73,6 +75,8 @@ describe('CoreNLP Library entry point', () => {
RelationExtractorAnnotator,
RegexNERAnnotator,
CorefAnnotator,
NaturalLogicAnnotator,
OpenIEAnnotator,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: same here, let's follow the same order.

});
});
});
Expand Down
4 changes: 4 additions & 0 deletions src/pipeline.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ import difference from 'lodash.difference';
import Service from './service';
import ConnectorServer from './connector/connector-server';
import tokenize from './simple/annotator/tokenize';
import natlog from './simple/annotator/natlog';
import openie from './simple/annotator/openie';
import ssplit from './simple/annotator/ssplit';
import pos from './simple/annotator/pos';
import lemma from './simple/annotator/lemma';
Expand Down Expand Up @@ -30,6 +32,8 @@ const ANNOTATORS_BY_KEY = {
relation,
regexner,
coref,
natlog,
openie,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: and here

};

const LANGUAGE_TO_ISO2 = {
Expand Down
40 changes: 40 additions & 0 deletions src/simple/annotator/natlog.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import Annotator from '../annotator';
import TokenizerAnnotator from './tokenize';
import WordsToSentenceAnnotator from './ssplit';
import POSTaggerAnnotator from './pos';
import MorphaAnnotator from './lemma';
import DependencyParseAnnotator from './depparse';

/**
* @class
* @classdesc Class representing an Natural Logic Annotator.
* @extends Annotator
* @memberof CoreNLP/simple/annotator
* @requires tokenize, ssplit, pos, lemma, depparse (Can also use parse)
* @see {@link https://stanfordnlp.github.io/CoreNLP/natlog.html|NaturalLogicAnnotator}
*/
class NaturalLogicAnnotator extends Annotator {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

/**
* Create an Annotator
* @param {Object} [options] a key-value map of options, without the annotator prefix
*/
constructor(options = {}) {
super(
'natlog',
{
// dopolarity: True by default. If set to false, the annotator will only annotate
// quantifiers and quantifier scopes, and not annotate the polarity of each token
...options,
},
[
new TokenizerAnnotator(),
new WordsToSentenceAnnotator(),
new POSTaggerAnnotator(),
new MorphaAnnotator(),
new DependencyParseAnnotator(),
],
);
}
}

export default NaturalLogicAnnotator;
20 changes: 20 additions & 0 deletions src/simple/annotator/natlog.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import NaturalLogicAnnotator from './natlog';

describe('Annotator', () => {
let annotator;

describe('NaturalLogicAnnotator', () => {
beforeEach(() => {
annotator = new NaturalLogicAnnotator();
});

it('should have a proper pipeline', () => {
expect(annotator.pipeline()).to.deep.equal(['tokenize', 'ssplit', 'pos', 'lemma', 'depparse', 'natlog']);
});

it('should have the proper default options', () => {
expect(annotator.options()).to.deep.equal({
});
});
});
});
51 changes: 51 additions & 0 deletions src/simple/annotator/openie.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import Annotator from '../annotator';
import NaturalLogicAnnotator from './natlog';
import DependencyParseAnnotator from './depparse';
import CorefAnnotator from './coref';

// TODO: Requirements are missing here https://stanfordnlp.github.io/CoreNLP/annotators.html, verify.

/**
* @class
* @classdesc Class representing an OpenIE Annotator.
* @extends Annotator
* @memberof CoreNLP/simple/annotator
* @requires natlog
* @see {@link https://stanfordnlp.github.io/CoreNLP/openie.html|OpenIEAnnotator}
*/
class OpenIEAnnotator extends Annotator {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

/**
* Create an Annotator
* @param {Object} [options] a key-value map of options, without the annotator prefix
*/
constructor(options = {}) {
super(
'openie',
{
// format (Enum) default One of {reverb, ollie, default, qa_srl}. Changes the output form
// filelist (filepath) null A path to a file, which contains files to annotate.
// threads (integer) number of cores The number of threads to run on.
// max_entailments_per_clause (integer) 1000 The maximum number of entailments to prod
resolve_coref: false, // (boolean) false If true, run coreference (and consequently
// ignore_affinity (boolean) false Whether to ignore the affinity model for preposi
// affinity_probability_cap (double) 1 /3 The affinity value above which confidence
// triple.strict (boolean) true If true, extract triples only if they consume the e
triple: { all_nominals: false }, // If true, extract nominal relations always and not o
// splitter.model (filepath) You can override the default location of the clause
// splitter.nomodel (boolean) false Run without a clause splitting model – that is,
// splitter.disable (boolean) false Don’t split clauses at all, and only extract re
// affinity_models (filepath)
...options,
},
options.resolve_coref
? [
new CorefAnnotator(),
new DependencyParseAnnotator(),
new NaturalLogicAnnotator(),
]
: [new NaturalLogicAnnotator()],
);
}
}

export default OpenIEAnnotator;
32 changes: 32 additions & 0 deletions src/simple/annotator/openie.spec.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
import OpenIEAnnotator from './openie';

describe('Annotator', () => {
let annotator;

describe('OpenIEAnnotator', () => {
beforeEach(() => {
annotator = new OpenIEAnnotator();
});

it('should have a proper pipeline', () => {
expect(annotator.pipeline()).to.deep.equal([
'tokenize',
'ssplit',
'pos',
'lemma',
'depparse',
'natlog',
'openie',
]);
});

it('should have the proper default options', () => {
expect(annotator.options()).to.deep.equal({
resolve_coref: false,
triple: {
all_nominals: false,
},
});
});
});
});
40 changes: 37 additions & 3 deletions src/simple/sentence.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import head from 'lodash.head';
import Annotable from './annotable';
import TokenizerAnnotator from './annotator/tokenize';
import ParserAnnotator from './annotator/parse';
import OpenIEAnnotator from './annotator/openie';
import DependencyParseAnnotator from './annotator/depparse';
import Token from './token';
import Governor from './governor';
Expand Down Expand Up @@ -30,6 +31,8 @@ class Sentence extends Annotable {
super(text);
this._tokens = [];
this._governors = [];
this._openie = [];
this._natLogPolarities = [];
}

/**
Expand Down Expand Up @@ -186,18 +189,45 @@ class Sentence extends Annotable {
}

// TODO
// eslint-disable-next-line class-methods-use-this
// eslint-disable-next-line class-methods-use-this, no-unused-vars
natlogPolarities() {
// if (!this.hasAnnotator(NaturalLogicAnnotator)) {
// throw new Error(
// 'Asked for PolarityAnnotation on Sentence, but there are unmet annotator dependencies.'
// );
// }
// return this._natLogPolarities;
}

// TODO
// eslint-disable-next-line class-methods-use-this, no-unused-vars
natlogPolarity(index) {
// if (!this.hasAnnotator(NaturalLogicAnnotator)) {
// throw new Error(
// 'Asked for a PolarityAnnotation on Sentence, but there are unmet annotator dependencies.'
// );
// }
// return this._natLogPolarities[index];
}

// TODO
// eslint-disable-next-line class-methods-use-this
/**
* Extract open-domain relation triples.
* @requires {@link OpenIEAnnotator}
* @throws {Error} in case the require annotator was not applied to the sentence
* @returns {Array.<OpenIETriple>} OpenIE-Triples
*/
openie() {
// TODO: Create OpenIE-Triples:
// object:"Constantin Hütterer"
// objectSpan:Array(2) [3, 5]
// relation:"is"
// relationSpan:Array(2) [2, 3]
// subject:"My name"
// subjectSpan:Array(2) [0, 2]
if (!this.hasAnnotator(OpenIEAnnotator)) {
throw new Error('Asked for a OpenIE-Annotation on Sentence, but there are unmet annotator dependencies.');
}
return this._openie;
}

// TODO
Expand Down Expand Up @@ -287,6 +317,10 @@ class Sentence extends Annotable {
this.addAnnotator(ParserAnnotator);
this._parse = sentence.parse;
}
if (sentence.openie) {
this.addAnnotator(OpenIEAnnotator);
this._openie = sentence.openie;
}
if (sentence.basicDependencies) {
this.addAnnotator(DependencyParseAnnotator);
this._governors = sentence.basicDependencies.map(gov =>
Expand Down