Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup analytics docs and test #388

Merged
merged 30 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
640c8e3
refactor: first draft of a small docs and test cleanup
AlasDiablo Dec 14, 2023
7deb2a0
test (analytics): add different data set to tune
AlasDiablo Dec 14, 2023
ae66698
test (analytics): fix typo
AlasDiablo Dec 14, 2023
93f1d31
Merge branch 'master' into docs-tests-analytics
AlasDiablo Jan 24, 2024
75f3521
docs (analytics): update tune doc
AlasDiablo Jan 24, 2024
eb7b3a3
docs (analytics): fix typo
AlasDiablo Jan 24, 2024
f8969d1
Merge branch 'master' into docs-tests-analytics
AlasDiablo Jan 24, 2024
5797e4f
docs (analytics): generify tests
AlasDiablo Jan 24, 2024
e95c6d3
docs (analytics): generated doc
AlasDiablo Jan 24, 2024
3084e28
test (analytics): fix test
AlasDiablo Jan 24, 2024
41d61e2
refactor (analytics): update value
AlasDiablo Jan 25, 2024
4182d47
docs (analytics): fix typo in json example
AlasDiablo Jan 25, 2024
e5399fe
docs (analytics): fix typo
AlasDiablo Jan 25, 2024
df570cd
refactor (analytics): update summing doc and add test
AlasDiablo Jan 25, 2024
dc721a2
test (analytics): add test how send badly structured data
AlasDiablo Jan 25, 2024
8ddd539
docs (analytics): add sort and statistics doc and fix some typo
AlasDiablo Jan 26, 2024
b93f5e4
Merge branch 'master' into docs-tests-analytics
AlasDiablo Jan 29, 2024
77bf346
docs (analytics): add slice doc
AlasDiablo Jan 29, 2024
d0f5093
docs: add Engine scope jsdoc type
AlasDiablo Jan 29, 2024
edb6743
docs (analytics): reformat segment doc
AlasDiablo Jan 29, 2024
25a8f62
docs (core): add missing private
AlasDiablo Jan 29, 2024
a776fe3
docs (analytics): fix typo
AlasDiablo Jan 30, 2024
164c119
docs (analytics): typo
AlasDiablo Jan 30, 2024
e0c1920
docs (analytics): add reducing documentation
AlasDiablo Jan 30, 2024
cff6018
docs (analytics): reformat pluck doc and fix reducing doc
AlasDiablo Jan 31, 2024
eaa00a5
docs (analytics): fix nitpick typo
AlasDiablo Jan 31, 2024
2947063
docs (analytics): reformat pair doc
AlasDiablo Jan 31, 2024
91d2410
refactor (analytics): remove code added for no reason
AlasDiablo Jan 31, 2024
67ee08d
refactor (analytics): reformat output doc
AlasDiablo Jan 31, 2024
a26ce90
Merge branch 'master' into docs-tests-analytics
touv Feb 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
955 changes: 591 additions & 364 deletions docs/plugin-analytics.md

Large diffs are not rendered by default.

955 changes: 591 additions & 364 deletions packages/analytics/README.md

Large diffs are not rendered by default.

109 changes: 76 additions & 33 deletions packages/analytics/src/output.js
Original file line number Diff line number Diff line change
@@ -1,47 +1,26 @@
import { get, unset } from 'lodash';
/**
* Format the output with data a meta
*
* @example <caption>Input</caption>
* [
* { _id: 1, value: 2, total: 2 },
* { _id: 2, value: 4, total: 2 }
* ]
*
* @example <caption>Script</caption>
* .pipe(ezs('output', { meta: 'total' }))
*
* @example <caption>Output</caption>
* {
* data: [
* { _id: 1, value: 2 },
* { _id: 2, value: 4 }
* ],
* meta: {
* total: 2
* }
* }
*
* @name output
* @param {boolean} [indent=false] indent or not
* @param {string[]} [meta] fields to be considered as metadata
* object
* @returns {string}
* Pair function see documentation at the end.
* This part of the doc is used for jsdoc typing
* @private
* @param data {unknown}
* @param feed {Feed}
* @param ctx {import('../../core/src/engine').EngineScope}
*/
function output(data, feed) {
const indent = this.getParam('indent', false);
const extract = this.getParam('meta');
const output = (data, feed, ctx) => {
const indent = ctx.getParam('indent', false);
const extract = ctx.getParam('meta');
const extracts = Array.isArray(extract) ? extract : [extract];
const keys = extracts.filter((x) => x);
const cr = indent ? '\n ' : '';

const json = (d) => JSON.stringify(d, null, indent ? ' ' : null);

if (this.isLast()) {
if (ctx.isLast()) {
feed.write(`]}${cr}`);
return feed.close();
}
if (this.isFirst() && !this.isLast()) {
if (ctx.isFirst() && !ctx.isLast()) {
const values = keys.map((p) => get(data, p));
feed.write(`{${cr}"meta":{${cr}`);
if (keys.length > 0) {
Expand All @@ -63,7 +42,71 @@ function output(data, feed) {
keys.forEach((p) => unset(data, p));
feed.write(json(data));
return feed.end();
}
};

/**
* Create an output string containing all incoming elements in a `data` array.
* with given `meta` extracted into an object called `meta`.
*
* Créer une sortie en chain de caratere avec les element entrent mise dans un tableau nommé `data`
* eyent les donnée `meta` extrais et mises dans un objet appelé `meta`.
*
* #### Script / Scénario
*
* ##### ini
*
* ```ini
* ; Import analytics plugin required to use "output"
* ; Importation du plugin analytique nécessaire pour utiliser "output"
* [use]
* plugin = analytics
*
* ; Using "output" with 'indent' as true and 'meta' as total
* ; Utilisation de "output" avec 'indent' à vrai et total comme paramètres de 'meta'
* [output]
* indent = true
* meta = total
*
* ```
*
* #### Input / Entrée
*
* ```json
* [
* { "_id": 1, "value": 2, "total": 2 },
* { "_id": 2, "value": 4, "total": 2 }
* ]
* ```
*
* #### Output / Sortie
*
* !!! Attention: This is an output function that can only be used at the end of an EZS script. !!!
* !!! The output is a string and can't be used with other EZS functions. !!!
*
* !!! Attention : Ceci est une fonction de sortie, Elle peut uniquement etre utilisé à la fin d'un script ezs !!!
* !!! Cette sortie est une chaine de carater et ne peut pas etre utilisé avec d'autre fonction ezs !!!
*
* ```json
* {
* "data": [
* { "_id": 1, "value": 2 },
* { "_id": 2, "value": 4 }
* ],
* "meta": {
* "total": 2
* }
* }
* ```
*
* @name output
* @param {Boolean} [indent=false]
* <ul><li>indent the output json</li></ul>
* <ul><li>indenté le json de sortie</li></ul>
* @param {String} [meta]
* <ul><li>element from the input to put it in the `meta` object</li></ul>
* <ul><li>élément a extraire de l'entrée et a mettre dans l'objet `meta`</li></ul>
* @returns {String}
*/
export default {
output,
};
108 changes: 65 additions & 43 deletions packages/analytics/src/pair.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,52 +2,19 @@ import { get } from 'lodash';
import core from './core';

/**
* Take `Object` object getting some fields with json path, and
* throw all pair of value from two fields
*
* ```json
* [
* { departure: ['tokyo', 'nancy'], arrival: 'toul' },
* { departure: ['paris', 'nancy'], arrival: 'toul' },
* { departure: ['london', 'berlin'], arrival: 'toul' },
* ]
* ```
*
* Script:
*
* ```ini
* [use]
* plugin = analytics
*
* [pair]
* path = departure
* path = arrival
*
* ```
*
* Output:
*
* ```json
* [
* { "id": [ "tokyo", "toul" ], "value": 1 },
* { "id": [ "nancy", "toul" ], "value": 1 },
* { "id": [ "paris", "toul" ], "value": 1 },
* { "id": [ "nancy", "toul" ], "value": 1 },
* { "id": [ "london", "toul" ], "value": 1 },
* { "id": [ "berlin", "toul" ], "value": 1 }
* ]
* ```
*
* @name pair
* @param {String} path
* @returns {Object}
* Pair function see documentation at the end.
* This part of the doc is used for jsdoc typing
* @private
* @param data {unknown}
* @param feed {Feed}
* @param ctx {import('../../core/src/engine').EngineScope}
*/
export default function pair(data, feed) {
if (this.isLast()) {
const pair = (data, feed, ctx) => {
if (ctx.isLast()) {
feed.close();
return;
}
let fields = this.getParam('path', []);
let fields = ctx.getParam('path', []);
if (!Array.isArray(fields)) {
fields = [fields];
}
Expand All @@ -68,4 +35,59 @@ export default function pair(data, feed) {
});

feed.end();
}
};

/**
* Create a pair with 'id' containing a pair of the given 'path's and 'value' set to 1.
*
* Créer un couple 'id' contenent un couple des 'path's donnée et 'value' mise à 1.
*
* #### Script / Scénario
*
* ```ini
* ; Import analytics plugin required to use "pair"
* ; Importation du plugin analytique nécessaire pour utiliser "pair"
* [use]
* plugin = analytics
*
* ; Using "pair" with 'departure' and 'arrival' as paths setttings
* ; Utilisation de "pair" avec 'departure' et 'arrival' comme paramètres de paths
* [pair]
* path = departure
* path = arrival
*
* ```
*
* #### Input / Entrée
*
* ```json
* [
* { "departure": ["tokyo", "nancy"], "arrival": "toul" },
* { "departure": ["paris", "nancy"], "arrival": "toul" },
* { "departure": ["london", "berlin"], "arrival": "toul" }
* ]
* ```
*
* #### Output / Sortie
*
* ```json
* [
* { "id": ["tokyo", "toul"], "value": 1 },
* { "id": ["nancy", "toul"], "value": 1 },
* { "id": ["paris", "toul"], "value": 1 },
* { "id": ["nancy", "toul"], "value": 1 },
* { "id": ["london", "toul"], "value": 1 },
* { "id": ["berlin", "toul"], "value": 1 }
* ]
* ```
*
* @name pair
* @param {String}
* <ul><li>path of the element who will be use to create the pair</li></ul>
* <ul><li>chemin de l'élément qui vas etre utilisé pour créer le couple</li></ul>
* @returns {{
* id: Array<String>,
* value: 1
* }}
*/
export default pair;
121 changes: 74 additions & 47 deletions packages/analytics/src/pluck.js
Original file line number Diff line number Diff line change
@@ -1,68 +1,95 @@
import { get } from 'lodash';
import core from './core';

/**
* Take `Object` object getting value of fields (with json `path`) and throws an
* object for each value
* Pluck function see documentation at the end.
* This part of the doc is used for jsdoc typing
* @private
* @param data {unknown}
* @param feed {Feed}
* @param ctx {import('../../core/src/engine').EngineScope}
*/
const pluck = (data, feed, ctx) => {
if (ctx.isLast()) {
feed.close();
return;
}
let fields = ctx.getParam('path', 'id');
if (!Array.isArray(fields)) {
fields = [fields];
}

fields
.filter((k) => typeof k === 'string')
.map((key) => [key, get(data, key)])
.filter((x) => x[1])
.map((item) => ([item[0], (item[1] instanceof Array ? item[1] : [item[1]])]))
.reduce((prev, cur) => prev.concat(cur[1].map((x) => ([cur[0], x]))), [])
.forEach((item) => feed.write(core(item[0], item[1])));
feed.end();
};

/**
* Extract the value of a given `path` and create a pair with the `path` as the `id`
* and `path` value as the `value`.
*
* ```json
* [
* { city: 'tokyo', year: 2000, count: 1 },
* { city: 'paris', year: 2001, count: 2 },
* { city: 'london', year: 2003, count: 3 },
* { city: 'nancy', year: 2005, count: 4 },
* { city: 'berlin', year: 2007, count: 5 },
* { city: 'madrid', year: 2009, count: 6 },
* { city: 'stockholm', year: 2011, count: 7 },
* { city: 'bruxelles', year: 2013, count: 8 },
* ]
* ```
* Extrais la valeur d'un `path` donnée et créer un couple avec pour identifient le `path`
* et comme `value` la valeur du `path`.
*
* Script:
* ### Example / Exemple
*
* #### Script / Scénario
*
* ```ini
* ; Import analytics plugin required to use "pluck"
* ; Importation du plugin analytique nécessaire pour utiliser "pluck"
* [use]
* plugin = analytics
*
* ; Using "pluck" with 'year' as path setttings instead of 'id' how is the default value
* ; Utilisation de "pluck" avec 'year' comme paramètres de path au lieux de la valeur par defaut qui et 'id'
* [pluck]
* path = year
*
* ```
*
* Output:
* #### Input / Entrée
*
* ```json
* [
* { "city": "tokyo", "year": 2000, "count": 1 },
* { "city": "paris", "year": 2001, "count": 2 },
* { "city": "london", "year": 2003, "count": 3 },
* { "city": "nancy", "year": 2005, "count": 4 },
* { "city": "berlin", "year": 2007, "count": 5 },
* { "city": "madrid", "year": 2009, "count": 6 },
* { "city": "stockholm", "year": 2011, "count": 7 },
* { "city": "bruxelles", "year": 2013, "count": 8 }
* ]
* ```
*
* #### Output / Sortie
*
* ```json
* [
* { "id": "year", "value": 2000 },
* { "id": "year", "value": 2001 },
* { "id": "year", "value": 2003 },
* { "id": "year", "value": 2005 },
* { "id": "year", "value": 2007 },
* { "id": "year", "value": 2009 },
* { "id": "year", "value": 2011 },
* { "id": "year", "value": 2013 }
* ]
* [
* { "id": "year", "value": 2000 },
* { "id": "year", "value": 2001 },
* { "id": "year", "value": 2003 },
* { "id": "year", "value": 2005 },
* { "id": "year", "value": 2007 },
* { "id": "year", "value": 2009 },
* { "id": "year", "value": 2011 },
* { "id": "year", "value": 2013 }
* ]
* ```
*
* @name pluck
* @param {String} [path=id] path to use form group by
* @returns {Object}
* @param {String} [path=id]
* <ul><li>path of the element who need to be extrated</li></ul>
* <ul><li>chemin de l'élément qui doit être extrais</li></ul>
* @returns {{
* id: String,
* value: Object
* }}
*/
export default function pluck(data, feed) {
if (this.isLast()) {
feed.close();
return;
}
let fields = this.getParam('path', 'id');
if (!Array.isArray(fields)) {
fields = [fields];
}

fields
.filter((k) => typeof k === 'string')
.map((key) => [key, get(data, key)])
.filter((x) => x[1])
.map((item) => ([item[0], (item[1] instanceof Array ? item[1] : [item[1]])]))
.reduce((prev, cur) => prev.concat(cur[1].map((x) => ([cur[0], x]))), [])
.forEach((item) => feed.write(core(item[0], item[1])));
feed.end();
}
export default pluck;
Loading
Loading