Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup analytics docs and test #388

Merged
merged 30 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
640c8e3
refactor: first draft of a small docs and test cleanup
AlasDiablo Dec 14, 2023
7deb2a0
test (analytics): add different data set to tune
AlasDiablo Dec 14, 2023
ae66698
test (analytics): fix typo
AlasDiablo Dec 14, 2023
93f1d31
Merge branch 'master' into docs-tests-analytics
AlasDiablo Jan 24, 2024
75f3521
docs (analytics): update tune doc
AlasDiablo Jan 24, 2024
eb7b3a3
docs (analytics): fix typo
AlasDiablo Jan 24, 2024
f8969d1
Merge branch 'master' into docs-tests-analytics
AlasDiablo Jan 24, 2024
5797e4f
docs (analytics): generify tests
AlasDiablo Jan 24, 2024
e95c6d3
docs (analytics): generated doc
AlasDiablo Jan 24, 2024
3084e28
test (analytics): fix test
AlasDiablo Jan 24, 2024
41d61e2
refactor (analytics): update value
AlasDiablo Jan 25, 2024
4182d47
docs (analytics): fix typo in json example
AlasDiablo Jan 25, 2024
e5399fe
docs (analytics): fix typo
AlasDiablo Jan 25, 2024
df570cd
refactor (analytics): update summing doc and add test
AlasDiablo Jan 25, 2024
dc721a2
test (analytics): add test how send badly structured data
AlasDiablo Jan 25, 2024
8ddd539
docs (analytics): add sort and statistics doc and fix some typo
AlasDiablo Jan 26, 2024
b93f5e4
Merge branch 'master' into docs-tests-analytics
AlasDiablo Jan 29, 2024
77bf346
docs (analytics): add slice doc
AlasDiablo Jan 29, 2024
d0f5093
docs: add Engine scope jsdoc type
AlasDiablo Jan 29, 2024
edb6743
docs (analytics): reformat segment doc
AlasDiablo Jan 29, 2024
25a8f62
docs (core): add missing private
AlasDiablo Jan 29, 2024
a776fe3
docs (analytics): fix typo
AlasDiablo Jan 30, 2024
164c119
docs (analytics): typo
AlasDiablo Jan 30, 2024
e0c1920
docs (analytics): add reducing documentation
AlasDiablo Jan 30, 2024
cff6018
docs (analytics): reformat pluck doc and fix reducing doc
AlasDiablo Jan 31, 2024
eaa00a5
docs (analytics): fix nitpick typo
AlasDiablo Jan 31, 2024
2947063
docs (analytics): reformat pair doc
AlasDiablo Jan 31, 2024
91d2410
refactor (analytics): remove code added for no reason
AlasDiablo Jan 31, 2024
67ee08d
refactor (analytics): reformat output doc
AlasDiablo Jan 31, 2024
a26ce90
Merge branch 'master' into docs-tests-analytics
touv Feb 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
568 changes: 363 additions & 205 deletions docs/plugin-analytics.md

Large diffs are not rendered by default.

568 changes: 363 additions & 205 deletions packages/analytics/README.md

Large diffs are not rendered by default.

123 changes: 81 additions & 42 deletions packages/analytics/src/slice.js
Original file line number Diff line number Diff line change
@@ -1,60 +1,99 @@

/**
* Take `Object` and throw the same object only if it is in the section of the
* stream between start and start + size. stream is numbered from 1
* Slice function see documentation at the end.
* This part of the doc is use for jsdoc typing
AlasDiablo marked this conversation as resolved.
Show resolved Hide resolved
* @private
* @param data {unknown}
* @param feed {Feed}
* @param ctx
*/
const slice = (data, feed, ctx) => {
if (ctx.isLast()) {
feed.close();
return;
}
const start = Number(ctx.getParam('start')) || 1;
const size = Number(ctx.getParam('size')) || 10;
const stop = start + size;
const index = Number(ctx.getIndex());

if (index >= stop) {
feed.close();
} else {
if (index >= start) {
feed.write(data);
}
feed.end();
}
[].slice();
};

/**
* Returns a copy of a section of a stream.
*
* ```json
* [{
* { id: 2000, value: 1 },
* { id: 2001, value: 2 },
* { id: 2003, value: 3 },
* { id: 2005, value: 4 },
* { id: 2007, value: 5 },
* { id: 2009, value: 6 },
* { id: 2011, value: 7 },
* { id: 2013, value: 8 },
* }]
* ```
* Renvoie une copie d'une section d'un flux.
*
* Script:
* ### Example / Exemple
*
* #### Script / Scénario
*
* ```ini
* ; Import analytics plugin required to use slice
* ; Importation du plugin analytique nécessaire pour utiliser slice
* [use]
* plugin = analytics
*
* [drop]
* ; Using "slice" with default settings
* ; Utilisation de "slice" avec les paramètres par défaut
* [slice]
* ; start = 1
* ; size = 10
*
* ```
*
* #### Input / Entrée
*
* ```json
* [
* { "id": 2023, "value": 12 },
* { "id": 2021, "value": 11 },
* { "id": 2019, "value": 10 },
* { "id": 2017, "value": 9 },
* { "id": 2013, "value": 8 },
* { "id": 2011, "value": 7 },
* { "id": 2009, "value": 6 },
* { "id": 2007, "value": 5 },
* { "id": 2005, "value": 4 },
* { "id": 2003, "value": 3 },
* { "id": 2001, "value": 2 },
* { "id": 2000, "value": 1 }
* ]
* ```
*
* Output:
* #### Output / Sortie
*
* ```json
* [
* { "id": 2001, "value": 2 },
* { "id": 2003, "value": 3 },
* ]
* [
* { "id": 2023, "value": 12 },
* { "id": 2021, "value": 11 },
* { "id": 2019, "value": 10 },
* { "id": 2017, "value": 9 },
* { "id": 2013, "value": 8 },
* { "id": 2011, "value": 7 },
* { "id": 2009, "value": 6 },
* { "id": 2007, "value": 5 },
* { "id": 2005, "value": 4 },
* { "id": 2003, "value": 3 }
* ]
* ```
*
* @name slice
* @param {Number} [start=0] start of the slice
* @param {Number} [size=10] size of the slice
* @param {Number} [start=1]
* <ul><li>the beginning index of the specified portion of the stream</li></ul>
* <ul><li>l'indice de début de la partie spécifiée du flux</li></ul>
* @param {Number} [size=10]
* <ul><li>the size of the specified portion of the stream</li></ul>
* <ul><li>la taille de début de la partie spécifiée du flux</li></ul>
* @returns {Object}
*/
export default function slice(data, feed) {
if (this.isLast()) {
feed.close();
return;
}
const start = Number(this.getParam('start')) || 1;
const size = Number(this.getParam('size')) || 10;
const stop = start + size;
const index = Number(this.getIndex());

if (index >= stop) {
feed.close();
} else {
if (index >= start) {
feed.write(data);
}
feed.end();
}
}
export default slice;
153 changes: 92 additions & 61 deletions packages/analytics/src/sort.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,83 +10,114 @@ const sorting = (arr, reverse = false) => {
}
return fastsort(arr).desc();
};

/**
* Sort function see documentation at the end.
* This part of the doc is use for jsdoc typing
* @private
* @param data {unknown}
* @param feed {Feed}
* @param ctx
*/
const sort = async (data, feed, ctx) => {
if (!ctx.store) {
const location = ctx.getParam('location');
ctx.store = createStore(ctx.ezs, 'sort', location);
ctx.table = [];
}
if (ctx.isLast()) {
const reverse = ctx.getParam('reverse', false);
const sorted = sorting(ctx.table, reverse);
await sorted.reduce(
async (previousPromise, cur) => {
await previousPromise;
return ctx.store.cut(cur)
.then((val) => feed.write(val));
},
Promise.resolve(),
);
await ctx.store.close();
feed.close();
} else {
const path = ctx.getParam('path', 'id');
const fields = Array.isArray(path) ? path : [path];
const keys = fields
.filter((k) => typeof k === 'string')
.map((key) => get(data, key));
const key = keys.length > 1 ? keys.join(',') : keys[0];
const idx = ctx.getIndex()
.toString()
.padStart(20, '0');
const hash = normalize(key)
.concat('~')
.concat(idx)
.replace(/\s/g, '~');
ctx.table.push(hash);
await ctx.store.put(hash, data);
feed.end();
}
};

/**
* Take all `Object` and sort them with dedicated key
* Sort incomming object base on the value of the given path.
AlasDiablo marked this conversation as resolved.
Show resolved Hide resolved
*
* ```json
* [{
* { id: 2000, value: 1 },
* { id: 2001, value: 2 },
* { id: 2003, value: 3 },
* { id: 2005, value: 4 },
* { id: 2007, value: 5 },
* { id: 2009, value: 6 },
* { id: 2011, value: 7 },
* { id: 2013, value: 8 },
* }]
* ```
* Trier les objets entrants sur la base de la valeur du chemin donné.
AlasDiablo marked this conversation as resolved.
Show resolved Hide resolved
*
* ### Example / Exemple
*
* Script:
* #### Script / Scénario
*
* ```ini
* ; Import analytics plugin required to use tune
AlasDiablo marked this conversation as resolved.
Show resolved Hide resolved
* ; Importation du plugin analytique nécessaire pour utiliser sort
* [use]
* plugin = analytics
*
* ; Using "sort" with default settings
* ; Utilisation de "sort" avec les paramètres par défaut
AlasDiablo marked this conversation as resolved.
Show resolved Hide resolved
* [sort]
* path = value
* reverse = true
* ; path = id
* ; reverse = false
*
* ```
*
* #### Input / Entrée
*
* ```json
* [
* { "id": 2013, "value": 8 },
* { "id": 2011, "value": 7 },
* { "id": 2009, "value": 6 },
* { "id": 2007, "value": 5 },
* { "id": 2005, "value": 4 },
* { "id": 2003, "value": 3 },
* { "id": 2001, "value": 2 },
* { "id": 2000, "value": 1 }
* ]
* ```
*
* Output:
* #### Output / Sortie
*
* ```json
* [
* { "id": 2013, "value": 8 },
* { "id": 2011, "value": 7 },
* { "id": 2009, "value": 6 },
* { "id": 2007, "value": 5 },
* { "id": 2005, "value": 4 },
* { "id": 2003, "value": 3 },
* { "id": 2001, "value": 2 },
* { "id": 2000, "value": 1 }
* ]
* [
* { "id": 2000, "value": 1 },
* { "id": 2001, "value": 2 },
* { "id": 2003, "value": 3 },
* { "id": 2005, "value": 4 },
* { "id": 2007, "value": 5 },
* { "id": 2009, "value": 6 },
* { "id": 2011, "value": 7 },
* { "id": 2013, "value": 8 },
* ]
* ```
*
* @name sort
* @param {String} [path=id] path to use for id
* @param {boolean} [reverse=false] reverser order
* @param {String} [path=id]
* <ul><li>path of the element used to as reference for the sort</li></ul>
* <ul><li>chemin de l'élément utilisé comme reference pour le trie</li></ul>
AlasDiablo marked this conversation as resolved.
Show resolved Hide resolved
* @param {boolean} [reverse=false]
* <ul><li>sort in ascending or descending order</li></ul>
* <ul><li>trier par ordre croissant ou décroissant</li></ul>
* @returns {Object}
*/
export default async function sort(data, feed) {
if (!this.store) {
const location = this.getParam('location');
this.store = createStore(this.ezs, 'sort', location);
this.table = [];
}
if (this.isLast()) {
const reverse = this.getParam('reverse', false);
const sorted = sorting(this.table, reverse);
await sorted.reduce(
async (previousPromise, cur) => {
await previousPromise;
return this.store.cut(cur).then((val) => feed.write(val));
},
Promise.resolve(),
);
await this.store.close();
feed.close();
} else {
const path = this.getParam('path', 'id');
const fields = Array.isArray(path) ? path : [path];
const keys = fields
.filter((k) => typeof k === 'string')
.map((key) => get(data, key));
const key = keys.length > 1 ? keys.join(',') : keys[0];
const idx = this.getIndex().toString().padStart(20, '0');
const hash = normalize(key).concat('~').concat(idx).replace(/\s/g, '~');
this.table.push(hash);
await this.store.put(hash, data);
feed.end();
}
}
export default sort;
Loading
Loading