Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load particles according to edm4hep #36

Merged
merged 39 commits into from
Jun 19, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
3cfbb4c
create initial types of collections defined in gsoc
brauliorivas May 29, 2024
186024e
loading function to check if types are correctly working
brauliorivas May 31, 2024
fbfb0ea
test suite for loadParticles and buildLoader
brauliorivas May 31, 2024
c7b270c
change example data for RecoParticle test
brauliorivas May 31, 2024
170de18
Merge branch 'main' into new-types
brauliorivas May 31, 2024
27d03ec
check if previews work
brauliorivas May 31, 2024
0a52737
small change
brauliorivas May 31, 2024
02392eb
another small change
brauliorivas May 31, 2024
13978a4
remove unnecessary exclamation mark
brauliorivas Jun 2, 2024
5f48b1d
eliminate try catch
brauliorivas Jun 3, 2024
00bb203
add version check + dynamic load of object
brauliorivas Jun 3, 2024
83a9bbd
update tests
brauliorivas Jun 3, 2024
3e64c41
types for RecoParticle relations including 1:1 and 1:many
brauliorivas Jun 4, 2024
f7a14c5
add links for types
brauliorivas Jun 5, 2024
904cd51
test for loading functions
brauliorivas Jun 6, 2024
ba9630c
test reconstruction
brauliorivas Jun 6, 2024
69f3fd6
remove a few things
brauliorivas Jun 7, 2024
a1afbfe
extract up-to-date types from edm4hep
brauliorivas Jun 9, 2024
1715a04
Merge branch 'new-types' into dynamic-types
brauliorivas Jun 9, 2024
cc58efa
load particles according to edm4hep.yaml definition
brauliorivas Jun 9, 2024
49d5e6e
improvements (advised) and create test a better test suite
brauliorivas Jun 10, 2024
0e30a99
add documentation to update-on-demand for dmx
brauliorivas Jun 11, 2024
1ff80b6
renaming from particle -> object
brauliorivas Jun 11, 2024
9ba9506
drop types for datatypes
brauliorivas Jun 11, 2024
57348db
correct file naming
brauliorivas Jun 11, 2024
c2b0742
typo in MODEl.md
brauliorivas Jun 12, 2024
c0c7387
Merge branch 'main' into dynamic-types
brauliorivas Jun 12, 2024
8e83913
Merge branch 'dynamic-types' of github.com:brauliorivas/dmx into dyna…
brauliorivas Jun 12, 2024
8e82181
[WIP] setup loading to later draw MCParticles
brauliorivas Jun 13, 2024
27ba065
fix typos in MODEL.md
brauliorivas Jun 13, 2024
e82ba4c
implement @tmadlener random color generator
brauliorivas Jun 13, 2024
9c0ee0a
minimal drawing for MCParticle
brauliorivas Jun 13, 2024
cd53b06
good working drawing
brauliorivas Jun 13, 2024
aeb04e2
dmx works with new loading functionality
brauliorivas Jun 13, 2024
fc45409
dmx completely using new loading function
brauliorivas Jun 14, 2024
bb0bc73
remove old files + fix filter function + fix lib
brauliorivas Jun 14, 2024
26597ee
new loading is done. [WIP] new test suite
brauliorivas Jun 14, 2024
0c1538b
add very basic tests for new loading logic
brauliorivas Jun 15, 2024
c37b2f3
check for correct object loading and links when reading a json file f…
brauliorivas Jun 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@
<div id="input-modal" class="modal-background">
<div class="modal-content">

<p>¡Welcome to <span id="logo">
<p>Welcome to <span id="logo">
<span id="logo-d">d</span><span id="logo-m">m</span><span id="logo-x">X</span>
</span>!</p>
</span></p>

<div id="input-message">
</div>
Expand Down
35 changes: 35 additions & 0 deletions js/types/dynamic.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import { createLink } from "./reconstruction.js";

export function loadMembers(object, data, membersToLoad) {
for (const member of membersToLoad) {
if (data[member.name] === undefined) continue; // load up to date data
tmadlener marked this conversation as resolved.
Show resolved Hide resolved
object.members[member.name] = data[member.name];
}
}

export function loadOneToOneRelations(object, data, relationsToLoad) {
object.oneToOneRelations = {};
for (const relation of relationsToLoad) {
const relationData = data[relation.name];
if (relationData === undefined) continue;

const link = createLink(object.id, object.id, relationData);
object.oneToOneRelations[relation.name] = link;
}
}

export function loadOneToManyRelations(object, data, relationsToLoad) {
object.oneToManyRelations = {};
for (const relation of relationsToLoad) {
const name = relation.name;
const relationData = data[name];

if (relationData === undefined) continue;
object.oneToManyRelations[name] = [];

for (const [index, relationElement] of relationData.entries()) {
const link = createLink(index + object.id, object.id, relationElement);
object.oneToManyRelations[name].push(link);
}
}
}
8 changes: 8 additions & 0 deletions js/types/edmobject.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
export class EDMObject {
constructor() {
this.members = {};
}

draw() {}
// more methods common to all particles
}
59 changes: 59 additions & 0 deletions js/types/load.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import { reconstructionTypes } from "./reconstruction.js";
import { datatypes } from "../../output/datatypes.js";
import {
loadMembers,
loadOneToOneRelations,
loadOneToManyRelations,
} from "./dynamic.js";

export function loadParticleType(collection, datatype, type) {
tmadlener marked this conversation as resolved.
Show resolved Hide resolved
const particles = [];

for (const [index, particle] of collection.entries()) {
const newParticle = new type();
newParticle.id = index;

loadMembers(newParticle, particle, datatype.members);

if (datatype.oneToOneRelations)
loadOneToOneRelations(newParticle, particle, datatype.oneToOneRelations);

if (datatype.oneToManyRelations)
loadOneToManyRelations(
newParticle,
particle,
datatype.oneToManyRelations
);

particles.push(newParticle);
}

return particles;
}

export function loadParticles(jsonData, event, particlesToLoad) {
tmadlener marked this conversation as resolved.
Show resolved Hide resolved
const eventData = jsonData["Event " + event];

const particles = {};

for (const type of particlesToLoad) {
const particlesType = Object.values(eventData).filter(
(element) => element.collType === `${type}Collection`
);

particlesType.forEach(({ collection }) => {
loadParticleType(collection, datatypes[type], reconstructionTypes[type]);
});
}

return particles;
}

const particlesToLoad = [
// subset of datatypes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am missing the edm4hep::MCParticle here (or I am missing something entirely?). It should be possible to load that through the same mechanism, right? That would also serve as a good "real world" test to see whether the scheme works to get to a useful representation in memory.

"edm4hep::Cluster",
"edm4hep::ReconstructedParticle",
"edm4hep::Vertex",
"edm4hep::Track",
"edm4hep::ParticleID",
];
53 changes: 53 additions & 0 deletions js/types/reconstruction.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
import { EDMObject } from "./edmobject.js";

export class Cluster extends EDMObject {
constructor() {
super();
}
}

export class ParticleID extends EDMObject {
constructor() {
super();
}
}

export class ReconstructedParticle extends EDMObject {
constructor() {
super();
}
}

export class Vertex extends EDMObject {
constructor() {
super();
}
}

export class Track extends EDMObject {
constructor() {
super();
}
}

export class GenericLink {
// we may create a specific class for each type if needed
constructor(id, from, to, collectionID) {
this.id = id;
this.from = from;
this.to = to;
this.collectionID = collectionID;
}
}

export function createLink(id, from, { collectionID, index }) {
return new GenericLink(id, from, index, collectionID);
}

export const reconstructionTypes = {
"edm4hep::Cluster": Cluster,
"edm4hep::ParticleID": ParticleID,
"edm4hep::ReconstructedParticle": ReconstructedParticle,
"edm4hep::Vertex": Vertex,
"edm4hep::Track": Track,
};
7 changes: 7 additions & 0 deletions js/types/units.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export const units = {
"charge": "e",
"mass": "GeV",
"time": "ns",
"momentum": "GeV",
"position": "mm", // more to come
};
2 changes: 2 additions & 0 deletions model/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*.yaml
*.yml
17 changes: 17 additions & 0 deletions model/download.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import https from "https";
import fs from "fs";

const URL =
"https://raw.githubusercontent.com/key4hep/EDM4hep/main/edm4hep.yaml";

const folder = process.argv[2] ?? "model";

const file = fs.createWriteStream(`${folder}/edm4hep.yaml`);
https.get(URL, (res) => {
res.pipe(file);

file.on("finish", () => {
file.close();
console.log("Download completed");
});
});
107 changes: 107 additions & 0 deletions model/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
import jsYaml from "js-yaml";
import fs from "fs/promises";

const fileRoute = process.argv[2] ?? "model/edm4hep.yaml";
const folderOutput = process.argv[3] ?? "output";

const definitionFile = await fs.readFile(fileRoute, "utf-8");
const definitionObject = jsYaml.load(definitionFile);

const components = definitionObject.components;
const datatypes = definitionObject.datatypes;

const configTypes = new Set([
"edm4hep::Cluster",
"edm4hep::ParticleID",
"edm4hep::MCParticle",
"edm4hep::Vertex",
"edm4hep::ReconstructedParticle",
"edm4hep::Track",
]);

const selectedTypes = Object.entries(datatypes).filter(([key, _]) =>
configTypes.has(key)
);

const componentsDefinition = {};
const datatypesDefinition = {};

class Component {}

class DataTypeMember {
constructor(type = null, name, unit = null) {
if (type) this.type = type;
this.name = name;
if (unit) this.unit = unit;
}
}

class Relation {
constructor(type = null, name) {
if (type) this.type = type;
this.name = name;
}
}

const parseString = (string) => {
return string
.split("//")[0]
.trim()
.split(" ")
.filter((substring) => substring !== "");
};

const parseDatatypesMembers = (members) => {
const newMembers = [];

for (const member of members) {
let [type, name, unit] = parseString(member);
if (unit) unit = unit.replace("[", "").replace("]", "");
if (type.includes("edm4hep::")) {
newMembers.push(new DataTypeMember(type, name, unit));
} else {
newMembers.push(new DataTypeMember(null, name, unit));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I have missed this bit before when I was talking about dropping the types. Do we need the type information for members that are components? (I.e. the things starting with edm4hep::? If yes, than we might as well just keep all the type information instead of selectively dropping stuff. If we don't need the type information for the components either, I think we can drop all of them here.

Checking whether we can fit the MCParticle into this dynamic scheme should give as a good indication of what is needed, I think.

Copy link
Member Author

@brauliorivas brauliorivas Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought to keep only types from objects that contain edm4hep:: because maybe when drawing relations, we may need to know towards what type of object the relation is. But you are right; currently it isn't necessary, and we may continue without them. Also, I've included MCParticle which loads seamlessly. Using the testing json file, I get this
Selection_005
when running the loading function, without error.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this also already used when I load a file in the viewer for displaying?

Copy link
Member Author

@brauliorivas brauliorivas Jun 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, not.

}

return newMembers;
};

const parseRelation = (relations) => {
return relations.map((relation) => {
const [type, name] = parseString(relation);
if (type.includes("edm4hep::")) {
return new Relation(type, name);
} else {
return new Relation(null, name);
}
});
};

selectedTypes.forEach(([name, values]) => {
const members = values["Members"] ?? false;
let parsedMembers;
if (members) parsedMembers = parseDatatypesMembers(members);
const oneToManyRelations = values["OneToManyRelations"] ?? false;
let parsedOneToManyRelations;
if (oneToManyRelations)
parsedOneToManyRelations = parseRelation(oneToManyRelations);
const oneToOneRelations = values["OneToOneRelations"] ?? false;
let parsedOneToOneRelations;
if (oneToOneRelations)
parsedOneToOneRelations = parseRelation(oneToOneRelations);

datatypesDefinition[name] = {
members: parsedMembers,
oneToManyRelations: parsedOneToManyRelations,
oneToOneRelations: parsedOneToOneRelations,
};
});

const output = `export const datatypes = ${JSON.stringify(
datatypesDefinition,
null,
2
)}`;

await fs.writeFile(`${folderOutput}/datatypes.js`, output);
Loading
Loading