Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add example w/ semantic decoding w/ neural field #53

Open
maedoc opened this issue Dec 7, 2023 · 2 comments
Open

Add example w/ semantic decoding w/ neural field #53

maedoc opened this issue Dec 7, 2023 · 2 comments
Labels
help wanted Extra attention is needed question Further information is requested

Comments

@maedoc
Copy link
Member

maedoc commented Dec 7, 2023

This is a use case brainstormed during the brain hack in Marseille w/ someone who's name is ftm forgotten.

  • acoustic model turns speech into a set of signals
  • signals provided as stimulation to part of a neural field
  • local & nonlocal connectivity (and perhaps even higher order coupling) do information processing
  • a decoder reads out semantic content a la sentence embedding

If instead of neural field it was a transformer, then the models already exist (e.g. whisper). The question is how to do this with from computational neuroscience first principles.

@maedoc maedoc added help wanted Extra attention is needed question Further information is requested labels Dec 7, 2023
@i-Zaak
Copy link

i-Zaak commented Dec 7, 2023

This is very similar in spirit to the D1.9 demonstrator: https://wiki.ebrains.eu/bin/view/Collabs/sga3-d1-9-poc-simulations-for-validity

@maedoc
Copy link
Member Author

maedoc commented Dec 7, 2023

Yeah I meant to ask you to post a link.. thanks!

These kinds of examples fit well within the JAX ecosystem since the NN training can be done simultaneously with the VB model training.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants