Decide on a method for frontend-backend primitive type communication

[mtrberzi]

The frontend currently has hard-coded definitions of all the digital circuit primitives. Obviously this isn't a thing we want to keep for too much longer, especially as I'm developing a new backend that I really want to get integrated with the frontend, and might even be starting a third backend (neural nets) pretty soon, time permitting.

One of our initial ideas was to serialize a "standard library" for each backend that defined some primitive functions in the frontend and mapped them to objects in the backend. However, I've identified a few problems with that approach.

For the microfluidics backend, I need to be able to support an infinite family of nodes that are all subtypes of the same "abstract base node" but that have a varying number of ports. This is also a problem in the digital circuits backend when we want to deal with RAM and ROM primitives -- code can be generated for these devices with arbitrary address and data widths. And in both of these cases (and potentially in other backends as well, neural nets being another use case that I won't explore in detail here), these are all things we really want to be able to describe in a flexible way.

Now, clearly if there are an infinite number of potential devices, we can't serialize all of them in finite space. So, either we choose some arbitrary limit on connectivity, or we have to do something more powerful than serialization in order to tell frontends what primitives they can use in the backend.

One possibility is to allow the frontend to query the backend directly when it encounters an undefined function. The backend could then provide the frontend with a definition of the primitive function and an intermediate type definition for the NodeTypeValue (or possibly ConstraintTypeValue) corresponding to that primitive function The same could be done for intermediate ConnectionValues (ask the backend "what kind of connection do you use to go from port type X to port type Y?").

Obviously this requires that the frontend have some way to communicate with the backend, which could be done through the service interfaces defined in core (and controlled/coordinated by some higher-level driver process, like the "engine" which doesn't exist yet but will soon).

One caveat is how the function signature will appear in the frontend. In the case of a RAM, we may want the address and data ports to be expressed as tuples in the frontend, but as a collection of single-bit ports in the intermediate and backend. In this case the backend will need to know something about frontend types in order to generate these function signatures. (This means putting existing "frontend-only" types such as tuples and enums into core, so that both the frontend and backend can refer to them.)

[mtrberzi]

It's also worth mentioning that I talked to my supervisor about doing this and he agreed that being able to do this is essential to having a flexible and usable language. I'll go ahead with a proposed implementation in a day or two (once I get serialization cleaned up).

[lucaswoj]

I'll need some more time to collect my thoughts on this. I take some offense to the assertion that there is no way to serialized a representation of all supported types by a backend. I would like to spend more time looking at possible alternatives before creating a system where the frontend is directly querying running backend programs. 

For the case of variable width busses, I think an implementation of inferred-width arrays would be much more elegant. 

 - Lucas
—
Sent from Mailbox

On Wed, Oct 1, 2014 at 8:41 AM, Murphy Berzish [email protected]
wrote:

The frontend currently has hard-coded definitions of all the digital circuit primitives. Obviously this isn't a thing we want to keep for too much longer, especially as I'm developing a new backend that I really want to get integrated with the frontend, and might even be starting a third backend (neural nets) pretty soon, time permitting.
One of our initial ideas was to serialize a "standard library" for each backend that defined some primitive functions in the frontend and mapped them to objects in the backend. However, I've identified a few problems with that approach.
For the microfluidics backend, I need to be able to support an infinite family of nodes that are all subtypes of the same "abstract base node" but that have a varying number of ports. This is also a problem in the digital circuits backend when we want to deal with RAM and ROM primitives -- code can be generated for these devices with arbitrary address and data widths. And in both of these cases (and potentially in other backends as well, neural nets being another use case that I won't explore in detail here), these are all things we really want to be able to describe in a flexible way.
Now, clearly if there are an infinite number of potential devices, we can't serialize all of them in finite space. So, either we choose some arbitrary limit on connectivity, or we have to do something more powerful than serialization in order to tell frontends what primitives they can use in the backend.
One possibility is to allow the frontend to query the backend directly when it encounters an undefined function. The backend could then provide the frontend with a definition of the primitive function and an intermediate type definition for the NodeTypeValue (or possibly ConstraintTypeValue) corresponding to that primitive function The same could be done for intermediate ConnectionValues (ask the backend "what kind of connection do you use to go from port type X to port type Y?").
Obviously this requires that the frontend have some way to communicate with the backend, which could be done through the service interfaces defined in core (and controlled/coordinated by some higher-level driver process, like the "engine" which doesn't exist yet but will soon).

One caveat is how the function signature will appear in the frontend. In the case of a RAM, we may want the address and data ports to be expressed as tuples in the frontend, but as a collection of single-bit ports in the intermediate and backend. In this case the backend will need to know something about frontend types in order to generate these function signatures. (This means putting existing "frontend-only" types such as tuples and enums into core, so that both the frontend and backend can refer to them.)

Reply to this email directly or view it on GitHub:
#24

[mtrberzi]

Inferred-width arrays would actually be much better than this -- if you can
think of a way to get those into the intermediate, that would be great.

On Wed, Oct 1, 2014 at 10:02 PM, lucaswoj [email protected] wrote:

I'll need some more time to collect my thoughts on this. I take some
offense to the assertion that there is no way to serialized a
representation of all supported types by a backend. I would like to spend
more time looking at possible alternatives before creating a system where
the frontend is directly querying running backend programs.

For the case of variable width busses, I think an implementation of
inferred-width arrays would be much more elegant.

• Lucas
  —
  Sent from Mailbox

On Wed, Oct 1, 2014 at 8:41 AM, Murphy Berzish [email protected]
wrote:

The frontend currently has hard-coded definitions of all the digital
circuit primitives. Obviously this isn't a thing we want to keep for too
much longer, especially as I'm developing a new backend that I really want
to get integrated with the frontend, and might even be starting a third
backend (neural nets) pretty soon, time permitting.
One of our initial ideas was to serialize a "standard library" for each
backend that defined some primitive functions in the frontend and mapped
them to objects in the backend. However, I've identified a few problems
with that approach.
For the microfluidics backend, I need to be able to support an infinite
family of nodes that are all subtypes of the same "abstract base node" but
that have a varying number of ports. This is also a problem in the digital
circuits backend when we want to deal with RAM and ROM primitives -- code
can be generated for these devices with arbitrary address and data widths.
And in both of these cases (and potentially in other backends as well,
neural nets being another use case that I won't explore in detail here),
these are all things we really want to be able to describe in a flexible
way.
Now, clearly if there are an infinite number of potential devices, we
can't serialize all of them in finite space. So, either we choose some
arbitrary limit on connectivity, or we have to do something more powerful
than serialization in order to tell frontends what primitives they can use
in the backend.
One possibility is to allow the frontend to query the backend directly
when it encounters an undefined function. The backend could then provide
the frontend with a definition of the primitive function and an
intermediate type definition for the NodeTypeValue (or possibly
ConstraintTypeValue) corresponding to that primitive function The same
could be done for intermediate ConnectionValues (ask the backend "what
kind of connection do you use to go from port type X to port type Y?").
Obviously this requires that the frontend have some way to communicate
with the backend, which could be done through the service interfaces
defined in core (and controlled/coordinated by some higher-level driver
process, like the "engine" which doesn't exist yet but will soon).
One caveat is how the function signature will appear in the frontend. In
the case of a RAM, we may want the address and data ports to be expressed
as tuples in the frontend, but as a collection of single-bit ports in the
intermediate and backend. In this case the backend will need to know
something about frontend types in order to generate these function
signatures. (This means putting existing "frontend-only" types such as
tuples and enums into core, so that both the frontend and backend can refer

to them.)

Reply to this email directly or view it on GitHub:
#24

—
Reply to this email directly or view it on GitHub
#24 (comment)
.

[lucaswoj]

By the time they hit the intermediate, they should be expanded to normal arrays. The only challenge is clerical: representing a function signature with a variable-width array in the backend type specification. —
Sent from Mailbox

On Wed, Oct 1, 2014 at 7:27 PM, Murphy Berzish [email protected]
wrote:

Inferred-width arrays would actually be much better than this -- if you can
think of a way to get those into the intermediate, that would be great.
On Wed, Oct 1, 2014 at 10:02 PM, lucaswoj [email protected] wrote:

I'll need some more time to collect my thoughts on this. I take some
offense to the assertion that there is no way to serialized a
representation of all supported types by a backend. I would like to spend
more time looking at possible alternatives before creating a system where
the frontend is directly querying running backend programs.

For the case of variable width busses, I think an implementation of
inferred-width arrays would be much more elegant.

• Lucas
  —
  Sent from Mailbox

On Wed, Oct 1, 2014 at 8:41 AM, Murphy Berzish [email protected]
wrote:

The frontend currently has hard-coded definitions of all the digital
circuit primitives. Obviously this isn't a thing we want to keep for too
much longer, especially as I'm developing a new backend that I really want
to get integrated with the frontend, and might even be starting a third
backend (neural nets) pretty soon, time permitting.
One of our initial ideas was to serialize a "standard library" for each
backend that defined some primitive functions in the frontend and mapped
them to objects in the backend. However, I've identified a few problems
with that approach.
For the microfluidics backend, I need to be able to support an infinite
family of nodes that are all subtypes of the same "abstract base node" but
that have a varying number of ports. This is also a problem in the digital
circuits backend when we want to deal with RAM and ROM primitives -- code
can be generated for these devices with arbitrary address and data widths.
And in both of these cases (and potentially in other backends as well,
neural nets being another use case that I won't explore in detail here),
these are all things we really want to be able to describe in a flexible
way.
Now, clearly if there are an infinite number of potential devices, we
can't serialize all of them in finite space. So, either we choose some
arbitrary limit on connectivity, or we have to do something more powerful
than serialization in order to tell frontends what primitives they can use
in the backend.
One possibility is to allow the frontend to query the backend directly
when it encounters an undefined function. The backend could then provide
the frontend with a definition of the primitive function and an
intermediate type definition for the NodeTypeValue (or possibly
ConstraintTypeValue) corresponding to that primitive function The same
could be done for intermediate ConnectionValues (ask the backend "what
kind of connection do you use to go from port type X to port type Y?").
Obviously this requires that the frontend have some way to communicate
with the backend, which could be done through the service interfaces
defined in core (and controlled/coordinated by some higher-level driver
process, like the "engine" which doesn't exist yet but will soon).
One caveat is how the function signature will appear in the frontend. In
the case of a RAM, we may want the address and data ports to be expressed
as tuples in the frontend, but as a collection of single-bit ports in the
intermediate and backend. In this case the backend will need to know
something about frontend types in order to generate these function
signatures. (This means putting existing "frontend-only" types such as
tuples and enums into core, so that both the frontend and backend can refer

to them.)

Reply to this email directly or view it on GitHub:
#24

—
Reply to this email directly or view it on GitHub
#24 (comment)
.

---

Reply to this email directly or view it on GitHub:
#24 (comment)