tracked-built-ins built-in

[NullVoxPopuli]

Propose making tracked-built-ins built-in, which unblocks the implementations of #1000 (array) and #999 (hash)

Rendered

Summary

This pull request is proposing a new RFC.

To succeed, it will need to pass into the Exploring Stage, followed by the Accepted Stage.

A Proposed or Exploring RFC may also move to the Closed Stage if it is withdrawn by the author or if it is rejected by the Ember team. This requires an "FCP to Close" period.

An FCP is required before merging this PR to advance to Accepted.

Upon merging this PR, automation will open a draft PR for this RFC to move to the Ready for Released Stage.

Exploring Stage Description

This stage is entered when the Ember team believes the concept described in the RFC should be pursued, but the RFC may still need some more work, discussion, answers to open questions, and/or a champion before it can move to the next stage.

An RFC is moved into Exploring with consensus of the relevant teams. The relevant team expects to spend time helping to refine the proposal. The RFC remains a PR and will have an Exploring label applied.

An Exploring RFC that is successfully completed can move to Accepted with an FCP is required as in the existing process. It may also be moved to Closed with an FCP.

Accepted Stage Description

To move into the "accepted stage" the RFC must have complete prose and have successfully passed through an "FCP to Accept" period in which the community has weighed in and consensus has been achieved on the direction. The relevant teams believe that the proposal is well-specified and ready for implementation. The RFC has a champion within one of the relevant teams.

If there are unanswered questions, we have outlined them and expect that they will be answered before Ready for Release.

When the RFC is accepted, the PR will be merged, and automation will open a new PR to move the RFC to the Ready for Release stage. That PR should be used to track implementation progress and gain consensus to move to the next stage.

Checklist to move to Exploring

• The team believes the concepts described in the RFC should be pursued.
• The label S-Proposed is removed from the PR and the label S-Exploring is added.
• The Ember team is willing to work on the proposal to get it to Accepted

Checklist to move to Accepted

• This PR has had the Final Comment Period label has been added to start the FCP
• The RFC is announced in #news-and-announcements in the Ember Discord.
• The RFC has complete prose, is well-specified and ready for implementation.
  • All sections of the RFC are filled out.
  • Any unanswered questions are outlined and expected to be answered before Ready for Release.
  • "How we teach this?" is sufficiently filled out.
• The RFC has a champion within one of the relevant teams.
• The RFC has consensus after the FCP period.

[ef4]

Notes from first discussing of this proposal in RFC review:

• regardless of whether we decide to accept this change, it would be good to replace the tracked storage polyfill with a real implementation first.
• it does seem implied by the built-in array and hash helper RFCs that tracked implementations of those things would exist at a fairly low level (assuming they get implemented as glimmer keywords, glimmer itself would necessarily know about implementation of TrackedArray and TrackedObject).

[NullVoxPopuli]

tracked-storage-primitive in implementation here: emberjs/ember.js#20814

[NullVoxPopuli]

related to non-reactive entries for array and hash: #1070 (Array and Object)

[josemarluedke]

Great work on this RFC! I’d like to propose a utility function, such as reactive, that could create the appropriate tracked data structure based on the input type. This could simplify the process of creating tracked values and make the API more flexible.

For example, if you pass an object to the reactive function, it would return a TrackedObject. If you pass an array, it would return a TrackedArray. This would allow developers to more easily create tracked data structures without needing to manually specify the type each time.

import { reactive } from '@ember/reactive';

const trackedObj = reactive({ key: 'value' }); // returns TrackedObject
const trackedArr = reactive([1, 2, 3]); // returns TrackedArray

Additionally, I’d suggest reconsidering the naming of Tracked*. To better align with the language used in other frameworks and make the terms more intuitive, perhaps these could be renamed to Reactive*. This would emphasize the reactive nature of these structures and help make the API more consistent with other reactive libraries.

[NullVoxPopuli]

realized that i don't need tracked-storage-primitives or cells to implement this.

PR here, work in progress: glimmerjs/glimmer-vm#1713.

By skipping that intermediary user-land abstraction, the TrackedArray implementation would likely be even faster.

(implementation happening in glimmer-vm, so that glimmer-vm can implement keywords that build on these structures)

[NullVoxPopuli]

All tests are ported over to glimmer-vm now -- tho I (re)learned that glimmer-vm doesn't have each-in, so I skipped those tests for now.

Now that the infra is ported for quickly testing collection reactivity, porting over Object, Map, Set, WeakMap, WeakSet will be trivial.

[Eredrim]

Hello!
I love the idea and I think these new reactive structure would be really useful for many purposes. However, it's not clear for me how TrackedArray is different of EmberArray (not the mutable one). EmberArray is currently what I use to track arrays. I fear it might be complicated to maintain both structures and suggest to use one in place of another one.

[kategengler]

Hello! I love the idea and I think these new reactive structure would be really useful for many purposes. However, it's not clear for me how TrackedArray is different of EmberArray (not the mutable one). EmberArray is currently what I use to track arrays. I fear it might be complicated to maintain both structures and suggest to use one in place of another one.

EmberArray should not be used. TrackedArray is preferred and is the way forward. This was addressed in the original RFC for tracked-built-ins https://github.com/emberjs/rfcs/blob/master/text/0812-tracked-built-ins.md?plain=1#L40 This RFC is about making the external implementation internal.

[Eredrim]

EmberArray should not be used. TrackedArray is preferred and is the way forward. This was addressed in the original RFC for tracked-built-ins https://github.com/emberjs/rfcs/blob/master/text/0812-tracked-built-ins.md?plain=1#L40 This RFC is about making the external implementation internal.

Thanks for your answer and for clarification. Considering that, maybe you should consider deprecating EmberArray usage or at least update API documentation to signify TrackedArray should be used instead.

[kategengler]

Thanks for your answer and for clarification. Considering that, maybe you should consider deprecating EmberArray usage or at least update API documentation to signify TrackedArray should be used instead.

We are certainly working towards that.

[gossi]

Great work on this RFC! I’d like to propose a utility function, such as reactive, that could create the appropriate tracked data structure based on the input type. This could simplify the process of creating tracked values and make the API more flexible.

For example, if you pass an object to the reactive function, it would return a TrackedObject. If you pass an array, it would return a TrackedArray. This would allow developers to more easily create tracked data structures without needing to manually specify the type each time.

import { reactive } from '@ember/reactive';

const trackedObj = reactive({ key: 'value' }); // returns TrackedObject
const trackedArr = reactive([1, 2, 3]); // returns TrackedArray

Additionally, I’d suggest reconsidering the naming of Tracked*. To better align with the language used in other frameworks and make the terms more intuitive, perhaps these could be renamed to Reactive*. This would emphasize the reactive nature of these structures and help make the API more consistent with other reactive libraries.

I second that. Compared to other frameworks, which use "one" function to do that.

• Svelte:
  • $state()
• Vue:
  • ref()
  • reactive()
• Angular:
  • signal()
• React:
  • useState() (I guess, according to their official website - I'm so not into react)

And then ember:

• tracked()
• trackedObject()
• trackedArray()
• trackedMap()
• trackedWeakMap()
• trackedSet()
• trackedWeakSet()

where one might say, that ember is really trying hard not wanting to be used... and I wouldn't disagree.

My preference would be with one reactive() here (similar to $state()) - but potentially the idea from vue where in unsupported cases a second function with a proxy might be the better option (I don't have deep glimmer VM knowledge to judge the edge cases).

[NullVoxPopuli]

implementing a reactive() (or tracked()) isn't technically a big deal -- it's more an issue of specification -- how to we define what it adapts?
Do we say:

• primitives
• objects
• arrays
• maps / sets
• weak maps / sets

and that's it?
and by it, I mean, the absolute minimal app someone can make would end up shipping all 7 reactive data structures.

IF we say only primitives, objects, and arrays, what is the justification for omitting maps/sets and their weak variants? less used?

What do we say about the future planned reactive data structures? (which are planned under separate imports as well)

What do folks think?

[gossi]

my hunch was to accept anything "native" to js. Then I looked what svelte is accepting, which comes close I'd say: https://github.com/sveltejs/svelte/blob/main/packages/svelte/src/ambient.d.ts - I think that's what I'd probably expect (more like gut feeling here), dunno if svelte has an RFC for that.

My questions:

• On an API surface level, does it compete with cell() in A new reactive primitive: cell #1071 ?
  ➡️ see ref() vs reactive() in vue
• Will it be deeply reactive?
  ➡️ see $state() in svelte

[NullVoxPopuli]

Thanks for providing those links! that was helpful 💪

On an API surface level, does it compete with

I think what you're proposing does, but the main thing that this RFC is concerned about is public (not quite low-level) API -- what you're proposing can totally wrap these built in tracked-built-ins as well as cell -- that could even (and this is my preference) be a separate subsequent RFC -- because there are more reactive utilities to implement beyond just the 6 that this RFC is concerned with.
What you want in a general "I don't care what I'm using, nor do I care for any nuance in perf / memory-pressure / updating, I just want it to be reactive" utility is probably even more stuff, and probably some way to describe how to more generically handle reactive wrapping as svelte as done. These would be implementation details of said subsequent RFC.

Will it be deeply reactive?

no, that is out of scope for this RFC.

---

For a deep-do-everything utility, a new RFC could be written that builds on all this that is maybe exported from

import { tracked } from '@ember/reactive/all'; 
//                       or something short like
//                      '@ember/x';
//                      if, during implementation of tracked-built-ins,
//                      we figure out how to more generally implement all these utilities
//                      without shipping so many bytes

[gossi]

I think what you're proposing does, but the main thing that this RFC is concerned about is public (not quite low-level) API

yes, I'm speaking exactly that. What will we make public and thus define the experience for people.

Currently we have this narrative: "Use JS as you are used to. Use @tracked to make your code reactive and ember will take care of the rest..." but we are about to continue the story with "... but you failed making your code reactive bcz from the 87 reactive exports we provide you, you have chosen the wrong one".

I see this parallel to react's shouldComponentUpdate() and to put ember devs in a similar situation to actively think about updates again.

Which is why I consider this question important:

Will it be deeply reactive?

no, that is out of scope for this RFC.

Let's put this together with an example and two approaches to deal with it.

Example:

const customer = reactive({
  givenName: 'Luke',
  familyName: 'Skywalker',
  address: {
    planet: 'Tatooine',
    location: 'At my uncles farm'
  }
});

function moveToHooth(customer) {
  customer.address.planet = 'Hooth';
  customer.address.location = 'Rebell base';
}

... and somewhere in the template we render the address.

This example would work in svelte, because the API was designed to work towards the expectations of consumers. It wouldn't in ember right now as deep reactivity is explicitely excluded, though this is unknown for consumers at first and breaks the narrative of "just use JS".

We find ourself somewhere between correctness (given implementation details) and consumer expectation, for which I have two options:

1. "Use JS"

In this case deep reactivity is a given and works with the idea of "performance optimization comes from measuring at first, then apply a treatment" and gives you two functions/parameters:

• reactive(data) - makes everything deeply reactive
• reactive(data, proxy) - provide a second param for fine tuning your reactive approach

this idea is a bit with the assumption that most reactive data is ok to be deeply reactive (this is very much subjective by me), but provides an escape hatch for those working on data-heavy apps

2. Separate deep and non-deep reactivity

Make it explicit in the API what you get:

• reactive(data) - makes everything reactive (non-deeply)
• deepReactive(data) - makes everything deeply reactive

The learning curve is a bit higher and comes with a couple rounds of frustrations and negative experiences at first.

Additional Comments:

• It might seem like I shifted a little to deep reactivity, but this is what collections are about. Whether this is a list or a dict.
• "Correctness" about the API is knowing the internals, which consumers do not need to know
• Keep the API surface as minimal as possible
• Design a good narrative

Happy to hop on a call to discuss.

[NullVoxPopuli]

I think you've convinced me to write an RFC for the everything-reactive (even deeply) util -- I am curious what others think tho. Our learning store would then become:

• use reactive(/* this is deep */) or @reactive (also deep), for when you have unknown-shaped or deep data, and use the fine-grained utils directly if you know ahead of time what optimization you want to make -- and the thing we'd maybe yak shave is the import path of this util -- unless .. "it's probably fine" to have on @ember/reactive, because we're only talking about 2KB (gzip) or so anyway

thanks for the detailed write up!

[gossi]

Based on the discussion from yesterday, I think it was Ed who dropped the idea for the API I'm playing around with. Here are my blurbs:

import { reactive } from 'ember/reactive';

// casual usage
const user = reactive({
  givenName: 'Luke',
  familyName: 'Skywalker',
  address: {
    planet: 'Tatooine',
    location: 'At my uncles farm'
  }
});

// advanced usage
const heavySet = reactive.set(new Set());

which comes down to a statement above: "measure your own system and improve upon that" for which explicit/narrowed "sub-functions" exist, that actually carry on this narrative through its syntax.

Going into deep nesting, the two options from above persist, but let me precise them.

1) Opt-out: "Just" Use JS

deeply reactive by default.

• reactive(data) - makes everything deeply reactive (casual mode)
• reactive(data, function) - provide a second param for fine tuning your reactive approach (advanced mode)

providing a function as a second parameter gives authors fine control over where they want to make improvements and more importantly how - so they can tailor this to their situation. I dunno how that function would look like and it is rather complex (I guess?), but we can yield supporting utils into that function for people to use, eg.

const data = { /* ... */ };

const optimized = reactive(data, (data, key, utils) => {
  utils.something();
});

ofc, this would work in "advanced mode" too:

• reactive.set(data)
• reactive.set(data, function)

2) Opt-in: Ember optimizes your data

Make it explicit in the API what you get:

• reactive(data) - makes everything reactive (non-deeply)
• reactive.date(data) - makes everything reactive (non-deeply)

for deep reactivity this can come in two forms:

a) two functions

• deepReactive(data) - makes everything deeply reactive
• deepReactive.date(data) - makes everything deeply reactive

b) with param

• reactive(data, true) - makes everything deeply reactive
• reactive.date(data, true) - makes everything deeply reactive

now, you will receive embers deep reactivity, no control over it though.

Thoughts

From these two I'd actually consider the opt-out version to have advantages over the opt-in version:

• It has a good narrative: "Just use JS"
• Optimization can happen according to your data
• Optimization function can become quite a challenge to design and to use - but if done right and people love to use it, then it is :chefs-kiss:
• Ember's deep reactivity (as in opt-in) might not be the best optimization for consumers
• Less error-prone, but more performance-prone
  • what's the assumption we see more often here: "casual" usage or data-heavy apps

Questions

• How does reactive() compete with other reactive exports?

  • Can they be reactive.something() - ok, now it sounds like I found the hammer and make every problem look like a nail, but its a question worth asking

• Is reactive.something() tree-shakeable?

• Would it work like so:

   function moveToHooth(user) {
     user.address.planet = 'Hooth';
     user.address.location = 'Rebell base';
   }
   
   function calculateDistanceTo(user, anotherPlanet) {
     const planet = user.adress.planet;
   }

  or is it more like a cell then:

   function moveToHooth(user) {
     user.set({ 
       ...user.current,
       adress: {
         ...user.current.address,
         planet: 'Hooth',
         location: 'Rebell base'
       }
     })
   }
   
   function calculateDistanceTo(user, anotherPlanet) {
     const planet = user.value.adress.planet;
   }

blurbing end.

[scottmessinger]

I'm very in favor of deep reactivity being the default (with an escape hatch). I can't think of many times where shallow reactivity would be assumed or wanted (outside of performance reasons).

I like the casual/advanced usage @gossi spec'd out:

import { reactive } from 'ember/reactive';

// casual usage
const user = reactive({});

// advanced usage
const heavySet = reactive.set(new Set());

I would think shallow tracking would be:

const shallow = reactive.shallow({...})
const shallowSet = reactive.shallow.set(new Set());

I like the idea of passing a function as the second argument for specifying the equality function or other tweaks.