Memoization

Overview

To make best use of an effect run-time like Live, you need to understand its <Component> tree. How it renders and how it updates can have significant effects on performance.

The goal is to keep component re-rendering to a minimum, and keep the app fully responsive.

The big difference between Live and React is that there is no DOM at all. So the main question is how that model applies to something that works more like an executable program than a UI tree.

Imperative vs Reactive

In Use.GPU it's fine to build a stack like this:

<WebGPU> // GPU command queue
  <OrbitControls> // a UI controller
    <OrbitCamera>   // a View that emits GPU data
      // ...
        <Pass>          // a render encoder pass <- imperative!
          <UI>            // a UI system
            <Layout>        // a layout system
              // ...
                <Block>         // a <div>
                  <Inline>        // a <span>
                    <Text />        // a styled string

This is a pretty ambitious mixing of concerns. The nesting might seem weird, but follows the rules of data-dependency: the entire view depends on the camera, so it sits near the top.

Ultimately, the goal is to produce a series of drawing commands and send them to the GPU. The device handle is owned by <WebGPU> and is passed downstream via DeviceContext. But, React-like tree updates are sparse and only affect components whose props/state changed. So if every component executed draw calls immediately, then you would produce incomplete frames.

Therefor, all the drawing code has to be gathered up somehow. This could be the responsibility of <Pass>. But imagine if the contents of Text changed. In the React model, data flow is one-way: parents are oblivious to the state of children. <Pass> would not be able to tell that anything happened. If the change was caused downstream of it, it would not re-render the screen.

The solution is easy to explain. Given a tree in JSX notation:

<Pass>
  <UI>
    // ...
  </UI>
</Pass>

Then in Live it is actually possible for <Pass> to put code inside the </Pass> part. These are continuations, labelled Resume(Pass) in the Live inspector.

Instead of rendering to DOM, children like <Text /> can yeet arbitrary values back upstream. These are gathered using a map-reduce like arrangement, which is incremental, i.e. cached. Parents can only respond to these gathered values in their </Continuation>, not in the main <Component>.

The purpose of a component like <Pass> is then literally to only gather a list of lambdas, that is () => { … }. These can be called one after the other, without knowing what they do. This executes all the GPU-side per-frame code in one place, in original tree order.

Whenever a drawing command changes, it will yield a new lambda, be gathered by the nearest <Pass> (or <Compute>), and be used for the next draw. Drawing commands can be added/removed freely by the mounting/unmounting of child nodes.

Memoization and Signals

In Live, as in React, components can be memoized with memo. This means that a component will stop a render in its tracks if none of its props have changed. All children will be left untouched and their code will not run. The same applies to memoized continuations: they are only re-run if a yeet inside has changed.

This is a problem because GPU data is passed by reference. When a <Data> source updates, the StorageSource it produces is the same as before. Similarly, the position of an <OrbitCamera> is passed by reference too.

Generally, passing values by reference is unsafe in a React-like and will cause stale renders. If there is no explicit data dependency between e.g. the camera and the drawing components, other than one being nested in the other, then the view won't be updated at all if there is a memo in the way.

To address this, there are two complementary mechanisms:

• FrameContext, which is a trigger for all per-frame activity that requires an immediate CPU-side response. Subscribing to the frame context with usePerFrame will cause a component to re-render when the view parameters change, or a new animation frame is rendered inside a <Loop>.

• Data sources put an empty drawing command in the queue, to trigger a GPU-side response. In practice, this is optimized by rendering a <Signal />, which will re-run the drawing code without invalidating caches.

Reconciliation and Quoting

Data sources and compute shaders can be declared anywhere, and don't necessarily live inside a single rendering pass. But if some data changes, all draws downstream should re-run. This implies that the lambdas need to be gathered up at the root <WebGPU>, pulling from and passing through every single node in the tree. This would be highly inefficient, and would also reserve yeeting for a single purpose in most of the tree.

Instead, we reconcile drawing commands separately into a new sub-tree. This is similar to how React reconciles HTML into a new DOM tree. Except it just reconciles normal Live components. This is done by quoting inside a reconciler. This works the same way as quoting and unquoting in Lisp.

In Use.GPU, the first tree's components are only involved with setting up the draw calls, while the second tree is mainly yeeted lambdas that execute draw calls, i.e. <Yeet>{() => { … }}</Yeet>:

<WebGPU>
  <Reconcile> // The CPU-side tree
    <Data>
      <Signal /> // quote-yeets `undefined`
      <Pass>
        <PointLayer />
        <LineLayer />
      <Resume(Pass)> // quote-yeets the code to draw PointLayer & LineLayer
  <Resume(Reconcile)> // The GPU-side tree
    <Gather>
      <Unquote>
        <Yeet /> // empty signal - (does not actually appear in the tree)
        <Yeet>{() => { … }}</Yeet> // code from Resume(Pass)
    <Resume(Gather)> // gathers up all the draw calls and runs them

A <Signal /> is then a shorthand for a <Quote><Yeet /></Quote>. Quoting transplants it into the second sub-tree, where it will yeet undefined, i.e. no value. This causes the surrounding Resume(...) to be re-run with the same lambdas as before.

Implicit Memoization

Live also replicates one of React's least understood features, namely implicit memoization. This is important because Use.GPU encourages deep component trees, with lots of context providers sprinkled throughout.

Many mistakenly believe that deep component stacks are automatically bad for performance. But there is a crucial nuance: the freshness of the JSX matters. This is difficult but important to wrap your head around.

Given a rendered component stack such as:

<App>
  <FooProvider>
    <BarProvider>
      <BazProvider>

One aspect that is not visible here is "who rendered who".

e.g. Did <App> render…

return (
  <FooProvider>
    <BarProvider>
      <BazProvider>
        //...
)

Or just…

return (
  <FooProvider />
);

…so that <FooProvider> actually rendered <BarProvider> and <BazProvider>?

It matters because children is just a regular prop in JSX. The two cases are:

1. <App> constructs the entire JSX expression. This means all the children props are created there.

2. The children of <BarProvider> and <BazProvider> are only constructed by <FooProvider>.

Despite there being no visible difference in the resulting tree, the two cases update very differently.

When <FooProvider> changes state:

1. It will render the same props.children value that App gave it previously. <BarProvider> and its children will not re-render.

2. It will render new JSX instances for <BarProvider> and <BazProvider>, with new children props. Both will re-render.

If a component renders the exact same JSX object as before, React will not re-render that child, even if the child is not memoized.

So there is a hidden semantic in React: when you construct a JSX expression anew, you are requesting that those children be re-rendered unless memoized. But if you reuse a JSX expression from before, you are implicitly giving permission to ignore any unchanged children.

This means when a component re-renders the same {props.children}, they will be skipped unless they are explicitly subscribed to some context that changed. This is true for all components, not just context providers.

Live inspector

A carefully designed sandwich of providers and components can be much more efficient than most people realize. The trick is to build your deep stacks in a few central places that don't re-render often. It will accordeon out into large chunks of tree that are frequently skipped.

To get a feel for this, it is most useful to observe re-renders in the Live inspector. It will also highlight dependencies and "who-rendered-who" relationships: