Layout and UI

Overview

The @use-gpu/layout package provides typical 2D layout as found in HTML and other UI toolkits.

It uses SDF shaders to render fully scalable vector graphics and text on the GPU.

It implements a subset of the CSS box model, using the same terminology:

• <Absolute> for absolute positioning
• <Block> elements for stacking
• <Inline> elements for paragraphs
• <Text> for styled font spans
• <Flex> for flex box (X or Y)
• <Overflow> for scrolling
• <Element> for generic rectangles / images
• <Inline> can contain inline <Block> elements
• <Transform> for transforms and masking

This is capable of doing every day layouts, with a much simpler set of nouns than HTML.

Center text inside a centered box, horizontally and vertically:

return (
  <Flat>

    <Pass overlay>
      <UI>

        <Flex width="100%" height="100%" align="center">
          <Flex width={500} height={150} fill="#3090ff" align="center">

            <Inline>
              <Text weight="black" size={48} color="#ffffff">-~ Use.GPU ~-</Text>
            </Inline>

          </Flex>
        </Flex>

      </UI>
    </Pass>

  </Flat>
);

Layouts are usually placed inside a <Flat> view, and will typically be rendering an overlay <Pass>.

Elements can have images and even shaders applied to them. This makes them excellent for both content and debug overlays.

These elements are rendered by <UI> into <UIRectangles> layers. These can render large amounts of text and shapes in a handful draw calls.

Layouts can be inspected using the Live inspector, similar to browser dev tools.

The layout system is extensible and reusable: @use-gpu/present uses it as the basis for its slides.

Limitations

• Layouts can have scrollable views, but are otherwise non-interactive. This is TBD.
• Looks best when using a <LinearRGB> target for gamma-correct RGB blending.
• Text is limited to left-to-right and does not support complex scripts.

(Can't build web UIs with this yet.)

Styling

There is no CSS-analogue: every element has its own style properties.

Properties do accept CSS colors, and include CSS-like shorthands.

e.g. margin={10} is the same as margin={[10, 10, 10, 10]}

Absolute elements can specify:

• left, top, right, bottom, width, height
• pixels (number) or percentages '100%'

Containers can specify:

• direction: x or y stacking
• border, radius, padding and margin
• number or [x, y] or [left, top, right, bottom]

Block-like elements can specify:

• width, height and aspect ratio
• grow and shrink
• stroke and fill
• image: TextureSource + alignment

Inline elements can specify:

• weight, size, color, lineHeight
• detail for SDF quality
• family

Flex boxes can specify:

• align content (start, center, justify-start, evenly, …)
• align shorthand for [alignX, alignY]
• anchor items on line
• gap

Make "styled components" yourself:

const StyledH1 = (props) => <Inline margin={[0, 10]}><Text size={32} weight="bold" {...props} /></Inline>;

Fonts

• Use <FontLoader> to load .ttf fonts
• Font stack with fallback fonts
• Emoji .png on-demand

Layout Fit

A <Layout> starts with the origin in the top-left and 100% width/height in logical pixel units.

If the current view's Y is up instead of down, a Y-flip is automatically applied.

• All root elements are positioned absolutely.
• To stack elements vertically, use a <Block> as a container.
• To set up a scrollable view, use an <Absolute> with an <Overflow> inside.

Data Flow

The layout process proceeds in several steps of tree reduction-and-expansion.

These apply outside-in and inside-out constraints as a one-way data flow.

• In -> Out: Children report their minimum and maximum width/height + desired flex
• Out -> In: Parents compute the target size for children to fit into
• In -> Out: Children lay out their contents and report their final size
• Out -> In: Parents align and place their children

Like CSS, this does not use a constraint solver. Children may refer to a parent's width or height only if that dimension is already pre-determined.

Most elements have a snap property to control snapping to whole logical pixels.

Shape Aggregation

Each laid-out shape is a <UIRectangle> (box) or <Glyphs> (text). These will emit a UIAggregate, a pure data structure that describes 1 to N rectangles with their attributes.

These are gathered up and emitted by <Layout> towards <UI>. <UI> will group all the shapes into distinct layers, each a <UIRectangles> (plural).

Each such layer is drawn using a single draw call. Elements may be re-ordered if there is an existing layer to fit into, but only if their relative z-order is still respected. For this, the aggregator checks element bounding boxes for non-overlap.

Elements with unique shaders applied will get their own unique draw call and layer.

Rendering and Scaling

The rendering is resolution independent: when the viewport changes scale, the same draw calls still produce crisp output. When zoomed out, thin edges will correctly anti-alias as well. When zoomed in, SDF text will eventually get blobby, unless you increase the detail.

Scrolling is handled directly on the GPU, by shifting the underlying transform matrix. Viewports are clipped off using shaders as well.

This means that all the content is always being drawn, even that which is off-screen. To mitigate this, clipping is done early in the vertex shader for fully invisible rectangles. Partially visible rectangles are clipped in the fragment shader.

Using React-style virtualization of longer content seems like a viable solution here, which remains to be explored.

Extensibility

The code for layout is inside the <Components> themselves. <Layout> simply gathers and calls.

Each renders a nested <BoxLayout> or <InlineLayout>, using whatever logic it wants to produce the boxes or lines.

This means that it's possible to extend the model with e.g. <Grid> or <Table> without changing the core.

Similarly, <Overflow> is not actually an element, but rather a pseudo-element that attaches to its parent's box. It can be used to make anything scrollable.