Routing and Parametrisation

In the High-level Model the user does not connect anything. Objects are placed, and the connections are derived from the placements by the Builder — and the latter not only compiles the structure laid out in the session and translates it into chains of Render Notes, but also applies a set of Rules while doing so. These rules are configurable.

A Placement attaches a media object into the session and carries a list of constraints. These resolve at least two dimensions: when the object appears, and where its output goes. The second is what concerns us here. A placement may carry a plug — that is a connection request naming an output designation. When no plug is given, the Builder searches the enclosing scopes, and finally applies a generic rule, matching by media stream type.

The counterpart is a claim: an element may assert that it constitutes such a named designation. Output and routing will only happen where a claim and a use coincide. A designation that nobody claims connects to nothing; a claim that nobody uses produces nothing. This symmetry is deliberate — it means that no part of the model can quietly become an output, and that routing can be expressed without any element needing to know who will listen.

Two consequences are worth drawing out at once.

Together the arrangement rules described here create a structural vocabulary. And whilst this structural vocabulary has been deliberately kept to a minimum, it can be applied wherever appropriate. A pipe is a linear chain of processing steps with an entrance and an exit — it emerges whenever effects and transitions attach to some other element by placement, and the chain is built in the order in which they are placed. Pipes occur as the local chains belonging to a clip, and as the global pipes of a timeline, which collect and mix like the subgroups and masters of a sound mixing desk. An effect stack may equally be attached at a fork, applying to everything within it, or at a subgroup bus, applying to the subgroup as a whole — and a subgroup may in turn be routed onward to another, by adding a plug into its placement.

Describing the signal flow in terms of these pipes imposes some defining structural properties: First and foremost, a pipe is linear. Anything that branches happens between pipes, not inside one. Only when the output from pipes is combined into another designation, a mixing or overlaying stage emerges. This is not a restriction rationalised afterwards, it is what makes the arrangement presentable in compact form. An effect stack or a strip in a mixer is the direct depiction of a linear chain.

Furthermore, a pipe is homogeneous in its stream type, and no conversion is allowed to happen within it. Conversions occur at the junctions between pipes, and only where the routing rules permit them: Wiring a monophonic effect into a pipe carrying an Ambisonics feed is not a small mismatch to be papered over — the channels of such a sound field are not separate signals but coefficients of one spatial encoding, and the operation of a monaural effect has no meaning in that context. It is reported as an error. Decoding that same field into a stereo bus however, intended for headphones, is a legitimate rendering operation, and it belongs at the junction where it can be seen.

This is also why a clip carrying image and sound is not a special construction. It anchors two pipes at the same point, and effects attached to it are sorted by the kind of media they apply to. There is no link to maintain between picture and sound, because there was never a division to bridge: mixed media multichannel material is the ordinary case, and material with a single stream is the exception.

Lumiera does translate everything into a graph of processing nodes. The question is whether that graph is what the user should work on, directly. The answer is no, and the reason is not distaste for node editors.

The vision document names a medium level of abstraction as a fundamental force of the design: the user manipulates placements, parameters, masks and layering directly — which is comparatively low-level and deliberately so — while being shielded from technical detail such as format conversion or access to individual channels. A node graph sits below that line. It exposes exactly what the principle sets out to conceal, and it invites the user to maintain by hand a structure that the Builder can derive. Feature-driven applications on the other hand, where one presses the button and the software decides autonomously, would operate way above that line of medium abstraction. And thus the structural vocabulary of Lumiera, the pipes, buses and placements are situated in that middle ground where a direct connection to the medium is still given.

And there is a practical argument, pointing into the same direction. Anyone who works seriously with node-based compositing will learn to impose discipline on the graph voluntarily — a grouping and naming, an arrangement to keep the subtrees self-contained — because an unstructured graph becomes unworkable at scale. The structure Lumiera prescribes is the structure practitioners adopt anyway. What is given up is the freedom to build arbitrary topologies; what is gained is that the arrangement stays legible, and that the same arrangement can be presented as a mixer rather than as a node tree.

However, where genuinely graph-like composition is desired, the model already provides the necessary expressive means: For one, the routing plugs can be added anywhere, even in the middle of a pipe, so that sidechains can be formed. And furthermore, a sequence bound into another sequence appears there as a virtual clip, with its own internal arrangement, behaving from the outside like any other media. The recursion is already present, and it isolates the complexity in the same way that practitioners do by hand.

Establishing a connection does more than join two points.
It creates degrees of freedom that did not exist before.

Connecting a monaural source into a spatial sound bus creates the possibility of placing that source in space: azimuth, elevation, distance, directivity. Placing one image on top of another will create an overlay mode and layering order. A connection carrying several multicam views may create a view selection. Which degrees of freedom appear is determined by the stream types meeting at that junction — and once those new dimensions exist, they are ordinary placement dimensions, resolved and inherited like any other.

In most software these forms of connectivity are provided, albeit in the form of distinct features, located in unrelated parts of the application, implemented separately, with asymmetric vocabulary. In Lumiera they are just one slot in the model whose available settings are derived from the type of what meets there: Summation with transparency is only the default way of overlaying two video streams, not the only one. Image compositing has a set of blend modes — the formulation introduced by the GIMP became a de-facto standard and was adopted by other applications. Masking provides the alpha input for these stream combinators; 3D images have a depth of stereoscopic window; sound has panning laws, and spatialisation schemes.

Which leaves an obvious question hanging. These degrees of freedom are control values — a position in space, an opacity, a blend mode — and they have to come from somewhere.

In Lumiera, a parameter is not a just stored value.
It is a data item that gets resolved when it is needed.

What a parameter carries is a disposition: a type together with a statement of how its value is to be obtained. The type has two aspects. Its domain fixes which values are meaningful at all — a value type, a range, a set of symbols, a structure — and it is the builder’s business to guarantee that whatever is delivered lies within it. The scale of the type fixes what a value means: decibels rather than a factor, seconds rather than frames. Scale is treated as part of the type and not as a matter of display, because a logarithmic quantity is a different quantity: it steps differently under the hand, and, as will be seen in a moment, it combines differently.

The second aspect is the provision of a parameter — where the value comes from. It may be a fixed value; it may be pulled from a function of time; it may come from a control being operated, or from a stream of external data. What matters is that the consumer of the parameter cannot tell which. A constant is simply the case where the function is trivial.

That is the point at which the first of the annoyances above dissolves. Several sources may contribute to one parameter, and the placement of each contribution states whether it replaces what the enclosing scope supplies or combines with it. Combining needs a joining operator, and the permissible operators follow from the domain — which is why scale belongs to the type: multiplying in a linear scale is adding in a logarithmic one, and a mechanism that treated scale as presentation would leave the question undefined. A contribution also states where it stands relative to the local one, because these operators do not generally commute.

So a gain may carry a value the user adjusts by hand, and a curve varying over time, and a correction applying to a whole group — all at once, without any of them seizing the control. And a scene-wide colour correction combined with a local adjustment is one transformation computed once, rather than two effects in series quantising the image twice.

This is also where Automation belongs, and it is deliberately mentioned this late. Automation is not a mechanism of its own; it is one provision among several — the case where the value is a function of time, typically defined mathematically as some kind of Spline, or step-wise interpolation. The habit of treating it as a separate faculty, a recording of knob movements replayed by machinery bolted on top of the mixer, is the source of much of the trouble it causes elsewhere: such a recording is attached to the surface of the application rather than to the material, and drifts against the material as soon as anything is edited. Here an automation source is a model object like any other, attached by placement, and therefore follows the clip or effect it belongs to whenever these move.

Taken together, the parameters form a second network spanning the processing, beside the flow of media data. Its practical significance is that it marks out what can change without rebuilding anything: the structure of the media graph is fixed when it is built, while the values flowing through the parameter network vary freely. A monitoring point that can be switched during playback is of this kind, or a mute toggle — the alternatives were built in advance, and the switching is a parameter.

This is a design in progress, and several of the concepts described here are firmer at the top than at the bottom.

Beyond that, three things are simply untouched: what takes over the role of muting or soloing a track once the global order of tracks is given up; how automation defined over nominal time behaves when material is retimed or nested; and how a model that routes by designation is to be exchanged with formats that assume explicit wiring between numbered tracks.