Lumiera
The new emerging NLE for GNU/Linux
State Idea
Date Mi 11 Mai 2011 17:53:16 CEST
Proposed by Ichthyostega <prg@ichthyostega.de>

The main service of the Renderengine is to deliver a stream of calculations bound by timing constraints. The parameters of this delivery can be reconfigured

Try to start the integration and overall design of the Lumiera Render Engine. Point of reference is the functionality other parts of the application are relying on.

Description

The Engine is driven by the Player subsystem and draws on the low-level Model (Render nodes network) for all local parameters and control data. The goal is to deliver all the typical playback and rendering operations commonly found in NLE applications (functional requirements). Moreover, this functionality shall be delivered in a robust and reliable fashion, while making optimal use of the available I/O bandwidth and computation power (non-functional requirements).

Requirements Specification

functional

  • simple calculation stream

  • with or without defined end point

  • deliver to fixed output buffer(s) with high timing precision

  • ability to deliver individual data chunks (e.g. single frames)

  • “free wheeling” operation for maximum calculation throughput

  • throttled operation using idle calculation or bandwidth resources

  • streams with regular stepping and arbitrary block sizes

  • looping, seamless chaining of calculation streams with ongoing timing

  • ability to pause and to change / combine any of the above any time

  • ability to abort or change, providing reliable feedback on completion

  • ability to relocate (shift in time) parts of an ongoing calculation stream

  • support for chase-and-lock synchronisation

non-functional

  • protection against overload and deadlocks

  • gracious degradation in case of problems

  • maintain a configurable quality-of-service level

  • utilise precisely the resources actually available

Functionality description in detail

→ see the Engine/Interface overview for a description of the involved entities and for definitions for common terms.

Definitions

Calculation Stream

A series of similar but parametrised calculations, bound to deliver results in sequence and in accordance to timing constraints

Timed Delivery

Calculation result data has to be delivered within a fixed time interval immediately preceding the delivery deadline, so it can be consumed by an (possibly external) output process without the need for further buffering

Output Slot

An abstracted facility receiving calculated data; including a scheme to organise the output buffers, which get handed over to an independent thread running in parallel, or maybe even to an external process

Operation description

When creating a calculation stream, the exit node and an already opened output slot are provided, plus the timing parameters (frame duration, stepping interval length, optional start and/or endtime)

A free wheeling calculation stream is created in a similar fashion, just without timing constraints on the output delivery; i.e. just the output slot is parametrised differently. This invocation is used to create a “mixdown” or “final render” to be saved into an output file.

A background rendering mandate is created analogous, but without providing an output slot. Rather, it is expected that the engine will cache the generated data internally.

When calculation stream definitions are chained, the follow-up calculation stream is expected to be delivered seamlessly after the preceding stream, without interrupting the output timings.

Looping is a special kind of chained calculations, where the same segment is delivered continuously. But note, the loop boundaries are not necessarily aligned with the frame spacing or the output timing requirements.

For supporting chase-and-lock, the engine needs a mechanism to follow an externally provided synchronisation goal, without altering the output delivery timings. Obviously, we need to build in a strategy for handling this problem (because the solution is bound to be different for different kinds of media).

The engine can expect the output slot to support de-clicking or flicker protection — yet the engine needs to signal precisely when this is necessary

Quality of service

The Engine is expected to provide different quality-of-service classes, which are requested as part of the definition parameters for a calculation stream.

  • SYNC_PRIORITY means to keep up to the delivery requirements, even if this means failing to deliver data altogether.

  • PERFECT_RESULT means to deliver data perfect up to the definition, even if this means violating the timing constraints.

  • COMPROMISE allows the engine to take some shortcuts in order to deliver an roughly satisfactory behaviour. Likely there will be multiple classes of compromise.

The quality of service is partially implemented directly by the engine and partially passed on as parameter to the individual node invocations. For example, the engine might decide to switch down to proxy media, while actually the node network will perform the actual switch and reconfiguration.

The quality of service could be implemented as a strategy, to be consulted at various decision points. The above cases would then be just some preconfigured default strategies.

Reconfiguration and abort

There needs to be a separate “control channel” to cause various reconfigurations during an ongoing calculation process. With the exception of the output slot, all parameters defining an calculation stream might be changed on-the-fly — including the possibility to abort calculation altogether.

The engine is not required to react on such change requests immediately or synchronously. The goal is rather to integrate such changes seamlessly. Yet we require…

  • a guarantee that the change request is observed within some tolerance interval (i.e. we may block waiting on the change to happen, without risking a deadlock)

  • a reliable feedback after the change has happened, by invoking a response signal (functor/callback provided with the change request)

  • a guarantee not to proceed with the original setup after this signalling (read: after receiving this feedback, resources required only by the initial setup may be deallocated)

Especially note that the following things might be changed in the middle of an ongoing calculation:

  • timing parameters of the calculation stream (frame durations, stepping interval)

  • start and end time

  • splitting and chaining of calculation streams (e.g introducing jumps)

  • adjust the looping boundaries

  • toggle paused state

  • change the exit node to use for pulling

  • relocate the nominal time position of parts of the calculation stream; especially we expect already calculated and cached data to be re-labeled

  • invalidate parts of the (nominal) time axis, forcing recalculation

  • abort individual calculation streams without interfering with others.

Tasks

  • analyse requirements of the player subsystem (✔ done)

  • determine further informations needed during calculation WIP

  • find out about timing requirements and constraints in detaill TODO

  • define the interface functions in detail TODO

  • prepare a test fixture with mock-up calculations TODO

  • implement the invocation backbone with stubbed functionality TODO

Discussion

Pros

Cons

The requirements placed on life changes are quite high

Alternatives

Rationale

The expectations for the playback and render functionality of a NLE are pretty much set. There isn’t much room for reducing functionality. So the goal for this RfC is to precisely define the inevitable and break it down into tangible functionality on the implementation level.

Comments

Discussed in the May developers meeting. Seems to be basically acceptable. Cehteh proposed some small adjustments:

  • making the QualityOfService rather a strategy to be queried

  • treating the rescheduling a bit separate from the other changes, because that is very common and needs to be performant.

  • introducing a separate scheduler/queue for time scheduled tasks, like with rater soft realtime requirements 

    So 15 Mai 2011 00:55:24 CEST Ichthyostega <prg@ichthyostega.de>

Back to Lumiera Design Process overview