The new emerging NLE for GNU/Linux

What follows is a summary of Lumiera’s Data Handling Backend

This is the foundation layer responsible for any high performance or high volume data access. Within Lumiera, there are two main kinds of data handling:

  • The Session and the object models manipulated through the GUI are kept in memory. They are backed by a storage backend, which provides database-like storage and especially logging, replaying and “Undo” of all ongoing modifications..

  • Media data is handled frame wise — as described below.

The backend uses memory mapping to make data available to the program. This is somewhat different to the more common open/read/write/close file access, while giving superior performance and much better memory utilization. The data backend must be able to handle more data than will fit into the memory or even address space on 32 bit architectures. Moreover, a project might access more files than the OS can keep open simultaneously, thus the for Files used by the Backend, it needs a FilehandleCache to manage file handle dynamically.

Which parts of a file are actually mapped to physical RAM is managed by the kernel; it keeps a FileMapCache to manage the FileMaps we’ve set up. In the End, the application itself only requests Data Frames from the Backend.

To minimize latency and optimize CPU utilization we have a Prefetch thread which operates a Scheduler to render and cache frames which are expected to be consumed soon. The intention is to manage the rendering just in time.

The prefetcher keeps Statistics for optimizing performance.

Accessing Files

FileDescriptor is the superclass of all possible filetypes, it has a weak reference to a FileHandle which is managed in within the FilehandleCache. On creation, only the existence (when reading) or access for write for new files are checked. The FileDescriptor stores some generic metadata about the underlying file and intended use. But the actual opening is done on demand.

The content of files is memory mapped into the process address space. This is managed by FileMap entries and a FileMapCache.

File Handles

A FilehandleCache serves to store a finite maximum number of FileHandles as a MRU list. FileHandles are managed by the FilehandleCache; basically they are just storing the underlying OS file handles and managed in a lazy/weak way, (re)opened when needed and aging in the cache when not needed, since the amount of open file handles is limited aged ones will be closed and reused when the system needs to open another file.

File Mapping

The FileMapCache keeps a list of FileMaps, which are currently not in use and subject of aging. Each FileMap object contains many Frames. The actual layout depends on the type of the File. Mappings need to be page aligned while Frames can be anywhere within a file and dynamically sized.

All established FileMaps are managed together in a central FileMapCache. Actually, FileMap objects are transparent to the application. The upper layers will just request Frames by position and size. Thus, the File entities associate a filename with the underlying low level File Descriptor and access


Frames are the smallest datablocks handled by the Backend. The application tells the Backend to make Files available and from then on just requests Frames. Actually, those Frames are (references to) blocks of continuous memory. They can be anything depending on the usage of the File (Video frames, encoder frames, blocks of sound samples). Frames are referenced by a smart-pointer like object which manages the lifetime and caching behavior.

Each frame referece can be in one out of three states:


the backing FileMap is checked out from the aging list, frames can be read


the backing FileMap is checked out from the aging list, frames can be read and written


the FileMap object is checked back into the aging list, the frame can’t be accessed but we can try to transform a weak reference into a readonly or readwrite reference

Frames can be addressed uniquely whenever a frame is not available. The backend can’t serve a cached version of the frame, a (probably recursive) rendering request will be issued.


There are 2 important points when we want to access data with low latency:

  1. Since we handle much more data than it will fit into most computers RAM. The data which is backed in files has to be paged in and available when needed. The Prefetch Thread manages page hinting to the kernel (posix_madvise()..)

  2. Intermediate Frames must eventually be rendered to the cache. The Backend will send Renderjobs to the Scheduler.

Whenever something queries a Frame from the backend it provides hints about what it is doing. These hints contain:

  • Timing constraints

    • When will the Frame be needed

    • could we drop the request if it won’t be available (rendered) in-time

  • Priority of this job (as soon as possible, or just in time?)

  • action (Playing forward, playing backward, tweaking, playback speed, recursive rendering of dependent frames)

  • The Backend will try to render related frames in groups.

  • This means that following frames are scheduled with lower priority.

  • Whenever the program really requests them the priority will be adjusted.