weaving-pattern-builder.hpp File Reference

Construction kit to establish an invocation scheme for media calculations. More...

Go to the source code of this file.

Description

Construction kit to establish an invocation scheme for media calculations.

Adapters and configuration is provided to invoke the actual media processing function in accordance to a wiring scheme as implied by the signature of the bound function.

• the function takes one to three arguments
• these are related to the parameters, the input and the output (always in that order)
• the specification of at least one output buffer is mandatory
• a function may omit input and / or the parameter «slot»
• multiple items of the same kind (output, input, parameter) can be packaged into a heterogeneous tuple, or given as an array of identically typed elements; yet a single value can be accepted directly as function argument.
• input/output buffers are recognisable as pointers, while parameters are value data.
• pointers and parameter values are typed, which is used internally to ensure passing the right value to the corresponding item and to ensure suitable memory allocations.

  Note

  steam::engine::Turnout mixes-in the steam::engine::MediaWeavingPattern, which in turn inherits from a FeedManifold given as template parameter. So this constitutes an extension point where other, more elaborate invocation schemes could be integrated.

Preparing a FeedManifold and handling invocation parameters

Detection of the processing function signature with all possible variations as detailed above is the responsibility of the FeedManifold template. For each distinct signature, a suitable data layout is generated, including storage to hold the processing-functor itself (which is embedded as a clone-copy to expose the actual invocation to the optimiser in the C++ compiler). The WeavingBuilder defined here is used to build a Port implementation and thus a specific »Weaving Pattern«, which — at the actual Node invocation — will in turn build the concrete FeedManifold instance into local stack memory. For this reason, the Port can be understood as the Level-1 builder, whereas the Port / Weaving Builder is classified as Level-2 and a processing and link-builder operating on top of the former is designated as Level-3.

The actual type of the FeedManifold, including all the specifics of the data layout, becomes embedded into the Port implementation (≙Weaving Pattern) by means of a FeedPrototype instance. Furthermore, a parameter-functor can be installed there, to generate actual parameter data whenever the FeedPrototype generates a new FeedManifold instance for the next render invocation. The parameter data (and a copy of the processing-functor) is stored alongside in this generation step, and thus available in local stack memory during an extended (possibly recursive) render invocation sequence.

Invocation parameters are a crucial ingredient for each invocation, yet the responsibility for the parameter-functor to produce these parameters lies in a different part of the system than the responsibility for configuring the processing functor. The reason is simply that the setup of actual parameters is an essential part of the user's work on the edit in the Session. The control flow for parameter thus traces back into the session, while on the other hand the processing-functor must be configured by an external media-library adapter Plug-in. So this creates the challenge that in actual use the PortBuilder will be passed through several realms. Firstly, the external library binding will be invoked to set up a processing-functor, and then, in a separate step, the same PortBuilder instance, unfinished at that point, will be passed to the code responsible for configuring parameters and Parameter Automation. Only after that, the terminal builder operation WeavingBuilder::build() will be invoked, and the control flow in the Lumiera Builder subsystem proceeds to outfitting the next Render Node. This intricate sequence of configuration step translates into the necessity to build the FeedPrototype first in its basic form, without a parameter-functor. The second configuration step performed later will then have to re-shape the FeedPrototype to add a parameter-functor. This amounts to a move-copy, thereby changing the FeedPrototype's template arguments to the full signature, including the type of the parameter functor. In this final shape, it can be integrated into a Turnout instance and dropped off into the PatternData, which is used to record configuration for the actual storage allocation and node generation step performed later.

Interplay of NodeBuider, PortBuilder and WeavingBuilder

The steam::engine::WeavingBuilder defined here serves as the low-level builder and adapter to prepare the wiring and invocation. The builder-API allows to setup the wiring of input and output-»slots« and control some detail aspects like caching. However, without defining any connections explicitly, a simple 1:1 wiring scheme is employed

• each input slot of the function gets an input buffer, which is filled by pulling (i.e. invoking) a predecessor node (a so called »lead«).
• for each output slot a buffer is allocated for the processing function to drop off the calculated media data
• only one of these output buffers is used as actual result, while the other buffers are just discarded (but may possibly be fed to the frame cache).

Each Processing Node represents one specific processing functionality on a logical level; yet such a node may be able to generate several „flavours“ of this processing, which are represented as ports on this node. Actually, each such port stands for one specific setup of a function invocation, with appropriate wiring of input and output connections. For example, an audio filtering function may be exposed on port-#1 for stereo sound, while port-#2 may process the left, and port-#3 the right channel in isolation. It is entirely up to the library-adapter-plug-in to decide what processing functions to expose, and in which flavours. The WeavingBuilder is used to generate a single Turnout object, which corresponds to the invocation of a single port and thus one flavour of processing.

At one architectural level above, the NodeBuilder exposes the ability to set up a ProcNode, complete with several ports and connected to possibly several predecessor nodes. Using a sequence of NodeBuilder invocations, the processing node graph can be built gradually, starting from the source (predecessors) and moving up to the exit nodes, which produce the desired calculation results. The NodeBuilder offers a function to define the predecessor nodes (also designated as lead nodes), and it offers an entrance point to descend into a PortBuilder, allowing to add the port definitions for this node step by step.

On the implementation level, the PortBuilder inherits from the NodeBuilder and embeds a WeavingBuilder instance. Moreover, the actual parametrisations of the NodeBuilder template are chained to create a functional data structure. This intricate setup is necessary because the actual data structure of the node graph comprises several small descriptor arrays and interconnected pointers, which are all placed into consecutive chunks of memory, using a custom allocator, the AllocationCluster. The lib::Several is used as front-end to access these small collections of related objects, and the associated lib::SeveralBuilder provides the low-level memory allocation and object creation functionality. The purpose of this admittedly quite elaborate scheme is to generate a compact data structure, with high cache locality and without wasting too much memory. Since the exact number of elements and the size of those elements can be deduced only after the builder-API usage has been completed, the aforementioned functional datastructure is used to collect the parametrisation information for all ports, while delaying the actual object creation. With this technique, it is possible to generate all descriptors or entries of one kind in a single run, and placed optimally and compact into the memory allocation.

See also

turnout.hpp

node-builder.hpp

NodeLink_test

Todo:

WIP-WIP-WIP as of 10/2024 prototyping how to build and invoke render nodes /////////////////////////TICKET #1371

Definition in file weaving-pattern-builder.hpp.

#include "lib/error.hpp"
#include "lib/several-builder.hpp"
#include "steam/engine/proc-id.hpp"
#include "steam/engine/engine-ctx.hpp"
#include "steam/engine/buffer-provider.hpp"
#include "steam/engine/buffhandle-attach.hpp"
#include "steam/engine/media-weaving-pattern.hpp"
#include "lib/meta/tuple-closure.hpp"
#include "lib/meta/tuple-helper.hpp"
#include "lib/format-string.hpp"
#include "lib/iter-zip.hpp"
#include "lib/util.hpp"
#include <functional>
#include <utility>
#include <vector>
#include <string>

Namespaces
namespace	steam
	Steam-Layer implementation namespace root.
namespace	steam::engine
	Lumiera's render engine core and operational control.

Typedefs
template<uint siz>
using	SizMark = std::integral_constant< uint, siz >

Classes
struct	PatternData< PAR, BUILD, siz >
	Recursive functional data structure to collect weaving pattern data and finally to emplace a Turnout instance into the data storage for each port, as specified by preceding builder-API invocations. More...
struct	PatternDataAnchor
	Data recursion end: prime the port data storage by reserving appropriate storage to hold all known Turnout elements. More...
struct	WeavingBuilder< POL, PROT >
	A low-level Builder to prepare and adapt for a specific node invocation. More...
struct	WeavingBuilder< POL, PROT >::BufferDescriptor< BU >