Collecting initial drafts
At the start of 2011, several design sketches emerged as an offspring of other discussions and brainstorming sessions.
-
the Engine Interface draft contains some fundamental terms and definitions.
-
discussions regarding time values and timecode handling shed some light on the requirements to be fulfilled by a player subsystem
-
Definition of Time entities from the implementation notes in the TiddlyWiki
Reasoning Behind the Player Subsystem’s Structure
The name “Player” might seem surprising at first. The full, complete and precise name would be something along the lines of “_render- and playback-process coordination subsystem_”. However we need to shorten that into a single word. The more obvious abbreviation (assuming importance of words to be the selection criteria) would be Render, but that would be strikingly misleading, because everyone would take that to refer to the render engine. Thus, we are left with Player. A second consideration highlights similarities to the structure of a typical software player — the term we selected turns out to be suitably aligned to the actual nature of that subsystem.
Influences and Requirements
In accordance with the spirit of modern software development, the analysis starts by determining the Service that is provided by such a player. Examination of the Use Cases determines the fundamental forces to be multiplicity, combined with differentiation in detail, all under the government of time-bound delivery, combined with live reconfiguration.
The basic service turns out to be performing of a preconfigured object model. This performance is time based. Multiple usage instances of this service can be expected to coexist, each of which can be broken down into a set of elementary streams to be delivered in sync. The parameters of delivery can be reconfigured on-the-fly.
Modes of Operation
Delivery can be either free-wheeling, so as to cover a predefined time interval with fixed quality calculations (final render), and it can be throttled for using excess computation power (background rendering). Or — alternatively — delivery can be time-bound (for classical playback), following the projection of wall clock time onto a predefined timeline interval (the so called “playhead” or “playback cursor” proceeding along a timeline).
Reconfiguration
Some of these operation modes need to be prepared to encounter an unpredictable live reconfiguration driven by user interactions:
-
any part of background rendering can be invalidated and restarted, while other parts should be re-integrated, possibly with a re-adjusted position
-
playback, which evolves in a linear manner, can be in one of any of these states at any particular time :
-
paused
-
reversed in direction
-
playback speed re-adjusted
-
-
playback can be looped, with unlimited adjustments of the loop boundaries at any time.
Conclusions for the Design
Based on these observations, the following design would appear to be pretty much obvious:
The overall player subsystem can be described as “play/render-this”-service. Given a suitable (high-level) object, the player has the ability to “perform” (play or render) the object.
-
the standard operation is playing a timeline.
-
it is conceivable that playing a selection of other objects would be permitted, for example, directly playing a clip or even a media asset. In these cases, it is the player’s job to prepare the scaffolding that is required on the fly.
Yet each such performance of an object is a stateful instance, a player application: on application of the player service, the client obtains a front-end handle, a play-controller, which is a state machine. This provides states and transitions such as play, pause, ffwd, rew, goto, step, scrub and similar. Moreover, it maintains (or connects to) a distinct playback location, and it can be hooked up to a play-control GUI widget (or something simpler in the case of a render process, which is free wheeling).
Each play-controller, in turn, is then associated with several play/render-processes, one for each independent media stream (channel) to be produced. Of course this isn’t an operating system process; rather, each such process is a compound of entries in a registration table, which serves the purpose of tying several other services together, which we initiate and use in order to make that render process happen. Most notably, we’ll use the services of the actual engine which provides us with a kind of a calculation stream service: the ability to deliver a sequence of calculated data frames in a timely fashion.
When a client requests such an instance of the player service, we build up these parts providing that service, which cascades down to the individual elements. At that point, we need to pull and combine two kinds of information:
-
the “what” to render: this information stems from the session/model.
-
the “how” to render: this information is guided by the derived output configuration.
Viewer and Output Connection
Creating a player instance binds three partners together: a timeline, a viewer and the engine. While the timeline provides the contents to play, the viewer connection is crucial for working out the actual output sink(s) and, hence, the output format to use. Thus, a viewer connection is a prerequisite in order to create a player instance.
Viewer connections exist as associations in the session/model — which are entities separate from the player. Usually, a timeline has (at least) one viewer connection. But in case such a connection is (still) missing, building a player instance recurs to the session to get a suitable viewer allocated. The viewer connection can’t be broken during the lifetime of that player instance (or putting it the other way: breaking that viewer connection, e.g. by forcing a different connection or by shutting down the viewer, immediately terminates the player.) This detaching works synchronously, i.e. it blocks until all the allocated output slots can be released.
Live switching
While the viewer connection can be treated as fixed during the lifespan of a player instance, several life switching and reconfiguration operations might happen at any time: the model port (the place where data is retrieved from calculation), the output characteristics (framerate, direction) and the delivery goals (playback position, loop playing, scrubbing) all may be changed during playback — we need a way for the player to “cancel” and reconfigure the vault layer services.
Frame quantisation
Quantisation is a kind of rounding; like any kind of rounding, quantisation is a dangerous operation because it kills information content.
Thus, there are three fundamental guidelines when it comes to rounding:
-
don’t do it
-
do it at most once
-
do it as late as possible
These may guide the process of finding the right place for the Quantiser(s) to apply. We need some information flows to be joined in order to be able to do the quantisation, which leaves us with just a few possible junction points where to place quantisation: the Vault, the GUI, the player and the session.
-
putting it into the vault layer seems to be the most reasonable at first sight: we can “do away” with nasty things soon, especially if they are technicalities, “get a clean state soon” — and hasn’t frame quantisation something to do with media data, which is handled in the vault?
Well, actually, all of those are pitfalls to trap the unwary. About cleanliness, well, sigh! Doing rounding soon will leave us with a huge amount of degraded information flows throughout the whole system; as a consequence, the general rule is to do it as late as possible. Uncrippled information is an enablement. And last but not least: the frame quantisation is connected to the output format — and the vault layer is likely within the whole application the subsystem most remote and unaware of output requirements.
-
rounding/quantising in the GUI is extremely common within media applications; unfortunately there does not appear to be a single rational argument in support of this habit. Most of all, it is a violation of the subsidiarity principle.
Which leaves us with the player and the session. Both positions could arguably be supported. Here, a more careful consideration shows, that the “act of frame rounding” can be decomposed: into the act of quantisation and the frame grid. Basically it is the session which has the ability to form the frame grid, but it is lacking crucial information about the output. Only when connecting both — which is the essence of the player — frame quantisation can actually be performed. Thus, the player is the natural location to perform that quantisation operation.