Jobs Data
Contents
Introduction
A ROAD Job is a description of an audio processing task, which is to run at any time, but as a concluded process. It has a CurrentState describing if it's valid, waiting, running, finished, etc. Its state changes by Transitions which are part of the job's data and which may be added at any time. The DirectShow filter graph, which is need for actual audio processing is specified by the JobDescription, which is part of a Transition and which may be modified by later Transitions. A Transition is realized when its Trigger activates it.
A ROAD job can be expressed in JSON and be stored in a web page, database or file. It must be independent from specific machines and should be portable at least inside a company installation context. Job definitions could be used as universal descriptors to be used in any interfaces between components.
The job and its transitions may be incomplete (as long as the job isn't to be processed) and each data member of it may be updated at any time (maybe with restrictions while running).
Data Structure Diagram
Terms and Data Type Names
We omit any "ROAD" prefixes for all terms.
Term / Type | Meaning |
---|---|
Job | A job is a single instance of a worker processing request. It has a current state like prepared, scheduled, running, finished, cancelled, etc. It cannot be repeated, but may change its state several times (e.g. run, pause, run again, etc.). It must have at least a creation entry in the history list which defines the transition from prepared → waiting or running. |
Job's Transitions | A sorted list of Transitions which lead into transitions from one state to another. At least one transition must be included in a job. Each transition starts with ProcState Invalid and changes it when done; at the same time the timestamp is set to a valid value. When finished the list of passed Transitions beginning with the initial one from prepared to wait or running represents the complete job history. |
Transition | A transition documents the change from one JobDescription to another one. Only the fields being changed need to be included in the new JobDescription, which may be omitted if it doesn't change. The new state is documented by referring the targeted JobDescription and the ProcState achieved. |
CurrentState | The last achieved ProcState. As long as this transition is not reached, the current state remains invalid or pending. |
Trigger | Describes when (in terms of time or other conditions) to do a transition and specifies the new requested job state. To be extensible inheritance is used to represent different kinds. |
RequestedState | The new ProcState requested when triggered. |
JobDescription | Contains information about what to do while in a state. See Job Description Data. As this information is incremental only changing fields need to be provided. If the JobDescription doesn't change, it can be omitted. |
PipeSpec | Specification of DirectShow pipe including mixer (e.g. for fading) |
JobTemplate | From data point of view identical with a Job, but without history. Will be copied to a job object when this is being prepared for run. |
History (not part of diagram) | The history of a job is the collection of all passed transitions. |
Data Fields
Jobs
A job which is to run (this applies to all real jobs, in difference to job templates) will have a history consisting of at least two so-called transitions (start and finish), but may have even more, depending on what's happening. Each transition contains exactly one so-called trigger, which defines when it is to occur and which new processing state is requested. A transition may contain a job description, which updates the actual input, output or mixer configuration.
Job's field | Description |
---|---|
JobId | Unique ID of this job. Is exclusively created by the ROAD Service's jobs/POST command and is required for all later access to this job object. |
Name | Descriptive name (at least for the type of job) intended for end user |
ProcessingHost | Name of host having processed this job - empty before job is closed |
Transitions | List of transitions |
CreatedTime | Time of job's creation (mostly short before job is being started in Paused or Running state) |
UpdatedTime | Time of job's last update (time of last transition) |
CurrentState | Most current state of this job (one of the CurrentProcStates described below) |
IsActive | false, if the job's CurrentState is one of Finished, Cancelled, Error |
LogInfos (invisible) | List of log file infos. Note these file paths are locally on the processing host. The files are purged after some time, so it's not guaranteed they still exist. This information has no use for clients and can be accessed via REST interface only by explicitly requesting it (a query for LogInfoCount might be useful). |
ClientData | Optional arbitrary info, which can be provided and used by a client creating and later accessing this job. This field can contain any valid JSON value or JSON object. It is possible to use it later for filtering or sorting in a query. This can also be applied on sub-fields of clientData. |
WorkerProcessId | Windows process ID if there's currently a worker process responsible for this job. 0 otherwise. |
StatusMessage | Status message of last status reported by the worker process, usually empty except there was an error. |
FriendlyStatusMessage | Short user-friendly variant of StatusMessage. |
Transitions
A job's Transitions field describes the past or planned Transitions the job ran or runs through while being processed. Each of them contains a specification of what and when to change and may contain a DirectShow pipe description for input and/or output side as well as a mixer specification for gain leveling and mixing matrix.
Transition's Field | Description |
---|---|
TransitionId | Unique ID of this transition. Is created by the ROAD Service only and is required for all later access to this transition. |
TimeStamp | Date and time of past transition - empty before that (for pending transitions) |
CurrentProcState | Processing State reached by this transition - "Invalid" before that |
Trigger | Trigger responsible for starting this transition |
JobDescription | Optional: Contains the actual processing specification (or parts of it updating the former one), e.g. DirectShow input and output graphs, mixer specification, ... |
Info | Optional additional information |
Volatile (write-only) | Pass volatile = true to avoid this transition to be stored, but only to be sent to the Worker while the job is already running. Notifications for this transition will also be omitted then. |
Triggers
Triggers describe when to go into a transition and which processing state is requested then.
A trigger leads into transition from one job state to another. One trigger can occur only once and is exhausted then.
Several triggers are elapsed in their sequence, if possible.
Triggers' field Trigger type | extra fields | Description |
---|---|---|
all | Base of all trigger types. The provided fields are part of every trigger. | |
Type | Type of trigger (Immediate, Time, Duration, ...). Description below in this table. | |
RequestedProcState | Requested processing state (Wait, Pause, Start, ...). None means to keep the current processing state (needed e.g. to change the mixer level while running or while waiting). | |
(Options) | Optional additional options. Currently a boolean value can be passed as Options.OutputSwitch in combination with a RequestedProcState Wait or Pause. | |
Implicit | Implicit triggers are created only by a job being processed reaching a transition which was not specified before, e.g. because of an input file running to its end or because of an error. The reason for this trigger is placed in the transition's info field. | |
OutputSwitch | OutputSwitch triggers are created when a job's output is directed to a new destination (file, database entry). This functionality is controlled by a part of the job description (see Output Switch Specification). Additional information about the new output etc. is placed in the transition's info field. | |
Immediate | Immediate triggers mean to activate this transition immediately, so as soon as possible. | |
Time | Time triggers are to be activated at a specific time, so they are for pre-planned transitions. Note time triggers consider the ROAD host's system clock, not the audio source's one. For best precision favor a duration trigger. | |
StartTime | UTC time (YYYY-MM-DD hh:mm:ss.xyz) to start this transition | |
Duration | Duration triggers define a time span between last triggering before and this one. The "last triggering" isn't finally defined and implemented yet For execution the real audio time is used rather than a system clock, so their precision is around one audio sample. | |
Duration | Number of milliseconds between last trigger and this one. | |
Level (not yet) | Not yet specified. | |
Event | Not yet specified. |
JobDescription
JobDescriptions describe how a job processing is specified after having gone into a transition.
JobDescriptions are differential, this means all former parts not mentioned later remain and only changing parts need to be specified. So the first transition may describe the DirectShow graphs needed to play a file and broadcast it via RTP and later transitions only contain leveling information.
See Job Description Data for more about JobDescriptions.
Processing States
We distinguish two job processing states - RequestedProcState and CurrentState - which are connected with each other as a change of requested state leads into an action in the processing, which mostly causes a change of its current state.
Requested ProcStates
Note: The described "follow-up current ProcState" state is the CurrentState, which will result from the requested one in the usual successful case. This may occur with some delay or even fail.
ProcState | Follow-up Current ProcState | Description |
---|---|---|
Invalid | --- | Internal state resulting from incomplete, missing or faulty initialization. Not to be requested by client. |
None | --- | Leave the processing state. This trigger provides other changes (usually changing the pipeSpec, e.g. for Mixer). |
Wait | Waiting | Start a job in waiting state (meaning to prepare it). |
(Pause) | (Paused) | (Pause a running job.) This requested and current state is currently not completely clear. Sending Pause is valid, the resulting current state is Waiting. |
Start | Running | Start the job. |
Finish | Finished | Finish a job. Re-starting is then not possible any more. |
Cancel | Cancelled | Cancel a job. Re-starting is then not possible any more. |
Current ProcStates
Note: The described "results from requested ProcState" is the RequestedState, which leads into the current one in a usual successful case. CurrentStates can of course change by other reasons, too (e.g. job finishes itself or runs into an error).
ProcState | Results from Requested ProcState | Description |
---|---|---|
Invalid | --- | Internal state resulting from missing or faulty initialization. |
None | --- | No defined state, probably intermediately while initialization. |
Waiting | Wait (or from another condition, e.g. timer trigger waiting) | Job is in waiting state, meaning it is verified and prepared, only the trigger is pending. |
Paused | Pause | |
Running | Start | Job is in progress. |
Finished | Finish | Job is finished successfully. Re-starting is not possible. |
Cancelled | Cancel | Job is cancelled, meaning finished without success. Re-starting is not possible. |
Updating | Worker is busy updating job (probably updating the DS pipe). (not yet used - do we need it?) | |
Error | --- | An error occurred. Further info should be provided in extra text fields. |
Terminated | --- | The ROAD Service finished without receiving a final update on an active job's state. Reasons might be a timeout when finishing the Service while a Worker was still running (which leads into a Finish request to all active Workers), corrupted Worker process or an erratic shutdown of the Service. |
Pending | Intermediate state while initialization of a job or until a transition is processed. |
Timers
Timer-triggers jobs will be sent to worker with a Running RequestedState as the timer is associated with the requested state. The Worker then prepares the DS pipe and set the job's CurrentProcState to Waiting.
Pipe Specifications
A pipe specification consists of
- input queue(s)
- mixer connections and configuration
- output queue(s)
Not all these parts have to be provided in every case as ROAD supports partial updates and provides defaults for some cases.
Modifications on Running Jobs
Any change on a running job must be triggered. So a running job can only be modified by adding transitions which realize a changed job specification, e.g. a new gain value or a changed PipeSpec. (Later: or removing still pending transitions. Adding here could also mean to insert it at a specific location in the sequence.)
"Running job" means any job being at least in waiting state, so potentially being cared for by a Worker process.
Job Templates
Job templates contain a general part of a type of jobs. They are used to create a complete job by duplicating and completing or overriding some fields.
Job JSON Examples
Start Recording
JSON Example - Start Recording
{
"jobId": "12345",
"name": "Recording Test 123",
"transitions": [
{
"trigger": {
"type": "Immediate",
"requestedProcState": "Start"
},
"jobDescription": {
"pipeSpec": {
"InputGraph": [
{
"Name": "Input",
"Filter":[
{
"Name": "ASIO Source",
"Device": {
"Name": "ASIO Hammerfall DSP",
"Channels": "1, 2, 7"
},
"Format": {
"SampleRate": "Sample44100",
"Resolution": "Res16Bit"
}
}
]
}
],
"Mixer": {
"Comment": "the mixer will be configurable in a later step",
"Matrix": {
"Comment": "define to connect input In1 with output Out1"
}
},
"OutputGraph": [
{
"Name": "Output",
"Filter": [
{
"Name": "WAVDest",
"Clsid": "{79376820-07D0-11CF-A24D-0020AFD79768}",
"Title": "This is a test BWF file",
"WriteRf64": true,
"WriteLevlChunk": true
},
{
"Name": "FileWriter",
"Clsid": "{79376820-07D0-11CF-A24D-0020AFD79769}",
"Filepath": "\\\\fileserver1\\test\\file1.wav"
}
],
"Comment": "Currently the graphs are connected in a queue. Later this may become configurable."
}
]
}
}
}
]
}
Stop Recording
JSON Example - Stop Recording
{
"transitions": [
{
"trigger": {
"type": "Immediate",
"requestedProcState": "Finish"
}
}
]
}
Implications
- A job always has at least one transition.
- A transition can be recognized as passed by its state (not invalid or pending) and timestamp (valid).
- Not every transition must be passed to finish a job.
- The current state of a job is always documented in the newest passed transition. But to access this information directly, there's the (redundant) CurrectState field of the job.
- The job's history is described by all passed transitions. As the trigger data remains, the "intended" flow remains documented (for passed and not passed transitions).
- The final transition may lead into a message text (+ friendly text) information, mostly used for errors.