Using Variables to Make Jobs More Flexible

This page explains how to use and maintain ROAD variables intending to make some fields of a job more flexible.

General Definitions

• Variables are used to replace any field content inside a jobDescription. This will be done generically regardless of the content (so even invalid JSON content isn't always detected). There’s no verification if this is resolvable, so errors may only be detected when job is being created or updated. The only verification being done is for still having a valid jobDescription JSON then.
• The Worker is not involved, so variable replacement occurs only on Service side and only when a jobDescription is updated and then sent to a Worker (when a job is created or updated).
• This means the job data being written to database also reflects the state after variable replacement.
• (So the original jobDescription and the used variable are not documented in the job. It's likely reasonable to write this into the Service.log file.)

Placeholders

Variables are a special variant of placeholders, which can be inserted into JobDescriptions. Other placeholders are filled with automatically generated content like the processing host's name or a timestamp. See Placeholders in JSON.

Syntax

• General syntax to embed placeholders: ${Var:VarName} where "Var" is case-insensitive
• VarName is restricted to usual identifier syntax, so allowing only alphanumeric characters and some special characters:  _ . @
• The variable's value can be a string or a JSON object.
• If the variable's content is a string:
  • If the the variable is included in a string in the jobDescription, its value will be inserted into that string at this location.
  • The value can contain surrounding quotes to mark it as string. Quotation marks inside the string must be escaped by \" as well as other characters that need escaping inside a JSON string.
  • A field can be defined as combination of placeholder with fix text (e.g. "dpeUrl": “https://${Var:@DpeUrl}”)
  • To concatenate variables combine them in a field (e.g. “dpeUrl”: "${Var:@DpeProtocol}${Var:@DpeDnsName}”)
• If the variable's content is a JSON object or array it will replace the original string completely. This means:
  • The value must be enclosed in curly or square brackets to mark it an object or array.
  • The JSON value at this location at this location becomes a JSON object or array (not any more included in quotes).
  • In case the string referring to the variable contains any more than this, an error will be reported.
• Variable names are evaluated case-sensitively.

Examples

This part of a jobDescription refers to two variables, the first intended to have a string, the second to have a JSON object:

{
	"name": "DPE Writer",
	"dpeUrl": "${var:@DpeUrl}",
    "metadata": "${var:@Meta}"
}

This refers to variables which could look like this:

{
	"name": "@DpeUrl",
	"value": "https://road-dpe"
},
{
    "name": "@Meta",
    "value": {
        "title", "...",
        "...": "..."
    }
}

Access and Persistence of Variables

REST API to Access Variables

• api/variables supporting POST, PUT, GET and DELETE
• content of variable can bet sent as REST body or as a REST parameter (POST api/variables/dpeUrl?value="dpe_host_42")
• content of variable must be a valid JSON value (so either a string surrounded by quotes, a JSON object surrounded by curly brackets, a JSON array surrounded by square brackets, or another JSON value)

Persisting Variables

• All updated variables are stored in the database.
• All variables are global inside a ROAD cluster.
• In case a variable should be kind of local (or other specific scope), this needs providing according variable names like "MyHost_MyVariable".