Reference¶
Runtime Settings¶
Runtime settings define common configuration parameters on how it should be executed.
Parameters¶
Version
Type: StringRequired: NoDefault: 1.0Description: Engine version number.
Name
Type: StringRequired: NoExample: “My Integration Config”Description: Configuration file name.
LogPayload
Type: BooleanRequired: NoDefault: falseExample: trueDescription: Determines whether payload (input and output) should be logged. It is recommended to do so to help with debugging. Some integrations may use up a lot of disk however.
EntryPipelineId
Type: StringRequired: YesExample: Pipeline1Description: Starting point of execution.
Connection¶
Connection represents a physical connection to a local or a remote resource. Following sections describe supported connection types in more details.
RESTful¶
Connection definition for HTTP RESTful API. A RESTful API is an application program interface (API) that uses HTTP requests to GET, PUT, POST and DELETE data.
Parameters¶
Type
Type: StringRequired: YesAccepts:Restful
,Datum
Description: Connection type.
Name
Type: StringRequired: YesExample: My connection nameDescription: Connection name. It will be referenced in logs.
Url
Type: StringRequired: YesValidation: Must be valid URLExample: https://api.my-company.com/v1Description: Connection base URL
Authentication
Type: IntegerRequired: YesExample: 5Description: Id of Authentication entity. Must be created in Integration > Authentication form.
Datum¶
Datum is the name of the internal Subscribe-HR data layer. It is used to perform direct read and write operations from and to Subscribe-HR. Unlike API connection Datum is fast and does not require additional parameters or authentication to be defined.
Parameters¶
Type
Type: StringRequired: YesAccepts:Restful
,Datum
Description: Connection type.
Name
Type: StringRequired: YesExample: My connection nameDescription: Connection name. It will be referenced in logs.
Operation¶
Operation represents an action that can be performed through a given connection. For example a GET requests through RESTful API or query operation through a database connection.
RESTful¶
Operation definition for HTTP RESTful API. Must reference RESTful connection.
Parameters¶
Name
Type: StringRequired: YesExample: My read operationDescription: Operation name. It will be referenced in logs.
Connection
Type: StringRequired: YesExample: MyRestfulConnectionDescription: Connection reference key that was used in the definition.
Method
Type: StringRequired: YesAccepts:GET
,POST
,PUT
,PATCH
,DELETE
Example: GETDescription: HTTP request method.
Path
Type: StringRequired: YesExample: /api/v1/employees/:IdDescription: HTTP end point path. Supports merge parameters e.g. :Id. Will be merged from Parameters.Url when passed in input.
ErrorHandlingStyle
Type: StringRequired: NoAccepts:warn
,halt
Default: haltExample: warnDescription: Indicates how to handle errors when they are encountered. For example if operation received an error when trying to write datawarn
will produce a warning and continue execution of next record. Halt will completely terminate the process.
ErrorStatusCodes
Type: ArrayRequired: NoDefault: [‘4xx’, ‘5xx’]Example: [400, ‘5xx’]Description: HTTP status codes that constitue errors. Supports wild card declaration e.g.5xx
will include all 500 error codes.
ErrorStatusCodeExceptions
Type: ArrayRequired: NoDefault: [404]Example: [403, 404]Description: HTTP status codes that should be ignored even if they are defined in ErrorStatusCodes field. Supports wild card declaration e.g.5xx
will include all 500 error codes.
OutputType
Type: StringRequired: NoAccepts:json
,raw
Default: jsonExample: jsonDescription: Output format to expect operation to return.
Datum¶
Operation definition for Datum connection type. Must reference Datum connection.
Parameters¶
Name
Type: StringRequired: YesExample: My read operationDescription: Operation name. It will be referenced in logs.
Connection
Type: StringRequired: YesExample: MyDatumConnectionDescription: Connection reference key that was used in the definition.
Action
Type: StringRequired: YesAccepts:Create
,Update
,Get
,Delete
,Query
Example: GetDescription: Type of operation to perform.
Entity
Type: StringRequired: Yes unless Action is QueryValidation: Valid Subscribe-HR entity nameExample: EmployeeDescription: Subscribe-HR entity name. Can be found in Development > Objects > System Name.
Query
Type: StringRequired: Yes only if Action is QueryValidation: Valid SSQL queryExample: SELECT e FROM Employee eDescription: Must provide full SSQL query ifAction is Query
. Otherwise only where clause condition e.g.Id = :Id
.
ErrorHandlingStyle
Type: StringRequired: NoAccepts:warn
,halt
Default: haltExample: warnDescription: Indicates how to handle errors when they are encountered. For example if operation received an error when trying to write datawarn
will produce a warning and continue execution of next record. Halt will completely terminate the process.
Pagination
Type: Object (Pagination)Required: NoExample:{ MaxItemsPerPage: 10 }
Description: Pagination parameters allow splitting requests into batches. There may be cases where a lot of data needs to be processed at once. Due to resource allocation limits it is best practice to paginate your operations to return 100 or less records at a time. The limit is not enforced. This may change in the future. Pagination parameter can only be used with Action of typeGet
andQuery
.
OutputType
Type: StringRequired: NoAccepts:json
,raw
Default: jsonExample: jsonDescription: Output format to expect operation to return.
Pagination¶
Pagination defines how operations paginate requests.
Mapping¶
Mapping defines how source field is transformed into destination field. Mappings utilise JsonPath syntax to perform transformations making it very flexible to manipulate data. Mappings only deal with JSON format. Any operation that returns any other type of data will need to be converted into JSON first.
Parameters¶
FromField
Type: StringRequired: Yes if no Default specifiedExample: $.idDescription: JsonPath pattern to extract source field value.
ToField
Type: StringRequired: YesExample: $.Data.SurnameDescription: JsonPath pattern for destination field.
Tag
Type: StringRequired: NoExample: MyEarlierTagDescription: Perform mapping from a data tag instead of input data.
Default
Type: StringRequired: Yes if no FromField specifiedExample: Some stringDescription: Default value to assign to a field if from field is NULL or an empty string. Alternatively if FromField is not specified default value will be written into destination field.
Translations
Type: ObjectRequired: NoExample:{"m": "male"}
Description: Transformation object if field needs to be transformed to a different value. For example source value may be set tom
which should be translated intomale
in destination field.
DateFormatFrom
Type: StringRequired: Yes if DateFormatTo is specifiedExample: d-m-YDescription: For date fields indicates what format to expect dates in.
DateFormatTo
Type: StringRequired: NoExample: Y-m-dDescription: For date fields indicates what format to output dates into.
Note
If only DateFormatFrom attribute is specified the default output format will be set to Y-m-d H:i:s
.
FunctionId
Type: Object (Function)Required: NoExample: MyMappingFunctionDescription: Reference to predefined function to use to perform the mapping.
Code
Type: FunctionRequired: NoExample: function(input) { const output = input; return output; }Description: Inline function definition to perform the mapping.
Function¶
Users can define javascript functions to use during pipeline execution.
The syntax is function(input) { const output = input; return output; }
. Javascript engine that we use behind the
scenes is v8 which has support for ECMAScript 2015 (ES6).
Logical¶
Logical functions control the flow of pipeline execution process and determine what pipelines should be executed next based on the input. One of the most common use cases will be to determine if API has returned a status code of 404 and then execute record creation pipeline otherwise perform an update. Expected output of the function is a string with a pipeline name or an array with multiple names to be executed next.
Parameters¶
Type
Type: StringRequired: YesAccepts:Logical
,Mapping
Example: LogicalDescription: Defines function type.
Code
Type: FunctionRequired: YesValidation: Valid javascript functionExample: function(input) { if (input.StatusCode == “404”) { return “Pipeline2”; } return “Pipeline3”; }Description: Function code.
Mapping¶
Mapping functions are used to assist with performing mapping operations. Input comes from a selector or a previous action. Function can then manipulate the input and produce an output that is then passed back to the execution action.
Parameters¶
Type
Type: StringRequired: YesAccepts:Logical
,Mapping
Example: MappingDescription: Defines function type.
Code
Type: FunctionRequired: YesValidation: Valid javascript functionExample: function(input) { if (input.EmploymentType == “f”) { return “fulltime”; } }Description: Function code.
Pipeline¶
Pipeline is a list of actions that need to be performed in a sequence. Pipelines can be nested and trigger other pipelines. There are a number of action types that can be performed within pipeline which are outlined below.
Common Action Parameters¶
Type
Type: StringRequired: YesAccepts:Operation
,Iterator
,Map
,Function
,Pipeline
Example: OperationDescription: Defines action type to execute.
InputTag
Type: StringRequired: NoExample: MyEmployeeRecordDescription: Tags input data to reuse later on in the pipeline.
OutputTag
Type: StringRequired: NoExample: MyEmployeeOutputRecordDescription: Tags output data to reuse later on in the pipeline.
Operation Action¶
Triggers an operation.
Parameters¶
Inherits all Common Action Parameters
Id
Type: StringRequired: YesExample: MyApiGetOperationDescription: Operation Id to execute.
Iterator Action¶
Iterates over a data set.
Parameters¶
Inherits all Common Action Parameters
Selector
Type: StringRequired: YesExample: $.Data.content[*]Description: JsonPath expression to indicate which data to iterate over.
Map Action¶
Iterates over a data set.
Parameters¶
Inherits all Common Action Parameters
Id
Type: StringRequired: YesExample: MyEmployeeMappingsDescription: Mapping Id to execute.
Function Action¶
Executes a javascript function. Can either be an inline function or reference to a predefined function.
Parameters¶
Inherits all Common Action Parameters
Id
Type: StringRequired: Yes if using a predefined functionExample: MyPredefinedFunction1Description: Predefined function Id.
FunctionType
Type: StringRequired: Yes if it is an inline functionAccepts:Logical
,Mapping
Example: MyPredefinedFunction1Description: Predefined function Id.
Code
Type: StringRequired: Yes if it is an inline functionExample: function(input) { const output = input; return output; }Description: Inline function definition.
Pipeline Action¶
Executes a pipeline or a series of pipelines.
Note
Pipeline action must be the last action in the sequence. It is not possible to return output from pipeline and continue executing another action. This design ensures that there is no retrace within execution plan to minimise errors and keep pipelines linear.
Parameters¶
Inherits all Common Action Parameters
Id
Type: String or ArrayRequired: YesExample: [“Pipeline2”, “Pipeline3”]Description: One or multiple pipelines to execute.