Split a file into individual records using SourceInstanceCount in Jitterbit Integration Studio
Introduction
This design pattern can be used to split a small set of source data in a multi-record JSON or XML file into multiple files, each containing a single record, using the SourceInstanceCount
function.
Tip
This pattern is recommended to be used only with a small set of source records when the source data is complex (hierarchical). For flat (non-hierarchical) source data, see Split a file into individual records using SCOPE_CHUNK. This pattern is not recommended for large sets of data because the payload must be repeatedly processed for each source record.
Use case
In this scenario, the source data is a complex JSON file that contains a small number of multiple records, and the target endpoint requires records to be processed individually.
Design pattern and example
This example operation chain is used to illustrate the design pattern, whose key characteristics are described below:
-
The operation Get File with Multiple Records retrieves the multi-record JSON file from the source and writes the contents to a variable (named
Variable
). -
On success of that operation, the operation Control executes a controller script, the parts of which are described below:
<trans> ordercount = CountSubString($Variable, '"assigned_location":'); $gv_record_no=1; While($gv_record_no<=ordercount, RunOperation("<TAG>operation:Split into Single-record Files</TAG>"); $gv_record_no++; ) </trans>
-
The
CountSubString
function is used to count the number of records from the source (replacingassigned_location
with any field that appears only once for each record in the source data). -
A global variable (
gv_record_no
) is initialized at1
and will be used to increment the number of records. -
The
While
function is used to step through the record count and execute the next operation for each record.
-
-
As the script cycles through records, the operation Split into Single-record Files is repeatedly executed. These are the steps of the operation:
- The contents of the multi-record JSON file contained in
Variable
are used as a source for the transformation. -
In the transformation, a condition script is defined on the item node to iterate through each record:
If(SourceInstanceCount() == $gv_record_no, true, false)
-
On each operation run, the transformation will output a single JSON record, which can be written to the desired target.
- The contents of the multi-record JSON file contained in