Skip to end of metadata
Go to start of metadata


A Local Storage Read activity interacts with a Local Storage connection to read data as a source in an operation or a script. After configuring a Local Storage connection, you can configure as many Local Storage activities as you like for each Local Storage connection.

Creating a Local Storage Activity

From the design canvas, open the Connectivity tab of the design component palette:

Within the Endpoints filter, click the Local Storage connection block to display activities that are available to be used with a Local Storage connection:

To create an activity that can be configured, the activity must first be added to an operation on the design canvas. To add an activity to an operation, drag the activity block from the palette to the operation.

For more information about the parts of an operation and adding activities to operations, see Operation Creation and Configuration.

Accessing Menu Actions

After a Local Storage activity has been added to an operation, menu actions for that activity are accessible from the project pane in both the Workflows and Components tabs, and from the design canvas:

  • Project Pane: In the Workflows or Components tab of the project pane, hover over an activity name and click the actions menu icon  to open the actions menu.

  • Design Canvas: Within the operation, click an existing activity block to open the actions menu.

Each of these menu actions is available:

  • View/Edit: This opens the activity configuration screen for you to configure the activity. For details, see Configuring a Local Storage Read Activity later on this page.
  • Delete: This is used to permanently delete the activity (see Component Dependencies, Deletion, and Removal).
  • Rename: This positions the cursor on the activity name in the project pane for you to make edits.
  • View Dependencies: This changes the view in the project pane to display any other parts of the project that the activity is dependent on (see Component Dependencies, Deletion, and Removal).
  • Remove: Available only from the actions menu on the design canvas, this removes the activity as a step in the operation without deleting it from the project. When you remove an activity that is adjacent to a transformation, if schemas are provided within that activity, they will no longer be referenced by the transformation. Removed components can be accessed or permanently deleted from the project pane (see Component Dependencies, Deletion, and Removal).
  • Deploy: This deploys the activity and any components it is dependent on (see Component Deployment).
  • Configurable Deploy: This opens the deployment screen, where you can select project components to deploy (see Component Deployment).
  • Add to group: This opens a prompt to create a new custom group or to add the component to an existing group. Custom groups are an organizational tool to help organize a project (see Component Groups).
  • Duplicate: This creates a copy of the activity as a new, unreferenced component. Upon creating the component copy, the cursor is positioned on the component name within the project pane for you to rename the component.

Configuring a Local Storage Read Activity

Follow these steps to configure a Local Storage Read activity:

Step 1 – Enter Information and Provide File Schema

TIP: Fields with a variable icon support using global variablesproject variables, and Jitterbit variables. Begin typing an open square bracket [ into the field or click the variable icon to display existing variables to choose from.

  • Name: Enter a name to use to identify the Local Storage Read activity. The name must be unique for each Local Storage Read activity and must not contain forward slashes (/) or colons (:).

  • Provide Response Schema: The response schema defines the structure of data that will be used by the Local Storage Read activity. Whether a response schema is required depends on if the activity will be used as the source of a transformation (see When to Use a Schema). For instructions on completing this section of activity configuration, refer to Schemas Defined in an Activity.

    NOTE: As an alternative to providing a schema during activity configuration, you can define a schema directly within a transformation (see Schemas Defined in a Transformation). File schemas defined directly in a transformation take precedence over those configured as part of an activity.
  • Get Files: Enter the name of the file(s) that you want to read from within the directory specified in the Local Storage connection. This field may also include an asterisk * to use as a wildcard (for example, *.txt or *.*) or a question mark ? to match exactly one character (for example, file?.txt). The use of wildcards is separate from the Use regular expression option described below. If specifying data in ZIP format to be automatically decompressed and read, see the Compression options described below.

  • Optional Settings: Click to expand additional optional settings:

    • File Filter Options: These options affect how the input entered in the Get Files field above will be interpreted:

      • Ignore Case: Select this checkbox if you have entered input in the Get Files field above that should be interpreted as case-insensitive.

      • Use regular expression: Select this checkbox if you have entered input in the Get Files field above that should be interpreted as a regular expression. This option is separate from the use of wildcards described in Get Files above.

    • Character Encoding: Specify character encoding to define a specific character encoding for the source file. If you leave the field blank, Jitterbit Harmony will try to detect the encoding. If you want to overwrite the default, you can choose from any of the encoding types listed in Character Encoding.

    • Success/Error Folder: These options are used to archive a copy of all files processed. This can be used in place of the After Processing option to Rename File on the file source to allow you to write to a separate location rather than to a sub-folder in the same location.

      • Use Success Folder: Select this checkbox to write a file to the specified folder after successful processing on the primary target path. If writing a file to multiple paths, the operation is successful only if all paths are successful. If this checkbox is selected, also provide the Success Folder in Absolute Path on the Private Agent machine.

      • Use Error Folder: Select this checkbox to write a file to the specified folder after unsuccessful processing. If at least one path is unsuccessful, then the file is written to this folder. If this checkbox is selected, also provide the Error Folder in Absolute Path on the Private Agent machine.

        CAUTION: A file is written only if the failure is due to errors processing the Local Storage Read activity. A file is not written if, for example, the Local Storage connection is unsuccessful, causing the entire operation to fail.

        NOTE: When using filename keywords in these fields, the keywords are not resolved. For a workaround, see Limitation and Workaround under Filename Keywords.

      The file(s) will be named as specified in the Get Files field above, with a trailing underscore and a timestamp added. If there is already a file with the same name in that folder, an additional underscore and a counter will be added.

      For example, if a file named data.csv is successfully processed, a copy named data.csv_2018-01-01_12-00-00-000 could be written to the folder specified as the success folder. If such a file already exists from a previous run, a new file data.csv_2018-01-01_12-00-00-000_1 would be added. In a similar fashion, if an error folder is specified, any files that fail will be written to a file in the error folder.

    • After processing: If you want to perform further actions after processing the data, these options are available:

      • Delete File: Select this checkbox if you want files that have been successfully processed to be deleted. When this option is selected, the Rename File option below is automatically populated with /\delete/\ to indicate that processed files will be deleted and as such, renaming a file does not apply. Clearing Delete File also clears the Rename File field.

      • Rename File: Enter a name if you want to rename files after processing. Any of the filename keywords are available to generate unique filenames for the source data after processing. Note the Rename File field is unavailable if Delete File above is selected. In addition, toggling Delete File above will clear any input entered in the Rename File field.

    • Trigger File: A trigger file is a file that must be present before a source file is retrieved by an operation for processing. The presence of a trigger file indicates that a source file is ready to be read and is not still in the process of being written by a previous operation.

      • Use Trigger: Select this checkbox if you want the system to look for a trigger file before processing source files. Note, when using a trigger file:

        • The operation will poll, looking for the trigger file. Polling time interval is fixed at one second.

        • The trigger file is deleted as processing starts.
        • If the trigger file is not found, the operation will keep attempting to find the trigger file until the operation times out.

        • Setting a trigger file blocks any operation or script that uses the source until either the trigger file condition is satisfied or the operation times out.

      • Absolute file path: Enter the absolute file path on the Private Agent machine where the trigger file resides. Select a file that meets these criteria:

        • The file can be of any type and may contain anything as its contents; only the presence of the file defines the trigger file behavior.

        • The use of wildcards is not permitted.

        • The trigger file cannot be the same file as a source file being processed.
        NOTE: When using filename keywords in this field, the keywords are not resolved. For a workaround, see Limitation and Workaround under Filename Keywords.
    • Ignore Lines: If there are headers and/or footers in the data, enter the number of lines to ignore in the First and Last fields. The system will ignore these lines in the source.

    • Compression: If the source data is in a compressed format, select The source data is compressed. These options become available:

      • Format: Use the dropdown to select the format of the compressed data. Currently, only ZIP format is supported.

      • Password: If the archive is encrypted, enter the password.

        NOTE: Jitterbit Harmony supports reading only from the root directory of an encrypted archive.

  • Save & Exit: If enabled, click to save the configuration for this step and close the activity configuration.
  • Next: Click to temporarily store the configuration for this step and continue to the next step. The configuration will not be saved until you click the Finished button on the last step.
  • Discard Changes: After making changes, click Discard Changes to close the configuration without saving changes made to any step. A message will ask you to confirm that you want to discard changes.

Step 2 – Review Data Schema

  • Data Schema: If provided during activity configuration, the response data schema is displayed. If the operation uses a transformation, the data schemas will be displayed again later during the transformation mapping process, where you can map to target fields using source objects, scripts, variables, custom values, and more. You can also define schemas directly in a transformation.

  • Add Plugin(s): Plugins are Jitterbit- or user-provided applications that extend Harmony's native capabilities. To apply a plugin to the activity, click to expand this section and select the checkbox next to the plugin to be used. For additional instructions on using plugins, including details on setting any required variables used by the plugin, see Plugins Added to an Activity.

  • Back: Click to temporarily store the configuration for this step and return to the previous step.
  • Finished: Click to save the configuration for all steps and close the activity configuration.
  • Discard Changes: After making changes, click Discard Changes to close the configuration without saving changes made to any step. A message will ask you to confirm that you want to discard changes.

Next Steps

After configuring a Local Storage Read activity, you can use it within an operation or script as described below.

Completing the Operation

After configuring a Local Storage Read activity, complete the configuration of the operation by adding and configuring other activitiestransformations, or scripts as operation steps (see Operation Creation and Configuration). You can also configure an operation's operation settings, which include the ability to chain operations together that are in the same or different workflows (see Operation Settings).

Local Storage Read activities can be used as a source with these operation patterns:

Other patterns are not valid using Local Storage Read activities. See the validation patterns on the Operation Validity page.

This example setup depicts one common operation setup using a Local Storage Read activity:

When ready, deploy and run the operation (see Operation Deployment and Execution) and validate behavior by checking the operation logs (see Operation Logs).

Using Local Storage Activities in Scripts

Local Storage Read activities that have already been configured as part of an operation can also be referenced in a script for use with certain script functions that use a sourceId as a parameter. For example:

To add a configured Local Storage activity to a script to be referenced by one of these functions, drag the configured activity from the Endpoints tab of the script component palette to the script. Or, if you already know the function you want to use, add it from the Functions tab first; then position the cursor after the opening parenthesis of the function and press Control+Space to display a list of autocomplete suggestions. Select an activity to insert the activity reference into the script.

For more details on referencing activities in scripts, see Endpoints on the Jitterbit Script page.

On This Page

Last updated:  Mar 09, 2020

  • No labels