Skip to Content

Schemas Defined in an Activity

Introduction

Schemas used in a transformation either are inherited from connector activities that are initially adjacent to a transformation (as described on this page), or they are defined in a transformation (see How Schemas Are Defined in Schema Usage). Schemas defined in an activity that are inherited by an adjacent transformation are not considered to be part of the transformation.

A schema defined in an adjacent activity (or an initially adjacent activity that has since been removed) takes precedence over a schema defined in a transformation.

If a transformation inheriting a schema from an activity is referenced in another operation and the activity providing the schema is not adjacent in the current operation, the transformation remains unchanged and continues to inherit the schema from the activity that is no longer adjacent (see Transformation Reuse).

If an activity providing a schema to a transformation is deleted, that schema then becomes part of the transformation and becomes defined directly in the transformation. If the deleted activity was providing a response schema to a transformation, that transformation would become invalid, as a transformation with a source schema is required to be preceded by an activity providing a response schema. Other rules and patterns for constructing a valid operation are covered in Operation Validity.

Define a Schema in an Activity

Depending on the connector, the ability for a user to provide a schema may be included as an activity configuration step. The interface for defining schemas is covered in each connector's documentation.

Some connectors where schemas can be defined by a user have a common interface, including these:

An example of defining a schema in a File Share Read activity is included below.

During configuration of the above activities, you select one of three options in a section called Provide Response Schema or Provide Request Schema:

ftp read provide response schema

  • No: Choose this option to skip providing a schema. You may want to select this option for a number of reasons:

    • If the connector provides a fixed schema when a user-defined schema is not provided.
    • If you define the schema directly in the transformation (see Schemas Defined in a Transformation).
    • If no schema is required for your use case (see Schema Usage).

  • Yes, Use Saved Schema: Choose this option to select an existing schema that has previously been defined in the current project. When this option is selected, this section becomes available:

    ftp read provide response schema saved schemas

    • Saved Schemas: Use the dropdown to select from an existing schema to reuse.

    • View Schema: After an existing uploaded schema is selected, you can view the schema directly within the text area below the dropdown. To view the schema in a larger area, click the popout icon popout (after opening that area, click the return icon popout to come back to this screen).

      Though a saved schema is not editable, this text area can be copied using Control+C (Windows or Linux) or Command+C (macOS).

      This text area is only for the display of existing uploaded schemas and does not display flat, hierarchical, or mirrored custom schemas.

    • Validation: Validation information is provided below the text area and is based on the file extension of the saved schema.

  • Yes, Provide New Schema: Choose this option to define a new schema by loading one from a URL, uploading a file, or manually entering one into the text area. When this option is selected, this section becomes available:

    ftp read provide response schema schema name

    • Schema Name: Enter a name for the schema into the upper text box, including the file extension (.csv, .json, .xml, .xsd, or .zip). If no file extension is provided, the content is analyzed to autodetect the file type to use for validation. If you are loading the schema from a URL or uploading a file, you can leave this blank, as the name will be populated once the file is loaded.

    • Load URL: Click to open a dialog where you can load a schema from a URL:

      upload schema url 2

      • File Type: Use the dropdown to select from the supported file types, including CSV, JSON, XML, XSD, and ZIP. Take note:

        • XSD: An XSD provided by URL can import/include other XSD URLs by reference.

        • ZIP: A ZIP archive can contain a collection of XSDs, which can import/include each other by reference. Any non-XSD files in the archive are ignored. Multiple directory levels are supported.

      • URL: Enter a URL that is accessible without authentication. Files up to 5 MB in size can be uploaded.

      • Load: Click to load the schema from the URL. Note that the file is retrieved only a single time for schema generation. In addition, be aware that some data may be converted during processing as described in Schema Processing.

        Note

        When uploading an XSD file that contains multiple top-level elements, on clicking Next to advance to the next step, you are prompted to select the desired root node. Select the desired element and then click Finish to upload the file:

        please select the root to use

      • Cancel: Click to close the Upload Schema URL dialog without saving.

    • Upload File: Click to open a dialog where you can load a schema from a file that is accessible from the current machine:

      upload schema file zip

      • File Type: Use the dropdown to select from the supported file types, including CSV, JSON, XML, XSD, and ZIP.

        Note

        A ZIP archive can contain a collection of XSDs, which can import/include each other by reference. Any non-XSD files in the archive are ignored. Multiple directory levels are supported.

      • File: Use the Browse button to the right to browse to a file that has not yet been used in the current project. Files up to 5 MB in size can be uploaded.

        Warning

        If you try to upload a file with the same name as an existing file already defined in the project, on clicking Next to advance to the next step, a dialog asks if you want to overwrite the existing file. If you click Continue, the file is replaced with the new file with the same name in all places where it is used in the project. If you don't want to overwrite the file, click Cancel. Then manually modify the file so it has a unique name and then try to upload it again.

        overwrite file

      • Load: Click to load the schema from the file. Note that some data may be converted during processing as described in Schema Processing.

        Note

        When uploading an XSD file that contains multiple top-level elements, you are prompted to select the desired root node. Select the desired element and then click Finish to upload the file:

        please select the root to use

      • Cancel: Click to close the Upload Schema File dialog without saving.

    • View/Edit Schema: If you are not uploading a schema, you can enter one manually into the text area below the Load URL and Upload File buttons. Otherwise, after uploading a schema, you can view or edit the schema directly within that text area.

      To view or edit the schema in a larger area, click the popout icon popout (after opening that area, click the return icon popout to come back to this screen).

      This text area can also be copied using Control+C (Windows or Linux) or Command+C (macOS).

    • Validation: As you edit a schema, validation information is provided below the text area, with any errors reported one line at a time. That is, after resolving an error on one line, additional syntax errors to resolve may be reported for subsequent lines. Validation is based on the file extension of the provided schema.

Schema Actions Menu

After a schema is defined in an activity, you can access additional menu options that are available for all schemas.

These include Delete, Rename, View Dependencies, Deploy, Configurable Deploy, and Add to Group, as well as Edit Schema and Clear Schema. For details on these actions, see Schema Actions Menu.