Skip to Content

Schema Usage

Introduction

Schemas are used to represent the request and response structures in a transformation. Those schemas can be inherited from activities that are initially adjacent to the transformation, or they can be defined directly in a transformation. This page describes when schemas are used and how schemas are defined.

When a Schema Is Used

Whether a schema is used depends on the activity and if it is being used as a source or a target of a transformation.

Sources

A source activity is any activity that provides data within an operation, which can then be transferred to a target unchanged or changed by a transformation prior to reaching a target.

Source activities require a data schema only if they will be used as input in a transformation:

  • If the source activity is providing data that will be transformed, it must have a response data schema, which is either automatically provided or can be defined while configuring the activity or transformation.
  • If the source activity is providing data that will not be transformed, a response schema does not need to be defined, as the response schema is not used in the operation.

Activities that are typically (but not necessarily) used as sources include those with one of these words in the name of the activity:

  • Download
  • Get
  • Read
  • Request
  • Query

Targets

A target activity is any activity that receives data within an operation. This data can come from a source activity, from a transformation, or from other input such as variables or scripts.

Target activities require a data schema only if they are receiving data that has been transformed or if they provide a response that you want to write to another target:

  • If the target activity is receiving data that has been transformed, it must have a request data schema, which is either automatically provided or can be defined while configuring the activity or transformation.
  • If the target activity is receiving data that has not been transformed, a request schema does not need to be defined, as the request schema is not used in the operation.
  • If a specific type of target activity provides a response, then you can also have a response data schema. If you want to write the response to another target, the response data schema is either automatically provided or can be defined while configuring the activity or transformation. If the target type provides a response, an operation is not required to write the response to another target; that is, a response schema may be provided but can be ignored within the operation if it is not required.

Activities that are typically (but not necessarily) used as targets include those with one of these words in the name of the activity:

  • Add
  • Create
  • Delete
  • Insert
  • Post
  • Put
  • Response
  • Upload
  • Update
  • Upsert
  • Write

How Schemas Are Defined

Schemas are defined either in an activity or in a transformation. In an operation, a schema provided in an adjacent activity (or an initially adjacent activity that has since been removed) takes precedence over a schema defined in a transformation.

This diagram shows an operation whose transformation inherits its source schema from a preceding adjacent activity and has a transformation-defined target schema:

schema definition diagram annotated pp

Schemas defined in an activity may be either connector-generated or user-defined:

  • Connector-generated: Connector-generated schemas are either fixed in the connector or are generated directly from the endpoint. Depending on the specific activity, either a request or a response schema, or both a request and a response schema may be provided. Connector-generated schemas have an automatically generated name that depends on whether the schema is a request or response:

    User-defined Endpoint Name→User-defined Activity Name→Request

    User-defined Endpoint Name→User-defined Activity Name→Response

    This transformation shows the names of connector-generated schemas being inherited from activities on both its source and target sides:

    schema names annotated

    Depending on the connector, the structure of some connector-generated schemas may be edited by the user, or the structure may dynamically change based on user input provided during activity configuration or based on changes in the endpoint itself.

    The names of connector-generated schemas cannot be edited by a user.

  • User-defined: User-defined schemas are provided by the user, either by creating a custom schema, by uploading a file or URL, or by manually entering the schema. Depending on the specific activity, either a request or a response schema, both a request and response schema, or no schemas may be defined.

    The names of user-defined schemas are based on the name of the provided file or are user-defined.

When used in an operation, activities that have a request schema don't necessarily need to be preceded by a transformation. Similarly, activities that have a response schema don't necessarily need to be followed by a transformation. In some cases, such as an operation designed to archive files from a data source to a target without a transformation, no schemas are required; if they are specified, they are simply ignored.

In an operation, if a transformation requires a source schema, it must be preceded by an activity providing a response schema. Other rules and patterns for constructing a valid operation are covered in Operation Validity.

Activity-provided response and request schemas always override transformation-provided source and target schemas, if present:

  • Activity-provided response schema: When a connector activity with a response schema is placed preceding a transformation, the activity response schema is inherited by the transformation as the source schema and overrides any transformation-provided schema.
  • Activity-provided request schema: When a connector activity with a request schema is placed following a transformation, the activity request schema is inherited by the preceding transformation as the target schema and overrides any transformation-provided schema.

All transformations are reusable regardless of where their schemas come from. However, there are special considerations when referencing in another operation a transformation that has a schema provided by an activity, as described in Transformation Reuse.

If a transformation-provided schema was present prior to configuration of an adjacent activity, that schema cannot be cleared directly. Though a transformation-provided schema remains referenced by the transformation, it is not used in processing. To clear a transformation-provided schema under these circumstances, you can temporarily remove the adjacent activity first. Once the adjacent activity is removed, a transformation-provided schema can be cleared by using the schema actions menu to select Clear Schema. Once you have cleared the transformation-provided schema, you can replace the activity in the operation.