Skip to Content

Transformation Basics

Overview

A transformation is a project component that is used as a step in an operation to map or transform inputs to a resulting output by moving data, cleaning data, or applying business logic. A transformation consists of source and target schemas that have been defined in the transformation and the transformation mapping that generates the output.

If you don't want to transform data and instead pass through data unchanged, you don't need to add a transformation. Valid operation patterns are provided in Validation Patterns under Operation Validity.

This topic covers general concepts that are important to understand about transformations.

These pages are included in this topic:

  • Data Structures
    Data structures can be provided during activity configuration or they can be defined within the transformation itself. Once the source and target schemas of a transformation are defined, you create transformation mappings between the source and target schemas to define how data should be processed.
  • Nodes and Fields
    Nodes and fields are contained within the schemas that are specified either during activity configuration or in the transformation. While configuring a transformation, nodes and fields are displayed the same regardless of the transformation display mode.
  • Transformation Processing
    Harmony supports streaming and chunking as methods of processing a transformation. A streaming transformation is the preferred method to use when the amount of memory a transformation uses needs to be limited. In cases where you are unable to use a streaming transformation, chunking may be applicable.
  • Transformation Reuse
    All transformations can be referenced multiple times as operation steps. The process involved in referencing a transformation depends on whether the schema is defined in a transformation or inherited from an activity.

Also see the topic Schema Basics for information about schemas:

  • Schema Usage
    Whether a schema is required depends on the type of activity and if it is being used as a source or a target of a transformation.
  • Schema Processing
    When defining a schema, some data may be converted during processing. This page describes how this data is converted.
  • Schema Actions Menu
    This page describes the actions available for schemas after they are created, accessible from the project pane or the transformation. These include deleting, viewing dependencies of, deploying, or configurable deploy of a schema, as well as editing the configuration of an activity providing a schema, editing a schema, or clearing a schema from a transformation.