Skip to end of metadata
Go to start of metadata

Introduction

A Snowflake put activity puts data into a table (either as a CSV file or directly mapped to columns of a table) on a Snowflake connection and is used as a target in an operation. After configuring a Snowflake connection, you can configure as many Snowflake activities as you like for each Snowflake connection.

Creating a Snowflake Activity

From the design canvas, open the Connectivity tab of the design component palette:

Connectivity tab

Within the Endpoints filter, click the Snowflake connection block to display activities that are available to be used with a Snowflake connection:

Connectivity tab

To create an activity that can be configured, the activity must first be added to an operation on the design canvas. To add an activity to an operation, drag the activity block from the palette to the operation.

For more information about the parts of an operation and adding activities to operations, see Operation Creation and Configuration.

Accessing Menu Actions

After a Snowflake activity has been added to an operation, menu actions for that activity are accessible from the project pane in either the Workflows or the Components tabs, and from the design canvas:

  • Project Pane: In the Workflows or Components tab of the project pane, hover over an activity name and click the actions menu icon to open the actions menu.

  • Design Canvas: Within the operation, click an existing activity block to open the actions menu.

Each of these menu actions is available:

  • View/Edit: This opens the activity configuration screen for you to configure the activity. For details, see Configuring a Snowflake Put Activity later on this page.

  • Delete: This is used to permanently delete the activity.

  • Rename: This positions the cursor on the activity name in the project pane for you to make edits.

  • View Dependencies: This changes the view in the project pane to display any other parts of the project that the activity is dependent on.

  • Remove: Available only from the actions menu on the design canvas, this removes the activity as a step in the operation without deleting it from the project. When you remove an activity that is adjacent to a transformation, if schemas are provided within that activity, they will no longer be referenced by the transformation. Removed components can be accessed or permanently deleted from the project pane.

  • Deploy: This deploys the activity and any components it is dependent on.

  • Configurable Deploy: This opens the deployment screen, where you can select project components to deploy.

  • Add to group: This opens a prompt to create a new custom group or to add the component to an existing group.

  • Duplicate: This creates a copy of the activity as a new, unreferenced component. Upon creating the component copy, the cursor is positioned on the component name within the project pane for you to rename the component.

Configuring a Snowflake Put Activity

Follow these steps to configure a Snowflake put activity:

  • Step 1 – Select a Snowflake Table
  • Step 2 – Select an Approach
  • Step 3 – Review Data Schema

Step 1 – Select a Snowflake Table

Connectivity tab

  • Name: Enter a name to use to identify the Snowflake put activity. The name must be unique for each put activity and must not contain forward slashes (/) or colons (:).

  • Select a Table: This section displays tables available in the Snowflake endpoint. When reopening an existing activity configuration, only the selected table is displayed instead of reloading the entire table list.

    • Selected Snowflake Table: After a table is selected, it will be listed here.
    • Search: Enter any part of the table name into the search box to filter the list of tables. The search is not case-sensitive. If tables are already displayed within the table, the table results will be filtered in real time with each keystroke. To reload tables from the endpoint when searching, enter search criteria and then refresh, as described below.
    • Refresh: Click the refresh icon Refresh icon or the word Refresh to reload tables from the Snowflake endpoint. This may be useful if you have recently added tables to the Snowflake instance. This action refreshes all metadata used to build the table of tables displayed in the configuration.
    • Select a Table: Within the table, click anywhere on a row to select a table. Only one table can be selected. The information available for each table is fetched from the Snowflake endpoint:
      • Name: The table name from Snowflake.
      • Type: The table type from Snowflake.

    TIP: If the table does not populate with available tables, the Snowflake connection may not be successful. Ensure you are connected by reopening the connection and retesting the credentials.

  • Save & Exit: If enabled, click to save the configuration for this step and close the activity configuration.

  • Next: Click to save the configuration for this step and continue to the next step.

  • Discard Changes: After making changes to this step, click Discard Changes to close the configuration without saving changes to this step. A message will ask you to confirm that you want to discard changes.

Step 2 – Select an Approach

Connectivity tab

  • Approach: Enter an approach to be used for putting the data to Snowflake. Choose one of Stage File or SQL Insert.

    • Stage File: This approach will allow a CSV file to be inserted into Snowflake. The file will first be staged and then copied into the table following the specifications of the request data schema.

    • SQL Insert: For this approach, the table columns will be shown in the following data schema step, allowing them to be mapped in a transformation.

  • Back: Click to save the configuration for this step and return to the previous step.

  • Next: Click to save the configuration for this step and continue to the next step.

  • Discard Changes: After making changes to this step, click Discard Changes to close the configuration without saving changes to this step. A message will ask you to confirm that you want to discard changes.

Step 3 – Review Data Schema

Approach: Stage File

Connectivity tab

Approach: SQL Insert

Connectivity tab

Data Schema

  • Data Schema: The request and response data schemas for Snowflake will be displayed. The schemas displayed depend on the Approach specified in the previous step.

    • If the approach is Stage File, the specifications for staging and inserting a CSV file will be shown so that they can be mapped in a transformation. The pattern used is to match only one file. If the pattern matches more than one file, the activity will error with a descriptive message.

      NOTE: The enclosingChar can be either a single quote character ('), or double quote character ("). To use the single quote character, use either the octal (\047) or the hex (0x27) representations or use the double single-quoted escape (''). When a field contains this character, escape it using the same character.

      • Stage File Request:

        Request Schema Field/NodeNotes
        stageNameInternal Snowflake stage, table name, or path
        destinationPrefixPath or prefix under which the data will be uploaded on the Snowflake stage
        fileContentData file contents, in CSV format, that is to be staged for uploading to the Snowflake table
        destinationFileNameDestination file name to be used on the Snowflake stage
        compressDataBoolean flag for whether to compress the data before uploading it to the internal Snowflake stage
        patternRegular-expression pattern used for finding the file on the stage; if compressData is true, [.]gz is appended to the pattern
        fileFormatNode name for the next two fields
        nullIfA string to be converted to SQL NULL; by default, it is an empty string. See the NULL_IF option of the Snowflake COPY INTO documentation.
        enclosingCharCharacter used to enclose data fields; see the FIELD_OPTIONALLY_ENCLOSED_BY option of the Snowflake COPY INTO documentation
      • Stage File Response:

        Response Schema Field/NodeNotes
        fileName of the staged CSV file processed when putting data into the Snowflake table
        statusStatus returned
        rowsParsedNumber of rows parsed from the CSV file
        rowsLoadedNumber of rows loaded from the CSV file into the Snowflake table without error
    • If the approach is SQL Insert, the table columns will be shown, allowing them to be mapped in a transformation.

      • SQL Insert Request:

        Request Schema Field/NodeNotes
        tableNode showing the table name
        column_AFirst table column name
        column_BSecond table column name
        . . .Succeeding table columns
      • SQL Insert Response:

        Response Schema Field/NodeNotes
        statusBoolean flag telling if record insertion was successful
        errorMessageDescriptive error message if a failure during insertion
        recordsInsertedNumber of records inserted if a successful insertion

    If the operation uses a transformation, the data schemas will be displayed again later during the transformation mapping process, where you can map to target fields using source objects, scripts, variables, custom values, and more.

    The Snowflake connector uses the Snowflake JDBC Driver and the Snowflake SQL commands.

  • Back: Click to save the configuration for this step and return to the previous step.

  • Finished: Click to save the configuration for this step and close the activity configuration.

  • Discard Changes: After making changes to this step, click Discard Changes to close the configuration without saving changes to this step. A message will ask you to confirm that you want to discard changes.

Next Steps

After configuring a Snowflake put activity, complete the configuration of the operation by adding and configuring other activities, transformations, or scripts as operation steps. You can also configure an operation's operation settings, which include the ability to chain operations together that are in the same or different workflows.

Snowflake put activities can be used as a target with these operation patterns:

Other patterns are not valid using Snowflake put activities. See the validation patterns on the Operation Validity page.

To use the activity with scripting functions, write the data to a temporary location and then use that temporary location in the scripting function.

When ready, deploy and run the operation and validate behavior by checking the operation logs.

On This Page

Last updated:  Nov 13, 2019

  • No labels