Skip to end of metadata
Go to start of metadata

Introduction

A Snowflake Get activity retrieves table data from a Snowflake connection and is used as a source in an operation. After configuring a Snowflake connection, you can configure as many Snowflake activities as you like for each Snowflake connection.

Creating a Snowflake Activity

From the design canvas, open the Connectivity tab of the design component palette:

Connectivity tab

Within the Endpoints filter, click the Snowflake connection block to display activities that are available to be used with a Snowflake connection:

Connectivity tab

To create an activity that can be configured, the activity must first be added to an operation on the design canvas. To add an activity to an operation, drag the activity block from the palette to the operation.

For more information about the parts of an operation and adding activities to operations, see Operation Creation and Configuration.

Accessing Menu Actions

After a Snowflake activity has been added to an operation, menu actions for that activity are accessible from the project pane in either the Workflows or the Components tabs, and from the design canvas:

  • Project Pane: In the Workflows or Components tab of the project pane, hover over an activity name and click the actions menu icon to open the actions menu.

  • Design Canvas: Within the operation, click an existing activity block to open the actions menu.

Each of these menu actions is available:

  • View/Edit: This opens the activity configuration screen for you to configure the activity. For details, see Configuring a Snowflake Get Activity later on this page.

  • Delete: This is used to permanently delete the activity.

  • Rename: This positions the cursor on the activity name in the project pane for you to make edits.

  • View Dependencies: This changes the view in the project pane to display any other parts of the project that the activity is dependent on.

  • Remove: Available only from the actions menu on the design canvas, this removes the activity as a step in the operation without deleting it from the project. When you remove an activity that is adjacent to a transformation, if schemas are provided within that activity, they will no longer be referenced by the transformation. Removed components can be accessed or permanently deleted from the project pane.

  • Deploy: This deploys the activity and any components it is dependent on.

  • Configurable Deploy: This opens the deployment screen, where you can select project components to deploy.

  • Add to group: This opens a prompt to create a new custom group or to add the component to an existing group.

  • Duplicate: This creates a copy of the activity as a new, unreferenced component. Upon creating the component copy, the cursor is positioned on the component name within the project pane for you to rename the component.

Configuring a Snowflake Get Activity

Follow these steps to configure a Snowflake Get activity:

  • Step 1 – Select a Snowflake Table
  • Step 2 – Review Data Schema

Step 1 – Select a Snowflake Table

Connectivity tab

  • Name: Enter a name to use to identify the Snowflake Get activity. The name must be unique for each get activity and must not contain forward slashes (/) or colons (:).

  • Select a Table: This section displays tables and views available in the Snowflake endpoint. When reopening an existing activity configuration, only the selected table or view is displayed instead of reloading the entire list.

    NOTE: For information on Snowflake views, see Overview of Views in the Snowflake documentation.

    • Selected Snowflake Table: After a table or view is selected, it will be listed here.
    • Search: Enter any part of the table or view name into the search box to filter the list of tables and views. The search is not case-sensitive. If the list of tables and views are already displayed within the table, the table results will be filtered in real time with each keystroke. To reload tables or views from the endpoint when searching, enter search criteria and then refresh, as described below.
    • Refresh: Click the refresh icon Refresh icon or the word Refresh to reload tables and views from the Snowflake endpoint. This may be useful if you have recently added tables to the Snowflake instance. This action refreshes all metadata used to build the table of tables and views displayed in the configuration.
    • Select a Table: Within the table, click anywhere on a row to select a table or view. Only one table or view can be selected. The information available for each table is fetched from the Snowflake endpoint:
      • Name: The table or view name from Snowflake.
      • Type: The table type from Snowflake.

    TIP: If the table does not populate with available tables and views, the Snowflake connection may not be successful. Ensure you are connected by reopening the connection and retesting the credentials.

  • Save & Exit: If enabled, click to save the configuration for this step and close the activity configuration.

  • Next: Click to temporarily store the configuration for this step and continue to the next step. The
    configuration will not be saved until you click the Finished button on the last step.

  • Discard Changes: After making changes, click Discard Changes to close the configuration without saving changes made to any step. A message will ask you to confirm that you want to discard changes.

Step 2 – Review Data Schema

Connectivity tab

  • Data Schema: The request and response data schemas for Snowflake will be displayed. If the operation uses a transformation, the data schemas will be displayed again later during the transformation mapping process, where you can map to target fields using source objects, scripts, variables, custom values, and more.

    The Snowflake connector uses the Snowflake JDBC Driver and the Snowflake SQL commands.

    NOTE: The stageName is the name of an internal Snowflake staging area, which needs to be pre-existing before running an operation using this activity. The file is first copied to the staging area before it is available to the connector.

    • Request:

      Request Schema Field/NodeNotes
      stageNameName of the internal Snowflake stage used while getting table data
      fileNameName of the file on the internal Snowflake stage to be used while getting table data
      compressDataBoolean flag for whether to compress the data before uploading it to the internal Snowflake stage
    • Response:

      Response Schema Field/NodeNotes
      fileNameName of the file on the internal Snowflake stage used while getting table data (identical to request)
      contentContent of the Snowflake table in CSV format
  • Back: Click to temporarily store the configuration for this step and return to the previous step.

  • Finished: Click to save the configuration for all steps and close the activity configuration.

  • Discard Changes: After making changes, click Discard Changes to close the configuration without saving changes made to any step. A message will ask you to confirm that you want to discard changes.

NOTE: When providing Snowflake database objects such as stageName or fileName in a request schema, the Snowflake connector automatically adds the namespace for those objects instead of requiring that you provide the namespace yourself. For more information on using database and schema names with Snowflake, see the Snowflake Database, Schema, & Share DDL documentation.

Next Steps

After configuring a Snowflake Get activity, complete the configuration of the operation by adding and configuring other activities, transformations, or scripts as operation steps. You can also configure an operation's operation settings, which include the ability to chain operations together that are in the same or different workflows.

Snowflake Get activities can be used as a source with these operation patterns:

Other patterns are not valid using Snowflake Get activities. See the validation patterns on the Operation Validity page.

To use the activity with scripting functions, write the data to a temporary location and then use that temporary location in the scripting function.

When ready, deploy and run the operation and validate behavior by checking the operation logs.

On This Page

Last updated:  Jan 15, 2020

  • No labels