Google BigQuery Connection¶

Introduction¶

A Google BigQuery connection, created using the Google BigQuery connector, establishes access to Google BigQuery. Once a connection is configured, you can create instances of Google BigQuery activities associated with that connection to be used either as sources (to provide data in an operation) or as targets (to consume data in an operation).

Create or Edit a Google BigQuery Connection¶

A new Google BigQuery connection is created using the Google BigQuery connector from one of these locations:

• The design component palette's Connections tab (see Design Component Palette).
• The Global Connections page (see Create a Global Connection in Global Connections).

An existing Google BigQuery connection can be edited from these locations:

• The design component palette's Connections tab (see Design Component Palette).
• The project pane's Components tab (see Component Actions Menu in Project Pane Components Tab).
• The Global Connections page (see Edit a Global Connection in Global Connections).

Configure a Google BigQuery Connection¶

Each user interface element of the Google BigQuery connection configuration screen is described below.

• Connection Name: Enter a name to use to identify the connection. The name must be unique for each Google BigQuery connection and must not contain forward slashes / or colons :. This name is also used to identify the Google BigQuery endpoint, which refers to both a specific connection and its activities.

• Project ID: Enter the project_id provided by the credentials JSON file as described in Google BigQuery Prerequisites.

• Private Key: Enter the private_key provided by the credentials JSON file as described in Google BigQuery Prerequisites. The private key should be entered in this format:

  -----BEGIN PRIVATE KEY-----
  <private_key>
  -----END PRIVATE KEY----
  

  Note

  All newline characters (\n) present in the private_key string must be replaced with a carriage return in the Private Key field. Otherwise, an error will occur when testing the connection.

• Client Email: Enter the client_email provided by the credentials JSON file as described in Google BigQuery Prerequisites.

• Test: Click to verify the connection using the provided configuration. When the connection is tested, the latest version of the connector is downloaded by the agent(s) in the agent group associated with the current environment. This connector supports suspending the download of the latest connector version by using the Disable Auto Connector Update organization policy.

• Save Changes: Click to save and close the connection configuration.

• Discard Changes: After making changes to a new or existing configuration, click to close the configuration without saving. A message asks you to confirm that you want to discard changes.

• Delete: After opening an existing connection configuration, click to permanently delete the connection from the project and close the configuration (see Component Dependencies, Deletion, and Removal). A message asks you to confirm that you want to delete the connection.

Next Steps¶

After a Google BigQuery connection has been created, you place an activity type on the design canvas to create activity instances to be used either as sources (to provide data in an operation) or as targets (to consume data in an operation).

Menu actions for a connection and its activity types are accessible from the project pane and design component palette. For details, see Actions Menus in Connector Basics.

These activity types are available:

• Data Transfer: Transfers data from a data source to a dataset in Google BigQuery and is intended to be used as a target in an operation.

• Invoke Routine: Invokes a routine (such as a stored procedure, user-defined function, or table function) for a dataset in Google BigQuery and is intended to be used as a target in an operation.

• Query: Queries a table in a dataset in Google BigQuery and is intended to be used as a source in an operation.

• Insert Record: Inserts records into a table in Google BigQuery and is intended to be used as a target in an operation.

• Create Structure: Creates datasets or tables in Google BigQuery and is intended to be used as a target in an operation.

• Update Structure: Updates structures in Google BigQuery and is intended to be used as a target in an operation.

• Update Record: Updates records in a dataset in Google BigQuery and is intended to be used as a target in an operation.

• Delete Structure: Deletes structures from Google BigQuery and is intended to be used as a target in an operation.

• Execute Custom Query: Executes custom queries against tables in datasets in Google BigQuery and is intended to be used as a source in an operation.