Skip to Content

Apache Parquet Connection

Introduction

An Apache Parquet connection, created using the Apache Parquet connector, establishes access to Apache Parquet. These activities, once configured, interact with Apache Parquet through the connection.

Create or Edit an Apache Parquet Connection

A new Apache Parquet connection is created using the Apache Parquet connector from one of these locations:

An existing Apache Parquet connection can be edited from these locations:

Configure an Apache Parquet Connection

Each user interface element of the Apache Parquet connection configuration screen is described below.

Tip

Fields with a variable icon Variable icon support using global variables, project variables, and Jitterbit variables. Begin either by typing an open square bracket [ into the field or by clicking the variable icon to display a list of the existing variables to choose from.

  • Connection Name: Enter a name to use to identify the connection. The name must be unique for each Apache Parquet connection and must not contain forward slashes (/) or colons (:). This name is also used to identify the Apache Parquet endpoint, which refers to both a specific connection and its activities.

  • Authentication: When selected (default), shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • Auth Scheme: Select the authorization scheme, one of:

      • None

      • Basic

      • Auto

      • Digest

      • NTLM

      • Negotiate

      • OAuth

      • OAuthJWT

      • SFTP

      • FTP

      • SSLCertificate

      • AwsRootKeys

      • AwsIAMRoles

      • AwsEC2Roles

      • AwsMFA

      • AwsTempCredentials

      • AwsCredentialsFile

      • AzureAD

      • AzureServicePrincipal

      • AzureMSI

      • OKTA

      • ADFS

      • OneLogin

      • PingFederate

      • HMAC

    • Access Key: Enter the access key.

    • Secret Key: Enter the secret key.

    • Api Key: Enter the API key.

    • User: Enter the user.

    • Password: Enter the password.

    • Share Point Edition: Select the SharePoint edition, one of:

      • SharePointOnline

      • SharePointOnPremise

  • Connection: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • URI: Enter the URI.

    • Data Model: Select the data model, one of:

      • None

      • Document

      • FlattenedDocuments

      • Relational

    • Region: Enter the region.

    • Project Id: Enter the project ID.

    • Oracle Namespace: Enter the Oracle namespace.

    • Storage Base URL: Enter the storage base URL.

    • Use Virtual Hosting: When selected (default), specifies to use virtual hosting.

  • AWS Authentication: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • AWS Access Key: Enter the AWS access key.

    • AWS Secret Key: Enter the AWS secret key.

    • AWS Role ARN: Enter the AWS role ARN.

    • AWS Principal ARN: Enter the AWS principal ARN.

    • AWS Region: Select the AWS region, one of:

      • OHIO

      • NORTHERNVIRGINIA

      • NORTHERNCALIFORNIA

      • OREGON

      • CAPETOWN

      • HONGKONG

      • MUMBAI

      • OSAKA

      • SEOUL

      • SINGAPORE

      • SYDNEY

      • TOKYO

      • CENTRAL

      • BEIJING

      • NINGXIA

      • FRANKFURT

      • IRELAND

      • LONDON

      • MILAN

      • PARIS

      • STOCKHOLM

      • BAHRAIN

      • SAOPAULO

      • GOVCLOUDEAST

      • GOVCLOUDWEST

    • AWS Session Token: Enter the AWS session token.

    • MFA Serial Number: Enter the MFA serial number.

    • MFA Token: Enter the MFA token.

  • Azure Authentication: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • Azure Storage Account: Enter the Azure storage account.

    • Azure Access Key: Enter the Azure access key.

    • Azure Shared Access Signature: Enter the Azure shared access signature.

    • Azure Tenant: Enter the Azure tenant.

    • Azure Environment: Select the Azure environment, one of:

      • GLOBAL

      • CHINA

      • GERMANY

      • USGOVT

      • USGOVTDOD

  • SSO: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • SSO Login URL: Enter the SSO login URL.

    • SSO Properties: Enter the SSO properties.

  • OAuth: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • Initiate OAuth: Select the OAuth setting, one of:

      • OFF

      • GETANDREFRESH

      • REFRESH

    • OAuth Version: Select the OAuth version, one of:

      • Disabled

      • 1.0

      • 2.0

    • OAuth Client Id: Enter the OAuth client ID.

    • OAuth Client Secret: Enter the OAuth client secret.

    • OAuth Access Token: Enter the OAuth access token.

    • OAuth Access Token Secret: Enter the OAuth access token secret.

    • Callback URL: Enter the callback URL.

    • OAuth Grant Type: Select the OAuth grant type, one of:

      • CODE

      • CLIENT

      • PASSWORD

    • OAuth Password Grant Mode: Select the OAuth password grant mode, one of:

      • Post

      • Basic

    • OAuth Include Callback URL: When selected (default), specifies to include the OAuth callback URL.

    • OAuth Authorization URL: Enter the OAuth authorization URL.

    • OAuth Access Token URL: Enter the OAuth access token URL.

    • OAuth Refresh Token URL: Enter the OAuth refresh token URL.

    • OAuth Request Token URL: Enter the OAuth request token URL.

    • OAuth Verifier: Enter the OAuth verifier.

    • Auth Token: Enter the auth token.

    • Auth Key: Enter the auth key.

    • OAuth Params: Enter the OAuth params.

    • OAuth Refresh Token: Enter the OAuth refresh token.

    • OAuth Expires In: Enter the time in seconds that the OAuth token expires in.

    • OAuth Token Timestamp: Enter the OAuth token timestamp.

  • JWT OAuth: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • OAuth JWT Cert: Enter the OAuth JWT certificate.

    • OAuth JWT Cert Type: Select the OAuth JWT certificate type, one of:

      • USER

      • MACHINE

      • PFXFILE

      • PFXBLOB

      • JKSFILE

      • JKSBLOB

      • PEMKEY_FILE

      • PEMKEY_BLOB

      • PUBLIC_KEY_FILE

      • PUBLIC_KEY_BLOB

      • SSHPUBLIC_KEY_FILE

      • SSHPUBLIC_KEY_BLOB

      • P7BFILE

      • PPKFILE

      • XMLFILE

      • XMLBLOB

      • GOOGLEJSON

      • GOOGLEJSONBLOB

    • OAuth JWT Cert Password: Enter the OAuth JWT certificate password.

    • OAuth JWT Cert Subject: Enter the OAuth JWT certificate subject.

    • OAuth JWT Issuer: Enter the OAuth JWT issuer.

    • OAuth JWT Subject: Enter the OAuth JWT subject.

  • Kerberos: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • Kerberos KDC: Enter the Kerberos KDC.

    • Kerberos Realm: Enter the Kerberos realm.

    • Kerberos SPN: Enter the Kerberos SPN.

    • Kerberos Keytab File: Enter the Kerberos keytab file.

    • Kerberos Service Realm: Enter the Kerberos service realm.

    • Kerberos Service KDC: Enter the Kerberos service KDC.

    • Kerberos Ticket Cache: Enter the Kerberos ticket cache.

  • SSL: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • SSL Client Cert: Enter the SSL client certificate.

    • SSL Client Cert Type: Select the SSL client certificate type, one of:

      • USER

      • MACHINE

      • PFXFILE

      • PFXBLOB

      • JKSFILE

      • JKSBLOB

      • PEMKEY_FILE

      • PEMKEY_BLOB

      • PUBLIC_KEY_FILE

      • PUBLIC_KEY_BLOB

      • SSHPUBLIC_KEY_FILE

      • SSHPUBLIC_KEY_BLOB

      • P7BFILE

      • PPKFILE

      • XMLFILE

      • XMLBLOB

    • SSL Client Cert Password: Enter the SSL client certificate password.

    • SSL Client Cert Subject: Enter the SSL client certificate subject.

    • SSL Mode: Select the SSL mode, one of:

      • AUTOMATIC

      • NONE

      • IMPLICIT

      • EXPLICIT

    • SSL Server Cert: Enter the SSL server certificate.

  • SSH: When selected, shows additional settings. For documentation, see Establishing a Connection and Additional Configurations Properties:

    • SSH Auth Mode: Select the SSH auth mode, one of:

      • None

      • Password

      • Public_Key

    • SSH Client Cert: Enter the SSH client certificate.

    • SSH Client Cert Password: Enter the SSH client certificate password.

    • SSH Client Cert Type: Select the SSH client certificate type, one of:

      • USER

      • MACHINE

      • PFXFILE

      • PFXBLOB

      • JKSFILE

      • JKSBLOB

      • PEMKEY_FILE

      • PEMKEY_BLOB

      • PUBLIC_KEY_FILE

      • PUBLIC_KEY_BLOB

      • SSHPUBLIC_KEY_BLOB

      • P7BFILE

      • P7BBLOB

      • SSHPUBLIC_KEY_FILE

      • PPKFILE

      • PPKBLOB

      • XMLFILE

      • XMLBLOB

  • Use Proxy Settings: Select to use Private Agent proxy settings.

  • Advanced Configurations: When selected, shows the Advanced Configurations Properties table, which is used to add additional configuration properties as key-value pairs.

    Click the add icon add icon to add a row to the table below and enter a key-value pair for each property.

    To save the row, click the submit icon submit icon in the rightmost column.

    To edit or delete a single row, hover over the rightmost column and use the edit icon edit icon or delete icon delete icon.

    To delete all rows, click Clear All.

    Important

    Fields in the Advanced Configurations table display the variable icon Variable icon only in edit mode. For these fields' variable values to be populated at runtime, the agent version must be at least 10.75 / 11.13.

  • Test: Click to verify the connection using the specified configuration. When the connection is tested, the latest version of the connector is downloaded by the agent(s) in the agent group associated with the current environment. This connector supports suspending the download of the latest connector version by using the Disable Auto Connector Update organization policy.

  • Save Changes: Click to save and close the connection configuration.

  • Discard Changes: After making changes to a new or existing configuration, click to close the configuration without saving. A message asks you to confirm that you want to discard changes.

  • Delete: After opening an existing connection configuration, click to permanently delete the connection from the project and close the configuration (see Component Dependencies, Deletion, and Removal). A message asks you to confirm that you want to delete the connection.

Configure OAuth Connections

This connector supports a connection using OAuth by using specific configuration properties. After selecting an auth scheme that includes Initiate OAuth, you will be able to specify the configuration in the fields that are displayed.

Note

Additional IDs, secrets, scopes, and other authentication tokens may be required in order to successfully complete the OAuth authentication. Check the endpoint documentation for additional information on their requirements.

For this connector, two scenarios are supported. In either scenario, you will need to use a third-party tool (such as Postman or a similar tool) to obtain the required access token or tokens from the endpoint involved. Both of these scenarios work only for single-agent groups. These are Agent Groups that consist of a single agent.

  • Scenario 1: Using an OAuth access token that expires after a set period of time.

    With this configuration, you supply an OAuth access token that lasts for a set period of time, as determined by the particular endpoint. The connector will use the token to make the connection. The connection, however, will no longer work once the access token expires. You would configure the connector to use the token by setting these properties:

    Configuration Property Value
    Initiate OAuth OFF
    OAuth Access Token Set to the value of the OAuth access token obtained from the endpoint
  • Scenario 2: Using an OAuth access token combined with an OAuth refresh token to enable automatic refresh of access.

    With this configuration, the connector will use a refresh token to obtain a new access token once the current access token has expired. This is generally the best method of configuring OAuth.

    You supply an OAuth access token and an OAuth refresh token, as determined by the particular endpoint. You would configure the connector to use these tokens by setting these properties:

    Configuration Property Value
    Initiate OAuth REFRESH
    OAuth Access Token Set to the value of the OAuth access token obtained from the endpoint
    OAuth Refresh Token Set to the value of the OAuth refresh token obtained from the endpoint

Note

If using multiple OAuth-based Apache Parquet connections on the same Private Agent, set the configuration property OAuthSettingsLocation to a unique writeable agent path for each connection. In the jitterbit.conf file for the agent, set EnableLocalFileLocation=true. If you don't perform these two steps, the multiple connections will use the same default path and overwrite each other.

Next Steps

After an Apache Parquet connector connection has been created, you place an activity type on the design canvas to create activity instances to be used either as sources (to provide data in an operation) or as targets (to consume data in an operation).

Menu actions for a connection and its activity types are accessible from the project pane and design component palette. For details, see Actions Menus in Connector Basics.

These activity types are available:

  • Query: Retrieves records from a table at Apache Parquet and is intended to be used as a source in an operation.

  • Create: Inserts a record into a table at Apache Parquet and is intended to be used as a target in an operation.

  • Execute: Executes a procedure at Apache Parquet and is intended to be used as a target in an operation.