Apache Parquet Connector¶
The Jitterbit Harmony Apache Parquet Connector establishes access to Apache Parquet.
The Apache Parquet connector provides an interface for creating an Apache Parquet connection, the foundation used for generating instances of Apache Parquet activities. Activities interact with Apache Parquet through the connection and are intended to be used as sources (to provide data in an operation) or targets (to consume data in an operation).
The Apache Parquet connector is accessed from the design component palette's Connections tab (see Design Component Palette).
This connector can be used only with Private Agents. In addition, it is a Connector SDK-based connector, which may be referred to by Jitterbit when communicating changes made to connectors built with the Connector SDK.
See Jitterbit's comprehensive Apache Parquet connection documentation, which is provided at a dedicated website. Configuration details such as these are included:
- Getting Started: Initial steps for establishing a connection.
- Advanced Features: User-defined views and SSL configuration.
- Data Model: The data model that the connector uses to represent the endpoint.
- Advanced Configurations Properties: Properties that can be defined to configure the connection for both basic and advanced configurations.
Together, a specific Apache Parquet connection and its activities are referred to as an Apache Parquet endpoint:
Query: Retrieves records from a table at Apache Parquet and is intended to be used as a source in an operation. (Generically documented in Query Activities.)
Create: Inserts a record into a table at Apache Parquet and is intended to be used as a target in an operation. (Generically documented in Create Activities.)
Prerequisites and Supported API Versions¶
The Apache Parquet connector requires the use of an agent version 10.1 or later. These agent versions automatically download the latest version of the connector when required.
This connector requires the use of a Private Agent.
Refer to the documentation for information on the schema nodes and fields.
OAuth authentication is supported as described in Connections under Configure OAuth Connections.
If you experience issues with the Apache Parquet connector, these troubleshooting steps are recommended:
Ensure the Apache Parquet connection is successful by using the Test button in the configuration screen. If the connection is not successful, the error returned may provide an indication as to the problem.
Check the operation logs for any information written during execution of the operation.
Enable operation debug logging for Private Agents to generate additional log files and data.
Enable connector verbose logging for this connector using this specific configuration entry of logger name and level:
<logger name="org.jitterbit.connector.verbose.logging.Parquet" level="TRACE"/>
Check the agent logs for more information.