Operation Debug Logging for Private Agents¶

Introduction¶

Operation debug logs are additional log files and data that are generated for operations when operation debug logging is enabled. Operation debug logging is used mainly for debugging problems during testing and should not be turned on in a production environment.

On Private Agents, enabling operation debug logging can result in the generation of three types of logs:

• Operation Debug Log Files: Files written to a Private Agent directory that are generated when operation debug logging is enabled either on an individual Cloud Studio or Design Studio operation or for all operations on a Private Agent through an agent's configuration file. The files are accessible directly on Private Agents and are downloadable through the Management Console Agents > Agents and Activities pages. The files are subject to the retention policy configured in the agent's cleanup rules, which is 1 day by default.

  Caution

  When operation debug logging is turned on, generated debug log files may be large in quantity and/or size. It is recommended to disable debug logging as soon as possible.

• Component Input and Output Data: Data written to a Cloud Studio operation log when operation debug logging is enabled on a Cloud Studio operation running on a 10.48 or later Private Agent. The data is retained for 30 days by Harmony.

  Warning

  When component input and output data are generated, all request and response data for that operation are logged to the Harmony cloud and remain there for 30 days. Be aware that personally identifiable information (PII) and sensitive data such as credentials provided in a request payload will be visible in clear text in the input and output data within the Harmony cloud logs.

• API Operation Logs: Operation logs for both successful and unsuccessful API operations (configured for Custom APIs or OData Services). When operation debug logging is not enabled, only unsuccessful API operations are logged in the operation logs.

  Note

  The steps to enable logging of all API operations for the entire agent differ from the steps outlined in Entire Private Agent later on this page. For information on enabling API operation logging for the entire agent, see the API Operation section in Edit the Agent Configuration File.

Enabling Operation Debug Logging¶

Operation debug logging can be enabled on an individual operation or on an entire Private Agent. Component input and output data are generated by default on Private Agents 10.48 or later but can be disabled using additional settings in the Private Agent configuration file.

Individual Operation¶

Operation-level debug logging can help if you are having issues with an individual operation and do not need to turn on debug logging at the agent level, which may generate files that are large in quantity and/or size.

You can enable debug logging for an individual operation using either Cloud Studio operation options or Design Studio operation options:

• Cloud Studio: Use an operation's actions menu to select Settings. In the Options tab, select Enable Debug Mode Until and set a date for debug logging to be turned off. The option to cascade the debug mode settings to any child operations is also available.

• Design Studio: Right-click the background of any operation and select Options. In the Operation Options window, select Enable Debug Mode Until and set a date for debug logging to be turned off.

This date is limited to 2 weeks from the current date. Operation debug logging will be turned off at the beginning of this date (that is, at 12:00 am) using the time zone of the agent.

Entire Private Agent¶

Agent-level operation debug logging can be enabled from the Private Agent configuration file for all operations running on a Private Agent.

To enable debug logging, edit the Private Agent configuration file as follows, and then restart Jitterbit services.

• Under the [TransformationEngine] section, set WriteLog=true.

• Under the [OperationEngine] section, set Debug=true.

The files created in this directory can become very large. Turn off debug logging as soon as you can, and delete the files when they are no longer needed.

Additional Settings¶

Private Agents that are version 10.48 and later have the section [VerboseLogging] present in their Private Agent configuration file with the setting verbose.logging.enable=true, which enables generation of component input and output data:

[VerboseLogging]
verbose.logging.enable=true

When upgrading an earlier Private Agent version to 10.48 or later, the generation of input and output data is automatically enabled by default, even if the [VerboseLogging] section and the verbose.logging.enable setting are not present in the configuration file.

To disable generation of component input and output data, you must add the [VerboseLogging] section and set verbose.logging.enable=false to prevent component input and output data from being logged to the Harmony cloud.

Additional settings in the [VerboseLogging] section should not be changed unless instructed by Jitterbit Support.

Operation Debug Log Files¶

Once operation debug logging is enabled on a Cloud Studio operation, Design Studio operation, or entire Private Agent, operation debug log files for the applicable operations are generated when those operations are executed.

Debug Log File Access¶

Operation debug log files can be accessed from the Management Console or directly from the agent machine:

• Management Console
  You can download a ZIP archive of the operation debug log files from these Management Console pages:
  • Agents: From the Agents > Agents page, select the agent, and then use the Actions dropdown to select Download Agent Log.
  • Activities: From the Activities page, select the operation, then go to the Debug Files tab and use the link to download the logs.

• Private Agent
  On a Private Agent, you can access a ZIP archive of the logs in these default locations:

  • Windows: C:\Program Files\Jitterbit Agent\DataInterchange\Temp\Debug
  • Linux: /usr/local/jitterbit/DataInterchange/Temp/Debug

  To change the operation debug log directory, edit the Private Agent configuration file as follows, and then restart Jitterbit services: - Under the [TransformationEngine] section, set WriteLogDir to the desired directory location.

Debug Log File Retention Policy¶

By default, operation debug log files are deleted automatically after 1 day.

This time period is configurable in CleanupRules.xml, which is accessible on Private Agents in these default locations:

• Windows: C:\Program Files\Jitterbit Agent

• Linux: /usr/local/jitterbit/

Within the CleanupRules.xml file, you can edit this cleanup rule to change the operation debug log retention policy:

  <CleanupRule>
      <DirectoryPath SearchSubDirectory = "YES" >DataInterchange/Temp/Debug</DirectoryPath>
      <Pattern>*</Pattern>
      <FileAge NumDays = "1" Comparator = "GE"/>
      <FileSize Size = "0" Comparator = "GE"/>
  </CleanupRule>

Debug Log File Contents¶

Operation debug log files contain debug messages for each operation activity and include information that is logged before and after each transformation is executed. Information about other actions such as reading from and writing to the Harmony cache is also included.

These files are typically requested by Jitterbit Support if more information is required during troubleshooting.

Component Input and Output Data¶

Component input and output data is generated when a Cloud Studio operation that has operation debug logging enabled at the operation level is executed on a 10.48 or later Private Agent that also has the default configuration settings. Input and output data are available only for activities and transformations used as operation steps.

Input and Output Data Access¶

Component input and output data is accessed from the Cloud Studio operation log screen in additional table rows shown for each operation step that has input and output data. In the Input/Output column, icons for activities and transformations used as operation steps can be clicked to show or hide the component input and output data for that individual operation step. Each input and output log entry is limited to 100MB. If the data for an individual input or output log entry exceeds 100MB, no data will be displayed.

• Copy: Click the copy icon to copy the log data to your clipboard. When clicked, a clipboard icon is temporarily displayed.
• Download: Click the download icon to download the input or output data. The file is named with the operation step name followed by _input or _output as appropriate.

Input and Output Data Retention Policy¶

Component input and output data is retained for 30 days by Harmony.

Input and Output Data Contents¶

Component input and output data is generated for certain steps of an operation. The data is appropriate to the function of a step. The input is the structure that the component consumes, while the output is the structure that the component produces.

Input and output data is generated for these components used as operation steps:

• Activities: Input and output data and any related information for the execution of the activity. Some activities may have multiple input and output files.
• Transformations: Input and output data for the execution of the transformation.

Input and output data is not applicable to scripts when they are used as operation steps. Script content, such as that produced by the WritetoOperationLog function, is displayed in the log messages of an operation log.

Input and Output Data Troubleshooting¶

Agent logs related to the input and output data should not normally need to be accessed. If you think there may be an issue with generating the input and output data, you can check the Verbose Log Shipper service log files on a Private Agent:

<JITTERBIT_HOME>/VerboseLogShipper/verbose-log-shipper.out.log

The Verbose Log Shipper service can be restarted by restarting a Private Agent.

On Linux Private Agents, you can also disable and re-enable this service using these Jitterbit command line options:

jitterbit start_verboselogshipper

jitterbit stop_verboselogshipper

These options can also be viewed using jitterbit --help.