Jitterbit private agent automatic registration and autoscaling

Introduction

Agent automatic registration allows an administrator to pre-configure agents in a secure manner to automatically start up and register. This allows multiple agents to be created automatically (automatic registration) and as demand increases (autoscaling).

The agent automatic registration and autoscaling process works with Linux, Windows, and Docker private agents version 10.14 or higher. It is not available for the cloud agent groups.

These additional pages are available in this section:

• Docker agent automatic registration tutorial

  How to set up two (or more) Docker private agents that will automatically register to a designated private agent group in your organization.

• Private agents and kubernetes

  How to configure a group of private agents to be auto-registering and autoscaling by using Kubernetes and Helm.

Overview

The normal agent startup flow expects a credentials.txt file to be present within the agent installation directory. This file contains encrypted credentials for a specific agent. As this credentials file is unique, it cannot be duplicated and used with other agents.

The automatic registration process replaces this startup flow and instead allows each instance of an agent to create its own credentials.txt file upon startup.

This process requires a register.json file, which isn't unique to an agent and hence can be copied, duplicated, or imaged to all the agents in an agent group.

The register.json file is instead unique to an organization's specific agent group.

The automatic registration process requires that both of these criteria be met:

• A register.json file is present on the agent

• A credentials.txt file is not present on the agent

Agent registration file

The Agent Registration File is a JSON file called register.json. It is placed in the same location as the credentials.txt file:

• On Linux or Docker, in /opt/jitterbit/Resources

• On Windows, in C:\Program Files (x86)\Jitterbit Agent\Resources

The register.json file has these properties:

• cloudUrl: URL of the Harmony Cloud (required property)

• token: Agent registration access token (required property if username and password aren't defined)

  Important

  If using agents version 10.83 / 11.21 or later, we recommend using Agent registration access tokens over encrypted Agent Installer user credentials. Agent Registration access tokens do not expire and can be managed in the Management Console at any time.

• username: Username (without token), and encrypted using the JitterbitUtils application packaged with the default agent installation, as described below (required property if token isn't defined)

• password: Password (without token), and encrypted using JitterbitUtils, as described below (required property if token isn't defined)

• agentGroupId: Agent group ID, visible on the Management Console Agents page

• agentNamePrefix: A prefix added to the UTC timestamp of agent creation to form the agent name, as shown in the Management Console. Variables that are automatically substituted can be used to create unique names that identify an agent both in the Management Console and in the host system (optional property).

• deregisterAgentOnDrainstop: When set to true, performs agent deregistration upon agent drainstop or JVM shutdown and removes the agent from the Management Console's Agents page. Defaults to false (optional property)

• retryCount: Number of retries to be attempted if the agent is having issues connecting to the Harmony Cloud for registration. Valid range: 0 to 300, defaults to 10 (optional property).

• retryIntervalSeconds: Number of seconds that the agent will wait before retrying a connection. This interval doubles with each new attempt, to a maximum of 600 seconds (10 minutes). Valid range: 5 to 600, defaults to 5 (optional property).

Variables in the agentNamePrefix are automatically substituted:

• %ip%: IP address of the system or container
• %host%: Host name of the system; for a Docker instance, this is the Docker Container ID; for Kubernetes, it is set to the container's pod name
• %guid%: A randomly generated GUID of length 8 characters. This is highly recommended in autoscaling scenarios to assign a random value to each agent's name.

• For example: an agentNamePrefix of auto_%ip%_%host%_%guid% will result in an agent name such as

  auto_172.17.0.2_b2a546cc080e_59a260ca_2020-02-14 17:43:03.033

Example register.json

{
  "cloudUrl": "https://example.jitterbit.com",
  "token": "jb_a1b23c45-d6ef-789a-b0cd-e12fabcd3e4f",
  "agentGroupId": "21",
  "agentNamePrefix": "auto_%ip%_%host%_%guid%",
  "deregisterAgentOnDrainstop": "true",
  "retryCount": "10",
  "retryIntervalSeconds": "5"
}

{
  "cloudUrl": "https://example.jitterbit.com",
  "username": "$000MTF/XSfev/KjlB/M1Q5vFkPLBKU5LKFYm.../PwoHRiSGGLlJ",
  "password": "$006WIdHtws7wrjgCNAoabUzu0f+HsyzAfAZtTpauvsHBE=",
  "agentGroupId": "21",
  "agentNamePrefix": "auto_%ip%_%host%_%guid%",
  "deregisterAgentOnDrainstop": "true",
  "retryCount": "10",
  "retryIntervalSeconds": "5"
}

Edit on a Windows operating system

If you are editing the register.json file on a Windows operating system for use with Docker, use an editor such as Notepad++ that allows you to set the line endings correctly. UNIX (LF) linefeeds are required for the register.json to work correctly on a Docker system. This is what you should see when editing the file in Notepad++: Unix (LF)

Encrypt the username and password

The username and password are encrypted using the JitterbitUtils application (for Linux and Docker, with the script jitterbit-utils) packaged with the default agent installation. Copy the results of this command into the JSON file in the locations as described above.

cd /opt/jitterbit/bin && jitterbit-utils -e "first.last@example.com" "somePassw0Rd1"

docker run --rm jitterbit/agent:latest jitterbit-utils -e "first.last@example.com" "somePassw0Rd1"

cd C:\Program Files\Jitterbit Agent\bin && JitterbitUtils.exe -e "first.last@example.com" "somePassw0Rd1"

$000MTF/XSfev/KjlB/M1Q5vFkPLBKU5LKFYm.../PwoHRiSGGLlJ
$006WIdHtws7wrjgCNAoabUzu0f+HsyzAfAZtTpauvsHBE=

docker run --rm jitterbit/agent echo "\"pa\$\$\`\\/\\/0rd\""

"pa$$`\/\/0rd"

Agent logic

The Jitterbit Agent's Jitterbit Tomcat service is responsible for checking and executing the automatic registration process. The process looks for a register.json file and if present, uses it to create a credentials.txt file. At the end of the process, the register.json file is deleted as it is no longer required for agent startup. Instead, the credentials.txt file that was created will be used.

Other agent services—such as clean up and the scheduler—will simply wait during startup until the credentials.txt file is available.

Agent autoscaling and provisioning

Configured automatic registration agents are required for autoscaling. Once configured, they can be orchestrated with different technologies, either manually or with systems such as Kubernetes.

Amazon EC2

1. Set up authentication credentials:
   • If using agents version 10.83 or later, generate an Agent Registration access token
   • If not, add a user to the organization and assign the user the Agent Install permission. Encrypt this new user's username and password using JitterbitUtils
2. Create an image box, install the agent, and install any other required software (monitoring, utils, etc.)
3. Stop the agent
4. Create the register.json fileand define its settings. For authentication, insert either:
   • An Agent Registration access token, or
   • An encrypted username and password
5. Copy the register.json file into the same directory as the credentials.txt file
6. Delete the credentials.txt file
7. Create an Amazon EC2 image

At this point, this Amazon EC2 image can be used either in an autoscaling group or used manually by creating a new Amazon EC2 instance from the image.

When these instances come online and the agent(s) start up, they will go through the automatic registration process described above.

Docker

When using a Docker agent for autoscaling, the process and flow is the same as described above but with two options: option 1, using mapped files; and option 2, using environment variables.

Option 1: Mapped files

Docker agents require a credentials.txt file to be mapped to the /conf directory.

Instead of providing and mapping a credentials.txt file in the /conf directory, provide the register.json file in that directory using an orchestration tool of choice (such as Kubernetes).

Option 2: Environment variables

If using encrypted agent installer user credentials, another option is to supply the contents of the register.json file using environment variables. This example shows the variables that define the automatic registration:

AUTO_REGISTER="true"
HARMONY_ORIGIN="https://example.jitterbit.com"
AUTO_REGISTER_ENCRYPTED_USERNAME="$000MTF/XSfev/KjlB/M1Q5vFkPLBKU5LKFYm.../PwoHRiSGGLlJ"
AUTO_REGISTER_ENCRYPTED_PASSWORD="$006WIdHtws7wrjgCNAoabUzu0f+HsyzAfAZtTpauvsHBE="
AUTO_REGISTER_AGENT_GROUP_ID="21"
AUTO_REGISTER_AGENT_NAME_PREFIX="auto_%ip%_%host%_%guid%"
AUTO_REGISTER_DEREGISTER_ON_DRAINSTOP="true"
AUTO_REGISTER_RETRY_COUNT="10"
AUTO_REGISTER_RETRY_INTERVAL_SECONDS="5"

QA and configuration testing

To test these features and confirm that you have configured an agent group correctly:

1. Set up authentication credentials:
   • If using agents version 10.83 or later, generate an Agent Registration access token
   • If not, add a user to the organization and assign the user the Agent Install permission. Encrypt this new user's username and password using JitterbitUtils
2. Install a private agent and test with sample operation runs to ensure that it is working correctly during normal installation and that it functions properly
3. Stop the agent and delete or move out the credentials.txt file for the agent
4. Create the register.json and define its settings. For authentication, insert either:
   • An Agent Registration access token, or
   • An encrypted username and password
5. Place the register.json file in the same place where the credentials.txt file was found
6. Start the agent
7. Check the agent log: the agent should automatically register itself as a new agent for the agent group ID that was provided
8. The Management Console Agents page should show the newly self-registered agent
9. Upon successful registration, the agent should delete the register.json file and show instead a credentials.txt file