Skip to Content

Agent Automatic Registration and Autoscaling

Introduction

Agent automatic registration allows an administrator to pre-configure agents in a secure manner to automatically start up and register. This allows multiple agents to be created automatically (automatic registration) and as demand increases (autoscaling).

The agent automatic registration and autoscaling process works with Linux, Windows, and Docker Private Agents version 10.14 or higher. It is not available for the Cloud Agent Groups.

These additional pages are available in this section:

Overview

The normal agent startup flow expects a credentials.txt file to be present within the agent installation directory. This file contains encrypted credentials for a specific agent. As this credentials file is unique, it cannot be duplicated and used with other agents.

The automatic registration process replaces this startup flow and instead allows each instance of an agent to create its own credentials.txt file upon startup.

This process requires a register.json file, which isn't unique to an agent and hence can be copied, duplicated, or imaged to all the agents in an Agent Group.

The register.json file is instead unique to an organization's specific Agent Group.

The automatic registration process requires that both of these criteria be met:

  • A register.json file is present on the agent

  • A credentials.txt file is not present on the agent

Agent Registration File

The Agent Registration File is a JSON file called register.json. It is placed in the same location as the credentials.txt file:

  • On Linux or Docker, in /opt/jitterbit/Resources

  • On Windows, in C:\Program Files (x86)\Jitterbit Agent\Resources

The register.json file has these properties:

  • cloudUrl: URL of the Harmony Cloud (required property)

  • token: Agent Registration access token (required property if username and password aren't defined)

    Important

    If using agents version 10.83 / 11.21 or later, we recommend using Agent Registration access tokens over encrypted Agent Installer user credentials. Agent Registration access tokens do not expire and can be managed in the Management Console at any time.

  • username: Username (without token), and encrypted using the JitterbitUtils application packaged with the default agent installation, as described below (required property if token isn't defined)

  • password: Password (without token), and encrypted using JitterbitUtils, as described below (required property if token isn't defined)

  • agentGroupId: Agent Group ID, available from the Management Console Agents > Agent Groups page, by hovering over the Agent Group name to see the Agent Group ID (required property)

  • agentNamePrefix: A prefix added to the UTC timestamp of agent creation to form the agent name, as shown in the Management Console. Variables that are automatically substituted can be used to create unique names that identify an agent both in the Management Console and in the host system (optional property).

  • deregisterAgentOnDrainstop: When set to true, performs agent deregistration upon agent drainstop or JVM shutdown and removes the agent from the Management Console's Agents page. Defaults to false (optional property)

  • retryCount: Number of retries to be attempted if the agent is having issues connecting to the Harmony Cloud for registration. Valid range: 0 to 300, defaults to 10 (optional property).

  • retryIntervalSeconds: Number of seconds that the agent will wait before retrying a connection. This interval doubles with each new attempt, to a maximum of 600 seconds (10 minutes). Valid range: 5 to 600, defaults to 5 (optional property).

Variables in the agentNamePrefix are automatically substituted:

  • %ip%: IP address of the system or container
  • %host%: Host name of the system; for a Docker instance, this is the Docker Container ID; for Kubernetes, it is set to the container's pod name
  • %guid%: A randomly generated GUID of length 8 characters. This is highly recommended in autoscaling scenarios to assign a random value to each agent's name.

  • For example: an agentNamePrefix of auto_%ip%_%host%_%guid% will result in an agent name such as

    auto_172.17.0.2_b2a546cc080e_59a260ca_2020-02-14 17:43:03.033

Example register.json

Important

Agent Registration access tokens require agents version 10.83 / 11.21 or later.

register.json
{
  "cloudUrl": "https://example.jitterbit.com",
  "token": "jb_a1b23c45-d6ef-789a-b0cd-e12fabcd3e4f",
  "agentGroupId": "21",
  "agentNamePrefix": "auto_%ip%_%host%_%guid%",
  "deregisterAgentOnDrainstop": "true",
  "retryCount": "10",
  "retryIntervalSeconds": "5"
}

Important

If using agents version 10.83 / 11.21 or later, we recommend using Agent Registration access tokens over encrypted Agent Installer user credentials. Agent Registration access tokens do not expire and can be managed in the Management Console at any time.

register.json
{
  "cloudUrl": "https://example.jitterbit.com",
  "username": "$000MTF/XSfev/KjlB/M1Q5vFkPLBKU5LKFYm.../PwoHRiSGGLlJ",
  "password": "$006WIdHtws7wrjgCNAoabUzu0f+HsyzAfAZtTpauvsHBE=",
  "agentGroupId": "21",
  "agentNamePrefix": "auto_%ip%_%host%_%guid%",
  "deregisterAgentOnDrainstop": "true",
  "retryCount": "10",
  "retryIntervalSeconds": "5"
}

For higher security, the account (username) used in the register.json file must have only the Agent Install permission. This is set in the Management Console. For examples of how to confirm and set an agent role, see Managing Permissions, Roles, and Members within Organizations.

Note

To configure a user with Agent Install permission, you will need to log in as a user with Admin permission to the Harmony Portal. When you have completed the configuration of the new account, log out of the Harmony Portal, as you will need to respond to the agent configuration email as that new user in order to complete the configuration. Failing to log out can result in you logging in as the admin user and being unsuccessful in completing the configuration. You can use a private browser window to respond to the agent configuration email to prevent logging back in as the user with Admin permission.

Tip

The cloudUrl required depends on the Harmony Region:

We recommend you define deregisterAgentOnDrainstop as true so that as agents shut down, they are automatically removed from the Management Console's Agents page. Failure to do so will prevent the agent from successfully restarting and registering.

Editing on a Windows Operating System

If you are editing the register.json file on a Windows operating system for use with Docker, use an editor such as Notepad++ that allows you to set the line endings correctly. UNIX (LF) linefeeds are required for the register.json to work correctly on a Docker system. This is what you should see when editing the file in Notepad++: Unix (LF)

attachment

Encrypting the Username and Password

Important

If using agents version 10.83 / 11.21 or later, we recommend using Agent Registration access tokens over encrypted Agent Installer user credentials. Agent Registration access tokens do not require this encryption step.

The username and password are encrypted using the JitterbitUtils application (for Linux and Docker, with the script jitterbit-utils) packaged with the default agent installation. Copy the results of this command into the JSON file in the locations as described above.

Examples

Example usage of jitterbit-utils on Linux to encrypt a username and password:

cd /opt/jitterbit/bin && jitterbit-utils -e "first.last@example.com" "somePassw0Rd1"

Example usage of jitterbit-utils on Docker to encrypt a username and password:

docker run --rm jitterbit/agent:latest jitterbit-utils -e "first.last@example.com" "somePassw0Rd1"

Example usage of JitterbitUtils.exe on Windows to encrypt a username and password:

cd C:\Program Files\Jitterbit Agent\bin && JitterbitUtils.exe -e "first.last@example.com" "somePassw0Rd1"

Example response (username and password, each on a line):

$000MTF/XSfev/KjlB/M1Q5vFkPLBKU5LKFYm.../PwoHRiSGGLlJ
$006WIdHtws7wrjgCNAoabUzu0f+HsyzAfAZtTpauvsHBE=

Tip

If your username or password contains characters normally interpreted by the shell, you must escape them, as in this example:

Linux or macOS shell:

docker run --rm jitterbit/agent echo "\"pa\$\$\`\\/\\/0rd\""

Output:

"pa$$`\/\/0rd"

Agent Logic

The Jitterbit Agent's Jitterbit Tomcat service is responsible for checking and executing the automatic registration process. The process looks for a register.json file and if present, uses it to create a credentials.txt file. At the end of the process, the register.json file is deleted as it is no longer required for agent startup. Instead, the credentials.txt file that was created will be used.

Other agent services—such as clean up and the scheduler—will simply wait during startup until the credentials.txt file is available.

Agent Autoscaling and Provisioning

Configured automatic registration agents are required for autoscaling. Once configured, they can be orchestrated with different technologies, either manually or with systems such as Kubernetes.

Amazon EC2

  1. Set up authentication credentials:
    • If using agents version 10.83 or later, generate an Agent Registration access token
    • If not, add a user to the organization and assign the user the Agent Install permission. Encrypt this new user's username and password using JitterbitUtils
  2. Create an image box, install the agent, and install any other required software (monitoring, utils, etc.)
  3. Stop the agent
  4. Create the register.json fileand define its settings. For authentication, insert either:
    • An Agent Registration access token, or
    • An encrypted username and password
  5. Copy the register.json file into the same directory as the credentials.txt file
  6. Delete the credentials.txt file
  7. Create an Amazon EC2 image

At this point, this Amazon EC2 image can be used either in an autoscaling group or used manually by creating a new Amazon EC2 instance from the image.

When these instances come online and the agent(s) start up, they will go through the automatic registration process described above.

Docker

When using a Docker agent for autoscaling, the process and flow is the same as described above but with two options: option 1, using mapped files; and option 2, using environment variables.

Option 1: Mapped Files

Docker agents require a credentials.txt file to be mapped to the /conf directory.

Instead of providing and mapping a credentials.txt file in the /conf directory, provide the register.json file in that directory using an orchestration tool of choice (such as Kubernetes).

Option 2: Environment Variables

If using encrypted Agent Installer user credentials, another option is to supply the contents of the register.json file using environment variables. This example shows the variables that define the automatic registration:

AUTO_REGISTER="true"
HARMONY_ORIGIN="https://example.jitterbit.com"
AUTO_REGISTER_ENCRYPTED_USERNAME="$000MTF/XSfev/KjlB/M1Q5vFkPLBKU5LKFYm.../PwoHRiSGGLlJ"
AUTO_REGISTER_ENCRYPTED_PASSWORD="$006WIdHtws7wrjgCNAoabUzu0f+HsyzAfAZtTpauvsHBE="
AUTO_REGISTER_AGENT_GROUP_ID="21"
AUTO_REGISTER_AGENT_NAME_PREFIX="auto_%ip%_%host%_%guid%"
AUTO_REGISTER_DEREGISTER_ON_DRAINSTOP="true"
AUTO_REGISTER_RETRY_COUNT="10"
AUTO_REGISTER_RETRY_INTERVAL_SECONDS="5"

Warning

A known issue with autoscaling is that if the number of agents you are licensed for is exceeded, no error message or warning is provided. Instead, the agent registration fails and a registration failure message is written to the agent log. However, no message will appear in the Management Console. As a workaround, monitor the Management Console dashboard, be aware of your license limits, and check that the number of agents provisioned does not exceed those limits.

QA and Configuration Testing

To test these features and confirm that you have configured an Agent Group correctly:

  1. Set up authentication credentials:
    • If using agents version 10.83 or later, generate an Agent Registration access token
    • If not, add a user to the organization and assign the user the Agent Install permission. Encrypt this new user's username and password using JitterbitUtils
  2. Install a Private Agent and test with sample operation runs to ensure that it is working correctly during normal installation and that it functions properly
  3. Stop the agent and delete or move out the credentials.txt file for the agent
  4. Create the register.json and define its settings. For authentication, insert either:
    • An Agent Registration access token, or
    • An encrypted username and password
  5. Place the register.json file in the same place where the credentials.txt file was found
  6. Start the agent
  7. Check the agent log: the agent should automatically register itself as a new agent for the Agent Group ID that was provided
  8. The Management Console Agents > Agent Groups page should show the newly self-registered agent
  9. Upon successful registration, the agent should delete the register.json file and show instead a credentials.txt file