High availability deployment in AWS for Jitterbit App Builder

Introduction

App Builder can leverage many AWS services to aid in the deployment of a high availability (HA) environment.

AWS elements

• S3 Bucket - configured to allow App Builder read/write access for the purpose of creating and sharing security keys.

• App Builder Database - Can be run using RDS or installed on EC2 instances.

• ElastiCache - Used as a message bus for information sharing between App Builder instance

• Elastic Beanstalk - Manages App Builder instances and provides a load balancer to distribute client requests.

S3 setup - Storing data encryption keys on S3

As noted in Data encryption configuration, EC2 instance local storage cannot be used for longterm storage. Consequently, Data Encryption Keys (DEKs) must be stored in S3 buckets.

S3 buckets used for data encryption key (DEK) storage

Within the bucket, DEKs are prefixed with the Elastic Beanstalk environment name.

Configuring Elastic Beanstalk environment Data Encryption Key (DEK) storage

App Builder ships with a .ebextensions script which registers the environment properties for DEK storage. These properties include:

App Builder will not start with the default Elastic Beanstalk environment properties. You must change the DataEncryptionKeyS3BucketEndpoint and DataEncryptionKeyS3KeyPrefix values.

Granting Elastic Beanstalk environments access to S3 buckets

EC2 instances within an Elastic Beanstalk environment are assigned to a role. The following example policy grants access to an S3 bucket.

{
  "Version": "2012-10-17",
  "Statement": [
    {
       "Effect": "Allow",
       "Action": "s3:*",
       "Resource": "arn:aws:s3:::{bucket-name}/*"
     },
     {
       "Effect": "Allow",
       "Action": [
         "s3:ListAllMyBuckets",
         "s3:GetBucketLocation",
         "s3:ListBucket"
       ],
       "Resource": "*"
     }
   ]
}

ElastiCache setup

Use ElastiCache to facilitate communication between App Builder servers. A small node, with no replicas or shards is all that is required.

The configuration information will be used when setting up the application servers in Beanstalk and Non-beanstalk configurations.

Database setup

Database setup is outside the scope of this document. Jitterbit can provide RDS setup instructions if desired.

Elastic Beanstalk setup

Amazon modifies their interface frequently. These screenshots and flow are accurate as of November 2016.

Create the environment:

Select "Create web server"

Select IIS.

The Configuration will be defaulted to a value, we do not want. Select "Change platform version", and select "8.5 on 64bit Windows Server 2012 R2".

Select the appropriate application version.

Enter the environment name. This will become part of one url that might be used to reach this environment. Keep in mind the namespace is across all elastic Beanstalk instances of all AWS customers.

Check "Create this environment inside a VPC".

Jitterbit uses T2.small and T2.medium instances for development and QA purposes, and larger instances for production environments.

If you want to be notified of environment issues, enter your email address below.

Configure VPC settings as appropriate. ELB visibility should be External

Review the summary:

Permissions

After the environment is up

The environment will take approximately 15 minutes to come up.

After it appears green on the dashboard, we will configure some additional parameters:

1. Configuration... Software Configuration... Set

   1. ConnectionInfo to a string that matches your database connection information

   2. DataEncryption:S3BucketEndpoint to the bucket you setup in Step 1

   3. DataEncryption:S3KeyPrefix to the Elastic Beanstalk environment name.

   4. DataEncryption:S3KeyPrefix

   5. SharedState:RedisConnectionString to the appropriate connection string for redis (ex: 10.0.2.139:16016)

   6. SharedState:Provider

After these steps you will go to the App Builder site. This will trigger a database installation or upgrade if required.

Configuring HTTPS

By default, new Elastic Beanstalk environments are not configured for HTTPS. HTTPS is strongly recommended for all new Elastic Beanstalk environments.

Elastic Beanstalk environments terminate the HTTPS connection at the Elastic Load Balancer. App Builder's support for reverse proxies allows it to detect and handle this scenario. App Builder recognizes three environment properties defined when the Elastic Beanstalk instance is deployed. These environment variables are scripted: you will not need to configure them yourself.

Configuring Elastic Beanstalk to accept HTTPS requests

1. Log into the Amazon Web Services Console.

2. Under Services, choose Elastic Beanstalk.

3. Select the environment you are updating.

4. Select Configuration on the left menu.

5. Select the gear next to Load Balancing.

6. Set the Secure Listener Port to 443.

7. Select the SSL Certificate ID that is applicable (App Builder uses wildcard.zudy.com – other sites will use something different)

   Note

   This assumes that your new Elastic Beanstalk environment will be accessible via *.zudy.com.

8. Click the Save button.

Non-Beanstalk application server configuration

If you are not using AWS Elastic Beanstalk, you are configuring application servers by hand.

You will need to make the follow changes to a default App Builder setup – these settings should be applied to every instance of App Builder:

The following should be setup in c:\inetpub\wwwroot\Vinyl\Config\AppSettings.config

<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
  <add key="SharedStateProvider" value="Redis"/>
  <add key="RedisConnectionString" value="YOUR_SERVER_HERE:6379"/>
  <add key="DataEncryptionKeyStorage" value="S3"/>
  <add key="DataEncryptionKeyS3BucketEndpoint" value="https://s3.amazonaws.com/YOUR_BUCKET_HERE"/>
  <add key="DataEncryptionKeyS3KeyPrefix" value="YOUR_ENVIRONMENT_NAME"/>
</appSettings>

Every instance of App Builder should have the same connection.xml file.

After making changes, and restarting App Builder, check the logs to verify that settings are being picked up correctly.