Background services in Jitterbit App Builder

Overview

The App Builder Background Services feature allows App Builder to run long running Events in a throttled background queue. The service itself is hosted by the App Builder web application in IIS. As long as the website is running, the background service is running. It does not require any additional installation or configuration.

There are two components involved in the App Builder Background Service:

Background queue

Often times a developer creates an Event that performs several Actions, possibly spanning multiple data sources, or even REST calls to external systems. The background queue allows the developer to specify that the Event should be run in the background instead of the foreground. There are a few benefits to doing this:

• When clicking a button to execute an Event, control is immediately returned to the client. The client may continue on with their work while the job continues to run on the host system.
• The background queue saves on resources. By placing Events into the background which would otherwise tie up resources, the developer allows App Builder to throttle the execution of those Events. As the load on the system grows, the Job Queue's level of parallelism reduces. This allows the web server operate with more threads and more access to the CPU. As an example, if a 100 people log onto the site and click a button that executes an Event, IIS would need to handle 100 concurrent executions of that Event. By submitting the Event to the job queue, we can reduce the concurrency to only execute a small number concurrently, which frees up resources for the web site.

• The background queue monitors events and stores history. We intend to extend this functionality to all Events in the future. For now, if you are interested in maintaining a log of the execution history of an Event including:

  • Who ran the event it
  • When did they run it
  • How long did it take to run
  • Was it successful (if not, what errors occurred)

This information is available in the Monitoring section of the IDE. On the initial page you will find a list of running services and a list of running schedules or events.

To specify that an Event should be run in the background, the developer should locate the Event, then double-click the Event, edit it and select an Execution Type of Launch Now (Background). From that point on, App Builder will queue the Event for execution whenever a user executes it.

If your Event has validation rules, and those rules return an error, App Builder will return the results immediately to the user along with the validation messages, and cancel the Event from running in the queue. Warning Validations are not supported when executing the Event in the background.

If appropriate consider changing the Success message for the Event so that it is conveys to the end user that the operation is running in the background.

The App Builder Job Queue sleeps indefinitely, using minimal resources until an event is queued. It does not poll for jobs to run. Instead it utilizes Microsoft's The Task-based Asynchronous Pattern, ensuring that system resources are fully utilized.

Scheduler

App Builder supports scheduling the execution of Events instead of requiring manual execution of Events. The scheduler, like the job queue is running in the Background Service as long as the web application is running. App Builder currently supports one mode of execution for a scheduled Event, called a multi-row execution. This is different than an Event executed via button click. When a user executes an Event against a business object, it is executed against a single row. When the scheduler executes an Event, it is executed against the entire business object. This means the queue loops through each row in the business object and runs the scheduled Event against that row. It does this by queuing each row for execution by the job queue.

Because the scheduled Events are submitted to the job queue for execution, App Builder is able to take advantage of the performance benefits that come with throttling the execution of these potentially large data sets. This also provides the same visibility to scheduled Events with regard to monitoring and logging. There will be an entry in the log for each row executed by the scheduled Event. The primary key information for each row will be included in the log to aid in finding and debugging issues.

To schedule an Event for execution, locate the Event, double-click it, and set the Execution Type to Launch via Schedule (Background). This will allow the Event to be attached to a Schedule.

To create a schedule, go to the Monitoring section in the IDE. Click Schedules and create a new Schedule.

Following are the types of schedules App Builder supports:

• One Time Execution - Pick a day and time in the future for the Event to be executed. App Builder will execute the Event at the scheduled time and then deactivate the schedule.
• Periodic Execution - Execute the Event every X seconds/minutes/hours.
• Daily Execution - Execute the Event at an exact time every X days. This can be configured to run every day, every 3rd day, every 7 days, and so on.
• Weekly Execution - Execute the Event at an exact time, on an exact day of the week, every X number of weeks. In this schedule the developer can pick specific days of the week to execute the Event. For example, execute every other Sunday and Wednesday at 7 PM.
• Triggered Execution - Execute the Event in response to a signal such as a user signing into the system.

To add an Event to a schedule, go to the Schedules page, and click on the Events icon. Add the Event to the schedule, specifying a maximum concurrency and the user id that will run the Event. If the Event business object returns multiple rows, App Builder will attempt to execute the Event against each row with the specified concurrency. For example, if 100 rows are returned by the business object and the maximum concurrency is 3, App Builder will attempt to run 3 instances event concurrently, until the Event has been run 100 times (once for each row).

Service instances

This page allows the user to stop/start the background service and monitor the active services and running jobs.

• It provides the start date of the service and the up-time.

• There are Stop and Start buttons for stopping the background service.

• Stop will drain all currently executing jobs to finish and cancel any exiting queued jobs. It will also stop any scheduled or invoked events from being queued.

• Start will start the background service back up, clear the activity table and begin scheduling and support of background event
• The page is not constantly updating its state in the UI, so there is a Refresh button. Clicking Refresh will load the latest information for the running service instances and running jobs without requiring the user to refresh the entire browser.

Logs page

• Shows messages from the current running instance of the service.
• Available from the Service Instances page

Canceling of a job

How App Builder cancels a job is different depending on the state of the job and the type of job being run.

• Pending State - The job has been created and queued, but is not yet executing. In this state App Builder immediately cancels the job.
• Running State - If the job is running, App Builder sends a cancellation request to the job. Some operations take longer to cancel than others, and some operations do not allow for interruption. When job reaches a point that supports cancellation, it will cancel and update its activity information.
• Schedule Job - If the job is a schedule job, and it has created n number of jobs to execute the scheduled event. Canceling the schedule job will also cancel the jobs it initiated. In other words, it cancels the entire execution of the schedule.