There are a number of unique issues and considerations that can arise when integrating to and from SAP instances, particularly when creating a bidirectional integration. This page looks at common integration patterns, their design considerations, change data capture, and provides tips on handling SAP-specific topics.
SAP and Jitterbit Terms
The first four of these terms are from SAP (SAP, IDoc, BAPI, RFC); the last two (Jitterbit SAP Connector, Jitterbit SAP Event Listener) are Jitterbit-specific.
SAP: When Jitterbit refers to “SAP”, we mean a version of SAP that is compatible with our Jitterbit SAP Connector. Typically, this is SAP ECC version 6 and above and SAP S/4HANA on-premises. The essential component is that Jitterbit requires a version of SAP that uses SAP’s proprietary JCo 3.0 (Java Connector version 3).
IDoc: Also known as an "Intermediate Document", it is a text file with a pre-defined format similar to an EDI document. (Be aware that an IDoc uses German abbreviations inside the document.) There are IDocs for many master data objects, such as customers and vendors, as well as transactional objects such as sales orders. Originally used to exchange data between SAP systems, IDocs can also be used for third-party integration.
BAPI: A Business API (BAPI) is an API—similar to a SOAP or REST API—and uses a request and response construct. One main difference from SOAP or REST is that a BAPI has a single method and is specific to an object. For example, there are 60 BAPIs related solely to the Customer object. In practice, only a few are actually used. BAPIs are delivered and maintained by SAP.
RFC: An RFC (Remote Function Call) is the same as a BAPI, except that an RFC may or may not be maintained across SAP upgrades. There are standard RFCs supplied by SAP. Custom RFCs can (and frequently are) built to support integrations, and must be maintained by the SAP customer.
Jitterbit SAP Connector: This is one of Jitterbit’s set of application connectors. It uses SAP’s proprietary JCo 3.0 (Java Connector) protocol to work with SAP integration objects. It is compatible with SAP ECC version 6 and above. The Jitterbit SAP Connector is also compatible with SAP S/4HANA on premises. SAP S/4HANA Cloud is not compatible with the connector, as that endpoint exposes APIs as web services. In the case of SAP S/4HANA Cloud, an integration would be similar to any other web service-based API. Other SAP applications, such as SAP By Design or SAP Business One also do not use the SAP Connector.
Jitterbit SAP Event Listener: This is a Jitterbit service that receives IDocs from an SAP instance. It is configured to match an inbound IDoc (outbound from SAP, inbound to Jitterbit) to specific Jitterbit project operations. The SAP Event Listener is different than a real-time Jitterbit API, where an external endpoint is making calls on an event-driven basis. The major difference is that an IDoc sent to the event listener does not require a response. Be aware that the default method of sending IDocs is to use tRFC (transactional Remote Function Call), in which the IDocs are sent in a batch without a defined order. The other method is qRFC (queued Remote Function Call), an extension of transactional RFC, in which the event listener responds back to SAP after receiving each IDoc, signaling to send the next IDoc in the transaction. This ensures that IDocs are processed in a particular order.
Updating SAP using BAPIs/RFCs versus using IDocs
When updating data in SAP, you generally have the choice of using either a BAPI/RFC or an IDoc. For integration purposes, the BAPI/RFC is preferable.
With the BAPI/RFC:
- There is a response that can be trapped. A successful update will usually have in the response the ID of the record (such as a sales order or customer) which can be used to update the source application.
- If an error is returned, that can be trapped and processed into the Jitterbit error handling processes.
- BAPIs/RFCs can be tested in SAP's SE37, as described here.
With an IDoc:
- When an IDoc is sent, the only acknowledgement sent back is a code indicating it was received; no indication of success or failure.
- The IDoc is processed in SAP asynchronously. If it fails, a user action can reprocess the IDoc.
There are certain scenarios where an IDoc is preferable:
- An IDoc can contain multiple records, while BAPIs/RFCs can usually only handle a single update. A bulk update scenario may require use of an IDoc.
- There may be functionality that an IDoc can invoke that is not available using a BAPI/RFC.
Working with IDocs
IDocs are not strictly hierarchical, in the sense that the repeating nodes are not actually nested within parent nodes. They are a flat structure.
The parent is
"IDOC" and all segments below are at the same sublevel. Header information is at the same level as item information.
For example, this is the
ORDERS05 XML structure:
Useful SAP Integration Patterns
Master Data Integration of Customers
While Customers are one of the most common objects in SAP, this can be one of the most challenging integrations. SAP Customers uses the concept of Partner Functions such as SoldTo, ShipTo, and BillTo.
In SAP, a Customer can be one of—or a combination of—a series of Partner Functions. A Customer could be its own SoldTo, ShipTo, and BillTo. A Customer can be a SoldTo and a BillTo, with a different Customer as the ShipTo. A Customer can be the SoldTo with many other Customers being the ShipTos (a common case where a franchise customer has many locations). The key consideration is that it’s essential to have a thorough understanding of the integration requirements for both Customers and the Partner Functions.
Example: With an initial load of data, in general the SoldTo customers need to be loaded first, since the other customers are dependent on the SoldTo to exist in the target endpoint in order to build the relationship. As the BillTo contains the billing address, frequently the endpoint customer definition needs main and shipping addresses, and the integration has to update the SoldTo customer with the BillTo address in the endpoint.
In the example below, a single IDoc from SAP results in an upsert to both an Account and a custom object Child Account in Salesforce:
The integration is much more challenging (and becomes a project risk factor) if the integration is bidirectional. In other words, SAP Customers flow to the target endpoint, and then any new Customers and Partner Functions have to be created in SAP from that target endpoint. As SAP’s object creation data validation rules are stricter than other endpoints, this requires considerable custom work on both the endpoint and SAP itself to handle the Partner Functions.
This example project moves Salesforce Accounts to SAP and handles the Partner Function management; it requires a total of nine operations:
Change Data Capture scenarios for Master Data objects
In general, IDocs are used for change data capture in SAP, as BAPIs do not handle a timestamp-based query (such as "all data changes since date X"). IDocs, however, can be configured to be generated using change pointers associated with an object, and doing so should be straightforward for most SAP implementations.
IDocs can be generated to contain either all the data or just changes. Getting all the data is generally a better choice since there may be relationships in the data that are needed for an upsert. But consider the actual business scenario; customers performing integrations may overlook that mass updates of data can trigger thousands of IDocs at a time, the processing of which can then exceed login or other limits in the target.
IDocs lend themselves to "straight-through" processing:
- IDoc is received by the Jitterbit SAP Event Listener
- SAP Event Listener moves the payload to the Operation queue
- Operation receives the IDoc
- The IDoc data is loaded into the target
- If thousands of IDocs are sent at once, then Jitterbit will spin up multiple threads and fire off multiple simultaneous calls. If the call is to update Salesforce, then the volume of calls can exceed login limits.
- If the target is unreachable, then the IDoc payload is not delivered to the end target and is then lost.
The alternative to "straight-through" processing is "store-and-forward", where the IDoc payload is written to a temporary file.
In the example below, the first operation receives an IDoc from the Event Listener and stores it locally. The second operation is on a fast schedule (such as every minute) and scans the file directory for files to process. When files are found, the files are looped through and each file is passed to the next operation. When the last operation in the chain is run, it deletes the file, and the loop is repeated until exhausted. If there is a technical failure during the process, the file will not be deleted and will be reprocessed a second time. Note that—by default—Jitterbit’s temporary files are stored for 24 hours before being deleted. This is configurable for a longer period if necessary.
Store-and-Forward Processing and Multi-Agent Environments
A consideration when using the "store-and-forward" approach is when it is run in a multi-agent environment. An IDoc may be received by Agent1 and stored locally, but the schedule may select Agent 2, which will find no files to process. It isn’t until Agent 1 is called by the scheduler that its files will be processed. Bear in mind that all the files in the temp directory will be processed until exhausted. If there is a requirement that IDocs must be guaranteed to be processed in order, then the use of a shared resource such as an FTP site, local file system, or database can be used. However, adding external data stores can create an additional point of failure. Agent clusters are used for failover and load balancing, so using external data stores that are not similarly configured can undermine overall system robustness.
Extracting an ID to Fetch Additional Data
Another common pattern is where the IDoc simply has an ID, and a BAPI is called to fetch additional data.
In the above example, looking at the third operation ("3. Mat-Get Details"), the IDoc is read to retrieve the Material ID (which is then used to call a custom RFC). This screenshot shows the transformation extracting the Material ID:
SAP Dates: GUI versus API
Though SAP can display dates in the GUI in different formats (following country conventions), the data format for APIs is always
SAP End Dates
If an end date is not filled in SAP (such as in an "evergreen" agreement), SAP will send
00000000 in the IDoc, which can be trapped for and handled:
Finding the Appropriate Customer/Partner Function
For example: when creating a Sales Order, you need to pass in appropriate Customer/Partner Functions. In the SAP GUI, you might see a name such as Sold-To Party or the language-specific equivalent Partner Function SP, but the SAP API instead uses the German abbreviation AG shown in the first column.
Language-specific Partner Function
Using the SAP Function Builder to model an API call
When mapping data in a transformation using a BAPI or RFC target, it helps to test the exact values SAP is looking for. Similar to other enterprise applications, what is seen in the SAP GUI is not what is used in an API. SAP has a "Function Builder" that can help with modeling a BAPI or RFC call.
Use t-code se37 to call up the Function Builder screen, and enter the BAPI or RFC to test it.
Here we show using
Select the "wrench" button, circled above in red. You are presented with the testing screen:
Filling in the query table (
"CDHR") and delimiter (
Set under options:
Set under fields:
Then run the test:
The values used in the test correspond to what is required for the transformation:
If additional fields are required, you can add more fields for mapping in the transformation:
See the example on Handling BAPI Conventions for different ways to handle multiple fields in mappings.
Handling BAPI Conventions
Certain BAPIs—such as
BAPI_SALESORDER_CREATEFROMDAT2—have sets of fields that require special handling.
As shown below, the fields that are mapped in the Header (
ORDER_HEADER_IN) are repeated in the
*_INX folder. To tell SAP that you intend to populate the header fields, insert an "
X" as the field value in each of the fields that are a repeat of a header field:
However, in this section (
ITM_NUMBER field under
ORDER_ITEMS_INX does not have an
"X" inserted; instead, it receives the same item number as used in the field
Handling Partner Functions with Additional Folders
In this section, we need to pass in the SoldTo, ShipTo, and BillTo. We've added additional folders for mapping them and defaulted the partner roles (AG, WE, RE):
Handling Partner Functions with the SetInstances Function
An alternative to adding new folders is to use the
SetInstances() function with no additional folders:
In the root condition, we would create a series of arrays inside a
PartnersArray and set it to the
In the folder condition under
ORDER_PARTNERS, to set the instances we would use:
Handling Incrementing Line Numbers
In the section
ORDER_SCHEDULES_IN, we would once again insert an
"X" in the data fields.
For the field
SCHED_LINE, SAP increments them by 10 (line 10, line 20, line 30, and so on). We can use a global variable counter in the condition under
ORDER_SCHEDULES_IN to calculate the value:
See Jitterbit function
Retrieving Values From an IDoc
Here we want to get start and end dates from an IDoc (
ORDERS05) to create a contract. The data is located here:
E1EDP03, we want the DATUM value where
IDDAT == 107 (or
108) for the start and end dates respectively. A handy function for this is the Jitterbit
FindValue() function. This function will cycle through the
IDDAT values, find the one with
"107" and retrieve the
See Jitterbit function
Last updated: Feb 05, 2020