Salesforce Record Limits¶

Limitation¶

Many web service APIs have size limitations. If you are running into record limits imposed by Salesforce's API, you may want to use chunking to split the source data into multiple chunks.

Use Chunking¶

In Harmony Cloud Studio, chunking can be enabled for each operation under the operation options.

When using chunking on an operation where the source is a Salesforce activity, the data will be chunked in the transformation, not when the data is retrieved. The target data will first be written to numerous temporary files (one for each chunk). These files will then be combined into one target file.

Similarly, when using chunking on an operation where the target is a Salesforce activity, the transformation is performed on each chunk separately, with each source chunk producing one target chunk. The resulting target chunks combine to produce the final target.

For example, a Salesforce upsert accepts only 200 records for each call. With sufficient memory, you could configure the operation to use a chunk size of 200. The source would be split into chunks of 200 records each, and each transformation would call the web service once with a 200-record chunk. This would be repeated until all records have been processed. The resulting target files would then be combined.

When using chunking with Salesforce bulk activities, it is recommended to change the default chunk size of 200 to a large number, such as 10,000.

For instructions and best practices on using chunking, see Operation Options.