Performance - Is there a way to perform bulk insert/update/delete in Scribe Online

[Kirill Yunussov]

Environment: Salesforce-to-Dynamics NAV 2017

Problem: Time taken by Scribe Online to integrate a single large order can be over 10 mins.

 

Right now, I am creating/updating records one at a time, and with large orders having a few hundred line items, that takes way too long - each line-item has to be inserted with its own insert statement, running various triggers in the database after each insert.

 

What I would like to do is collect all of the line-items into a list/array, and then insert the list, with just one DML statement. Is that possible

[Albert Baiungo Jr.]

Hi Kirill,

 

Thank you for posting your question. My name is Albert and I am with the Scribe Support Team. Support handles mostly break/fix types of issues as design is usually handled by our consulting services. However, I wanted to provide some articles for you to view regarding batch processing which is available in scribe online through the Odata service.

 

https://www.scribesoft.com/blog/2014/01/online-batch-processing/

 

https://help.scribesoft.com/scribe/en/index.htm#sol/conn/nav.htm%3FTocPa...

 

in the help guide scroll to find information about NAV batch processing

 

Thanks,

 

Albert

[Liz Neiley]

Hello Kirill,

I am the product manager for connectivity here at Scribe Software. I have put in a product idea for supporting bulk for Dynamics NAV. This will be reviewed for the next release. Thank you for submitting this to us! Please see Albert's response for another option in the meantime.

 

Best regards,

Liz

[Kirill Yunussov]

Thanks for looking into this.

 

I have tried Albert's solution of setting the "Batch" option in the Create element to various numbers. This does not seem to do much in the NAV connector:

 

1) Line records are still being inserted one at a time. I kept refreshing the parent record in NAV (Sales Order), and the number of lines is growing one at a time, at about 1 line per second for the first 50 lines, then it slows down dramatically. The slow-down is another issue, not Scribe-related.

 

2) Scribe Execution Statisttics shows a reduced number of "Create" calls, equal to the number of batches, but execution time is the same as without batch processing.

 

3) When trying to set the batch size to 100 or more, I get the following error in Scribe, seems to be dealing with OData request size limitation:

 

Error in calling Operation Create Microsoft.Data.OData.ODataException: The maximum number of bytes allowed to be read from the stream has been exceeded. After the last read operation, a total of 1058485 bytes has been read from the stream; however a maximum of 1048576 bytes is allowed.

[Ash Arendsz]

Update on batch processing with NAV: https://help.scribesoft.com/scribe/en/index.htm#sol/conn/nav.htm 

 

Batch Processing

 

The Dynamics 365 Business Central / NAV Connector supports batch processing for Create, Update, and Delete operations if the Dynamics 365 Business Central / NAV OData V4 service also supports Batch Processing for those operations. The maximum number of records allowed per batch is 100.

The Dynamics 365 Business Central / NAV OData V3 service does not support Batch Processing.