Efficiency drop-off with large message count

We have a large number of integrations moving various data from ERP. When doing inventory count a script is run in the ERP that triggers updates to over 100k records and consequently results in a huge message queue.

 

Because this is a queue, first in equals first out. We have set varying priority level on Publishers but this does not affect queue order.

 

The problem is that more urgent messages are not processed until large inventory batch first processed, which takes time. So we are forced to turn off ​Inventory Integration so messages skipped and more urgent ones processed.

 

Unfortunately the other side effect is that the messages now process at a MUCH slower pace. I theorize because each thread starts at top of queue every pass, skips inventory messages, then processes other, then repeats.

 

Anyone know how to better deal with this?

(1) Answer

Login