Use Trickle Loading with Exact Online Webhooks

Wiki (en)
,
3

Invantive Data Replicator can use trickle loading, which means that only deltas (changes consisting of insert, update and delete) are applied on the previous table partition version (on Exact Online; a completed load of a table and an Exact Online company) leading to a new table partition version. For instance:

  • old table partition version Exact Online company 123, table TransactionLinesIncremental, ready version 8
  • take deltas and apply on old version
  • make new version 9 and assign it state ‘Ready’.

The trickle loading process is also elaborated on in the topic Enable webhook trickle loading on OEM licenses. More precise, for Exact Online the delivery of webhooks is a sensitive process since Exact Online currently lacks any form of heart beat to report that the loop has been established:

Therefore Data Replicator on Exact Online uses the first delta as a heart beat signal to indicate the loop has been closed.

A request has been registered with Exact for some years to improve the reliability of webhook event delivery, but as of 2021 there is no final plan to add such functionality. As an alternative to reduce loading times, there is an option to resort to the new *Incremental tables such as ItemsIncremental. They combine a reduced API consumption with always consistent data contents. For more background refer to:

Small number of companies

For small numbers of Exact Online companies, the easiest way is to wait a few days and then check in dc_table_partition_versions that the actual approach used is trickle loading (value T).

Note that dc_table_partition_versions contains a column for the request seeding approach and the actual approach used.

For those tables still not using tricke loading, you can ask a key user to create and delete a transaction such as updating the description of a General Ledger Account.

Large number of Exact Online companies

The manual approach to get the first signal over the wire can be labor intensive, especially when replicating hundreds or thousands of Exact Online companies. This holds especially for accounting firms where many Exact Online companies are only updated once per year, such as during the yearly accounting and/or regulatory reporting on holdings. Till the first message is processed, trickle loading on these companies will resort to full copy approach.

There is no supported way to manually force the loading approach for large number of Exact Online companies from Copy to Trickle Loading. However, an Invantive consultant can make the necessary internal changes to the Data Replicator repository on basis of Time & Material to simulate that a message was received. This typically takes 30 minutes per Data Replicator repository.