Summary
The introduction of rate limits on Exact Online in 2016 will cause imminent failure on all Invantive SQL-based installations. This note describes the concept, the Invantive support of rate limits and the cause of the imminent failure. Users are urged to upgrades as soon as their environment allows to avoid a production stop.
Looking for Exact Online Rate Limits in 2021?
For the 2021 rate limits, please refer to Itgeneor192/itgeneor229 Exact Online: no remaining capacity - 2 van forums (Dutch).
Rate Limits
The amount of API calls on Exact Online has increased considerably over last years, despite efforts by large consumers to reduce their traffic and the introduction of new APIs like the bulk APIs for initial loading. These compensate a number of poorly designed APIs that require frequent small calls such as item pricing.
Allegedly over 15 million calls are made every day. To create a more level playing field, Exact announced in November 2017 that they would introduce API rate limits to reduce the daily maximum number of calls per combination of application (identified by client ID) and company to 5,000. No calls can be made above this volume. This project was stalled due to stability issues on the implementation.
Currently Exact is activating the rate limits across it’s user base, starting of with a rate limit of 500,000 calls per day per combination of application and company. The official published statement by Exact Online is that these rate limits will be reduced again to 5,000 calls per days unless an exemption has been agreed upon.
Details on the actual implementation and daily rate limits are available at Exact Online API documentation.
Besides these changes, a number of security-related changes are being made that depending on your configuration can require changes. Please consult your local dealer for assistance.
Invantive Support for Rate Limits
Invantive made their software products compliant with these rate limits and specifications in 2017. The original plans are described in Keep your lane on Exact Online.
Even better, Invantive SQL uses a number of advanced features to reduce the number of API calls by at least one and sometimes two orders of magnitude compared to it’s competitors. Products such as Invantive Data Replicator for mass replication of companies also use advanced techniques such as auto-tuning webhooks and smart sampling to further improve performance and reduce strain on the Exact Online API system.
This performance gains of Invantive SQL with Exact Online also apply to insert, update and delete operations. Samples are given in for instance Does Invantive SQL match the speed of Infinite Probability Drive, but in general adding solely the bulk keyword before for instance insert into gives you an speed increase combined with a reduction of API calls per day.
Failure Imminent
However: the actual current implementation by Exact of the rate limits differs in an essential part from the specifications, which causes ALL installations of Invantive to fail during the next days upon next use.
The API gateway returns the date and time at which the number of remaining API calls will be reset to it’s daily allowance. The associated UNIX epoch time returned in the HTTP header X-RateLimit-Reset is however in milliseconds instead of the seconds described in the specification. As a company of people originating from regulatory and operational reporting, our software includes many application controls to ensure that data is consistent (of cause we also try to take care of security and availability). Therefor, all releases checked that the reset moment in UNIX epoch time are within the allowed value range and would fail otherwise with an error such as (taken from our free tool to load exchange rates into Exact Online companies):
This fatal error will be raised some time during the next days when you don’t upgrade your Invantive software products before use.
You can check your software version for correct handling of the rate limits, by querying the new view RateLimits. RateLimits displays per division the calls consumed, calls available and the reset moment, such as:
As can be seen from this picture, not all companies and apps yet have rate limits applied. Also, only Exact Online companies on which a call has been executed will have their statistics listed.
Upgrades Available
A new software version of Invantive brand products is available within subscriptions that continue after January 1. When your subscription doesn’t continue after January 1, please contact Exact Support to request an exemption of API rate limits.
Consulting for making your scripts, reports and configurations compatible can be acquired from our dealers and Invantive based upon Time & Material rates and availability.
The currently recommended software versions are:
- Invantive Control for Excel
- Invantive Composition for Word
- Invantive Business FTP Server
- Invantive Data Loader
- Invantive Query Tool
- Invantive Data Hub (also for Invantive Data Replicator)
- Invantive Studio
- Invantive Producer Runtime
- Invantive Business for Outlook
- Invantive Business for Windows
- All users of Invantive BusinessDrive and Invantive Data Access Point have upgraded already.
- Get My Report for Exact Online has already been upgraded.
- Valuta Tools for exchange rates loading has already been upgraded.
- The problems with Exact Online rate limits have no influence on users of Invantive Estate, Invantive Vision or other platforms such as Company.info or Teamleader.
Updated agreement keys can be acquired by calling +31880026599.
Support capacity is restricted, so please plan your upgrade timely and allow for up to 5 working days for support calls to be handled in the next weeks.
Release notes are available on https://releasenotes.invantive.com. Updates to these releases will be announced in the beta section of releasenotes.invantive.com and on Twitter.
Versions released are betas from the 17.30 release scheduled for first week of January. Currently there are only few known issues in these which cause previously running configurations to fail with this new version.