Exact Online has indicated that as of July 1, 2021 a number of generic modifications to the Exact Online APIs will be implemented. These modifications are necessary to improve scalability and increased usage of the ecosystem.
See below for the latest updates.
The modifications cannot be pre-tested due to lack of a representative environment and poor specifications. Invantive has decided to inform users that around July 1, a production stop may occur that may last several weeks and cause problems into September. Subscriptions to Data Hub and Data Replicator will be dissolved and an alternative with lower service requirements will be available.
Invantive will make releases available as soon as possible. Based on current information and approach, it is not yet possible to determine the changes needed. Releases that address the issues will also become available from Invantive Data Hub and Data Replicator for direct interfacing with Exact Online when feasible.
Also modifications to reports and chains using Invantive SQL will be required; please book consultancy with dealers or Invantive for this well before July 1 2021.
New Exact Online API limits.
The following limits are expected to be implemented by July 1, 2021:
- Maximum 200 token endpoint calls per app, user and day: implicit grant flow at login and every ten minutes, retrieve access token based on code from code grant flow, retrieve access token via refresh token from code grant flow.
- Maximum of 1 access token generation every 9.5 minutes, but no waiting for HTTP 401 error message (see next point).
- No more than 10 error messages per app, administration, endpoint and hour.
- Mandatory filtering on single and bulk endpoints if a sync API is available (see Exact Online: is verplicht filteren per client ID instelbaar? (Dutch)): known by the working name “mandatory filtering”.
- Maximum 60 calls per minute instead of 300 calls per minute: this is a limit across both XML and REST APIs. If exceeded, the number of calls per minute is automatically reduced.
- Throttling for usage above fair use: this was already in place.
- Issued refresh tokens will expire 30 days after the last access token exchange.
- No more exceptions regarding the use of old refresh tokens (workaround for interactive products described at Circumvent Exact Online 2FA to avoid entering PIN-code every ten minutes).
In addition, we were made aware that the limit of 5,000 API calls per day announced years ago will be enabled. We do not expect any problems with this because Invantive SQL has been supporting the stable Sync APIs and bulk APIs for quite some time.
However, it has been found that the rate limit header X-RateLimit-Remaining containing the remaining number of API calls until the next daily reset time does not always reflect the correct number of remaining calls with up to a factor of 4 difference. However, Invantive SQL will also insert an increasingly long pause in the event of an HTTP 429 Too Many Requests error, reducing the likelihood of a failure.
We did recently find that Deleted items in the sync APIs are cleaned up after 2 months. Logic has been added in 20.1.432 and later to automatically correct for this. Thus, the sync APIs and related *Incremental tables are not reliable in terms of completeness on older releases.
The limitation on the number of calls per day will cause problems for users who want to process, for example, 2 million article updates in terms of prices and texts per quarter in few days. These users will have to spread the work out over a longer period by having the software fine-tuned or switch to another package.
The specifications can be found as follows:
- Last version specifications
- Per July 2021: https://archive.is/wz9ca
- Per February 2021: https://archive.is/NHWff
Risks.
The following risks have been identified and may require adjustments.
Lack of Test Environment.
The impact of the changes cannot be tested until July 1. Exact Online has no sandbox or versions of the APIs like Salesforce where app developers can validate their software’s compliance with the APIs before going live. The signaling emails from the API ecosystem appear to be at least partially inaccurate and not traceable to the origin.
Exact Online also has no gradual go-live for these changes, for example by application, accountant, server or customer clusters.
Some of the changes are incalculable in terms of impact and some of the changes are not achievable anyway without workarounds.
Delivering prior to July 1 with adjustments on the gamble and then again afterwards to further smooth over hundreds of servers is cost unrealistic and carries the risk that the random adjustments introduce errors or worse: security issues since many restrictions are related to authentication.
Action: the risk of not having a test environment has been reduced by not starting the work for any needed modifications until July 1.
Long Duration due to VAT Closing and Holiday Period
Work, barring preliminary activities such as upgrading central infrastructure, as described above does not start until July 1. Going live is also the start of reporting on the preceding VAT period, which traditionally also requires manpower. In addition, manpower is limited due to planned vacations, both at Invantive partners and Invantive itself.
The duration of the adjustments - to the extent feasible - is therefore long. We are counting on this to continue into September before stability occurs.
Action: the risk has been mitigated as best as possible by informing users in advance so they can prepare and by classifying some of the subscriptions at a lower service level, see Stel tijdig uw rapporten op: tools voor Exact Online mogelijk niet beschikbaar na 1 juli (Dutch), Laad tijdig uw wisselkoersen: Valuta Tools voor Exact Online mogelijk niet beschikbaar na 1 juli (Dutch) and Change conditions Invantive Data Hub and Invantive Data Replicator.
It is not desirable to add thousands of companies or more per customer to Invantive Cloud. The infrastructure is not provisioned for this load and cost-wise this is not achievable.
Cluster configurations with Invantive SQL, Data Hub or Data Replicator
- Maximum 200 token endpoint calls per app, user and day: this means maximum 2000 connection minutes per day and this is not realistic in a cluster setup.
- Maximum 1 access token generation per 10 minutes: in cluster setups with Invantive SQL this is not realistic.
- No more exceptions in terms of using old refresh tokens.
The limit of maximum 1 access token per 10 minutes is not feasible if multiple Data Hub jobs run simultaneously against Exact Online or are started shortly after each other. Possibly consider scheduling the next execution of a job at least 10 minutes after the previous job ends.
Action: part of the problem is solved because Invantive Keychain will hold the now rapidly expiring refresh tokens. For large environments with high concurrency or clusters, there is a solution approach based on consultancy and based on Bring Your Own App.
Action: There will be no new links between Data Hub and Replicator and Exact Online for the time being. Large accountancy organizations that currently want to switch we recommend postponing this until after September. Small users can load data via the Invantive Cloud driver.
Invantive Control for Excel
- Maximum of 1 access token generation per 10 minutes.
Action: After July 1, the logic waiting for a 401 will be converted. This will take into account clock differences between local devices and the Exact servers, as is already done with authentication token generation.
Overall
The following problems apply in all deployment scenarios:
- No more than 10 error messages per app, division, endpoint and hour: it is not yet clear which events are counted as “error messages”; therefore, they cannot yet be adequately accounted for.
- Mandatory filter on single and bulk endpoints if a sync API is available: this is not practically feasible and, in our view, a nonsensical approach to an overload issue.
Action: after July 1, it will be experimentally determined when error messages are really present and a solution direction will be determined on a case-by-case basis - if feasible.
Action: If a filter becomes mandatory, then as a workaround an artificial filter will be introduced which will effectively retrieve everything if desired. If necessary, the artificial filter will be automatically varied. In addition, Invantive users will continue to receive automatic advice on how to use the APIs.
Other Apps
All Exact Online applications and partners will face this change simultaneously on July 1. From reports, we understand that other apps are also at risk of discontinuity. We encourage you to check with vendors in a timely manner to see if and what issues are anticipated.
Advices to Users
We advise users to upgrade to at least 20.1.432 (BETA) in anticipation of further modifications.
As many modifications cannot be tested by us in advance, users should assume that using Invantive products around July 1 will not be possible and there will be a production freeze that may last several weeks. It cannot be ruled out that the problems will persist into September.
Invantive will make releases available as soon as possible. At this time it is not possible for us to estimate the adjustments.
Adjustments in reports and chains using Invantive SQL will also be required; please book consultancy with dealers or Invantive for this well before July 1.
Possible Long Term Improvements Exact Online
From Invantive we contribute the following improvement suggestions towards Exact Online to improve availability and make changes easier. We cannot judge the relevance and feasibility:
Test Environment and/or Versioning Exact Online APIs.
Currently, changes (including breaking changes) are applied to Exact Online without app developers having a chance to test and certify their applications against the new situation beforehand. Currently, there is no testing environment, nor can app developers (like on Salesforce, Teamleader or Loket, for example) decide when to move from one API version to another.
Suggestion(s): adding or a sandbox/test environment starting 6 months before a breaking change goes live allows developers and users to avoid unplanned and avoidable downtime. Alternatively, versioning can be used, for example, by introducing a v2 alongside the current v1. The v2 is largely identical to v1, but includes breaking changes as mentioned above, expired APIs and new APIs. In sync with the Falls and Spring releases, an old version can be taken offline each time, so there is always 6 months of overlap and enough time to mitigate risks. A policy per company is therefore not necessary; the app developer can control which administrations use which version.
Better Planning
In recent years, breaking changes were often planned and implemented on the 1st of the month. This coincides exactly with the period calendar of most companies and thus traditionally the busiest time for Exact Online users for VAT returns or management accounting, for example. It is not uncommon for figures to be reported on the 4th or 8th day of the month. Therefore, to avoid risks, you will find frozen periods at most listed companies: no new software may be put into production around the period start. This sometimes goes so far that even traditional transactions such as payments are briefly suspended.
Suggestion(s): going live of breaking changes would therefore be better done after the completion of period reports in a period when staff is available. This is usually spring and fall, for example with the long weekends around April 15 and around September 15.
Smarter Horizon
Horizon to limits are usually set up based on round numbers, such as 30 days for refresh tokens or 60 days for empty deleted items from the sync APIs. A multiple of 30 days is inconvenient. For example, when reissuing bank account numbers and phone numbers, we therefore often work with the period length plus something extra, for example, 12 months + 1 month. Even with the MOT inspection, the time horizon is slightly stretched from exactly 365 days so that people have a chance to pick a convenient time for their MOT inspection.
Suggestion(s): if a limit of about 30 days is needed, then it would be prudent to assume, for example, a minimum of 31 days plus the typical absence of crucial personnel, such as weekends. A better choice would therefore be, say, 38 days or 45 days instead of 30 days. This avoids reaching a horizon every month and causing stress.
Communicate well
Breaking changes and changes are rarely communicated and are usually not or not well documented. This creates disruptions, in part because, thanks to its commercial success, many external parties want to work with Exact Online.
To illustrate: Invantive may be a large partner of Exact, but it is not desirable for Invantive to rank #1 on Google for many of the Exact Online API related search terms. If the brand owner does not rank #1, it means that the brand owner has a serious information gap despite Google’s valued ownership of the brand.
Suggestion(s): it would be nice if Exact API support communicates actively and early with the active users. Part of this could be to promote the community and also make it findable via Google. It would be wise to announce breaking changes in advance or at least communicate after implementation. If a breaking change has to be implemented immediately because of force majeure, it is better to start the conversation as a dialogue instead of a proposing monologue. Stelling - and certainly from a position of burden - generally yields a greater chance for relationship breakdown and escalation, according to Zuidema.
Commercial Model
Specific to the Exact Online API rate limits are very tight for the top 1% of users. However, the Exact Online infrastructure can easily handle large users, but the APIs do not provide for such heavy usage.
Suggestion(s): it may be commercially interesting to offer additional API calls like Yuki and Salesforce at an additional cost. This also encourages app developers and users to push for this. “If something is free, it’s not worth anything,” says an old salesman’s saying.
Return Error Status via HTTP Headers.
The status of rate limits is communicated via HTTP headers. The values do not always correspond to actual usage, but give a good indication. Production failures can be prevented by client software if the limits on the number of error messages can also be returned via an HTTP header, for example, the maximum number per hour on the endpoint, the number of errors encountered and the next reset time in UTC.
Further
This document will be updated as needed.
