Abstract
Exact completed on July 5, 2021 the rollout an API rate limitation in terms of the number of errors per API key, per user, per division, per endpoint, and per hour as part of a larger program of changes. An analysis was made on the impact of Invantive SQL-based applications using Implicit Grant Flow and/or Code Grant Flow to acquire an access token.
It was found that this specific new Exact Online API limit is not expected to have an impact on a simple environment without concurrent execution. For enterprise environments, a production stop can occur. Restrain to non-concurrent execution in such scenarios till Invantive provides a release that is less lazy with OAuth refresh.
New API Limit on error messages per hour
The Exact site specifies in the current specification that the following API limit will be applied as part of several other new Exact Online API rate limits:
No more than 10 errors per API key, per user, per company, per endpoint, and per hour. When you exceed this limit, your API key will be temporarily blocked from making further requests. The block will automatically be lifted after one hour and will gradually increase when you continue making these errors. Response code 400, 401, 403, and 404 are counted as errors.
This API limit has completed rollout on July 5. In a short test, we have analyzed the impact of the change on existing scripts by business logic. This test is part of an Invantive program to analyze and minimize impact.
Currently, there have been no reported production stops due to the change.
It was already assessed that 401 and 403 errors will be triggered up to 6 times per hour both during Implicit Grant Flow as well as Code Grant Flow to acquire a new access token for the Exact Online API, due to the lazy refresh mechanism of Invantive SQL. For the Exact Online platform, this mechanism will be altered in an upcoming 20.1 BETA release to pre-emptively acquire a new access token when the existing access token is expected to expire within 30 seconds.
The number of errors per hour must be multiplied by the number of independent concurrent processes refreshing access tokens. This of course only applies to client IDs that allow the use of an old refresh token, such as with Data Hub and large-scale Data Replicator applications. Most new clients do not allow reuse of a previously expired refresh token.
Other new rate limits such as on maximum life time of a refresh token, mandatory filtering, and restrictions on the daily and minutely rate limit window are not covered in this analysis. Please consult the base topic for more information.
Scenario
Exactly 50 errors, evenly distributed across two categories, have been generated using Exact Online API calls within one minute clock time using the included Invantive SQL script.
The first category was the deletion of a contact using an API call on crm/Contacts, the second category was the creation of a journal with insufficient data through an API call on financial/Journals.
HTTP Status Codes Standard Deviation
The expectation was that the events would raise a 4xx HTTP status code, possibly matching the list provided, since (source Wikipedia, but similar in 3.1 on Best Practices for REST API Error handling):
4xx
This class of status code is intended for situations in which the error seems to have been caused by the client.
5xx
Response status codes beginning with the digit “5” indicate cases in which the server is aware that it has encountered an error or is otherwise incapable of performing the request. Except when responding to a HEAD request, the server should include an entity containing an explanation of the error situation, and indicate whether it is a temporary or permanent condition. Likewise, user agents should display any included entity to the user. These response codes are applicable to any request method.
It was however found that both business events raised errors communicated with a 5xx HTTP status code. The deletion of the contact raises an “HTTP 503 ServiceUnavailable” instead of a 4xx error. The attempt to create a journal raises an “HTTP 500 InternalServerError” instead of a 4xx error.
Conclusion
Since Exact Online communicates business logic events through HTTP 5xx status codes, the new API rate limit will not impact errors in business logic. The lazy refresh mechanism of Invantive SQL with Exact Online can generate up to 6 errors per hour per independent concurrent process.
For a simple environment without concurrent execution, the new API limit is not expected to have a measurable impact. For enterprise environments, a production stop can occur. Restrain to non-concurrent execution in such scenarios.
Invantive SQL Script
use 999@eol
declare
l_id guid;
begin
set use-http-disk-cache@eol false;
set use-http-memory-cache@eol false;
--
select /*+ low_cost */ id
into l_id
from exactonlinerest..contacts@eol
where lastname = 'Contact'
and firstname = 'Primary'
;
--
-- Generate error on deleting the main contact.
--
for i in 1..25
loop
dbms_output.put_line(rpad('*', 80));
dbms_output.put_line('*** LOOP ' || to_char(i) || ' ***');
dbms_output.put_line(rpad('*', 80));
--
-- This DELETE generates a ServiceUnavailable (503)
-- without any explanation, but does so always, even on
-- the first hit.
--
begin
delete from exactonlinerest..contacts@eol
where id = l_id
;
exception
when others
then
dbms_output.put_line('Error on delete:' || sqlerrm);
end;
--
-- Generate error on missing GL account.
--
begin
insert into exactonlinerest..journals@eol(code) values ('BRAS')
;
exception
when others
then
dbms_output.put_line('Error on insert:' || sqlerrm);
end;
end loop;
end;
Output of Script
The output from the script contained:
...
06-07-2021 18:59:15 *
06-07-2021 18:59:15 *** LOOP 25 ***
06-07-2021 18:59:15 *
06-07-2021 18:59:15 Error on delete:itgenoda131: The OData server is currently unavailable (because it is overloaded or down for maintenance). Generally, this is a temporary state. Please try again later. (https://start.exactonline.nl/api/v1/999/crm/Contacts(guid'78eccc5d-a2cf-400a-aa72-7e8282605759')).
The remote server returned an error: (503) Server Unavailable.
06-07-2021 18:59:16 Error on insert:itgenoda059: Verplicht: Grootboekrekening (CODE = BRAS).
The remote server returned an error: (500) Internal Server Error.