QlikへのTransactionLinesIncrementalのロードで「System.OutOfMemoryException」が発生

Ik heb geprobeerd om in Qlik (Cloud) de gegevens van de Exact Online API (Invantive Cloud) op te halen:

ExactOnlineREST.Incremental.TransactionLinesIncremental

Helaas krijg ik in Qlik de volgende foutmelding:

De volgende fout is opgetreden:

Interne fout. (Connectorfout: Er is een uitzondering van het type 'System.OutOfMemoryException' opgetreden.)

De fout trad hier op:

SELECT * FROM JsonV4GetData WITH PROPERTIES ( oDataResourcePath='ExactOnlineREST_Incremental_TransactionLinesIncremental_eol', oDataQueryOptions='', maxDepth='', maxResults='' )

Kan iemand mij hierbij helpen?

Complete information

Is it possible to add a (anonymized) screenshot of the request details in Invantive Bridge Online Monitoring as described in More insight with new Bridge Online Monitoring?

Details can be found by clicking on the download request which represents the issue discussed in this topic.

The download request typically has a SQL-statement with table name visible.

Please leave at least the following data visible from both columns:

  • the title bar with the request ID,
  • the status code, network size, path and times in the left column,
  • the error code and error message at the very bottom of the left column,
  • the entire right column including the SQL statement, table name and parameter values.

For example:

Verify correct server and user

Check carefully that you are logging into the Bridge Online website that is also used from Power BI and with the same user name. You will only see the requests from the Invantive Cloud user you are logging into the website with.

There are four servers in use:

  • bridge-online.cloud
  • app-online.cloud
  • bridge-online.invantive.com
  • app-online.invantive.com

You can see the server used in your script or source code of reporting.

Verify correct request and details

Please make sure to select the request to display the details. Only one single request should be visible in the screenshot.

Also check carefully that the request has a path with odata4 or apps. Other requests are in general non-relevant for this purpose.

Error message and tips per email

In addition, the Invantive Cloud user who has the error message will usually receive on his email address an email with an error message and tips if there is an error message in Power BI, Power Query, Azure Data Factory, Qlik or Tableau.

Advice is to check the affected user’s spam for such emails sent from support@invantive.com.

Hier is de screenshot:

image

Ook heb ik al gecontroleerd dat ik gegevens kan ophalen (andere API’s zoals de Receivables waren geen probleem).

Misschien is het nuttig, maar de gevraagde tabel zou 1-2 miljoen regels moeten hebben.

The request 0HNE7KSAM1E3M:00000003 on Invantive App Online was terminated by the OData consumer (Qlik in this case) dropping the connection after 19 minutes and 14 seconds. It received 78.028 rows sofar on 1 partition.

The actual row count can easily be assessed in typically less than a second using the approach explained in:

using a query such as:

select *
from   rowcounts@eol
where  table_name = 'SyncTransactionLines'

For now, it will be assumed that 1-2 million rows are to be expected. Also, the nature of the design of *Incremental tables means that data can only start to return once all rows have been collected for a partition. Since there is only 1 partition selected, this means that the data collection has already ended at this moment in time and Invantive App Online is streaming rows to the OData consumer typically limited only by the bandwidth between producer and consumer.

A volume of 1-2 million rows should not be an issue. As of July 2025, Invantive Cloud is typically starting to get overstretched when there are 4 million or more rows per partition. Larger volumes with tens or hundreds of millions of rows on Exact Online or other cloud platforms will definitely exceed the current specifications for the current configuration of the Invantive Cloud. For these scenarios, the enterprise products are recommended such as on-premises Data Hub or on-premises UniversalSQL Server.

For now, there is no reason to expect any problem on the Invantive Cloud side.

A Google search revealed that the OutOfMemoryException sometimes appears on Qlik. See for instance:

The last one describes an Out of Memory-exception with 500.000 rows. This Qlik community post refers to Solved: " System.OutofMemoryException " appears when i loa... - Qlik Community - 1977929 which might be helpful.

Please note the typical uncompressed size of a dataset of 2 million transaction line rows will be in the range of 2 to 4 GiB.

Het probleem is opgelost. In plaats van de OData-verbinding in Qlik aan te roepen, roep ik nu direct de REST API aan voor TransactionLinesIncremental.

Nu werkt het.

Oké, dat is geweldig!

Misschien downloadt de OData-connector van Qlik eerst de volledige dataset en verwerkt deze vervolgens als geheel, wat leidt tot hogere geheugeneisen. Dit in plaats van dit formaat te behandelen als een speciaal geval van streaming data, zoals de REST-connector lijkt te doen.

Hallo,

Helaas heb ik weer hetzelfde probleem.

image

Zoals je in de screenshot kunt zien, werd de fetch na 16 minuten afgebroken. Ik heb ook de tabel gecontroleerd, het zou minder dan 1 miljoen regels moeten zijn.

Soms haalt het de gegevens na 1 minuut op, soms duurt het langer.

Kun je me hierbij helpen? Ik zou graag een stabiele verbinding willen opzetten om dagelijks de gegevens op te halen.