Fout itgenclr078 bij ophalen documentbijlagen uit Exact Online

Vragen
,
1

Bij het uitvoeren van de instructie:

select count(*) 
from   ExactOnlineREST..DocumentAttachmentFilesIncremental

via een SQL-script in Invantive Data Hub versie (22.0.191-PROD+3392) ontstaat onderstaande fout.

Hoe kan ik dit oplossen?

2022-06-14 11:34:45.130 Information itgendhb044: SQL-statement uitvoeren:
use 1396556
2022-06-14 11:34:45.134 Information itgendhb018: Gekozen partities veranderd (1 rijen aangepast). (4 ms).
2022-06-14 11:34:45.135 Information itgendhb044: SQL-statement uitvoeren:
select count(*) from ExactOnlineREST..DocumentAttachmentFilesIncremental
2022-06-14 11:57:35.240 Warning itgendhb173: Context: D:\INVANTIVE-EOL\queries\DataHub\01\DataHubQuery9.sql: select count(*) from ExactOnlineREST..DocumentAttachmentFilesIncremental
2022-06-14 11:57:35.610 Error itgencun016: Fout itgenclr078: Externe fout in Invantive Data Hub.

Unterminated string. Expected delimiter: ". Path '[50667].AttachmentFromUrl', line 1, position 710065067.
2022-06-14 11:57:37.633 Error itgenclr078: Newtonsoft.Json.JsonReaderException
ValidationException
   at Newtonsoft.Json.JsonTextReader.ReadStringIntoBuffer(Char quote)
   at Newtonsoft.Json.JsonTextReader.ParseValue()
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateDictionary(IDictionary dictionary, JsonReader reader, JsonDictionaryContract contract, JsonProperty containerProperty, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
   at Invantive.Data.Providers.ExactOnline.ExactOnlineRestProvider.TQZ.MoveNext()
   at Invantive.Data.Providers.ExactOnline.ExactOnlineRestProvider.CQZ.MoveNext()
   at Invantive.Data.GenericProvider.VRB.MoveNext()
   at (Object )
   at IBC.ZK.M(Object )
   at Invantive.Basics.ExtensionMethods.NU`1.MoveNext()
   at Invantive.Data.Providers.DataCache.DataCacheProvider.MaintainTablePartitionVersionFactsOnePartition(GlobalState owner, ExecutionOptions executionOptions, ObjectDefinition objectDefinition, DatabaseColumnDefinitionCollection resultFields, Lazy`1 fetchFullDataSet, Boolean mayReplaceWithCachedData, String sourceDataContainerId, String partitionCode, IDataProvider dataSourceProvider, QueryObject originalQueryObject, Nullable`1 maxAgeDataMs, Decimal version, DcTablePartitionVersion previousReadyTablePartitionVersion, String refreshOrigin, PartitionData& replacementPartitionData)
   at Invantive.Data.Providers.DataCache.DataCacheProvider.MaintainTablePartitionVersionFactsOnePartitionAllVersions(GlobalState owner, ExecutionOptions executionOptions, ObjectDefinition objectDefinition, DatabaseColumnDefinitionCollection resultFields, Boolean mayReplaceWithCachedData, String sourceDataContainerId, String partitionCode, IDataProvider dataSourceProvider, QueryObject originalQueryObject, Nullable`1 maxAgeDataMs, String refreshOrigin)
   at Invantive.Data.Providers.DataCache.DataCacheProvider.AUN(GlobalState , ExecutionOptions , ObjectDefinition , DatabaseColumnDefinitionCollection , PartitionData , QueryObject , Nullable`1 , String )
   at Invantive.Data.Providers.DataCache.DataCacheProvider.GEZ.B(PartitionData , ParallelLoopState , Int64 )
   at Invantive.Basics.ParallelUtility.<>c__DisplayClass1_0`1.<ForEachWithAggregatedExceptions>b__0(TSource tSource, ParallelLoopState parallelLoopState, Int64 index)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at (Object )
   at IBC.SUI.M(Object )
   at Invantive.Basics.ErrorConstants.ThrowOneOrMoreExceptions(GlobalState owner, ExecutionOptions executionOptions, IEnumerable`1 exceptions)
   at IBC.UER.M(GlobalState , ExecutionOptions , IEnumerable`1 )
   at Invantive.Basics.ParallelUtility.ForEachWithAggregatedExceptions[TSource](GlobalState owner, ExecutionOptions executionOptions, Boolean shuffle, IEnumerable`1 source, ParallelOptions parallelOptions, Action`3 body)
   at Invantive.Data.Providers.DataCache.DataCacheProvider.WUN(GlobalState , ExecutionOptions , IDataProvider , ObjectDefinition , QueryObject , ParameterList , YieldedResultSet , Nullable`1 , Boolean , Int32 , String )
   at Invantive.Data.Providers.DataCache.DataCacheProvider.HandleOds(GlobalState owner, ExecutionOptions executionOptions, IDataProvider dataSourceProvider, ObjectDefinition objectDefinition, QueryObject queryObject, ParameterList parameters, YieldedResultSet result, Nullable`1 maxAgeDataMs, Boolean forceUseOds, Int32 maximumNumberOfParallelRequests, String refreshOrigin)
   at Invantive.Sql.V1.OperationalDataStoreExecutionHint.Handle(GlobalState owner, ExecutionOptions executionOptions, IProviderManager manager, YieldedResultSet result, ExecutionHints executionHints, IDataProvider dataSourceProvider, ObjectDefinition objectDefinition, QueryObject queryObject, ParameterList parameters, Int32 maximumNumberOfParallelRequests, String refreshOrigin)
   at Invantive.Sql.V1.DataSourceOrFunctionTree.<GetData>d__51.MoveNext()
   at Invantive.Data.CompressedEnumerable`1.<GetEnumerator>d__10.MoveNext()
   at Invantive.Sql.V1.FirehoseResultSet.RS.MoveNext()
   at Invantive.Sql.V1.FirehoseResultSet.CS.MoveNext()
   at Invantive.Data.CompressedEnumerable`1.<GetEnumerator>d__10.MoveNext()
   at Invantive.Sql.V1.FirehoseResultSet.RS.MoveNext()
   at Invantive.Sql.V1.FirehoseResultSet.CS.MoveNext()
   at Invantive.Sql.V1.ChainedFirehose.XS.MoveNext()
   at Invantive.Sql.V1.FirehoseResultSet.CS.MoveNext()
   at System.Linq.Lookup`2.Create[TSource](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
   at System.Linq.Enumerable.ToLookup[TSource,TKey](IEnumerable`1 source, Func`2 keySelector)
   at Invantive.Sql.V1.GroupByIterator.CK.MoveNext()
   at Invantive.Sql.V1.FirehoseResultSet.CS.MoveNext()
   at Invantive.Sql.V1.SelectListIterator.AC.MoveNext()
   at Invantive.Sql.V1.FirehoseResultSet.CS.MoveNext()
   at Invantive.Data.CompressedEnumerable`1.<GetEnumerator>d__10.MoveNext()
   at Invantive.Sql.V1.FirehoseResultSet.RS.MoveNext()
   at Invantive.Sql.V1.MaterializeSparseArrayIterator.<Iterator>d__1.MoveNext()
   at Invantive.Sql.V1.QueryPlan.IK.MoveNext()
   at Invantive.Data.ExtensionMethods.ATB`1.WTB.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Invantive.Data.ConnectionManager.JR.M(QueryPlan )
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at Invantive.Data.ConnectionManager.ExecuteAndCombineQueryPlansV1(GlobalState owner, ExecutionOptions executionOptions, QueryPlan[] queryPlans, String sqlStatement, ParameterList parameters)
   at Invantive.Data.ConnectionManager.ExecuteProviderPassthroughSqlActionTable(GlobalState owner, ExecutionOptions executionOptions, String actionSql, ParameterList parameters, String callSafeNameOverrule)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Invantive.Data.ConnectionManager.ExecuteProviderPassthroughSqlActionTable(GlobalState owner, ExecutionOptions executionOptions, String actionSql, ParameterList parameters, String callSafeNameOverrule)
   at Invantive.Data.ConnectionManager.PassthroughSqlActionTable(GlobalState owner, ExecutionOptions executionOptions, String actionSql, ParameterList parameters, String callSafeNameOverrule)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Invantive.Data.ConnectionManager.PassthroughSqlActionTable(GlobalState owner, ExecutionOptions executionOptions, String actionSql, ParameterList parameters, String callSafeNameOverrule)
   at Invantive.Data.ActionProceduresBase.PassthroughSqlActionTable(GlobalState owner, ExecutionOptions executionOptions, String actionSql, ParameterList parameters)
   at IDH.H.ProcessStatement(GlobalState owner, ExecutionOptions executionOptions, String statement, ResultSet& resultSet, Action`2 fetchMoreRows, Action`9 executeStatement, Boolean isInInteractiveMode, ScriptingLanguageNextStepSpecification& nextStepSpecification)
   at IDH.H.ProcessStatements(GlobalState owner, ExecutionOptions executionOptions, Action`2 fetchMoreRows, Action`9 executeStatement, IEnumerable`1 statements, String sourceFileName, ScriptingLanguageNextStepSpecification& exitSpecification)
2022-06-14 11:57:37.672 Information itgendhb221: Einde programma-uitvoering met exitcode 6.
2

De foutmelding is:

Unterminated string.
Expected delimiter: ".
Path ‘[50667].AttachmentFromUrl’, line 1, position 710065067.
Error itgenclr078: Newtonsoft.Json.JsonReaderException
…
at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
at Invantive.Data.Providers.ExactOnline.ExactOnlineRestProvider.TQZ.MoveNext()

Dit betekent dat het omzetten van de JSON-image van de vorige versie van de incrementele tabel DocumentAttachmentFilesIncremental niet geslaagd is.

Wat wel opvalt is de enorme omvang van deze JSON-image; er is sprake van positie 710.065.067, oftewel ruim 700 MB. Normaliter is de JSON-image verspreid over meerdere “chunks” en de maximale omvang zonder compressie is 100 MB.

Advies is om de volledige inhoud van de map met JSON-images voor incremental data te verwijderen na het maken en testen van een systeembackup. De volgende laadactie kan dan bijzonder lang duren.

De map is normaliter te vinden in %USERPROFILE%\invantive\Cache\incdata. De bestanden hebben het formaat:

ExactOnlineREST_Incremental_Docu-<HEXNUMBER>.cache

Advies is verder om te kijken in DocumentAttachmentIncremental of er enorme documenten zijn groter dan 50 MB) en die te verwijderen uit Exact Online.

Laatste advies is om het gebruik van DocumentAttachmentFilesIncremental te heroverwegen, evenals het gebruik van replicatie. Het voegt normaliter relatief weinig tot geen waarde toe ten opzicht van DocumentAttachmentsIncremental terwijl de omvang enorm kan zijn en een groot additioneel beslag op schijfcapaciteit kan leggen.

3

Deze vraag is automatisch gesloten na tenminste 2 weken inactiviteit nadat een mogelijk passend antwoord is gegeven. Het laatste gegeven antwoord is gemarkeerd als oplossing.

Gelieve een nieuwe vraag te stellen via een apart topic als het probleem opnieuw optreedt. Gelieve in de nieuwe vraag een link naar dit topic op te nemen door de URL er van in de tekst te plakken.