I’m loading data from Invantive Bridge through Azure Data Factory. To decrease the volume of data, I’ve tested the theory of this article.
I did two loads of table ExactOnlineREST.Incremental.AccountsIncremental@eol
(table with 144 columns). The first one, I used an OData select statement:
?$select=City%2CCode%2CCountry%2CDivision%2CID%2CIsSupplier%2CName%2CPostcode%2CStatus%2CVATNumber%2CAddressLine1%2CAddressLine2%2CEmail%2CPhone%2CPhoneExtension%2CWebsite%2CPaymentConditionPurchase%2CPaymentConditionSales
The second test, I just ran the same, but with a basic path (so not explicitly excluding the other columns):
I see no difference in volume (Network Size), only in duration (where the second version takes more time).
Can it be that using the Mapping in the copy pipeline implicitly does the same as using the select statement (although in Monitoring, I don’t see an explicit reflection in the full path load)? Or am I looking at this wrong?