We have a similar scenario and see the same problem as the OP, described in this thread (in Dutch).
In our case, the target table contains 100M rows (55GB of on disk data). We also only synchronize insert or update. We’re first dumping OData data (from Twinfield) in a staging table, so deletes cannot be detected anyway. That’s a common limitation of batch ETL processing (streaming CDC processing however… but that’s an entirely different chapter).
The full table scan on the target table cannot help us with that, would take a very significant amount of time and would require significantly more resources (probably more than we have available) on the machine running the Invantive Query Tool.