Summary
Invantive Data Replicator performs automatic upgrades of the repository. The note describes how an administrator can influence the upgrade process.
Data Replicator Repository
Invantive Data Replicator consists of two elements as shown below in an image:
- an application such as Invantive Data Hub with a license for the Data Replicator connector.
- a database containing both a repository as well as current facts and historical facts.
The application executes the logic provided by Invantive, retrieving data from the source systems and uploading them into the target database.
The repository defines the source systems, the source tables and partitions and the way in which the replication process is configured. The repository is typically filled using Invantive SQL statements starting with ‘alter persistent cache set …’
The facts tables contain the current (or better: last registered) contents of the source systems, whereas the history tables contain a registration of the source systems which allows time-travel. The history tables are delta compressed.
The contents of the repository can be queried from within the relational databases, using views starting their name with dc_, such as dc_table_partition_versions_r. The repository’s contents are also available directly from within all products that connect to Invantive SQL, such as Power BI or the Invantive Query Tool using views such as dc_table_partition_versions@datacache. The current data model can be found on documentation.invantive.com.
The size of the repository varies; some sites only have 20 table partitions defined, whereas others have literally hundreds of thousands combinations of tables and partitions they replicate. For sizing purposes, we generally recommend to reserve at least 1 GB of database storage capacity for the repository.
Repository Upgrades
The repository of Invantive Data Replicator has been enhanced over time to provide more functionality. Each change causes an increment in the version number. The current version number is 18 and it typically changes at least on every production release.
When Invantive client software with active Data Replicator license connects to an empty database, the first SQL statement that triggers Data Replicator activity will install the latest by the software supported repository data model. This can be a simple select statement on a data container or a repository query like select * from dc_versions@datacache.
When Invantive client software with active Data Replicator license connects to a database with an existing Data Replicator repository, the first SQL statement that triggers Data Replicator activity will upgrade the repository to the latest by the software supported repository data model.
Forcing an Upgrade
The upgrade process is fully automatic; upon connection Invantive Data Replicator establishes whether an upgrade must be tried. This upgrade check is triggered by using a different Invantive SQL version than previously with the repository, by setting auto upgrade or one-time auto upgrade one with a DDL statement or by forcing an upgrade check.
The upgrade check can be forced without changing the repository or Invantive SQL version by specifying upgrade-force-execute=true in your connection string, such as:
<connection
name="database-name"
dataCacheConnectionString="upgrade-force-execute=true;upgrade-force-repository-version-start=17;maximum-length-logical-view-column-name=30;maximum-length-logical-view-name=30;forced-casing-logical-view-column-name=Lower;forced-casing-logical-view-name=Lower;provider=SqlServer;connectionString={...}"
>
A full list of the available properties for use in the Data Replicator connection string can be found on documentation.invantive.com.
Version Selection
The upgrade typically starts at the last version installed and then progresses to the version supported by the client software. However, you can choose to start at a different (typically earlier) version to ensure that specific steps get executed.
To have the upgrade check use a different version as starting point for the upgrade path, you can set the attribute upgrade-force-repository-version-start as displayed in the example above.
Specials
After the upgrade check and a possibly upgrade, the connection string can require that a number of special actions are to be taken. There are currently two specials, neither of which does anything special with the optional attribute value:
- drop-repository-views: drop all views in the backing database for the repository, such as
dc_table_partition_versions. - drop-facts-views: drop all logical views for facts and history tables, such as
eol_transactionlinesbulk_r.
For example, the following XML fragment contained within dataCacheConnectionString will cause execution of both:
upgrade-force-specials={drop-repository-views;drop-facts-views=no relevant value}