Invantive Cloud provides a free Power BI connector Freshdesk, but sites with over 30.000 tickets of history typically were confronted with limitations that made use unrealistic. This was typically visible by an itgenfpr121 or itgenfpr117 error. Additionally, performance was sometimes less than optimal.
No Freshdesk 30.000 Tickets Limit
Starting release 22.0.461 from December 9, 2022, the maximum practical data volume of Freshdesk tickets has increased to over 500.000 tickets using the new table tickets_incremental.
The old table tickets remains available with a maximum of 30.000 tickets retrieved in one query for data.
Improved Ticket Download Performance
The initial download time for large data volumes of tickets from Freshdesk remains identical to previous releases, such as 15 minutes for 150.000 tickets. Your actual throughput for the initial download depends on Freshdesk subscription level and current Invantive Cloud load.
Downloads from OData-cache into Power BI, Power Query, Power Automate or Azure Data Factory still return at line speed retrieving up to over 1 gigabyte of data per second, which allows single digit second downloads for hundreds of thousands tickets when your OData-client (Power BI) or SQL-client has sufficient resources and network bandwidth.
A significant performance improvement is seen in subsequent real-time updates not coming from the OData-cache. These Freshdesk ticket downloads from tickets_incremental typically complete within 1 minutes per one hundred thousand tickets.
Internal Workings
The new table tickets_incremental employs an incremental strategy to first download all tickets in a long sequence of Freshdesk API calls. On subsequent downloads, the initial set is mutated to include all changes: new tickets, changed tickets and deleted tickets. The resulting set of tickets is then returned to the user.
The internal workings can be analyzed by querying the table incremental_statistics.