DataConnector | ComponentOne
In This Topic
    Incremental Caching
    In This Topic

    In case of working with very large data sets, there may be scenarios to completely refresh the cache data each time. At times it is required to only update the recent data. This can be achieved using the caching mode called incremental caching.

    Incremental cache is a way to update the changing data from server to local cache base on timestamp. Each table that wants to apply the incremental, must have a timestamp column, and when a row is inserted or updated, this column must be updated. 

    The below code example implements incremental caching, by setting IncrementalUpdate property to True, by default it is False.

    Copy Code
    IncrementalUpdate = true;

    In C1ConnectionStringBuilder, we have provide two properties, IncrementalUpdate And IncrementalUpdateColumn.

    DataConnector Default TimeStamp Column
    OData modifiedon
    D365S modifiedon
    Salesforce LastModifiedDate
    Kintone Updated_datetime
    QuickBooksOnline MetaData_LastUpdatedTime
    Google Analytics Updated
    Note: Incremental cache only insert new rows and update the changing rows, it cannot delete rows.

    Applications that work with large amount of data need local copy of data for enhanced performance. Usually historical records of such data do not change, the application needs to add new records to the cache regularly and refresh the complete cache periodically. Another application type may need to update historical data and add new data on a regular basis.

    Incremental Cache can be used to solve the above mentioned problems in the following ways.

    The table below explains the CRUD strategies for the above scenarios.

    Database Operation First Scenario Second Scenario
    Create Append to client based on unique identifier.This operation should be executed post expiry of cache tolerance. Append to Client based on unique identifier.This operation should be executed post expiry of cache tolerance.
    Read Get all data at first Get all data at first
    Update NA Update modified records based on timestamp and unique identifier. This operation should be executed post expiry of cache tolerance.
    Delete NA NA

    After performing CRUD operations on a table, to query modified records, you can use implicit cache. When you execute Insert/Update/Delete a table, the corresponding table in local cache will be marked over tolerance time. In offline mode, we still consider tolenrance time. If the table exists, but not in tolerance time, then the exception will be thrown.