Data Integration

When upgrading to a new version of a database or application, or when switching to a new system, data needs to be preserved in the new system. The purpose of data migration is to transfer existing data to the new environment. It needs to be transformed to a format suitable for the new system, at the same time preserving the information present in the old.

Data Integration

In most organizations, operational data integration is addressed by implementing custom programs or routines, completed on-demand for a specific need. Data migration/loading and data synchronization/replication are the most common applications of operational data integration. You may have the data but it needs to be accessed, merged, checked and understood which can often be difficult and time consuming.
Data integration takes the form of conforming dimensions and conforming facts. Conforming dimensions establish common dimensional attributes across separate databases so the drill-across reports can be generated using these attributes. Conforming facts are the common business metrics, such as KPIs across separate databases, which allow numbers to be compared mathematically by calculating differences and ratios.

Extract Transform Load (ETL)

Inteliment is capable of providing transformations using multiple techniques - ETL (extract, transform, load). With this approach the data is transformed on the target after being loaded. This is especially pertinent if the target database is powerful enough where it can be used to perform all transformations and optimize both performance and investment. In this case there is no 'useless' data transferred on the network and it takes full advantage of the power of the RDBMS. As is the case with all such tools, the flexibility to switch to a more traditional ETL architecture is always there.

Data Latency

Data latency refers to how quickly the data must be delivered to the end users. Data latency will have a significant impact on the design and implementation of the ETL system. Traditional batch-oriented data flows may be sufficient to meet the needs of the business. However, if time is critical, the ETL process may have to be implemented using a streaming data flow.