Importing Reference Data from other Sources


Summary

This page discusses risks and mitigation around importing data in Tideway Foundation. With Foundation's provenance feature which shows exactly how data is captured and where it came from, it is important that any imported data goes through the same level of data quality checking.

==

Impact

Risk

Mitigation

Credibility of Source Data

* Data source collects manually inputted data
* Data source is inconsistent

* Workshop with data source owners and project
Sponsors to confirm the value of the reference data
* Document limitations on data accuracy in the Tideway solution design document
* Script input process to include test and notification of changes to data for validation with previously imported versions of the data

Availability of Source Data

* Data source is not available on a scheduled basis
* Data source is not available for all regions

* Reduce import frequency to reflect how often the source data is available
* If exporting imported data from Foundation, amend export mapping files to auto-populate 'missing data' flag in the receiving system

Representation of Data in TWF

* Data is not represented in Foundation as found in original source (i.e. BTU data for a model spec where discovered spec differs)
* No legal permission to represent the data in Tideway (3rd party reference data)

* Document the data mapping and review with technology teams who will use the data and Sponsors
* Check legal ownership of all imported data before importing into Foundation
* Review Foundation CSV reports and Visualisation reports to closely match data source representation within Foundation

Reconciliation

* Data in Foundation does not reconcile with source data

* Test and document all import processes in the solution architecture
* Establish a process to proactively review source data against data imported into Foundation every 3 months
* Define owner and responsible party for all source data

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*

BMC Discovery content reference