Adding and managing entity catalogs

To create an entity catalog

1. Select Administration > ETL & System Tasks > Entity catalogs.
2. Click Add new entity catalog. 
3. Enter a unique name for the entity catalog.
4. Enter a description for the entity catalog that can be used to identify the relation between shares.
   
5. Click Save.

To view the details of an entity catalog

An entity catalog shows the details of entities (systems, business drivers, and domains) that are imported by all the ETLs. The details include their data sources, type, and the date when they were imported.

Perform the following steps:

1. Select Administration > ETL & System Tasks > Entity catalogs.
2. Click the required entity catalog.
   The entities imported by all the ETLs that use the entity catalog are displayed.
3. (Optional) To view the entities imported by a specific ETL:
   
   • In the Show only entities used by list, select the required ETL.
     
     
     
   • On the ETL tasks page, select an ETL, and then click Entity catalog.
     You can filter entities by using the filter on the Entity, Id, and Type columns.

To edit an entity catalog

You can modify the name and description of an entity catalog.

Perform the following steps:

1. Select Administration > ETL & System Tasks > Entity catalogs.
2. Clickcorresponding to the entity catalog that you want to edit.
   Or
   Click the entity catalog, and then click Edit entity catalog.
3. Update the name or description, and save the changes.

To delete an entity catalog

1. Select Administration > ETL & System Tasks > Entity catalogs.
2. Clickcorresponding to the entity catalog that you want to delete.

To reconcile entities

When two or more ETLs share an entity catalog, they might create and import data for two different entities that represent the same entity in the real world. To avoid the duplication of entities and data, you can keep only one entity and delete the duplicate entities using entity reconciliation.

Perform the following steps to reconcile duplicate entities:

1. Navigate to Administration > ETL & SYSTEM TASKS > Entity catalogs and locate the entity catalog that contains the entity to reconcile.
2. Click the entity catalog name to access its detail page.
3. Click Reconcile entities. The Entity Reconciliation page is displayed.

4. Specify the following reconciliation criteria:

   Field	Description
   Datasource filter	Select the entities generated by all the datasources (ETLs) of the current catalog or only the specific entities.  All datasources of this catalog: Select all the datasources (ETLs) available for this catalog.  Select datasources:  Select a datasource to reconcile specific entities available for specific ETLs. Datasource: Select the entities that you want to reconcile.
   Entity types	Use the default value (include all the system types) or select the entity type from the list.
   Compare entities by	Choose which fields the entities to reconcile must have in common. You can search the entities by using entity name, entity name and type, or lookup fields.  The lookup fields are populated only for the entities that are part of the selected entity catalog.

5. Click Search.
   The search results display the Entities to reconcile table with the entities that are detected as duplicate. The Destination entity column lists the entities that will remain after the reconciliation is performed. Use the filter icon  next to the Destination entity column to filter the entities by name or entity ID. 
   To view additional details about the entity, click the icon. Details such as the entity creation date, last activity date, lookup fields, and the name of the ETL that created this entity are displayed.
   

1. From the Entities to reconcile table, select the appropriate entities. 
2. Click Reconcile selected rows or Reconcile all to reconcile the entities.
   A pop-up message asking for confirmation about the entity reconciliation is displayed. After confirmation, the reconciliation request is submitted.

After the reconciliation request is submitted, the following message is displayed: Reconciliation in progress. See auditing logs for details or kill the current process.

You can click the auditing logs option to verify the details of all the reconciliations that have been performed. On the Auditing page, the Filter action groups and Action column contain auto-populated data. In the Filter action groups, User Administration category is displayed and in the Auditing messages table, the Action column displays reconciliation in the search. The page displays the reconciliation requests that were submitted. 

You can click kill to terminate the current reconciliation process. This option can be helpful if an incorrect system or too many systems are selected for reconciliation. 

The duplicate entities are merged and are displayed as a single entity in the Workspace > Domains, Services, & Applications. The lookup details display the lookup fields for the ETLs that were reconciled. 

To prevent duplication issues

To avoid these problems, the correct procedure for creating a new ETL task is:

1. Create the new ETL task with simulation mode turned on and the maximum log level (10).
2. Manually run the ETL task and check its execution log to find out if it created any new entities. You can use this information to understand if the automatic lookup process is safe and if you need to use shared lookup from another ETL.
3. If you notice an issue, you can also manually add a line in the lookup table.
4. Toggle simulation mode off.
5. Run the ETL task to import new data.