How FTS indexing works

Action Request System uses the AR System server Java process to insert, update, or delete data in the FTS indexes. The full text Indexing queue includes the threads, one by default, that perform the indexing. You can configure more indexing threads for FTS. However, you must exercise care. 

The indexing mechanism is based on an asynchronous queue, which is based on an internal system table (ft_pending) in the database during the originating transaction. The originating transaction involves a create entry, set entry, merge entry, or delete entry operation on a form where a field indexed for FTS exists.

A full text dispatcher thread processes the indexing tasks in real-time on a first-in, first-out basis, queuing the tasks for the indexing threads to process. As a result of this indexing model, the performance of the originating transaction is affected only marginally by inserting the indexing task record into the system table and is not subject to delays associated with full text indexing. However, the data might not be immediately available for search. The size of the delay depends on the size of the indexing queue and the availability of system resources to perform the indexing.

Related topics

Enabling-and-disabling-full-text-search


The Full Text Search (FTS) has smart processing for all the entries to be indexed. The smart process optimizes the processing of redundant entries and results in better indexing performance. It provides the following functionality:

  • Avoids processing duplicate indexing requests from JMS and ft_pending.
  • Does not depend on the JMS messaging-based communication for the FTS indexing. It uses only JMS as a wakeup signal and also adds a scheduler for any periodic missed entries.
  • Avoids logically redundant requests by cleaning the requests from ft_pending during the insertion or processing. The system makes sure that only the redundant entries are deleted from the system so that the work required for these entries is completed by some other entry. 
  • Optimizes the synchronization of global or form and entry-level re-indexing in such a way that the resources in the server (CPU and threads) are optimally used for indexing.

Best practice

We recommend that you perform a global re-index or schema re-index during off-peak hours, such as during a maintenance window.

Re-indexing of an entire schema or form deletes the existing indexes and creates the indexes again. So, the re-indexing of a form might be faster as compared to individually updating the index for a larger number of entries. 

If the administrator has disabled indexing, indexing tasks are still recorded, preserving the changes for later inclusion when indexing is enabled.

The following table lists the topics that help you understand how you can use FTS for attachments and localization, assign weights, add rules, and estimate FTS index sizes:

Action

Reference

Use FTSfor attachments.

Use FTS for localization and attachment file formats.

Assign weights in an accrued operation.

Add tokenization rules in FTS.

Estimate size of FTS indexes.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*