Page tree
Skip to end of metadata
Go to start of metadata

The full text of this page is only available to our customers.

Please login or sign up. You may also need to provide your support ID if you have not already done so.

Product Name
IBM Fluid Query
Publisher Page
IBM
Category
End-User Query, Reporting, and Analysis
Release
TKU 2019-04-1
More Information
Publisher Link
IBM

Product Description

The product enables you to query data across key data platforms, such as PureData System for Analytics, dashDB in the Cloud, IBM BigInsights, DB2 and Software Defined Environments. It also provides data

movement capabilities between NPS and Hadoop.

IBM Fluid Query allows queries to be sent to other database sources for processing in the Cloud, on-premise, Hadoop, as well as distributed and software defined

 environments. You can quickly query more data for discovery and deeper insights within systems across your data ecosystem.

Architecture

When you register the data connection functions, you can specify whether the functions run in local or remote mode.

In remote mode connector will have constantly running process.

  •  One Local database  can have several connectors (connector configurations)
  •  One connector can connect to several local databases
  •  You can have connectors for different remote JDBC sources.

Diagram:

Known Versions

  • 1.0
  • 1.5
  • 1.6
  • 1.7

Software pattern summary

Product componentOS typeVersioningPattern Depth
FluidQueryUNIXactive, fileInstance-based

Platforms Supported by the Pattern

Unix based

Software Instance Triggers

PatternTrigger NodeAttributeConditionArgument
FluidQueryDiscoveredProcess
cmdmatchesunix_cmd 'java'
and
argsmatchesregex '\scom\.ibm\.nz\.fq\.'

Software Instance Type Attributes Created

Simple Identification Mappings

The patterns in this module will set the following attributes:

NameCondition
IBM Fluid Queryunix_cmd 'java' and args '\scom\.ibm\.nz\.fq\.'

Сonfirmation of the existence:

  • Running process does not mean that Fluid Query is configured and available. At list one installed and configured jdbc connector should work.
  •  Fluid Query can't be installed and run without IBM Netezza Analytics.

Pattern runs command where nz_path is a link that always point to home directory of Netezza SQL server :

 

Сonfirmation CommandPATH=$PATH:%nz_path% %install_root%/fqRemote.sh ping

Command result example:

  • If Netezza Analytics is not works we will get error of fqRemote.

  • If Fluid Query is stopped, or no active connectors, command result will not has sub word 'fluidquery'.

Install Root

Pattern extracts the installation root running search command line using following regular expression:

 

PatternCommand
IBM Fluid Querygrep -m 1 "Package installed in" /nz/var/log/fluidquery_install/fluidquery_install.*

 If command failed, intall_root will be '/nz/export/ae/products/fluidquery' - Home Path that is always the same for versions 1.5+, regarding manual.

Versioning

The methods are tried in an order of precedence based on likely success and/or accuracy of the information that can be gathered. Once a result is obtained, the methods lower in precedence are not attempted. In order of precedence the methods are:

Configuration script Versioning:

Versioning of this product is attempted by running command line using following regular expression:

 

CommandRegular Expression
%install_root%/fqConfigure.sh --version(?i)FluidQuery\sVersion\:\s+(\d+(?:\.\d+)*)

Log versioning:

 Versioning of this product is attempted by running command line using following expression:

CommandExpression
grep -r "INZA java version" %log_root%(?i)version\s+\((\d+(?:\.\d+)*)\)

Fluid Query have the same version as built-in jdbc connector (but without build ).

Connector Details:

Running product means that at list one connector is configured.

Connection details for JDBC connector is attempted running command line using following expression:

  •  One Local database  can have several connectors (connector configurations)
  •  One connector can connect to several local databases
  •  You can have connectors for different remote JDBC sources.
    Left column is registered function, then local database and configuration name.
CommandOutput exampleExpression
'PATH=$PATH:%nz_path% %install_root%/fqAdmin.sh list'

           name     | database  |  config

       -------------+-----------+-----------                                         

        HDPREAD     | MY_NPS_DB | my_dashDB

        READ_DASHDB | MY_NPS_DB | my_dashDB

(?m)^\s*^[^name](\S+)\s+\|\s+(\S+)\s+\|\s+(\S+)

Extracting Details:

if previous command was successful, all details parsed in 3 lists.

The pattern performs searches for XML Path that match the following regular expression by reading output for each config name in connector details.

Command to obtain configuration detailsPATH=$PATH:%nz_path% %install_root%/fqAdmin.sh details %config% 

Where nz_path is a link that always point to home directory of Netezza SQL server,  %config% is name of connector configuration.

 

  1. The IBM Fluid Query pattern searches for <config xml path> that match the following regular expression:

  •  connectionXmlPath\s+\|\s+(\S+\.xml)

    Alternative way to obtain XML Path:

      The IBM Fluid Query pattern obtains <config xml path> from the properties file (%config%.properties) located in the <install root> directory, using regular expression:

  •  connectionXmlPath=(\S+\.xml)
        
       

 2. The IBM Fluid Query pattern searches for <connection URL> that match the following regular expression:
  • connectionURL\W+(\S+)

       Alternative way to obtain <connection URL>:

          The IBM Fluid Query pattern obtains <connection URL> from  <config xml path> file using xpath expression.
  •   //connectionURL

 

   

SI depth

There may be several processes of JDBC connector running on a host, but only one example of Fluid Query can be installed for Host and for Netezza Analytics software. If the product version is unknown, the key based on install root, si type and host.key.

Relationship Creation

Client-server communication relationship

  • If connection urls was extracted by Fluid Query pattern, "IBM Fluid Query" pattern tries to search related SI nodes on remote hosts using SearchFunctions.
    If related Si were found Communication relation is created between 'IBM Fluid Query'  SI (Client) and related Si node (Server).

  • IBM Fluid Query uses internal PostgreSQL server database (Netezza SQL) for storing it's metadata. Pattern tries to ensure that PostgreSQL is the one that needs to be linked, it should contain "/nz/kit.VERSION" in process cmd.

If Netezza SQL Server were found, client-server communication relationship is created between 'IBM Fluid Query' SI (Client) and 'PostgreSQL Database Server' (server)

 

Note:Currently Netezza SQL is discovered by ADDM as PostgreSQL Database Server without databases due to custom jdbc drivers that uses Netezza. We add list of internal databases with which Fluid Query works, as additional attribute.

 

Subject Matter Expertise

Subject Matter Expert input will be welcome on any other potential approaches not discussed.

Testing

The patterns have been tested against the live installations on UNIX platforms, with Netezza Analytics versions 3.2.1 and Netezza NPS 7.2.

Information Sources

Open Issues

There are no known open issues with this pattern.
The full text of this page is only available to our customers.

Please login or sign up. You may also need to provide your support ID if you have not already done so.