Integrating with Prometheus to collect events via webhook

Task 1: To plan for the connection

Review the following prerequisites to help you plan and configure a connection with Prometheus via a webhook.

Prometheus prerequisite

If you want to collect events data by using the custom JSLT mapping, prepare the events JSLT mapping, as shown in the
following example:

BMC Helix Intelligent Integrations prerequisites

• Depending on the location of the third-party product (SaaS, on-premises), choose one or more BMC Helix Intelligent Integrations deployment modes and review the corresponding port requirements. For information about various deployment modes and port requirements, see Deployment scenarios.
• Based on the deployment mode, use the BMC Helix Intelligent Integrations SaaS deployment or the BMC Helix Intelligent Integrations on-premises gateway or both. For more information about the gateway, see Deploying the BMC Helix Intelligent Integrations on-premises gateway.

In the preceding list, third-party product refers to Prometheus.
 

Task 2: To configure the connection with Prometheus via a webhook

1. Depending on the deployment mode, perform one of the following steps to access BMC Helix Intelligent Integrations:
   1. BMC Helix Intelligent Integrations SaaS – Log on to BMC Helix Portal, and click Launch on BMC Helix Intelligent Integrations.
   2. BMC Helix Intelligent Integrations on-premises gateway – Use the following URL to access BMC Helix Intelligent Integrations: https://<hostName>:<portNumber>/swpui
2. On the CONNECTORS tab, clickin the SOURCES panel.

3. Click 

    the 

   Prometheus

   Alert Manager Webhook tile.

4. Specify a unique instance name for the source connection:

   Best practice
   We recommend that you specify the instance name in the following format:
   <sourceType>_<sourceControllerServerName>_<InstanceQualifier>
   The instance qualifier helps you to distinguish the multiple instances configured from the same source
   server. For example, you can name your instances Prometheus_Host_PROD and Prometheus_Host_TEST.

5. Click VALIDATE AND CREATE.
   The specified connection details are validated and the corresponding source connection is created in the Source Connection list.
6. Verify that the source connection you created is selected.

   Important
   The destination host connection is created and configured automatically for each tenant when the source
   connection is created.

7. Click CREATE COLLECTORS to create the required collector streams for the selected data types.

8. Click a data type and specify the configuration parameters in the Distributors section as described in the following table:

   Parameter name	Description
   Max Batching Size	Specify the maximum number of data items to send in a single POST request to the destination API. The batch size depends on the destination’s ability to buffer the incoming data. Default: 250
   Max Batching Delay	Specify the maximum time (in seconds) to wait before the system builds and processes a batch. Default: 3 seconds
   Base Retry Delay	Specify the initial time (in seconds) for which to wait before the system tries to rebuild and process a batch. The waiting time increases in the following sequence: n1, n2, n3, and so on, where n indicates the number of seconds. Default: 2 seconds Example: Base Retry Delay is set to 2 seconds. Retry is performed after 2, 4, 8, 16, and so on, seconds.
   Max Intra-Retry Delay	Specify the maximum limit for the base retry delay.  Default: 60 seconds Example: Max Intra-Retry Delay is set to 60 seconds. Base Retry Delay is set to 2 seconds. Retries are performed 2, 4, 8, 16, 32, and so on, seconds later.
   Max Retry Duration	Specify the total time for which the system retries a delivery. For REST destinations, a delivery is a batch of data items in one POST request.  Default: 5 minutes Example: Max Retry Duration is set to 8 hours. Base Retry Delay is set to 2 seconds. Requests are sent for 2+4+8+16+32+64+132, and so on, until 8 hours in total duration is reached. After that, no subsequent attempts are made to retry the delivery. The assumption here is that if there is an outage or other issue with the destination tool, the recovery time should be less than the value of the Max Retry Duration parameter.
   Attributes To Be Dropped When Updating Events	Specify the event attributes that you do not want to be updated in BMC Helix Operations Management when events are updated.  For example, if you do not want an event's severity, source address, source category, and subcategory to be updated in BMC Helix Operations Management, you need to specify those attributes in a comma-separated format: severity,source_address,source_category,source_subcategory Important: You can obtain the event attribute names in BMC Helix Operations Management by exporting any event data in JSON, BAROC, XML, or CSV format. The exported file contains all attributes of the event data, and from there you can identify the attributes to be dropped.

9. Click CREATE DISTRIBUTORS to create the required distributor streams for the selected data types.

10. Click VALIDATE AND CREATE and then click SAVE STREAM to save the stream.
    After you save the stream, the connection that you just created is listed on the SOURCES panel.

11. Perform the following steps to enter the custom JSLT mapping: 

    1. On the SOURCES panel, click Configure Mediator for the source connection that you created and then expand PROMETHEUS ALERTS WEBHOOK and navigate to the DISTRIBUTOR CONFIGURATION tab.
    2. To view JSON in the edit mode, click Edit JSON.
    3. Search for the following line: "jsltField": "//NO JSLT",
    4. Delete the existing value from the jsltField field; the result should look as follows:"jsltField": "",
    5. To disable the edit mode for JSON, click Edit JSON.
    6. In the JSLT field, enter the JSLT that you have prepared according to the sample shown in Prometheus prerequisite.
    7. Click SAVE & CLOSE.

12. Navigate to the COLLECTOR CONFIGURATION tab.

13. Click copy to copy the auto-generated Prometheus webhook collector URL and save it in a temporary file.

    Example

    https://hostA/hii/api/mediator/v3/push/5a092cf4-e795-43f6-a3af-e2c3d9ee1e9a?token=API-KEY

14. Depending on whether you are using only SaaS deployment of BMC Helix Intelligent Integrations or BMC Helix Intelligent Integrations on-premises gateway, perform the following steps:

    • If you are using only SaaS deployment of BMC Helix Intelligent Integrations or the on-premises gateway with authentication enabled, perform the following steps:

      1. Log on to BMC Helix Portal and generate an access key.
         For instructions, see Setting up access keys for programmatic access.

      2. Copy the generated access key and save it in a temporary file.
         The key is generated in the following format: <accessKey>::<secretKey>,tenant id:<tenantID>.

         Example

         Y40OSC49QZA11Q8A1H9H6::MnVLk69TNyCEponsthHJ1Hj1uKcjTB,tenant id:385261281

      3. Change the format of the access key to <tenantID>::<accessKey>::<secretKey>.|

         Example

         385261281::Y40OSC49QZA11Q8A1H9H6::MnVLk69TNyCEponsthHJ1Hj1uKcjTB

      4. In a temporary file, modify the auto-generated collector URL by replacing API-KEY with the access key that you formatted in the previous step.

         Example

         https://hostA/hii/api/mediator/v3/push/5a092cf4-e795-43f6-a3af-e2c3d9ee1e9a?token=385261281::Y40OSC49QZA11Q8A1H9H6::MnVLk69TNyCEponsthHJ1Hj1uKcjTB

      5. Configure Prometheus to forward incidents data to BMC Helix Intelligent Integrations.

    • If you are using the on-premises gateway with authentication disabled, perform the following steps:

      1. Save the URL in a temporary file.

      2. Remove the following string from the collector URL:?token=API-KEY
         The updated collector URL looks like the following example:

         Example

         https://hostA/hii/api/mediator/v3/push/5a092cf4-e795-43f6-a3af-e2c3d9ee1e9a

      3. Configure Prometheusto forward incidents data to BMC Helix Intelligent Integrations.

15. On the SOURCES panel, move the slider to the right to start the data stream for the connector you created.

    Important
    For a data stream, the Run Latency (max/avg), Items (Avg per Run), and Last Run Status columns on the
    Streams page might show the status as No Runs during the data collection process. After completion of
    the process, these columns are updated with an appropriate status.