Performing deep pagination search by using the REST API
The following section provides a list of supported endpoints and an overview about running these endpoints. Before you run an endpoint, you must authenticate yourself. For more information, see Access and authentication for the REST API
Use the msearch API to perform a deep pagination search. Deep pagination refers to retrieving a large number of results beyond the default page or document limits.
POST
/logs/msearch
Authorization: Bearer <JWT_token>
Permission: loganalytics.logs.manage
For instructions about obtaining the JWT token, see Access and authentication for the REST API
Parameter details
Parameter name | Value type | Mandatory | Description |
|---|---|---|---|
size | Integer | Yes | The number of search results to be returned in a single API call. |
query | Object | Yes | The query to filter search results. This parameter supports the Elasticsearch Domain Specific Language query (DSL). |
sort | Array | Yes | The sorting criteria for the search results. At least one field is mandatory. For example: [{"@timestamp": "asc"}]. |
| search_after | Array | No | The sort values of the last document from the previous response. This parameter is used for pagination. It uses the sort values of the last document in the current result set as a reference point to determine where to begin the next search. |
| @timestamp | DateTime | Yes | The time range filter for search results. |
First msearch API invocation sample request
Use the search_after parameter to take the output from the first API request and use it as an input for the subsequent request. The search_after parameter is essential for pagination and helps retrieve subsequent results in a result set.
- The search_after parameter paginates through the results by using the sort values of the last document from the previous response.
- The sort values are an array of values that correspond to the fields specified in the sort parameter of the request.
Send an initial request to the API with a sort parameter, for example, "sort": [{"@timestamp": "asc"},{"_id": "asc"}].
First API invocation response returning sort array
The response contains a list of documents with a sort array. The sort values [1742973656518,7wdT0ZUBsdpwZrf8aLkU] are extracted from the last document.
"took": 228,
"responses": [
{
"took": 227,
"timed_out": false,
"_shards": {
"total": 102,
"successful": 102,
"skipped": 99,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": [
{
"_index": "logarc_454758490-00_r7_v1-000229",
"_id": "0wdT0ZUBsdpwZrf8aLoU",
"_score": null,
"_source": {
"bmc_user_groups": [""],
"message": "Some Log Message",
"log.file.path": "/opt/logfile.log",
"bmc_tags": "",
"bmc_integration_name": "43532-a89c3753",
"bmc_integration_id": "b57f1453-f901-48e5-8865-b6d2ee2708f6",
"bmc_connector_name": "connector name",
"bmc_connector_id": "a89c3753-8e9b-4f3c-b017-8d785317420b",
"bmc_policy_name": "Policy",
"log_source_host": "host.domain.com",
"file_format_type": "none",
"@timestamp": "2025-03-26T07:20:56.518Z"
},
"sort": [1742973656518, "0wdT0ZUBsdpwZrf8aLoU"]
},
{
"_index": "logarc_454758490-00_r7_v1-000229",
"_id": "1AdT0ZUBsdpwZrf8aLkU",
"_score": null,
"_source": {
"bmc_user_groups": [""],
"file_format_type": "none",
"@timestamp": "2025-03-26T07:20:56.518Z",
"message": "Some Log Message",
"log.file.path": "/opt/logfile.log",
"bmc_tags": "",
"bmc_integration_name": "43532-a89c3753",
"bmc_integration_id": "b57f1453-f901-48e5-8865-b6d2ee2708f6",
"bmc_connector_name": "connector name",
"bmc_connector_id": "a89c3753-8e9b-4f3c-b017-8d785317420b",
"bmc_policy_name": "Policy",
"log_source_host": "host.domain.com",
},
"sort": [1742973656518, "1AdT0ZUBsdpwZrf8aLkU"]
}
]
},
"status": 200
}
]
}
Subsequent msearch API invocation sample request
Use the extracted sort value in the search_after parameter of the next request.
Subsequent API invocation response
This response lists the documents that appear after the record with a sort value [1742973656518,FQdT0ZUBsdpwZrf8aLsU].
Subsequent request:
{
"size": 10,
"query": {
"bool": {
"must": [],
"filter": [
{
"bool": {
"should": [
{ "match": { "log_source_host": "pa-psr-lnx099.bmc.com" } }
],
"minimum_should_match": 1
}
},
{
"range": {
"@timestamp": {
"gte": "2025-03-26T07:20:56.518Z",
"lte": "2025-03-26T07:21:56.518Z",
"format": "strict_date_optional_time"
}
}
}
],
"should": [],
"must_not": []
}
},
"sort": [
{ "@timestamp": "asc" },
{ "_id": "asc" }
],
"search_after": [1742973656518, "7wdT0ZUBsdpwZrf8aLkU"]
}
Response:
{
"took": 1407,
"responses": [
{
"took": 1407,
"timed_out": false,
"_shards": {
"total": 102,
"successful": 102,
"skipped": 99,
"failed": 0
},
"hits": {
"total": { "value": 10000, "relation": "gte" },
"max_score": null,
"hits": [
{
"_index": "logarc_454758490-00_r7_v1-000229",
"_id": "8QdT0ZUBsdpwZrf8aLoU",
"_score": null,
"_source": {
"bmc_user_groups": [""],
"message": "Some Log Message",
"log.file.path": "/opt/logfile.log",
"bmc_tags": "",
"bmc_integration_name": "43532-a89c3753",
"bmc_integration_id": "b57f1453-f901-48e5-8865-b6d2ee2708f6",
"bmc_connector_name": "connector name",
"bmc_connector_id": "a89c3753-8e9b-4f3c-b017-8d785317420b",
"bmc_policy_name": "Policy",
"log_source_host": "host.domain.com",
"file_format_type": "none",
"@timestamp": "2025-03-26T07:20:56.518Z"
},
"sort": [1742973656518, "8QdT0ZUBsdpwZrf8aLoU"]
},
{
"_index": "logarc_454758490-00_r7_v1-000229",
"_id": "9AdT0ZUBsdpwZrf8aLoU",
"_score": null,
"_source": {
"bmc_user_groups": [""],
"log_source_host": "pa-psr-lnx099.bmc.com",
"file_format_type": "none",
"@timestamp": "2025-03-26T07:20:56.518Z",
"message": "Some Log Message",
"log.file.path": "/opt/logfile.log",
"bmc_tags": "",
"bmc_integration_name": "43532-a89c3753",
"bmc_integration_id": "b57f1453-f901-48e5-8865-b6d2ee2708f6",
"bmc_connector_name": "connector name",
"bmc_connector_id": "a89c3753-8e9b-4f3c-b017-8d785317420b",
"bmc_policy_name": "Policy",
"log_source_host": "host.domain.com",
},
"sort": [1742973656518, "9AdT0ZUBsdpwZrf8aLoU"]
}
]
},
"status": 200
}
]
}
Response codes
Code | Message | Description |
|---|---|---|
200 | Ok | The request was successful, and the response contains the requested data. |
400 | Bad request | The request is invalid because of missing or incorrect parameters, such as an invalid sort clause or a missing search_after value. |
401 | Authentication has failed | The request lacks valid authentication credentials. Make sure that the authorization header contains a valid bearer token. Verify JWT. |
| 403 | Forbidden | The user does not have the required permissions to access the endpoint. Make sure that the permission header is correctly set. Verify that the user has the loganalytics.logs.manage permission. |
| 404 | Not Found | The requested resource does not exist. |
500 | Unable to connect to a server | No records are received. Contact BMC Support. |