Best practices for improving the overall Remedy performance to optimum level
As BMC Remedy AR System administrators and security administrators, use the recommendations and configuration checklists described in this section to optimize the performance of AR System and its components.
The recommendations are derived based on testing outcomes. For optimum performance, we recommend that you test all the configuration settings in a testing environment before applying them to the production environment.
Warning
Contact your system administrator before making any configuration change specific to your setup.
Make sure to back up the settings before making changes to the configuration.
Environment and sizing considerations
For information about the sizing configurations that are specific to small, medium and large models, see Sizing baselines for the basic deployment model in
Remedy on-premises solution architecture
in the Remedy Deployment online documentation.
Web proxy or load balancer for Remedy Single Sign-On service
The following network segments are available for web proxy or load balancer for Remedy Single Sign-On service:
- From the browser proxy or the actual client (if proxy is not available) to the F5 load balancer.
- From the F5 load balancer to the Single Sign-On service.
You need not activate the HTTP keep-alive for either of the network segments because the interaction between the browser and the Remedy Single Sign-On service is brief.
For the F5 load balancer, we recommend the round-robin load balancing and no session binding.
Important
- You must set the client HTTP protocol version for the HTTP monitoring to HTTP 1.1. The return HTTP code for alive is HTTP 200.
For performance testing, the granularity of the ping interval was 5 seconds with three consecutive non-HTTP 200 errors, indicating the node is down.
For deployment, configure the ping interval to a lower threshold as required.
JVM options for Remedy Single Sign-On
The following table provides information about the Java virtual machine (JVM) options in tomcat/bin/setenv.sh for Remedy Single Sign-On:
Component | Small environment | Medium environment | Large environment |
---|---|---|---|
BMC Remedy Single Sign-On |
|
|
|
Java settings: Java 1.8 (Concurrent Mark Sweep (CMS) Garbage Collector (GC)) |
|
|
|
Use the following settings on all the three environment sizes:
| |||
Java settings: Java 11 or later (Using G1GC) |
|
|
|
Use the following settings on all the three environment sizes:
| |||
Tomcat/conf/server.xml | maxThreads=1600 | maxThreads=3000 | maxThreads=3000 |
Tomcat/conf/logging.properties | Change all log level from INFO to SEVERE | Change all log level from INFO to SEVERE | Change all log level from INFO to SEVERE |
Web proxy or load balancer for Mid Tier
The following network segments are available for Web proxy or load balancer for Mid Tier:
- From the client browser proxy to the F5 load balancer.
- From the F5 load balancer to the Mid Tier Tomcat cluster.
Important
You must turn on the HTTP keep-alive feature for both the network segments.
The following table provides the optional values for the HTTP keep-alive feature supported in Tomcat:
Keep-alive property | Value |
---|---|
Keep-alive count | 5,000 |
Keep-alive timeout | 60 seconds |
For the F5 load balancer, we recommend the round-robin load-balancing scheme with HTTP session persistence, where a cookie is inserted for that session.
To detect whether a node is alive, use HTTP monitoring and issue a GET on the following URL:
/arsys/shared/images/bkgd_image.gif
Important
- You must set the client HTTP protocol version for the HTTP monitoring to HTTP 1.1. The return HTTP code for alive is HTTP 200.
- For performance testing, the granularity of ping interval was 15 seconds with 3 consecutive non-HTTP 200 errors, indicating the node is down.
For deployment, configure the ping interval to a lower threshold as required.
JVM options for Mid Tier
The setting options listed in this section assume that each Tomcat instance in the cluster hosts only the Mid Tier and the Remedy Single Sign-On server agent.
For Windows, use the Configure Tomcat tool and select the Java tab to set the options.
BMC AR Mid Tier server configuration settings
The following table provides information about the JVM Options in tomcat/bin/startup.sh for Mid Tier Tomcat:
Component | Small environment | Medium environment | Large environment |
---|---|---|---|
BMC Remedy AR System Mid Tier servers |
|
|
|
Java settings: Java 1.8 (Using CMS GC) |
|
|
|
Use the following settings on all the three environment sizes:
| |||
Java settings: Java 11 or later (Using G1GC) |
|
|
|
Use the following settings on all the three environment sizes:
| |||
Tomcat/conf/server.xml (connector configuration) |
|
|
|
Tomcat/conf/logging.properties | Change all log levels from INFO and FINE to SEVERE | Change all log levels from INFO and FINE to SEVERE | Change all log levels from INFO and FINE to SEVERE |
MID TIER/WEB-INF/classes/config.properties You can change the settings in the Connection settings section on the config.jsp page. |
|
|
|
JVM Options in tomcat/bin/startup.sh for the Mid Tier + Remedy Single Sign-On server Tomcat
The settings listed in this section assume that each Tomcat instance in the cluster hosts both the Mid Tier (with the Remedy Single Sign-On server agent) and the Remedy Single Sign-On server web app.
All settings mentioned in BMC AR Mid Tier server configuration settings are applicable except for the following row:
Component | Small environment | Medium environment | Large environment |
---|---|---|---|
Tomcat/conf/server.xml (connector configuration) | maxThreads=1600 | maxThreads=3000 | maxThreads=3000 |
Plug-in servers configuration
This topic provides the recommendations for the plug-in server configuration.
Important
Plug-in servers can run with the default Garbage Collector (GC) settings unless they act primarily as servers and are configured for >= 4 GB heap.
The FTS plug-in server requires requires additional GC settings as listed in the following table:
Filename | Change to |
---|---|
<ar install dir>/pluginsvr/pluginsvr_config.xml | Add or alter the file to include: <numCoreThreads>30</numCoreThreads> |
<atrium core install dir>/cmdb/plugins/shared/pluginsvr_config.xml | Add or alter the file to include: <numCoreThreads>10</numCoreThreads> |
AR Dir/pluginsvr/fts/secondary/pluginsvr_config.xml |
Plug-in server configuration settings
The following table lists the plug-in server configuration settings for the different environments:
Plug-in server settings | Small environment | Medium environment | Large environment |
---|---|---|---|
Number of Users | 800 | 2,000 | 5,000 |
Plug-in Server Heap Size | 4 GB | 6 GB | 10 GB |
numCoreThreads** | 10 | 12 | 16 |
Garbage Collector (GC) settings
We recommend using the following GC parameters with the plug-in server and FTS when using Java version 1.8:
- XX:+DisableExplicitGC
- XX:+UseParNewGC
- XX:+UseConcMarkSweepGC
- XX:NewRatio=2
- XX:+UseCompressedOops
For Java version 11 or later, use the following GC parameters:
- XX:+DisableExplicitGC
- XX:MaxMetaspaceSize=512m
- XX:MetaspaceSize=384m
- XX:+UseCompressedOops
- XX:+UseG1GC
- XX:+UseStringDeduplication
- XX:G1NewSizePercent=40
- XX:+UnlockExperimentalVMOptions
- XX:MaxGCPauseMillis=200
For more information about configuring Java plug-in server options, see Updating the AR Monitor configurations to add or modify Java arguments for plug-in servers.
Database server configuration settings
Use these recommendations for the database server settings. All recommendations made in this section are based on testing outcomes and customer feedback. Certain values might vary based on specific environments.
Consult your Database Administrator (DBA) before making changes to your database environment. Whenever possible, we recommend that you test the settings before applying them to production.
For more information about database server configuration settings, see Setting plugin server configuration options.
The following table lists the database server configuration settings:
Component | Small environment | Medium environment | Large environment |
---|---|---|---|
BMC Remedy AR Database server |
|
|
|
Oracle database | |||
memory_target (>11g) | 8 | 16 | 32 |
sga_target (10g) | 5 | 10 | 20 |
Db_cache_size | 3 | 7 | 15 |
Shared_pool_size | 2 | 3 | 5 |
Pga_aggregate_target | 3 | 6 | 12 |
Cursor_sharing | EXACT | EXACT | EXACT |
_b_tree_bitmap_plans | False | False | False |
SQL server database | |||
Parameterization | Simple | Simple | Simple |
READ_COMMITTED_SNAPSHOT | On | On | On |
Max Degrees of Parallelism | 1 to 8 | 1 to 8 | 1 to 8 |
Cost Threshold for Parallelism | 50 | 50 | 50 |
PostgreSQL database | |||
max_connections | 200 | 300 | 700 |
shared_buffers | 4 GB | 8 GB | 16 GB |
effective_cache_size | 12 GB | 24 GB | 48 GB |
maintenance_work_mem | 1 GB | 2 GB | 2 GB |
checkpoint_completion_target | 0.9 | 0.9 | 0.9 |
wal_buffers | 16 MB | 16 MB | 16 MB |
default_statistics_target | 100 | 100 | 100 |
random_page_cost | 1.1 | 1.1 | 1.1 |
effective_io_concurrency | 200 | 200 | 200 |
work_mem | 5242 kB | 6990 kB | 5991 kB |
min_wal_size | 2 GB | 2 GB | 2 GB |
max_wal_size | 8 GB | 8 GB | 8 GB |
max_worker_processes | 8 | 16 | 32 |
max_parallel_workers_per_gather | 4 | 4 | 4 |
max_parallel_workers | 8 | 16 | 32 |
Important
If you choose to work with PostgreSQL (pgSQL) databases, have your DBA schedule and configure the auto-vacuum feature based on your business needs and usage patterns. After performing an upgrade, we recommend running a VACUUM FULL ANALYZE command from pgAdmin for every AR System database. Keeping pgSQL vacuumed is required to reclaim unused disk space and keep your statistics up to date. To learn more about the vacuum feature, see
Routine Vacuuming
in the PostgreSQL documentation.
Dedicated AR System server for Smart Reporting
We recommend setting up a dedicated AR Server for high-volume reporting environments to connect to the Smart Reporting server.
You can first set up this server similar to a small-size, user-facing server, and then adjust memory and CPU based on Report and Data volume.
Tomcat/conf/server.xml for smart reporting
The following table provides AR System sever recommendations for Smart Reporting:
Component | Small environment | Medium environment | Large environment |
---|---|---|---|
BMC Remedy Smart Reporting Server |
|
|
|
Concurrent users supported | 50 | 100 | 250 |
Java settings (arserver.config): Java 1.8 (Using CMS GC) |
|
|
|
Use the following on all the three environment sizes:
| |||
Java settings (arserver.config): Java11 or later (Using G1GC) |
|
|
|
Use the following on all the three environment sizes
| |||
maxThreads | 600 | 1200 | 2400 |
connectionTimeout | 90000 | 90000 | 90000 |
maxHttpHeaderSize | 8192 | 8192 | 8192 |
compression | ON | ON | ON |
compressionMinSize | 2048 | 2048 | 2048 |
compressibleMimeType | text/html,text/xml,text/plain, text/css*, text/javascript*, text/json*, application/x-javascript*, application/javascript*, application/json* | text/html,text/xml,text/plain, text/css*, text/javascript*, text/json*, application/x-javascript*, application/javascript*, application/json* | text/html,text/xml,text/plain, text/css*,text/javascript*, text/json*, application/x-javascript*, application/javascript*, application/json* |
noCompressionUserAgents | gozilla, traviata | gozilla, traviata | gozilla, traviata |
disableUploadTimeout | True | True | True |
enableLookups | False | False | False |
minSpareThreads | 25 | 25 | 25 |
* SP4 and above only. |
Connection pool
Log on to Remedy home page and navigate to Smart Reporting home page > Administration > Admin Console > Data Sources > AR System.
The following table lists the connection pool settings:
Small environment | Medium environment | Large environment |
---|---|---|
Min Connections - 50 | Min Connections - 50 | Min Connections - 50 |
Max Connections - 100 | Max Connections - 200 | Max Connections - 300 |
Refresh Time - 3 hours | Refresh Time - 3 hours | Refresh Time - 3 hours |
Timeout - 6400 seconds | Timeout - 6400 seconds | Timeout - 6400 seconds |
WEB-INF settings
Access the web.xml file in <Smart-Reporting-Dir>/BMCReporting/SmartReporting/appserver/webapps/ROOT/WEB-INF/.
The following table lists the WEB-INF settings:
Small environment | Medium environment | Large environment |
---|---|---|
JDBCMinCount 50 | JDBCMinCount 50 | JDBCMinCount 50 |
JDBCMaxCount 100 | JDBCMaxCount 200 | JDBCMaxCount 300 |
JDBCTimeout 1800 | JDBCTimeout 1800 | JDBCTimeout 1800 |
Smart IT
The following table lists the Smart IT configuration settings:
Component | Small environment | Medium environment | Large environment |
---|---|---|---|
BMC Remedy Smart IT | One server | Two servers | Three servers 8-CPU core 16 GB RAM 200 GB disk space |
Java settings (arserver.config): Java 1.8 (Using CMS GC) |
|
|
|
Use the following on all the three environment sizes:
| |||
Java settings (arserver.config): Java 11 or later (Using G1GC) |
|
|
|
Use the following on all the three environment sizes:
| |||
ulimit (Linux only) | ulimit -n 21000 | ulimit -n 21000 | ulimit -n 21000 |
Tomcat/conf/server.xml | acceptCount= 250 connectionTimeout= 60000 maxKeepAliveRequests= 5000 maxThreads= 1200 minSpareThreads= 50 | acceptCount= 250 connectionTimeout= 60000 maxKeepAliveRequests= 5000 maxThreads= 2000 minSpareThreads =50 | acceptCount= 250 connectionTimeout= 60000 maxKeepAliveRequests= 5000 maxThreads= 2000 minSpareThreads= 50 |
BMC Remedy AR System Server configuration settings
The following tables provide details about the BMC Remedy AR System server configuration settings for user-facing, backend integration, and common features:
BMC Remedy AR System Server user-facing configuration settings
The following table lists the user-facing configuration settings for BMC Remedy AR System server:
Component | Small environment | Medium environment | Large environment |
---|---|---|---|
BMC Remedy AR System servers |
|
|
|
Java settings (arserver.config): Java 1.8 (Using CMS GC) |
|
|
|
Use the following on all the three environment sizes:
| |||
Java settings (arserver.config): Java 11 or later (Using G1GC) |
|
|
|
Use the following settings on all the three environment sizes:
| |||
Fast queue | Private-RPC-Socket: 390620 4 6 | Private-RPC-Socket: 390620 8 12 | Private-RPC-Socket: 390620 16 24 |
Loop back or private socket queue | Private-RPC-Socket: 390626 5 5 | Private-RPC-Socket: 390626 5 7 | Private-RPC-Socket: 390626 5 9 |
List queue | Private-RPC-Socket: 390635 6 10 | Private-RPC-Socket: 390635 12 20 | Private-RPC-Socket: 390635 24 40 |
BMC Remedy AR System Server backend integration configuration settings
Component | Small Environment | Medium Environment | Large Environment |
---|---|---|---|
BMC Remedy AR System servers |
|
|
|
Java settings (arserver.config): Java 1.8 (Using CMS GC) |
|
|
|
Use the following settings on all the three environment sizes:
| |||
Java settings (arserver.config): Java 11 or later (Using G1GC) |
|
|
|
Use the following settings on all the three environment sizes:
| |||
FTS queue (FTS Indexer servers) | Private-RPC-Socket: 390602 3 3 | Private-RPC-Socket: 390602 5 5 | Private-RPC-Socket: 390602 7 7 |
Reconciliation queue | Private-RPC-Socket: 390698 2 4 | Private-RPC-Socket: 390698 2 4 | Private-RPC-Socket: 390698 2 8 |
CMDB RPC Normalization queue | Private-RPC-Socket: 390699 2 4 | Private-RPC-Socket: 390699 2 4 | Private-RPC-Socket: 390699 2 8 |
AR RPC Normalization queue | Private-RPC-Socket: 390681 2 2 | Private-RPC-Socket: 390681 2 2 | Private-RPC-Socket: 390681 2 2 |
List queue | Private-RPC-Socket: 390635 12 20 | Private-RPC-Socket: 390635 12 20 | Private-RPC-Socket: 390635 24 40 |
Fast queue | Private-RPC-Socket: 390620 8 12 | Private-RPC-Socket: 390620 8 12* | Private-RPC-Socket: 390620 16 24 |
Loop back or private socket queue | Private-RPC-Socket: 390626 5 5 | Private-RPC-Socket: 390626 5 7* | Private-RPC-Socket: 390626 5 9 |
CMDB-Inline-Normalization: T | CMDB-Inline-Normalization: T | CMDB-Inline-Normalization: T | |
Alert queue | Private-RPC-Socket: 390601 1 1 | Private-RPC-Socket: 390601 1 1 | Private-RPC-Socket: 390601 1 1 |
Escalation queue | Private-RPC-Socket: 390603 3 3 | Private-RPC-Socket: 390603 6 6 | Private-RPC-Socket: 390603 9 9 |
Approval | Private-RPC-Socket: 390680 2 2 | Private-RPC-Socket: 390680 3 3 | Private-RPC-Socket: 390680 4 4 |
Plugin-ARDBC-Threads: 4 12 | Plugin-ARDBC-Threads: 4 12 | Plugin-ARDBC-Threads: 4 12 | |
Plugin-AREA-Threads: 4 12 | Plugin-AREA-Threads: 4 12 | Plugin-AREA-Threads: 4 12 | |
Plugin-Filter-API-Threads: 4 12 | Plugin-Filter-API-Threads: 4 12 | Plugin-Filter-API-Threads: 4 12 |
* Denotes new default value in 9.1 SP2. If this value is not set, the new default is used.
** The number of backend servers may vary by customer needs. Customers with higher backend server requirements may consider using four or more servers. For information about assigning operations to servers, see article
000142769
.
Important
We recommend that you start with fast threads set to a minimum of 2 times and a maximum of 3 times the number of CPU cores and list threads set to a minimum of 3 times and a maximum of 5 times the number of CPU cores.
BMC Remedy AR System Server common configuration settings for user-facing and backend integration features
The following table lists the common configuration settings for user-facing and backend integration features:
Small/Medium/Large environment | Value |
---|---|
Max-Entries-Per-Query | 2000* |
Next-ID-Block-Size | 100 |
Server-side-table-chunk-size | 1000 |
Allow unqual queries | F* |
Cache mode | 0* |
Submitter mode | 1* |
Authentication chaining mode | 1* |
Server plugin default timeout | 120* |
External authentication return data capabilities | 31* |
RE RPC socket | 390698 |
Debug mode | 131120 |
Max log history | 8 |
Max log file size | 134217728 |
Plugin log level | 1000 |
Version-control object modification log mode | 10 |
Version-control object modification log save definition files | 0 |
* Denotes new default value in 9.1 SP2. If this value is not set, the new default is used.
Comments
Log in or register to comment.