Benchmark XML files

This topic contains the following sections:

Overview

Benchmarks are used by the Benchmark Alignment system task to gather ratings for systems. Use XML files that adhere to the attached XSD schema when adding or updating benchmarks. 

Benchmark definition

The benchmark definition section of the XML file contains definitions of new benchmarks. To add custom benchmark definitions or redefine existing definitions, use the Update Benchmarks page. For information, see Updating-benchmark-data.

When you upload new definitions, it overwrites any previously uploaded data. Therefore, maintain all the custom definitions and upload them again whenever changes are done.

The following table lists the attributes of the CPU_MODEL node that is available in the HARDWARE node.

The CPU_MODEL node must contain the DESCRIPTION node, which contains the string that matches the HW_MODEL of the system or CPU_MODEL value. For example,

<DESCRIPTION>Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz</DESCRIPTION>

The following table lists the attributes of the PERFORMANCE-RATING node which should reside under the CPU_MODEL node.


Note

Benchmarks with the same key are considered identical and only the latest result is maintained in the database. The previous results are overwritten.

Available benchmark rating types

The following table lists the benchmark rating types that are delivered in BMC Helix Continuous Optimization.

  • SPECINT95
  • SPECINT2000
  • MIPS
  • SPECINTRATE95
  • SPECINTRATE2000
  • SPECINT2006
  • SPECINTRATE2006
  • RPERF
  • SPECINT2017
  • SPECINTRATE2017

You can also add a new benchmark rating type. 

Adding a custom rating type

Add a PERF-RATING-TYPES section within the HARDWARE element in the upload file and add a new PERF-RATING-TYPE. For example:
<PERF-RATING-TYPES>
<PERF-RATING-TYPE Name="NEW-RATING-TYPE" />
</PERF-RATING-TYPES>
Then you can add new ratings for this rating type in your PERFORMANCE-RATING entries.

Custom benchmark example

The following XML example shows the custom defined CPU Model entry with a newly defined vendor:

<HARDWARE xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xsi:noNamespaceSchemaLocation="user-hardware-schema.xsd">
<!--user hardware file-->

<!-- if you submit this file through the 'Update benchmark data' page - it will overwrite any previous file (thus any added ratings) added through this page.  So if you wish to add hardware - update a single file and always submit that file -->  

<!-- if we are adding a new vendor - we can add it here - otherwise we can leave vendors section out -->
<VENDORS>
<VENDOR Name="NEW-VENDOR-NAME" />
</VENDORS>

<!-- list of benchmark cpu models to add-->
<CPU-MODELS>
<!-- the id here must be unique across all cpu-models - you'll be safe by prefacing it with USER_ and then whatever you wish to make it unique-->
<CPU-MODEL ID="USER_ID1" Vendor="NEW-VENDOR-NAME" Category="System" Min-Processors="1" Max-Processors="32">

<!-- this description will be used to map the system to this rating during benchmark alignment task run -->
<!-- needs to be exact match of hrw model that is shown in system's HW_MODEL or CPU_MODEL  config metric -->
<DESCRIPTION>Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz</DESCRIPTION>

<!-- can add as many performance ratings here as are valid for this system -->
<PERFORMANCE-RATING RatingType="SPECINTRATE2017" Processors="2" Rating="4350" OS="NT" />
<PERFORMANCE-RATING RatingType="SPECINTRATE2017" Processors="2" Rating="4330" OS="UNIX" />
<PERFORMANCE-RATING RatingType="SPECINTRATE2017" Processors="24" Rating="69350" OS="NT" />
<PERFORMANCE-RATING RatingType="SPECINTRATE2017" Processors="24" Rating="69280" OS="UNIX" />
<!-- e.g. <PERFORMANCE-RATING RatingType="SPECINTRATE2006" Processors="32" Rating="92373" OS="UNIX" /> -->
<!-- note that "NT" is OS key for all Windows types and "UNIX" is OS key for all unix variants -->
<!-- look into user_hardware_schema.xsd for list of currently added rating types (custom types may -->
<!-- be added as well)-->
<!-- comments are for your clarity only - not seen in UI.  Can add as many comment lines as you want -->
<COMMENT>Source: Customer added rating</COMMENT>
</CPU-MODEL>
<!-- add any other cpu models below -->
<!--
<CPU-MODEL ID="USER_ID2" Vendor="HPE" Category="System" Min-Processors="1" Max-Processors="32">
<DESCRIPTION>systemname</DESCRIPTION>
<PERFORMANCE-RATING RatingType="SPECINTRATE2017" Processors="24" Rating="69280" OS="UNIX" />
<COMMENT>Source: Customer added rating</COMMENT>
</CPU-MODEL> -->
</CPU-MODELS>
</HARDWARE>


 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*