VMware ESXi 4.0 and 4.1 and ESX 4.0 and 4.1 allow remote attackers to cause a denial of service (socket exhaustion) via unspecified network traffic. This issue has been defined as CVE-2011-1785.
BMC Atrium Discovery identifies Hosts which are vulnerable to this issue and avoids port scanning them in future Discovery runs. Full data is still collected. This avoids the socket exhaustion issue being triggered by BMC Atrium Discovery, though you should note that the Hosts remain vulnerable.
How the problem can be triggered
The socket exhaustion issue is caused by a defect in the VMware software which fails to handle TCP disconnection correctly. When a connection is made to the VMware Authentication Daemon (port 902), the daemon sends some banner text. If the connecting application does not read the text and then disconnects, the daemon fails to release the socket, eventually leading to exhaustion.
The BMC Atrium Discovery port scan uses a standard TCP
connect() system call to determine if the Discovery target has the port open. However, for performance reasons, BMC Atrium Discovery does NOT read any data sent by the remote system, it simply disconnects. This could trigger the socket exhaustion issue.
By default, every version of BMC Atrium Discovery up to and including 8.2 (and its maintenance releases) do NOT scan port 902 so those versions cannot trigger the socket exhaustion issue.
How BMC Atrium Discovery 8.3 avoids the problem
On the first scan of any endpoint IP address, every version of BMC Atrium Discovery performs a "port scan" of common access ports to determine how the device may be accessed. From BMC Atrium Discovery 8.3 onwards, this includes port 902 (the VMware Authentication Daemon) to identify potential ESX/ESXi targets. When the port is seen to be open, BMC Atrium Discovery performs a vSphere API query to obtain the ESX/ESXi version information. This is done without credentials so always succeeds for any ESX/ESXi 3.5 or later system. The version information allows BMC Atrium Discovery to identify vulnerable systems which it flags. The discovery scan continues as normal, looking for working vSphere or ssh credentials, which are also recorded.
In summary, on the first scan of an endpoint IP address, BMC Atrium Discovery 8.3 will scan port 902 ONCE.
On a subsequent scan of the endpoint IP address, Discovery refers to the information it previously gathered. If previous credentials (vSphere or ssh) still work, they are used and no port scan is performed. If the previous credential information is not present, or no longer works, BMC Atrium Discovery performs a port scan. However, if the endpoint was previously flagged (safe or not) port 902 is EXCLUDED from the port scan and simply considered as available. The scan then continues as the first scan case using a credential-less query. If this query succeeds, BMC Atrium Discovery updates the vulnerability flag. If the query fails, BMC Atrium Discovery assumes that the endpoint is no longer an ESX system.
In summary, on a subsequent scan of an ESX host, BMC Atrium Discovery 8.3 will NOT scan port 902.
If BMC Atrium Discovery has scanned an ESX host but it is not found on a subsequent scan (that is, the host is down or unreachable) it is recorded as "No Such Device" as usual. If the device then comes back onto the network, it is treated as a First Scan (described above).
Administrators can resolve this issue by installing one of the patches supplied by VMware (see links below)