×
$helper.renderConfluenceMacro('{bmc-global-announcement:$space.key}')
 
 
  •  
BMC Discovery content reference Supported Cloud Providers

Discovering Amazon Web Services

Amazon Web Services (AWS) is a cloud service provided by Amazon.com that gives you virtualized computing platforms accessible through the internet. It is divided into several regions around the world. 

Services and regulatory domains discovered

The following regulatory domains can be discovered:

  • AWS public cloud.
  • AWS GovCloud (US)—Introduced in the March 2018 product content update.

You must set up separate appropriate credentials for the AWS public cloud and the AWS GovCloud (US).

BMC Helix Discovery enables you to discover your cloud services running in AWS. The following set of AWS services can be found with the latest product content update:

More detailed information on the discovery of AWS services is provided on the following Configipedia pages:

Before you begin

For accurate scanning of AWS services, we strongly recommend that you follow this process:

  1. Create an AWS user.
  2. Configure Discovery credentials.
  3. Configure Roles in AWS Console and Discovery. Note that role creation is an optional step. The information on creating Roles in AWS and Discovery is available later in this manual in the AWS roles paragraph.

You can use BMC Helix Discovery to scan your AWS environment when all required configuration is complete.

Creating AWS credentials

Before you start making discovery on AWS, you should provide an access key (credential) with the help that can access the AWS cloud. It is available to create an access key using the AWS Identity and Access Management (IAM) console. Then, you can add the cloud discovery credential using the access key created in the IAM console to BMC Helix Discovery. 

Create an Access key in the IAM console

To create an Access key in the IAM console that is used to make secure queries to the AWS APIs, do the following steps:

  1. Navigate to the IAM console. 

    Note

    Amazon Web Services recommend not to call GetSessionToken with AWS account root user credentials. Instead, follow AWS best practices by creating one or more IAM users, giving them the necessary permissions, and using IAM users for everyday interaction with AWS.

    For more information about using API, see GetSessionToken.

  2. Grant the discovery user the ReadOnlyAccess permission (this simplifies and replaces the previously documented individual permissions.)

    Note

    Default ReadOnlyAccess permission does not provide access to Lake Formation Service. To access the Lake Formation Service, grant a user an additional inline policy for Lake Formation with a List and Read access level for all resources.

  3. From the discovery user account, create an Access key. For detailed information about the Access key, see Managing access keys for IAM users.

  4. Copy the Access key ID and the Secret access key. You can also download the Access Key ID and Access Secret Key as a csv. file and then import them while creating a cloud credential in BMC Helix Discovery.

    Note

    If you lose a Secret access key, you cannot retrieve it from the IAM console. In such a case, you must create a new Access key and use it in the BMC Helix Discovery cloud credential. It would be best to note the Secret access key until you successfully test the cloud credential.

Create a cloud credential in BMC Helix Discovery

The cloud credential uses the Access keys/IDs/passwords as the equivalent of a username and password combination. 

Create the cloud credential in the same way as any other credential:

  1. Click Add on the BMC Helix Discovery Device Credentials page and select Cloud Provider from the drop-down list.
    The Add Credential page is displayed.
  2. Click the + icon next to Credential Types to see the available Cloud Providers. Select Amazon Web Services from the drop-down list.
  3. Add the usual credential information:
    • Label.
    • Description.
  4. Add the information in the additional fields for AWS:
    1. Access Key ID
      You can import the csv files downloaded from the IAM console, reducing the scope for cut-and-paste errors when creating AWS credentials in BMC Helix Discovery. To upload a csv file containing the Key ID and Secret, click Upload CSV, select the file, and click Open.
    2. Secret Access Key
    3. CyberArk–If the CyberArk integration is enabled, do not enter a key ID and secret. Instead, enter a CyberArk search string in this field to extract a CyberArk credential. An example of a search string is:
      Object=Cloud Service-AWSAccessKeys-ABCDEFGABC1ABCDE3AB
      For more information on the integration, see Integrating with CyberArk Enterprise Password Vault.

    4. Assume Roles. Use the Amazon Resource Name (ARN) only if you want to apply role-based authentication for a user, application, or service.

      Note

      If you do not specify the ARN, you will discover AWS resources associated with the Access Key ID credentials. 

      For information on roles in IAM, see the AWS roles paragraph later on this page.

    5. To enable role-switching (multiple roles), enter each role as a new-line separated list. 

    6. To add an initial account when role switching, checks the appropriate checkbox.
  5. (Optional) Specify a proxy to use for access. For that, specify the following:
    • Hostname.
    • Port.
    • Username (only for authenticating proxies).
    • Password (only for authenticating proxies).
  6. Maximum Request Retries
    You can set several attempts to retry sending a request. The default number of retries is 3. The maximum value is 10.

  7. TLS Certificate Check
    This option can be disabled if your proxy uses self-signed certificates. 

    If you disable the certificate check, your credentials could be intercepted by a man-in-the-middle attack.

  8. Click Apply to save the credential.


Master Account read only permissions

By adding credentials (or a role) with Master Account (ReadOnly) permissions, enable Discovery to discover and model your Organization and other Amazon Accounts in your organization. For more details, see Amazon Organizations discovery


Master Account read only permissions

With the addition of the ability to use roles and 'wildcard' roles, it became possible to model services within child accounts and the hierarchical structure of an organization. But this still requires two scans. One with user credentials with elevated permissions and one using roles.
To reduce the number of scans, use the new checkbox 'Scan the initial account when role switching.'


Duration of account session

Sessions for AWS account owners are restricted to 3,600 seconds (one hour). If the duration is longer than one hour, the session for AWS account owners defaults to one hour.

AWS Roles

Using AWS roles is optional but is a recommended best practice. AWS supports multiple roles for a single credential. Scans can use only one credential. If you configure role switching for a credential, that one credential can use roles to discover diverse targets in a single scan. Without roles, you would need to set up multiple credentials and use multiple scans to achieve the same coverage.

An AWS role has a set of permissions to access specific AWS resources or make AWS service requests. A role is not uniquely identified with one person but is temporarily assumed by any user who needs to use the role permissions for a session. You can specify multiple roles for a single AWS credential.

A role temporarily sets aside the permissions associated with the username and grants access to trusted entities, such as a user, an application, or a service, to explore your AWS resources. For example, you might assign a role to a third party that needs to perform an audit of your resources. A user cannot simultaneously exercise user and role permissions granted to them. When a user switches to a role, the user temporarily gives up the permissions associated with their user credentials and only uses the permissions assigned to the role. When the user exits the role, the user permissions are automatically restored.

You may assign a user any number of AWS roles, but the user can only act as one role when requesting AWS services. 

The section below contains links to AWS documentation providing detailed information on managing roles.

You create and manage AWS roles in AWS using the IAM console. The following procedure contains links to the relevant AWS documentation.

  1. To create a new AWS role, see Creating a role to delegate permissions to an IAM user.
  2. To delegate access for other users to your AWS resources, see Creating a role to delegate permissions to an AWS service.

    After creating the trust relationship, an IAM user or an application from the trusted account can use the AWS Security Token Service (AWS STS) 'AssumeRole' API    . This operation provides temporary security credentials enabling access to your account's AWS resources.
  3. To assume roles, see Assume role.

         You should set appropriate IAM policies when the roles are created and assumed.

      4. To create an IAM policy, see Creating IAM policies (console).

STS activation status

For some regions, the STS can be inactive, which might cause an error RegionDisabled - HTTP Status Code: 403. "STS is not activated in the requested region for the account being asked to generate credentials. The account administrator must use the IAM console to activate STS in that region". For more information, see Activating and Deactivating AWS STS in an AWS Region.

When all configurations with roles are done, you can assign these roles for the role-based discovery of AWS resources with the help of the Add Credential screen in the BMC Helix Discovery Outpost.
Depending on the type of discovery required, you can switch roles for a user, application, or service. Role-switching enables you to use multiple roles for a single credential or scan. However, if you do not specify the ARN (Amazon Resource Name), you will discover AWS resources associated with the Access Key ID credentials. 

Configuring AWS Roles using Stack and StackSets

Automated configuration and deployment of a templated role are possible using Stack and StackSet. Configuring permissions, preparing roles for hundreds or thousands of accounts, and keeping these roles and permissions up to date can be technically challenging. Below, we presented the most superficial and recommended Amazon method of deploying roles and permissions within the AWS Organization.

Since the manual is extensive, it is attached as a file you can download directly from this page.


CloudFormation template is mentioned in the document above as a separate file for download.

bmc_discovery_Custom_StackSet.yaml

Wildcard roles

Starting from TKU November 2021, we have introduced the wildcard entries for roles in the UI credential page.

New opportunity to use 'wildcard' roles BMC Helix Discovery tries to get a list of accounts and substitute them into the template, having received the resulting list of roles to switch.

Wildcard role:     arn:aws:iam::*:role/BMC_HELIX_Trusted
List of accounts: 123456789012
                           111222333444
                           999888777666
Result role list:  arn:aws:iam::123456789012:role/BMC_HELIX_Trusted
                         arn:aws:iam::111222333444:role/BMC_HELIX_Trusted
                         arn:aws:iam:: 999888777666:role/BMC_HELIX_Trusted

Before you begin:

During discovery, BMC Helix Discovery tries to discover AWS Accounts list using API request: organizations.list_accounts.

Thus, AWS credentials added in BMC Helix Discovery should have access to the accounts list to work correctly.

We assume the use of Master account credentials or an elevated user to launch the request: “organizations. list_accounts”

Combined use:

       1)    BMC Helix Discovery accepts and uses ‘wildcard’ roles with ‘common’ roles.

       2)    BMC Helix Discovery accepts more than 1 ‘wildcard’ role.

Example:

Syntax:

The ordinary role looks like this:

arn:aws:iam::<account>:role/<name>

where <account> is the account number like 123456789012 (12 digits)

and <name> is the AWS role name.

The role with Wildcard looks like this:

arn:aws:iam::<*>:role/<name>

The <*> symbol means all the account numbers that can be found. (Using AWS ID and secret key)

For example: arn:aws:iam::*:role/BMC_HELIX_Trusted

Important note

  • Each add ‘Wildcard’ role adds a minimum of <21*accounts> endpoints to the scan, which can significantly slow down the scan due to a large amount of incoming data.
  • BMC Helix Discovery cannot guarantee that the Role works. The administrator is responsible for properly configuring the AWS role switching on the AWS side and the roles themselves.
  • Scanning with "wildcards' is limited to 1 AWS Organization and the scope and rights configured by the administrator on the AWS side (the template and instructions for setting are attached above).


The difference between regular scanning, scanning using roles, and using wildcards is displayed on the images below.


Complete service discovery in AWS organization using roles and initial account scan.

To discover all services of child accounts, Amazon Accounts, and benefits of the master account itself, you can use the new checkbox button in AWS credentials. 

Therefore, you can discover the role and primary account services with a single scan. 

In the Endpoints list, you will see 'ordinary' roles endpoints and 'No role switching' endpoints. These endpoints will present the data and services of the initial account used in the scan.

Testing the credential 

Time synchronization is essential

You must ensure that your appliance time is synchronized using NTP. If you do not use NTP, you must ensure that the time is no further than five minutes from when AWS is used.
AWS uses timestamped authentication, and any discrepancy will result in authentication failures.

Once you have created the credentials, you should test them to ensure they work.

For that, do the following steps:

  1. From the credentials page, click Devices.

  2. Filter the list to show cloud credentials.
  3. Click Actions for the AWS cloud credential you added, and then click Test.
  4. The default region is US EAST (N. Virginia). All valid AWS public cloud credentials should work with this region. However, you may choose a local region. It would help if you used separate appropriate credentials for the AWS public cloud and the AWS GovCloud it is AWS GovCloud (US).
  5. Click Test.
    The screen below shows a successful test.

If the credential test was unsuccessful, click on the Failure status to see the details. Ensure that you copy the secret access key correctly. You should also ensure that the appliance time is no longer than five minutes of the time AWS uses. For more information, see Time setting.

The BMC Helix Discovery appliance must be able to access AWS using HTTPS (port 443).

Testing credentials with roles and 'Wildcards'

In the case of using roles, 'wildcards' or a 'scan of the initial account', the testing logic will be as follows:

  1. ‘Wildcard’ is used, and the checkbox 'scan of the initial account' is not checked:
    1. First, non Wildcard roles will be tested.
    2. If no, the first created role will be tested (from the roles list we extract using a wildcard).
  2. ‘Wildcard’ is used, and the checkbox is checked:
    1. ‘Initial’ account id/secret will be used for testing.
  3. ‘Wildcard’ is not used, the role list exists, and the checkbox is checked:
    1. ‘Initial’ account id/secret will be used for testing.
  4. ‘Wildcard’ is not used, the role list exists, and the checkbox is not checked:
    1. First, non Wildcard roles will be tested.

Discovering EC2 hosts by using AWS Systems Manager

You can also discover EC2 hosts running in AWS using AWS Systems Manager (SSM). BMC Helix Discovery uses an existing AWS credential to access AWS and SSM. SSM returns the EC2 hosts that can be accessed by using the AWS credential, and BMC Helix Discovery creates implicit scans to discover those hosts. The advantages of using  SSM to discover EC2 hosts are as follows:

  • Your entire AWS estate can be discovered by using your existing AWS credentials; no additional credentials to manage.
  • Irrespective of how your AWS deployment's network is segmented, the single AWS SSM credential enables you to discover all of it.
  • No requirement for ssh configuration and EC2 key pairs.

For more information, see Discovering EC2 hosts by using AWS Systems Manager.

Time setting

Time synchronization is essential

You must ensure that your appliance time is synchronized using NTP. If you do not use NTP, you must ensure that the time is no further than five minutes from when AWS is used.
AWS uses timestamped authentication, and any discrepancy will result in authentication failures.

Run a cloud scan

To perform cloud discovery from the Discovery Status page, use the Add New Run control:

  1. Click Add New Run.
    The Add a New Run dialog is displayed.


  2. Update the fields as described in the following table:

    Field nameDetails
    LabelEnter a label for the discovery run. This label is shown where the discovery run is referred to in the UI.
    TimingSelect Snapshot to run an immediate cloud scan, or select Scheduled and fill in the scheduling information to run a scheduled cloud run.
    TargetingSelect the target for the discovery run. In this case, select Cloud.
    ProviderSpecify the type of cloud provider. In this case, select Amazon Web Services. The dialog refreshes with fields appropriate to the provider selected.
    Company(Optional) If you have CMDB synchronization configured with multi-tenancy, select the Company to which to assign the discovery run.
    CredentialSelect the credential to use for the discovery run. The list is populated with valid credentials for the selected provider.
    RegionsClick List of regions to scan for a complete list and select regions to scan. AWS also provides service and regulatory domain groups to scan, enabling you to select all regions in that service or domain.
    System Manager SessionsSelect whether to enable the use of the AWS Systems Manager for the scan.
    Sessions Per SecondSelect the number of AWS sessions permitted each second. The default value is three.
    Active SessionsSelect the number of active AWS sessions permitted each second. The default value is five.
    Session LoggingChoose whether to enable session logging for this scan. Session logging captures raw discovery data that can be used to diagnose discovery and data quality issues. The default is not to capture session logs.
    You only need to capture session logs when raising a customer support case.     This option is not available for Scheduled runs. For information on viewing session logs, see If you encounter a problem.
  3. Click OK to save the cloud scan settings and close the dialog.
    If you have configured a snapshot run, you can see it running immediately in the Currently Processing Runs tab. If you have configured a scheduled run, it is listed in the Scheduled Runs tab.

Examining results

Once you have scanned, you can examine the results. The screen below shows a discovered VM running in AWS.

Scanning the hosts

Perform a regular scan on the hosts running the VMs discovered in the cloud scan. Use the Unscanned Cloud Hosts report on the Cloud Overview dashboard to find these.

Scanning the hosts assumes that the appliance or proxy has network access to hosts running in the cloud, for example, using a VPN.

Note

Public IP addresses do not respond to ICMP pings. It would help if you disabled Ping before scanning, otherwise, all scans are dropped, reporting no response.

Common errors

For information about the errors that are common to all actions, see Common Errors.

Database discovery

You can discover all supported databases in AWS, for example: 

  • MySQL
  • Amazon Aurora (MySQL and PostgreSQL)
  • MariaDB
  • PostgreSQL
  • Oracle
  • Microsoft SQL Server.

The following information is required to discover databases in AWS:

  • Endpoint – you can identify the database endpoint using the RDS Dashboard in the AWS Management Console. The endpoint is of the form:
    test-rds.xyxyxyxy.us-east-1.amazonaws.com:3306
    To scan the endpoint, you must be able to resolve it to an IP address.
  • Security groups
    • If the endpoint is publicly accessible, you must still set up a security group with a rule to allow access from the IP address BMC Helix Discovery connects.

    • If the database is not publicly accessible, discovery must be running in AWS. You must set up security to allow access from the Virtual Private Cloud (VPC) in which BMC Helix Discovery is running and be part of a security group with a rule to allow access from the IP address BMC Helix Discovery connects to.

      Note

      In AWS, all security groups prevent access by default, you must enable access ports in a security group before any access is allowed.

    • To summarize, you must configure security groups that enable the BMC Helix Discovery appliance to access the database. This depends on how you have configured your AWS cloud services. 

  • Incoming connections – you must permit incoming connections with a rule for an IP address or set of IP addresses. For example, to permit access to a MySQL database from a single IP address, you would add a rule with the following parameters:
    • Type - MySQL/Aurora
    • Protocol - TCP
    • Port Range - 3306
    • Source - 77.168.1.100/32

Then the database can be discovered as any MySQL database in your estate.

BMC Helix Discovery database credential

Note

To discover a Database, appropriate Database credentials must be created.

For more details, see the Database credentials paragraph on the following page: Adding credentials. 

AWS discovery patterns

The AWS discovery patterns are available on the Manage>Knowledge page. They are in the Pattern modules list under Cloud>Amazon Web Services.

AWS tags discovery

For detailed information, see Discovering Cloud Tags

Event-Driven discovery with AWS Lambda

You can also use event-driven discovery with AWS using a Lambda function. An example function archive can be downloaded from Manage>Discovery Tools. The archive contains a Python Lambda function which you can upload into AWS Lambda. To use the function, you must provide a Python 3.x runtime, and the handler must be set to lambda_function.process_event.

The Lambda function receives events from AWS and uses the BMC Helix Discovery REST API to create an ExternalEvent node. The ExternalEvent node contains all the details of the AWS event and can be used to trigger a custom pattern. See Using external events for more details.

The Lambda function is configured using environment variables.

NameRequiredDefaultDetails
BMC_DISCOVERY_INSTANCEYes

The IP address or hostname of the BMC Helix Discovery instance. This must be reachable from the AWS region, an instance running in the same AWS VPC.

BMC_DISCOVERY_TOKENYes
The REST API authentication token. See Authentication and permissions in the REST API for more details.
BMC_DISCOVERY_API_PROTOCOLNoHTTPSThe REST API protocol to use. Defaults to HTTPS, but HTTP can also be used.
BMC_DISCOVERY_EVENT_SOURCENoawsThe value to use for the "source" attribute of the ExternalEvent node
BMC_DISCOVERY_EVENT_TYPENoAWSThe value to use for the "type" attribute of the ExternalEvent node


Information sources



Was this page helpful? Yes No Submitting... Thank you
Last modified by Anna Bondaruk on Dec 14, 2022
pc-2017-sep-1 aws amazon cloud

Comments

Log in or register to comment.

Alibaba Elastic Compute Service
Amazon Accounts
© Copyright 2004 - 2021 BMC Software, Inc.
Legal notices

Powered by Atlassian Confluence and Scroll Viewport

© Copyright 2005-2023 BMC Software, Inc. Use of this site signifies your acceptance of BMC’s Terms of Use . BMC, the BMC logo, and other BMC marks are assets of BMC Software, Inc. These trademarks are registered and may be registered in the U.S. and in other countries.