Discovering Amazon Web Services

Amazon Web Services (AWS) is a cloud service provided by Amazon.com that provides you with virtualized computing platforms accessible through the internet. It is divided into a number of regions around the world. You can access and configure all of your services using the AWS Management Console. 

Discovering Amazon Web Services

This section describes the settings and procedures required to discover services running in AWS. It contains the following topics:

• • Discovering Amazon Web Services
  • Time synchronization
  • Services and regulatory domains discovered
  • Prerequisites
  • Creating AWS credentials
    • Create Access key in IAM console
    • Create a cloud credential in BMC Discovery 
    • AWS Roles
    • Discovering EC2 hosts by using AWS Systems Manager
  • Time setting
  • Run a cloud scan
  • Examining results
  • Scanning the hosts
  • Common errors
  • Database discovery
    • BMC Discovery database credential
  • AWS discovery patterns
  • AWS tags discovery
  • Event Driven Discovery with AWS Lambda
  • Related Topics

Time synchronization

Services and regulatory domains discovered

The following regulatory domains can be discovered:

• AWS public cloud.
• AWS GovCloud (US) – Introduced in the March 2018 product content update.

You need to set up separate appropriate credentials for the AWS public cloud and the AWS GovCloud (US).

BMC Discovery enables you to discover your cloud services running in AWS. The following set of AWS services can be discovered with the latest product content update:

• Elastic Cloud Compute (EC2)
• Elastic Block Storage (EBS)
• Elastic File System (EFS)
• Amazon ElastiCache – Introduced in the March 2018 product content update.
• Relational Database Service (RDS) and clusters
• Amazon DynamoDB
• Elastic Load Balancer (ELB) including linkage from load balancer members to Hosts and Software Instances
• Virtual Private Cloud (VPC)
• Elastic Container Service (ECS)
• AWS CloudFormation
• Amazon API Gateway 
• Amazon Simple Storage Service (S3) 
• Amazon Kinesis Data Streams (KDS)
• Amazon Kinesis Data Analytics 
• Amazon Kinesis Data Firehose 
• Amazon Kinesis Video Streams
• Amazon Simple Notification Service (Amazon SNS) 
• Amazon Simple Queue Service (SQS) 
• Amazon MQ 
• Amazon Redshift 
• AWS Step Functions 
• AWS Elastic Beanstalk 
• Amazon Athena 
• Amazon EMR 
• Amazon Glue
• Amazon Glacier

More detailed information on discovery of AWS services is provided in the following Configipedia pages:

• 22.4 enhancements and patches
• Amazon Accounts
• Amazon action pack
• Amazon Amazon API Gateway - Reports & Attributes
• Amazon API Gateway
• Amazon AppSync
• Amazon Athena
• Amazon Backup
• Amazon Certificate Manager
• Amazon Cloud Formation
• Amazon Cloud Kubernetes
• Amazon CloudFront
• Amazon CloudSearch
• Amazon CloudWatch

Prerequisites

For correct scanning of AWS services, we strongly recommend that you follow this process:

1. Create an AWS user.
2. Configure Discovery credentials.
3. Configure Roles in AWS Console and Discovery. Note, that role creation is an optional step. The information on how to create Roles in AWS and Discovery is available in the AWS roles paragraph later in this manual.

When all required configuration is complete, you can use BMC Discovery to scan your AWS environment.

Creating AWS credentials

Before you start performing discovery on AWS, you should provide an access key (credential) with help of which BMC Discovery can access the AWS cloud. It is available to create an access key using the AWS Identity and Access Management (IAM) console. Then, you can add the cloud discovery credential using the access key create in the IAM console to BMC Discovery. 

Create Access key in IAM console

To create an Access key in the IAM console that are used to make secure queries to the AWS APIs, do the the following steps:

1. Navigate to the IAM console. 

   Warning

   Note

   Amazon Web Services recommend not to call GetSessionToken with AWS account root user credentials. Instead, follow AWS best practices by creating one or more IAM users, giving them the necessary permissions, and using IAM users for everyday interaction with AWS.

   For more information about using API, see GetSessionToken.

2. Grant the discovery user the "ReadOnlyAccess" permission (this simplifies and replaces the previously documented individual permissions.)

   Warning

   Note

   Default "ReadOnlyAccess" permission does not provide access to Lake Formation Service. To access the Lake Formation Service, grant a user an additional inline policy for Lake Formation with a List and Read access level for all resources.

3. From the discovery user account, create an Access key. For the detailed information about the Access key, see Managing access keys for IAM users.

4. Copy the Access key ID and the Secret access key. You can also download the Access Key ID and Access Secret Key as a csv. file and then import them during creation of a cloud credential in BMC Discovery.

   Warning

   Note

   If you lose a Secret access key, you cannot retrieve it from the IAM console. In such a case, you must create a new Access key and use it in the BMC Discovery cloud credential. It would be best to keep a note of the Secret access key until you have successfully tested the cloud credential.

Create a cloud credential in BMC Discovery 

The cloud credential uses the Access keys/IDs/passwords as the equivalent of a username and password combination. 

Create the cloud credential in the same way as any other credential:

1. On the BMC Discovery Device Credentials page, click Add and select Cloud Provider from the drop-down list.
   The Add Credential page is displayed.
2. Click the "+" icon next to Credential Types to see the available Cloud Providers. Select Amazon Web Services from the drop-down list.
3. Add the usual credential information:
   • Label.
   • Description.
4. Add the information in the additional fields for AWS:
   1. Access Key ID
      You can import the csv files downloaded from the IAM console, reducing scope for cut and paste errors when creating AWS credentials in BMC Discovery. To upload a csv file containing the Key ID and Secret, click Upload CSV, select the file, and then click Open.
   2. Secret Access Key
   3. CyberArk–If the CyberArk integration is enabled, do not enter a key ID and secret, insteфd, enter a CyberArk search string in this field to extract a CyberArk credential. An example of a search string is: Object=Cloud Service-AWSAccessKeys-ABCDEFGABC1ABCDE3ABFor more information on the integration, Integrating-with-CyberArk-Enterprise-Password-Vault.

   4. Assume Roles. Use the Amazon Resource Name (ARN) only if you want to a apply role-based authentication for a user, application, or service.

      Warning

      Note

      If you do not specify the ARN, you will discover AWS resources associated with the Access Key ID credentials. 

      For information on roles in IAM, see the AWS roles paragraph later on this page.

   5. To enable role-switching (multiple roles), enter each role as a new-line separated list. 
5. Optionally, specify a proxy to use for access. For that, specify the following:
   • Hostname.
   • Port.
   • Username (only for authenticating proxies).
   • Password (only for authenticating proxies).
6. Click Apply to save the credential.
   
   

AWS Roles

Using AWS roles is optional, but is a recommended best practice. AWS supports multiple roles for a single credential. Scans can use only one credential. If you configure role switching for a credential, that one credential can use roles to discover diverse targets, in a single scan. Without roles, you would need to set up multiple credentials, and use multiple scans to achieve the same coverage.

An AWS role has a set of permissions associated with it to access specific AWS resources or for making AWS service requests. A role is not uniquely identified with one person but is temporarily assumed by any user who needs to use the role permissions for a session. You can specify multiple roles for a single AWS credential.

A role temporarily sets aside the permissions associated with the username and grants access to trusted entities, such as a user, an application, or a service to explore your AWS resources. For example, you might assign a role to a third party that needs to perform an audit of your resources. A user cannot simultaneously exercise user and role permissions granted to them. When a user switches to a role, the user temporarily gives up the permissions associated with their user credentials and only uses the permissions assigned to the role. When the user exits the role, the user permissions are automatically restored.

You may assign any number of AWS roles to a user, but the user can only act as one role when making requests to AWS services. 

The section below contains links to AWS documentation providing detailed information on managing roles.

You create and manage AWS roles in AWS using the IAM console. The following procedure contains links to the relevant AWS documentation.

1. To create a new AWS role, see Creating a role to delegate permissions to an IAM user.
2. To delegate access for other users to your AWS resources, see Creating a role to delegate permissions to an AWS service.
   
   After you create the trust relationship, an IAM user or an application from the trusted account can use the AWS Security Token Service (AWS STS) AssumeRole API operation. This operation provides temporary security credentials that enable access to AWS resources in your account.
   
3. To assume roles, see Assume role.

         When the roles are created and assumed, you should set appropriate IAM policies.

      4. To create an IAM policy, see Creating IAM policies (console).

When all configurations with roles are done, you can assign these roles for role-based discovery of AWS resources, with the help of the Add Credential screen in BMC Discovery Outpost.
You can switch roles for a user, application, or service, depending on the type of discovery required. Role-switching enables you to use multiple roles for a single credential or scan. However, if you do not specify the ARN (Amazon Resource Name), you will discover AWS resources associated with the Access Key ID credentials. 

For information on how to switch the roles, see Switching to a role (console).

Testing the credential

Once you have created the credentials, you should test it to ensure that it works.

For that:

1. From the credentials page, click Devices.
2. Filter the list to show cloud credentials.
3. Click Actions for the AWS cloud credential you added, and then click Test.
4. The default region is US EAST (N. Virginia). All valid AWS public cloud credentials should work with this region. However, you may choose a local region. You need to use separate appropriate credentials for the AWS public cloud and the AWS GovCloud it is AWS GovCloud (US).
5. Click Test.
   The screen below shows a successful test.
   

If the credential test was unsuccessful, click on the "Failure" status to see the details. Ensure that you copied the secret access key correctly. You should also ensure that the appliance time is no further than five minutes of the time AWS is using. See Time settingfor more information.

The BMC Discovery appliance must be able to access AWS using https (port 443).

Discovering EC2 hosts by using AWS Systems Manager

You can also discover EC2 hosts running in AWS by using AWS Systems Manager (SSM). BMC Discovery uses an existing AWS credential to access AWS and SSM. SSM returns the EC2 hosts that can be accessed by using the AWS credential, and BMC Discovery creates implicit scans to discover those hosts. The advantages of using  SSM to discover EC2 hosts are as follows:

• Your entire AWS estate can be discovered by using your existing AWS credentials; no additional credentials to manage.
• Irrespective of how your AWS deployment's network is segmented, the single AWS SSM credential enables you to discover all of it.
• No requirement for ssh configuration and EC2 key pairs.

See Discovering-EC2-hosts-by-using-AWS-Systems-Manager for more information.

Time setting

Run a cloud scan

To perform cloud discovery, from the Discovery Status page, use the Add New Run control:

1. Click Add New Run.
   The Add a New Run dialog is displayed.
   
   
   

2. Update the fields as described in the following table:

   Field name	Details
   Label	Enter a label for the discovery run. Where the discovery run is referred to in the UI, it is this label that is shown.
   Timing	Select Snapshot to run an immediate cloud scan, or select Scheduled and fill in the scheduling information to run a scheduled cloud run.
   Targeting	Select the target for the discovery run. In this case, select Cloud.
   Provider	Specify the type of cloud provider. In this case, select Amazon Web Services. The dialog refreshes with fields appropriate to the provider selected.
   Company	(Optional) If you have CMDB synchronization configured with multi-tenancy, select the Company to which to assign the discovery run.
   Credential	Select the credential to use for the discovery run. The list is populated with valid credentials for the selected provider.
   Regions	Click List of regions to scan for a full list and select regions to scan. AWS also provides service and regulatory domain groups to scan, enabling you to select all regions in that service or domain.
   System Manager Sessions	Select whether to enable use of the AWS Systems Manager for the scan.
   Sessions Per Second	Select the number of AWS sessions permitted each second. The default value is three.
   Active Sessions	Select the number of active AWS sessions permitted each second. The default value is five.
   Session Logging	Choose whether to enable session logging for this scan. Session logging captures raw discovery data that can be used to diagnose discovery and data quality issues. The default is not to capture session logs. You need to capture session logs only when raising a case with Customer Support. This option is not available for Scheduled runs. For information on viewing session logs, see If-you-encounter-a-problem.

3. Click OK to save the cloud scan settings and close the dialog.
   If you have configured a snapshot run, you can see it running immediately in the Currently Processing Runs tab. If you have configured a scheduled run, it is listed in the Scheduled Runs tab.

Examining results

Once you have scanned, you can examine the results. The screen below shows a discovered VM running in AWS.

Scanning the hosts

Perform a normal scan on the hosts running the VMs discovered in the cloud scan. Use the Unscanned Cloud Hosts report on the Cloud Overview dashboard to find these.

Scanning the hosts assumes that the appliance or proxy has network access to hosts running in the cloud, for example, using a VPN.

Common errors

For information about the errors that are common to all actions, see Common Errors.

Database discovery

You can discover all supported databases in AWS. At the time of release of BMC Discovery 11.3, the following are supported:

• MySQL
• Amazon Aurora (MySQL and PostgreSQL)
• MariaDB
• PostgreSQL
• Oracle
• Microsoft SQL Server.

The following information is required to discover databases in AWS:

• Endpoint – you can identify the database endpoint using the RDS Dashboard in the AWS Management Console. The endpoint is of the form:
  test-rds.xyxyxyxy.us-east-1.amazonaws.com:3306
   To scan the endpoint, you must be able to resolve it to an IP address.
• Security groups
  • If the endpoint is publicly accessible, you still must set up a security group with a rule to allow access from the IP address from which BMC Discovery connects.

  • If the database is not publicly accessible, discovery must be running in AWS. You must set up security to allow access from the Virtual Private Cloud (VPC) in which BMC Discovery is running, and be part of a security group with a rule to allow access from the IP address from which BMC Discovery connects.

    Warning

    Note

    In AWS, all security groups prevent access by default, you must enable access ports in a security group before any access is allowed.

  • To summarize, you must configure security groups which enable the BMC Discovery appliance to access the database. This is entirely dependent on the manner in which you have configured your AWS cloud services.
• Incoming connections – you must permit incoming connections with a rule for an IP address or set of IP addresses. For example, to permit access to a MySQL database, from a single IP address, you would add a rule with the following parameters:
  • Type - MySQL/Aurora
  • Protocol - TCP
  • Port Range - 3306
  • Source - 77.168.1.100/32

Then the database can be discovered as any MySQL database in your estate.

BMC Discovery database credential

Information about Database credentials is available here in the Database credentials paragraph.

AWS discovery patterns

The AWS discovery patterns are available on the Manage>Knowledge page. They are located in the Pattern modules list, under Cloud>Amazon Web Services.

AWS tags discovery

Information about tags is available here.

Event Driven Discovery with AWS Lambda

You can also use event driven discovery with AWS using a Lambda function. An example function archive can be downloaded from Manage > Discovery Tools. The archive contains a Python Lambda function which you can upload into AWS Lambda. To use the function, you must provide a Python 3.x runtime, and the handler must be set to lambda_function.process_event.

The Lambda function receives events from AWS and uses the BMC Discovery REST API to create an ExternalEvent node. The ExternalEvent node contains all the details of the AWS event and can be used to trigger a custom pattern. See Using external events for more details.

The Lambda function is configured using environment variables.

Related Topics

Cloud providers

Amazon-Supported-Cloud-Regions