Which AWS service provides a feature that can be used to proactively monitor and plan for the service quotas of AWS resources?

Quota Monitor for AWS

🚀Solution Landing Page | 🚧Feature request | 🐛Bug Report | 📜Documentation Improvement

Note: For any relevant information outside the scope of this readme, please refer to the solution landing page and implementation guide.

Table of content

• Solution Overview
• Architecture
  • Deployment Scenarios
• Installation
• Customization
  • Setup
  • Unit Test
  • Build
  • Deploy
• Spoke Templates
• SSM Parameter Store
• File Structure
• License
• Operational Metrics

Solution overview

Quota Monitor for AWS is a reference implementation that provides a foundation for monitoring AWS services' quotas usage. Customers can leverage the solution to monitor quotas across services supported by Amazon Trusted Advisor and AWS Service Quotas; in multiple regions and multiple AWS accounts. The solution integrates with Amazon SNS and Slack to notify customers for service quotas approaching thresholds.

Architecture

The architecture can be broken down into different components, few which are installed in the monitoring account and others are installed in the monitored accounts. The monitoring account also known as hub account, collects all usage events from the monitored accounts (spokes) and raises notifications. Additionally, all the usage events are put on DynamoDB table in the hub account, which can be used to view historical trend of resource usage across all accounts.

Deployment scenarios:

The solution follows hub-spoke model and supports different deployment scenarios

• Environments where all AWS accounts are part of your AWS Organization
• Hybrid environments with AWS Organization and independent AWS accounts
• Environments not using AWS Organizations

hub: For the first two scenarios use quota-monitor-hub.template. For environments not using Organizations, use quota-monitor-hub-no-ou.template. Note: Hub template should be deployed in the monitoring account. For the first two scenarios, this account should also be your delegated administrator for StackSets in the organization.

spoke: Spoke templates are automatically deployed by StackSets for targeted Organizational Units. For hybrid environments and environments not using Organizations, deploy the spoke templates individually in the accounts where monitoring is needed. Note: ta-spoke.template should be deployed in us-east-1 ONLY. sq-spoke.template can be deployed in any region.

Installing pre-packaged solution template

• quota-monitor-hub.template
• quota-monitor-hub-no-ou.template
• quota-monitor-ta-spoke.template
• quota-monitor-sq-spoke.template
• quota-monitor-prerequisite.template

Note: hub, hub-no-ou and sq-spoke templates can be deployed in ANY region; prerequisite and ta-spoke template can be deployed in us-east-1 ONLY.

Parameters for hub template

• Deployment Configuration: Choose Organizations or Hybrid based on your use-case
• Notification Configuration: Choose the notifications you want to receive

Note: Deployment Configuration parameter is not available in hub-no-ou template.

Parameters for spoke templates

• EventBridge bus arn: Arn for the EventBridge bus where you want to send usage events

Note: You may leave rest of the parameters to default.

Customization

The steps given below can be followed if you are looking to customize the solution or extend the solution with newer capabilities

Setup

• Javascript Pre-requisite: node=v16.17.0 | npm=8.15.0

Clone the repository and run the following commands to install dependencies

git clone aws-solutions/quota-monitor-for-aws
cd ./quota-monitor-for-aws
npm ci

(optional) Run the following commands to format and lint the project per the project standards

npm run prettier-format
npm run lint

Note: Following steps have been tested under above pre-requisites

Running unit tests for customization

Run unit tests to make sure added customization passes the tests.

cd ./deployment
chmod +x ./run-unit-tests.sh
./run-unit-tests.sh

✅ Ensure all unit tests pass. Review the generated coverage report.

Build

To build your customized distributable run build from the project root

✅ All assets are now built.

Deploy

Run the following command from the root of the project

cd ./source/resources
npm ci

Bootstrap your CDK environment

npm run cdk -- bootstrap --profile <PROFILE_NAME>

npm run cdk -- deploy <STACK_NAME> --profile <PROFILE_NAME>

Note:

• STACK_NAME, substitute the name of the stack that you want to deploy, check cdk app
• PROFILE_NAME, substitute the name of an AWS CLI profile that contains appropriate credentials for deploying in your preferred region

✅ Solution stack is deployed with your customized code.

Independent spoke templates

There are two spoke templates packaged with the solution

• ta-spoke: provisions resources to support Trusted Advisor quota checks
• sq-spoke: provisions resources to support Service Quotas checks

Both spoke templates are independent standalone stacks that can be individually deployed. You can deploy the spoke stack and route usage events and notifications to your preferred destinations. Additionally, in sq-spoke stack you can control which services to monitor, by toggling monitored status of the services in the DynamoDB table ServiceTable. For deploying sq-spoke stack:

npm run cdk -- deploy quota-monitor-sq-spoke --parameters EventBusArn=<BUS_ARN> --profile <PROFILE_NAME>

Note: BUS_ARN, substitute the arn of the EventBridge bus where you want to send usage events

SSM Parameter Store based workflow

The solution provisions /QuotaMonitor/OUs and /QuotaMonitor/Accounts SSM Parameter Store. You can modify the parameters at any point after deployment to update the list of targeted organizational-units and accounts for monitoring.

• /QuotaMonitor/OUs: Once you update the parameter, StackSets takes care of deploying the spoke templates in the targeted OUs

• /QuotaMonitor/Accounts: Once you update the parameter, you need to deploy the spoke templates individually in the targeted accounts

File Structure

The project consists of several microservices, shared utility lambda layer and CDK resources

|-deployment/
  |-run-unit-test.sh              [ run all unit tests ]
  |-templates                     [ solution pre-baked templates ]
|-source/
  |-lambda
    |-services/
      |-cwPoller/                 [ microservice for polling CloudWatch metrics for quotas usage ]
      |-deploymentManager/        [ microservice for managing CloudFormation StackSet deployments ]
      |-helper/                   [ microservice for helper modules ]
      |-preReqManager/            [ microservice for fulfilling pre-requisites in the management account ]
      |-quotaListManager/         [ microservice for managing quota list that supports usage monitoring ]
      |-reporter/                 [ microservice for putting quota usage details on dynamodb ]
      |-slackNotifier/            [ microservice for raising alerts on slack ]
      |-taRefresher/              [ microservice for refreshing trusted advisor checks ]
    |-utilsLayer/                 [ lambda layer with shared modules, like logger, metrics, try/catch wrapper ]
  |-resources                     [ cdk resources to provision infrastructure ]
|-README.md
|-additional_files                [ CHANGELOG, CODE_OF_CONDUCT, LICENSE, NOTICE, sonar-project.properties etc.]

License

See license here

Collection of operational metrics

This solution collects anonymous operational metrics to help AWS improve the quality and features of the solution. For more information, including how to disable this capability, please see the implementation guide.

Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

Licensed under the Apache License Version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at

http://www.apache.org/licenses/LICENSE-2.0

or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions and limitations under the License.