TotalCloud Insights: Protect Your AWS Environment by Managing Access Keys Securely

Introduction

With the average cost of a data breach coming in at $4.45M in 2023, safeguarding sensitive information and maintaining the security of cloud environments is more critical than ever. Instances of compromised access keys, not exclusive to AWS (Amazon Web Services) but prevalent across cloud platforms, underscore the pressing need for robust security measures.

This blog takes a deep dive into an actual case of AWS access key theft, offering insights into the detailed steps taken to detect, respond to, and mitigate the breach. The article then provides best practices to avoid these types of attacks, some data points around average failure rates of AWS IAM-related controls focused on access keys, and showcases the power of Qualys TotalCloud to secure against these types of misconfigurations.

What are AWS access keys?

Access keys are essential credentials used for programmatic interactions with AWS services. They consist of an access key ID and a secret access key, which must be used together to authenticate requests.

Components of Access Keys

The access key ID is a unique identifier, while the secret access key is a password-like string used to sign requests. Losing a secret access key necessitates revoking the key and creating a new one.

Attack Recap: Real-World Case of AWS Access Key Theft

A security alert flagged an AWS user account breach initiated by an unfamiliar Kali Linux user agent – specifically aws-cli/1.22.34 Python/3.9.11 Linux/5.15.0-kali3-amd64 botocore/1.27.84. The subsequent investigation uncovered three denied AWS API calls, including an intriguingGetSendQuota, all originating from the same access key but with different user agents. This incident was quickly escalated, prompting swift remediation actions like credential resets and access key disablement.

Further examination revealed seven compromised AWS IAM accounts/access keys, all of which underwent similar remediation procedures. The unexpected twist—the compromised AWS keys were traced back to a publicly exposed Postman server with stored access key credentials.

The investigation extended to the associated IP address, revealing it was allocated to a hosting provider outside of AWS, Google, or Microsoft. Moreover, its location was atypical for the customer, adding another layer of complexity to the breach.

Potential Services Impacted

Unauthorized access key theft in AWS can have broad-reaching consequences, potentially impacting various AWS services.

• **IAM (Identity and Access Management) -**Compromised access keys can lead to unsanctioned entry into AWS resources and administrative actions, presenting a substantial security risk.
• **EC2 (Elastic Compute Cloud) -**Illegitimate access may result in the launch of harmful instances, data breaches, or service interruptions.
• **S3 (Simple Storage Service) -**Intruders can control S3 buckets, potentially revealing sensitive data or causing disruptions in data storage.
• **RDS (Relational Database Service) -**Malefactors may interfere with or gain access to valuable database information, potentially resulting in data breaches or data loss.
• CloudTrail - Attackers could tamper with or disable CloudTrail, covering their tracks and making incident detection more challenging.
• CloudWatch - Improper access can disrupt monitoring and potentially obscure malicious activities.
• ECS (Elastic Container Service) - Illicit intrusion can result in the manipulation of container services and unapproved deployments.
• EKS (Elastic Kubernetes Service): Attackers may manipulate Kubernetes clusters, leading to unapproved container deployments or disruptions.

In the above case, the attacker’s actions primarily revolved around AWS SES (Simple Email Service) with a focus on the “**GetSendQuota”**action. This kind of action offers the potential for a range of malicious activities.

GetSendQuota

• This call allows users to retrieve email sending limits and usage information.
• However, in the wrong hands, information obtained through this call could potentially be misused by attackers for various purposes, including:
  • Email Abuse – Attackers could use this information to determine how many emails they can send without triggering alarms or rate limits. This could be used for sending phishing emails, spam, or other malicious content.
  • Service Disruption – Attackers could use this information to target an organization's SES service to disrupt legitimate email communication by pushing the service to its limits.
  • Social Engineering - Knowledge of an organization's email sending limits and usage could be used in social engineering attacks or to craft more convincing phishing emails.

The attacker also misused the “**UpdateAccountSendingEnabled”API action.**This is the action that allows users to turn email on or off across the entire Amazon SES account in the current AWS region. This action is used by legitimate users to manage their email-sending capabilities, ensuring compliance with AWS SES limits and controlling email sending for various use cases.

However, if malicious actors gain unauthorized access to an AWS SES account, they can abuse the "UpdateAccountSendingEnabled" API action in several ways:

• Email Flooding - Attackers can enable email sending without any restrictions, leading to the rapid and excessive sending of emails. This could be used for email flooding or spam campaigns, overwhelming recipient inboxes and potentially disrupting email services.
• Phishing Campaigns - Attackers can use the enabled sending capability to launch phishing campaigns, sending fraudulent emails that mimic trusted entities to deceive recipients and steal sensitive information.
• Spam Distribution - Enabling sending can allow attackers to distribute spam emails at scale, promoting malicious products, services, or links to unsuspecting recipients.
• Harassment - Attackers may use the "UpdateAccountSendingEnabled" action to send harassing or offensive content to specific individuals or organizations.
• Resource Abuse - Continuous and excessive email sending can lead to resource abuse, causing AWS SES to incur additional costs and potentially violating AWS terms of service.

Best Practices for Access Key Management

Here are some best practices for managing access keys in cloud environments to prevent this kind of attack on an AWS environment. Note that these best practices also apply for other cloud environments, and to assist in maintaining the security of those environments, the following includes recommendations for Azure and GCP.

• Access Key Rotation - Regularly rotate access keys to limit exposure in case of a breach. AWS recommends key rotation every 90 days. Use the AWS Management Console or AWS CLI to generate new keys and disable or delete old ones. Azure and GCP also support key rotation for security.
• Avoid Hardcoding Credentials - Do not hardcode access keys and client secrets in your application code or scripts. Instead, utilize cloud-specific identity and access management services like IAM (AWS), Microsoft Entra ID (Azure), or Identity and Access Management (IAM) in GCP.
• Multi-Factor Authentication (MFA) - Enable MFA for accessing cloud accounts and managing access keys. This provides an additional layer of security.
• Principle of Least Privilege - Assign permissions according to the principle of least privilege, ensuring that users and services have only the minimum required access to perform their tasks.
• Audit and Monitor - Activate the cloud provider's monitoring and auditing services, such as AWS CloudTrail, Azure Activity Logs, and GCP Audit Logs, to track access credential usage and detect suspicious activities.
• **Secure Credential Storage -**Securely store access keys and client secrets using the respective cloud's key management services, like AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager. Avoid sharing keys via insecure methods like email.
• Limit Key Exposure - Minimize the exposure of access keys. Only provide access keys to those who need them, and never expose them to public repositories or web pages.
• **Monitor Key Usage -**Set up alerts or alarms for unusual access key usage patterns, such as unexpected increases in API calls or access from unique locations.
• **Credential Deactivation -**Promptly deactivate access keys or revoke client secrets that are compromised, or no longer needed, without deleting them. AWS, Azure, and GCP provide options to deactivate and revoke credentials for easy reactivation if required.
• Incident Response Plan - Develop a comprehensive incident response plan that includes specific steps for handling compromised access keys in each cloud provider.

Rates of Misconfigured Access Keys in AWS Environments

To illustrate the current state of access key security, the Qualys research team has conducted an in-depth analysis of the average failure rates of AWS IAM-related controls, focusing mainly on access keys.

While your environment is unique, and you should make your own assessments of which best practices to focus on first, the following graph provides a comprehensive overview of misconfigurations, enabling users to assess and address security vulnerabilities effectively.

Comprehensive Cloud Security with Qualys TotalCloud

To address the above misconfigurations and effectively secure access keys, Qualys TotalCloud offers a robust set of controls designed to implement best practices and advanced preventive measures, fortifying cloud environments like AWS against misconfigurations. As highlighted below, our security best practices cover vital areas such as IAM (Identity and Access Management), public accessibility, logging, and more. These controls are meticulously aligned with industry-wide security standards, ensuring comprehensive protection for your infrastructure. Here are a few more notable examples:

Conclusion

The incident described in this post is a stark reminder of the critical importance of safeguarding access keys and maintaining the security of cloud environments.

By consistently rotating keys, avoiding hardcoding, implementing multi-factor authentication, and following the principle of least privilege, you can significantly enhance the security of your cloud resources. Moreover, robust monitoring, secure key storage, and a well-prepared incident response plan are indispensable elements of a comprehensive security strategy. A solution like Qualys TotalCloud is a comprehensive way to implement these types of controls.

In today's threat landscape, these best practices are critical for maintaining the highest level of protection for your cloud infrastructure.

Start your Qualys TotalCloud experience to fortify your AWS environment against misconfigurations and take advantage of the best practices already programmed into the solution that will notify you when you need to take action to secure your access keys.

Contributors

• Rahul Pareek, Security QA Engineer, Cloud, Qualys