AWS Cross - Account S3 Bucket Policy for Elasticsearch Snapshot

In the AWS ecosystem, Elasticsearch is a powerful open - source search and analytics engine, and Amazon S3 is a highly scalable object storage service. Taking snapshots of Elasticsearch indices is crucial for data backup, migration, and disaster recovery. However, there are scenarios where you might need to store these snapshots in an S3 bucket that belongs to a different AWS account. This is where cross - account S3 bucket policies for Elasticsearch snapshots come into play. This blog post will explore the core concepts, usage scenarios, common practices, and best practices related to this topic, providing software engineers with a comprehensive understanding.

Table of Contents#

  1. Core Concepts
  2. Typical Usage Scenarios
  3. Common Practice
  4. Best Practices
  5. Conclusion
  6. FAQ
  7. References

Article#

Core Concepts#

Amazon S3#

Amazon S3 is an object storage service that offers industry - leading scalability, data availability, security, and performance. It stores data as objects within buckets, where each object consists of a file and optional metadata. Buckets are used to organize and store objects, and they have a globally unique name.

Elasticsearch Snapshots#

Elasticsearch snapshots are a way to back up your indices. You can take a snapshot of one or more indices and store it in a repository. For AWS Elasticsearch Service, S3 is a popular choice for a snapshot repository because of its durability and scalability.

Cross - Account Access#

Cross - account access in AWS allows resources in one AWS account to access resources in another AWS account. This is achieved through the use of IAM (Identity and Access Management) policies, which can be attached to IAM roles or users. In the context of Elasticsearch snapshots, cross - account access means that an Elasticsearch domain in one account can write snapshots to an S3 bucket in another account.

S3 Bucket Policy#

An S3 bucket policy is a JSON - based access policy that you can attach to an S3 bucket. It allows you to control who can access the bucket and what actions they can perform. For cross - account access, the bucket policy is used to grant permissions to the Elasticsearch service in another account to access the bucket.

Typical Usage Scenarios#

Centralized Data Storage#

Many organizations have a central AWS account dedicated to data storage. Elasticsearch domains in different accounts (e.g., development, testing, production) can take snapshots and store them in the central S3 bucket. This simplifies data management and compliance.

Disaster Recovery#

In case of a disaster in one AWS account, having Elasticsearch snapshots stored in an S3 bucket in another account ensures that the data can be restored. This provides an additional layer of protection against data loss.

Data Sharing#

If multiple teams within an organization have their own AWS accounts, they may need to share Elasticsearch data. By allowing cross - account access to the S3 bucket, teams can share snapshots easily.

Common Practice#

Step 1: Create an IAM Role in the Elasticsearch Account#

In the AWS account where the Elasticsearch domain resides, create an IAM role with the necessary permissions to access the S3 bucket in the other account. The role should have permissions to list, put, and delete objects in the S3 bucket.

{
    "Version": "2012 - 10 - 17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::your - target - bucket",
                "arn:aws:s3:::your - target - bucket/*"
            ]
        }
    ]
}

Step 2: Configure the Elasticsearch Domain#

Associate the IAM role created in step 1 with the Elasticsearch domain. In the Elasticsearch domain settings, specify the ARN of the IAM role.

Step 3: Set Up the S3 Bucket Policy in the S3 Account#

In the AWS account where the S3 bucket is located, create a bucket policy that allows the Elasticsearch service in the other account to access the bucket. The policy should include the ARN of the IAM role from the Elasticsearch account.

{
    "Version": "2012 - 10 - 17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::your - elasticsearch - account - id:role/your - iam - role"
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:PutObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::your - target - bucket",
                "arn:aws:s3:::your - target - bucket/*"
            ]
        }
    ]
}

Step 4: Register the S3 Repository in Elasticsearch#

Use the Elasticsearch API to register the S3 bucket as a snapshot repository.

curl -X PUT "https://your - elasticsearch - endpoint/_snapshot/your - repository - name" -H 'Content - Type: application/json' -d'
{
    "type": "s3",
    "settings": {
        "bucket": "your - target - bucket",
        "region": "your - bucket - region"
    }
}
'

Step 5: Take a Snapshot#

Use the Elasticsearch API to take a snapshot of the indices.

curl -X PUT "https://your - elasticsearch - endpoint/_snapshot/your - repository - name/your - snapshot - name"

Best Practices#

Least Privilege Principle#

Only grant the minimum permissions required for the Elasticsearch service to access the S3 bucket. For example, if you only need to take snapshots, don't grant permissions to delete the entire bucket.

Regular Monitoring#

Set up AWS CloudWatch alarms to monitor the snapshot process. This can help you detect any issues early, such as failed snapshot attempts.

Encryption#

Enable server - side encryption for the S3 bucket to protect your data at rest. You can use AWS - managed keys or your own customer - managed keys.

Versioning#

Enable versioning for the S3 bucket. This allows you to keep multiple versions of an object, which can be useful for recovery in case of accidental deletions or overwrites.

Conclusion#

AWS cross - account S3 bucket policies for Elasticsearch snapshots provide a flexible and secure way to manage Elasticsearch data across multiple accounts. By understanding the core concepts, typical usage scenarios, common practices, and best practices, software engineers can effectively implement cross - account snapshot storage. This not only enhances data management but also provides additional protection against data loss.

FAQ#

Q1: Can I use the same IAM role for multiple Elasticsearch domains?#

Yes, you can use the same IAM role for multiple Elasticsearch domains as long as they are in the same account and have the same access requirements.

Q2: What if the S3 bucket policy is misconfigured?#

If the S3 bucket policy is misconfigured, the Elasticsearch service will not be able to access the bucket, and snapshot operations will fail. You should carefully review and test the policy to ensure it is correct.

Q3: Can I take snapshots of only specific indices?#

Yes, when taking a snapshot using the Elasticsearch API, you can specify the indices you want to include in the snapshot.

References#

2025-10