PreviewLab7 hours

Search & Aggregations

Build an OpenSearch cluster for property search, mappings, indexing, queries, aggregations, and DynamoDB sync.

This lab uses a Booking.com-style property search domain. Properties (hotels, apartments, villas) are stored in DynamoDB as the source of truth. OpenSearch holds a copy optimised for search: full-text on name and description, geo-distance queries, faceted filters on amenities and property type, price range filters, and aggregations for the search results page.

Mental model

DynamoDB is where bookings are written. OpenSearch is where guests search. They serve different masters. DynamoDB optimises for write throughput and key lookups; OpenSearch optimises for finding the right property from 50,000 options using partial text, filters, and relevance.

Prerequisites

0 of 5 checked

IAM, Cluster Setup, and the Access Model

Goal

OpenSearch has the most complex access model of any service in this course. Three independent layers evaluate every request. Understand them before writing a single API call, all three failures produce identical 403s.

Estimated time: 60 min

The three-layer access model

What's happening here

Every request to OpenSearch passes through three independent access checks:

Domain access policy: a resource-based JSON policy on the OpenSearch domain. Controls which IAM principals are allowed to make HTTP requests to this domain at all. If this denies: 403, no further checks.
IAM policy on the caller: standard IAM policy on the calling identity (Lambda role, your IAM user). Must grant es:ESHttp* on the domain ARN. If this denies: 403, no further checks.
Fine-grained access control (FGAC): OpenSearch’s internal role system. A master user manages internal users, roles, and role mappings. If the IAM role isn’t mapped to an OpenSearch role: 403.

All three 403s look identical from the client. Debugging order: domain policy → IAM policy → FGAC role mapping. For this lab: public endpoint scoped by IAM + FGAC (acceptable for development). Production: VPC deployment with bastion (same pattern as ElastiCache).

bash

export AWS_REGION=ap-south-1
export ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

# IAM role for Lambda (used in Session 5)
cat > /tmp/lambda-trust.json << 'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF

aws iam create-role \
  --role-name opensearch-lab-lambda-role \
  --assume-role-policy-document file:///tmp/lambda-trust.json

aws iam attach-role-policy \
  --role-name opensearch-lab-lambda-role \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

export LAMBDA_ROLE_ARN=$(aws iam get-role \
  --role-name opensearch-lab-lambda-role \
  --query 'Role.Arn' --output text)

echo "Lambda role: $LAMBDA_ROLE_ARN"

Create the OpenSearch domain

What's happening here

Creating an OpenSearch domain takes 10–15 minutes. Key parameters:

engine-version: OpenSearch 2.x is current. Avoid legacy Elasticsearch versions.
cluster-config: t3.small.search is Free Tier eligible. instance-count: 1 for dev.
ebs-options: EBS-backed storage. gp3 with 10GB is enough for this lab.
access-policies: the domain access policy. Allows your IAM user and the Lambda role.
advanced-security-options: enables FGAC with a master user who manages internal users and role mappings.
node-to-node-encryption and encryption-at-rest: always enable. Required when FGAC is enabled.

bash

export MASTER_USER="opensearch-admin"
export MASTER_PASS="OpenSearch@Lab1!"
export MY_USER_ARN=$(aws sts get-caller-identity --query Arn --output text)

echo "Creating OpenSearch domain (takes 10-15 minutes)..."

aws opensearch create-domain \
  --domain-name opensearch-lab \
  --engine-version OpenSearch_2.11 \
  --cluster-config InstanceType=t3.small.search,InstanceCount=1 \
  --ebs-options EBSEnabled=true,VolumeType=gp3,VolumeSize=10 \
  --access-policies "{
    \"Version\": \"2012-10-17\",
    \"Statement\": [{
      \"Effect\": \"Allow\",
      \"Principal\": {\"AWS\": [\"$MY_USER_ARN\", \"$LAMBDA_ROLE_ARN\"]},
      \"Action\": \"es:*\",
      \"Resource\": \"arn:aws:es:$AWS_REGION:$ACCOUNT_ID:domain/opensearch-lab/*\"
    }]
  }" \
  --node-to-node-encryption-options Enabled=true \
  --encryption-at-rest-options Enabled=true \
  --domain-endpoint-options EnforceHTTPS=true \
  --advanced-security-options "Enabled=true,InternalUserDatabaseEnabled=true,MasterUserOptions={MasterUserName=$MASTER_USER,MasterUserPassword=$MASTER_PASS}" \
  --region $AWS_REGION

echo "Polling until active..."
until aws opensearch describe-domain \
  --domain-name opensearch-lab \
  --region $AWS_REGION \
  --query 'DomainStatus.Processing' \
  --output text | grep -q False; do
  echo "Still creating..."
  sleep 30
done

export OS_ENDPOINT=$(aws opensearch describe-domain \
  --domain-name opensearch-lab \
  --region $AWS_REGION \
  --query 'DomainStatus.Endpoints.vpc // DomainStatus.Endpoint' \
  --output text)
export OS_URL="https://$OS_ENDPOINT"
echo "OpenSearch endpoint: $OS_URL"

Checkpoint: curl -s -u "$MASTER_USER:$MASTER_PASS" "$OS_URL" returns JSON with cluster_name: opensearch-lab.

Configure FGAC: map Lambda IAM role to OpenSearch role

bash

# Map Lambda IAM role to all_access OpenSearch role (FGAC layer)
curl -s -X PUT \
  -u "$MASTER_USER:$MASTER_PASS" \
  -H "Content-Type: application/json" \
  "$OS_URL/_plugins/_security/api/rolesmapping/all_access" \
  -d "{\"backend_roles\": [\"$LAMBDA_ROLE_ARN\"]}" | jq

# Verify
curl -s -u "$MASTER_USER:$MASTER_PASS" \
  "$OS_URL/_plugins/_security/api/rolesmapping/all_access" \
  | jq '.all_access.backend_roles'

Observe: the Lambda role ARN is now in backend_roles. Without this, Lambda has correct IAM permissions and the domain policy allows it, but FGAC blocks it with a 403 identical to the other two failure modes.

Break It: IAM, Cluster Setup, and the Access Model

Break 1: All three 403 layers look identical

bash

# Wrong credentials (domain policy + auth failure)
curl -s -u "wrong-user:wrong-pass" "$OS_URL/_cluster/health" -w "\nHTTP: %{http_code}\n"

# Create a read-only internal user (FGAC layer)
curl -s -X PUT \
  -u "$MASTER_USER:$MASTER_PASS" \
  -H "Content-Type: application/json" \
  "$OS_URL/_plugins/_security/api/internalusers/readonly-user" \
  -d '{"password": "ReadOnly@1!", "backend_roles": ["readall"]}' | jq '.status'

# Try to create an index as read-only user (FGAC deny)
curl -s -X PUT \
  -u "readonly-user:ReadOnly@1!" \
  -H "Content-Type: application/json" \
  "$OS_URL/test-index" \
  -d '{"settings": {"number_of_shards": 1}}' \
  -w "\nHTTP: %{http_code}\n"

both wrong credentials and FGAC denial return 403. Debugging order: (1) verify domain access policy allows the caller ARN, (2) verify caller’s IAM policy has es:ESHttp*, (3) check _plugins/_security/api/rolesmapping/ to confirm IAM role is mapped to an OpenSearch role.

Index Design: Mappings and Settings

Goal

Design the properties index for a property search platform. Define explicit mappings for every field type before indexing any data. Understand why this is the single most consequential decision in the lab.