Introduction

In this tutorial, you will create an Amazon DocumentDB cluster. Operations on the cluster using CLI commands using CLI commands. For more information about Amazon DocumentDB, see Amazon DocumentDB Developer Guide.

Prerequisites

Before starting, you should have the following prerequisites configured

• An AWS account
• AWS CLI on your computer

Amazon DocumentDB tutorial

• Create an Amazon DocumentDB cluster using AWS CLI
• Adding an Amazon DocumentDB instance to a cluster using AWS CLI
• Describing Clusters and Instances using AWS CLI
• Install the mongo shell on MacOS
• Connecting to Amazon DocumentDB
• Performing Amazon DocumentDB CRUD operations using Mongo Shell
• Performing Amazon DocumentDB CRUD operations using python
• Adding a Replica to an Amazon DocumentDB Cluster using AWS CLI
• Amazon DocumentDB High Availability Failover using AWS CLI
• Creating an Amazon DocumentDB global cluster using AWS CLI
• Delete an Instance from a Cluster using AWS CLI
• Delete an Amazon DocumentDB global cluster using AWS CLI
• Removing Global Clusters using AWS CLI

Create an Amazon DocumentDB cluster using AWS CLI

Before you begin, If you have not installed the AWS CLI, see Setting up the Amazon Redshift CLI. This tutorial uses the us-east-1 region.

Now we’re ready to launch a Amazon DocumentDB cluster by using the AWS CLI.

An Amazon DocumentDB cluster consists of instances and a cluster volume that represents the data for the cluster. The cluster volume is replicated six ways across three Availability Zones as a single, virtual volume. The cluster contains a primary instance and, optionally, up to 15 replica instances. 

The following sections show how to create an Amazon DocumentDB cluster using the AWS CLI. You can then add additional replica instances for that cluster.

• When you use the console to create your Amazon DocumentDB cluster, a primary instance is automatically created for you at the same time.
• When you use the AWS CLI to create your Amazon DocumentDB cluster, after the cluster’s status is available, you must then create the primary instance for that cluster.

The following procedures describe how to use the AWS CLI to launch an Amazon DocumentDB cluster and create an Amazon DocumentDB replica.

To create an Amazon DocumentDB cluster, call the create-db-cluster AWS CLI.

aws docdb create-db-cluster \
      --db-cluster-identifier sample-cluster \
      --engine docdb \
      --engine-version 5.0.0 \
      --master-username masteruser \
      --master-user-password masteruser123

The db-subnet-group-name or vpc-security-group-id parameter is not specified, Amazon DocumentDB will use the default subnet group and Amazon VPC security group for the given region.

This command returns the following result.

It takes several minutes to create the cluster. You can use the following AWS CLI to monitor the status of your cluster. 

aws docdb describe-db-clusters \
--filter Name=engine,Values=docdb \
--db-cluster-identifier sample-cluster \
--query 'DBClusters[*].Status'

Adding an Amazon DocumentDB instance to a cluster using AWS CLI

Use the create-db-instance AWS CLI operation with the following parameters to create the primary instance for your cluster.

You can choice instance class from result of following command

aws docdb describe-orderable-db-instance-options --engine docdb --query 'OrderableDBInstanceOptions[*].DBInstanceClass'

aws docdb create-db-instance \
--db-cluster-identifier sample-cluster \
--db-instance-identifier primary-instance \
--db-instance-class db.t3.medium \
--engine docdb

This command returns the following result.

The following AWS CLI command lists the details for Amazon DocumentDB instances in a region.

aws docdb describe-db-instances --db-instance-identifier primary-instance

This command returns the following result.

Describing Clusters and Instances using AWS CLI

To view the details of your Amazon DocumentDB clusters using the AWS CLI, use the describe-db-clusters command. 

The following AWS CLI command list information about the Amazon DocumentDB cluster identification, status, and endpoint.

aws docdb describe-db-clusters --db-cluster-identifier sample-cluster --query 'DBClusters[*].[DBClusterIdentifier,Status,Endpoint]'

This command returns the following result.

Install the mongo shell on MacOS

Install the mongo shell with the following command:

brew tap mongodb/brew
brew install mongosh

To encrypt data in transit, download the public key for Amazon DocumentDB. The following command downloads a file named global-bundle.pem:

cd Downloads
curl -O https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem

You must explicitly grant inbound access to your client in order to connect to the cluster. When you created a cluster in the previous step, because you did not specify a security group, you associated the default cluster security group with the cluster.

The default cluster security group contains no rules to authorize any inbound traffic to the cluster. To access the new cluster, you must add rules for inbound traffic, which are called ingress rules, to the cluster security group. If you are accessing your cluster from the Internet, you will need to authorize a Classless Inter-Domain Routing IP (CIDR/IP) address range.

Run the following command to enable your computer to connect to your Redshift cluster. Then login into your cluster using mongo shell.

#get VpcSecurityGroupId
aws docdb describe-clusters --cluster-identifier sample-cluster --query 'DBClusters[*].[VpcSecurityGroups]'

#allow connect to DocumentDB cluster from my computer
aws ec2 authorize-security-group-ingress --group-id sg-083f2ca0560111a3b --protocol tcp --port 27017 --cidr 111.111.111.111/32

This command returns the following result.

Connecting to Amazon DocumentDB

 Run the following command to connect the Amazon DocumentDB cluster

docdbEndpoint=sample-cluster.cluster-cy1qzrkhqwpp.us-east-1.docdb.amazonaws.com:27017
docdbUser=masteruser
docdbPass=masteruser123
mongosh --tls --host $docdbEndpoint --tlsCAFile global-bundle.pem --username $docdbUser --password $docdbPass

This command returns the following result.

Use the below command to view the available databases in the your Amazon DocumentDB cluster

show dbs

Performing Amazon DocumentDB CRUD operations using Mongo Shell

MongoDB database concepts:

• A record in MongoDB is a document, which is a data structure composed of field and value pairs, similar to JSON objects. The value of a field can include other documents, arrays, and arrays of documents. A document is roughly equivalent to a row in a relational database table.
• A collection in MongoDB is a group of documents, and is roughly equivalent to a relational database table.
• A database in MongoDB is a group of collections, and is similar to a relational database with a group of related tables.

To show current database name

db

To create a database in Amazon DocumentDB, execute the use command, specifying a database name. Create a new database called docdbdemo.

use docdbdemo

When you create a new database in Amazon DocumentDB, there are no collections created for you. You can see this on your cluster by running the following command.

show collections

Creating Documents

You will now insert a document to a new collection called products in your docdbdemo database using the below query.

db.products.insert({
  "name":"java cookbook",
  "sku":"222222",
  "description":"Problems and Solutions for Java Developers",
  "price":200
})

You should see output that looks like this

You can insert multiple documents in a single batch to bulk load products. Use the insertMany command below. 

db.products.insertMany([
{
  "name":"Python3 boto",
  "sku":"222223",
  "description":"basic boto3 and python for everyone",
  "price":100
},
{
  "name":"C# Programmer's Handbook",
  "sku":"222224",
  "description":"complete coverage of features of C#",
  "price":100
}
])

Reading Documents

Use the below query to read data inserted to Amazon DocumentDB. The find command takes a filter criteria and returns the document matching the criteria. The pretty command is appended to display the results in an easy-to-read format.

db.products.find({"sku":"222223"}).pretty()

The matched document is returned as the output of the above query.

Use the find() command to return all the documents in the profiles collection. Input the following:

db.products.find().pretty()

Updating Documents

You will now update a document to add reviews using the $set operator with the update command. Reviews is a new array containing review and rating fields.

db.products.update(
  {
    "sku":"222223"
  },
  {
    $set:{    
      "reviews":[
        {
          "rating":4,
          "review":"perfect book"
        },
        {
          "rating":4.5,
          "review":"very good"
        },
        {
          "rating":5,
          "review":"Just love it"
        }
      ]
    }
  }
)

The output indicates the number of documents that were matched, upserted, and modified.

You can read the document modified above to ensure that the changes are applied.

db.products.find({"sku":"222223"}).pretty()

Deleting Documents

You can delete a document using the below code.

db.products.remove({"sku":"222224"})

Performing Amazon DocumentDB CRUD operations using python

Prerequisites

Before starting, you should have the following prerequisites configured

• Install pymongo
• Download source code sample python CRUD

To install pymongo, execute following command on MacOS

pip3 install pymongo

Edit sample_python_documentdb.py

Open sample_python_documentdb.py, edit variable with current DocumentDB value

username = "masteruser"
password = "masteruser@123"
clusterendpoint = "your_endpoint:27017"
tlsCAFile = "global-bundle.pem"

Execute python file

Execute the code and examine the output

python3 sample_python_documentdb.py

Adding a Replica to an Amazon DocumentDB Cluster using AWS CLI

To add an instance to your Amazon DocumentDB cluster, run the following command

aws docdb create-db-instance \
       --db-cluster-identifier sample-cluster \
       --db-instance-identifier instance-2 \
       --availability-zone us-east-1b \
       --promotion-tier 1 \
       --db-instance-class db.t3.medium \
       --engine docdb

The following example returns the DBClusterIdentifier, DBInstanceIdentifier  for sample-cluster

aws docdb describe-db-clusters \
    --db-cluster-identifier sample-cluster \
    --query 'DBClusters[*].[DBClusterIdentifier,DBClusterMembers[*].DBInstanceIdentifier]'

Amazon DocumentDB High Availability Failover using AWS CLI

A failover for a cluster promotes one of the Amazon DocumentDB replicas (read-only instances) in the cluster to be the primary instance (the cluster writer).When the primary instance fails, Amazon DocumentDB automatically fails over to an Amazon DocumentDB replica

The following operation forces a failover of the sample-cluster cluster. 

aws docdb failover-db-cluster --db-cluster-identifier sample-cluster

Creating an Amazon DocumentDB global cluster using AWS CLI

To create an Amazon DocumentDB regional cluster, call the create-db-clusterAWS CLI. The following AWS CLI command creates an Amazon DocumentDB cluster named global-cluster-id

aws docdb create-db-cluster \
--global-cluster-identifier global-cluster-id \
--source-db-cluster-identifier arn:aws:rds:us-east-1:111122223333:cluster-id

Output from this operation looks something like the following

A global cluster needs at least one secondary cluster in a different region than the primary cluster, and you can add up to five secondary clusters. 

I’m not adding a secondary cluster in this tutorial, but to add an AWS Region to an Amazon DocumentDB global cluster you can use:

aws docdb --region us-east-2 \
  create-db-cluster \
    --db-cluster-identifier cluster-id \
    --global-cluster-identifier global-cluster-id \
    --engine-version version

aws docdb --region us-east-2 \
  create-db-instance \
    --db-cluster-identifier cluster-id \
    --global-cluster-identifier global-cluster-id \
    --engine-version version \
    --engine docdb
      

Delete an Instance from a Cluster using AWS CLI

The following procedure deletes an Amazon DocumentDB instance using the AWS CLI.

aws docdb delete-db-instance --db-instance-identifier instance-2

Output from this operation looks something like the following.

Removing Global Clusters using AWS CLI

You can’t delete the global cluster until after you detach all associated clusters, leaving the primary for last. 

To remove a cluster from a global cluster, run the remove-from-global-clusterCLI command with the following parameters:

• --global-cluster-identifier — The name (identifier) of your global cluster.
• --db-cluster-identifier — The name of each cluster to remove from the global cluster.

Example:

aws docdb --region secondary_region \
  remove-from-global-cluster \
    --db-cluster-identifier secondary_cluster_ARN \
    --global-cluster-identifier global_cluster_id

aws docdb --region primary_region \
  remove-from-global-cluster \
    --db-cluster-identifier primary_cluster_ARN \
    --global-cluster-identifier global_cluster_id

In my case

aws docdb remove-from-global-cluster --db-cluster-identifier arn:aws:rds:us-east-1:111122223333:cluster-id --global-cluster-identifier global_cluster_id

To delete a global cluster, run the delete-global-cluster CLI command with the name of the AWS Region and the global cluster identifier, as shown in the following example.

aws docdb –region us-eas-1 delete-global-cluster \
–global-cluster-identifier global_cluster_id

If you also want to delete cluster, run the following command.

#list instance
aws docdb describe-db-clusters \
--db-cluster-identifier sample-cluster \
--query 'DBClusters[].[DBClusterIdentifier,DBClusterMembers[].DBInstanceIdentifier]'

#delete instance
aws docdb delete-db-instance \
    --db-instance-identifier sample-instance

#delete cluster
aws docdb delete-db-cluster \
    --db-cluster-identifier sample-cluster \
    -skip-final-snapshot

Conclusion

These steps provide an example to manage Amazon DocumentDB cluster. The specific configuration details may vary depending on your environment and setup. It’s recommended to consult the relevant documentation from AWS for detailed instructions on setting up. I hope will this your helpful. Thank you for reading the DevopsRoles page!

Refer

https://docs.aws.amazon.com/documentdb/latest/developerguide/what-is.html

https://catalog.us-east-1.prod.workshops.aws/workshops/464d6c17-9faa-4fef-ac9f-dd49610174d3/en-US

https://docs.aws.amazon.com/lambda/latest/dg/with-documentdb-tutorial.html#docdb-prerequisites

Introduction

In this tutorial, you will create a Lambda to access ElastiCache cluster. When you create the Lambda function, you provide subnet IDs in your Amazon VPC and a VPC security group to allow the Lambda function to access resources in your VPC. For illustration in this tutorial, the Lambda function generates a UUID, writes it to the cache, and retrieves it from the cache.

Invoke the Lambda function and verify that it accessed the ElastiCache cluster in your VPC.

Prerequisites

Before starting, you should have the following prerequisites configured

• An AWS account
• AWS CLI on your computer
• A Memcached cluster (refer Memcached tutorial to create a Memcached cluster )

Create a Lambda to access ElastiCache in an Amazon VPC

• Create the execution role
• Create an ElastiCache cluster
• Create a deployment package
• Create the Lambda function
• Test the Lambda function
• Clean up

Create the execution role

Create the execution role that gives your function permission to access AWS resources. To create an execution role with the AWS CLI, use the create-role command.

In the following example, you specify the trust policy inline.

aws iam create-role --role-name lambda-vpc-role --assume-role-policy-document '{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'

You can also define the trust policy for the role using a JSON file. In the following example, trust-policy.json is a file in the current directory. Example trust-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Add permissions to the role, and use the attach-policy-to-role command. Start by adding the AWSLambdaVPCAccessExecutionRole managed policy.

aws iam attach-role-policy --role-name lambda-vpc-role --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole

Create an ElastiCache cluster

Refer Memcached tutorial to create a Memcached cluster.

The following command retrieves the configuration endpoint (ConfigurationEndpoint)

aws elasticache describe-cache-clusters \
    --cache-cluster-id my-cluster --query 'CacheClusters[].ConfigurationEndpoint'

Create a deployment package

In the following example, create app.py a file in the current directory. Example app.py

from __future__ import print_function
import time
import uuid
import sys
import socket
import elasticache_auto_discovery
from pymemcache.client.hash import HashClient

#elasticache settings
elasticache_config_endpoint = "your-elasticache-cluster-endpoint:port"
nodes = elasticache_auto_discovery.discover(elasticache_config_endpoint)
nodes = map(lambda x: (x[1], int(x[2])), nodes)
memcache_client = HashClient(nodes)

def handler(event, context):
    """
    This function puts into memcache and get from it.
    Memcache is hosted using elasticache
    """

    #Create a random UUID... this will be the sample element we add to the cache.
    uuid_inserted = uuid.uuid4().hex
    #Put the UUID to the cache.
    memcache_client.set('uuid', uuid_inserted)
    #Get item (UUID) from the cache.
    uuid_obtained = memcache_client.get('uuid')
    if uuid_obtained.decode("utf-8") == uuid_inserted:
        # this print should go to the CloudWatch Logs and Lambda console.
        print ("Success: Fetched value %s from memcache" %(uuid_inserted))
    else:
        raise Exception("Value is not the same as we put :(. Expected %s got %s" %(uuid_inserted, uuid_obtained))

    return "Fetched value from memcache: " + uuid_obtained.decode("utf-8")

Dependencies

• elasticache-auto-discovery – The Lambda function uses this library to get the nodes in your Amazon ElastiCache cluster.

• pymemcache – The Lambda function code uses this library to create an HashClientobject to set and get items from memcache.

Create a deployment package.

zip -r function.zip app.py pymemcache/* elasticache_auto_discovery/*

Create the Lambda function

Create the Lambda function with the create-function command.

aws lambda create-function --function-name AccessMemCache --timeout 30 --memory-size 1024 \
--zip-file fileb://function.zip --handler app.handler --runtime python3.8 \
--role arn:aws:iam::123456789012:role/lambda-vpc-role \
--vpc-config SubnetIds=subnet-0a8aaace20a7efd26,subnet-0daa531c4e748062d,subnet-0de820fd0f0efded5,SecurityGroupIds=sg-083f2ca0560111a3b

Test the Lambda function

In this step, you invoke the Lambda function manually using the invoke command. When the Lambda function runs, it generates a UUID and writes it to the ElastiCache cluster specified in your Lambda code. The Lambda function then retrieves the item from the cache.

Invoke the Lambda function with the invoke the command includes getting log stream from CloudWatch

aws lambda invoke --function-name AccessMemCache --cli-binary-format raw-in-base64-out --payload '{"key": "value"}' out

Clean up

Run the following delete-function command to delete the AccessMemCache function.

aws lambda delete-function --function-name AccessMemCache

Run the following command to delete an IAM role

aws iam list-attached-role-policies --role-name lambda-vpc-role
aws iam detach-role-policy --role-name lambda-vpc-role --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
aws iam delete-role --role-name lambda-vpc-role

Conclusion

These steps provide an example to manage the Memcached cluster. The specific configuration details may vary depending on your environment and setup. It’s recommended to consult the relevant documentation from AWS for detailed instructions on setting up. I hope this will your helpful. Thank you for reading the DevopsRoles page!

Refer

https://docs.aws.amazon.com/lambda/latest/dg/services-elasticache-tutorial.html#vpc-ec-deployment-pkg

https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-awscli.html#with-userapp-walkthrough-custom-events-upload

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_manage_delete.html#roles-managingrole-deleting-cli

Introduction

In this Memcached tutorial, you will create an Amazon ElastiCache for the Memcached cluster in your default Amazon Virtual Private Cloud. Operations on the cluster using CLI commands and node management using CLI commands. For more information about Amazon ElastiCache, see Amazon ElastiCache.

Prerequisites

Before starting, you should have the following prerequisites configured

• An AWS account
• AWS CLI on your computer

Memcached tutorial

• Creating a Memcached cluster with AWS CLI
• Modifying a Memcached cluster with AWS CLI
• Viewing the elements in a Memcached cluster with AWS CLI
• Rebooting a Memcached cluster with AWS CLI
• Discovering the endpoints of Memcached cluster with AWS CLI
• Adding nodes to a Memcached cluster with AWS CLI
• Removing nodes from a Memcached cluster with AWS CLI
• Scaling Memcached vertically
• Configuring a Lambda function to access Amazon ElastiCache in an Amazon VPC
• Deleting a Memcached cluster with AWS CLI

Creating a Memcached cluster with AWS CLI

Before you begin, If you have not installed the AWS CLI, see Setting up the Amazon Redshift CLI. This tutorial uses the us-ease-1 region.

Now we’re ready to launch a Memcached cluster by using the AWS CLI.

You can set up a cluster with a specific number of nodes and a parameter group that controls the properties for each node. All nodes within a cluster are designed to be of the same node type and have the same parameter and security group settings. 

Every cluster must have a cluster identifier. The cluster identifier is a customer-supplied name for the cluster. This identifier specifies a particular cluster when interacting with the ElastiCache API and AWS CLI commands. The cluster identifier must be unique for that customer in an AWS Region. For more information, see create-cache-cluster

• Supported –engine-version
• –cache-parameter-group-name: If this argument is omitted, the default parameter group for the specified engine is used. Or you can use create-cache-parameter-group command to create a parameter group.
• If you’re going to launch your cluster in a VPC, make sure to create a subnet group in the same VPC before you start creating a cluster.

The following CLI code creates a Memcached cache cluster with 3 nodes.

#Creating a subnet group
aws elasticache create-cache-subnet-group \
    --cache-subnet-group-name my-subnetgroup \
    --cache-subnet-group-description "Testing" \
    --subnet-ids "subnet-0a8aaace20a7efd26" "subnet-0daa531c4e748062d" "subnet-0de820fd0f0efded5"

#Creating cluster
aws elasticache create-cache-cluster \
--cache-cluster-id my-cluster \
--cache-node-type cache.t2.medium \
--engine memcached \
--engine-version 1.5.16 \
--cache-subnet-group-name my-subnetgroup
--num-cache-nodes 3

This command returns the following result.

Modifying a Memcached cluster with AWS CLI

In addition to adding or removing nodes from a cluster, there can be times when you need to make other changes to an existing cluster, such as, adding a security group, changing the maintenance window, or a parameter group.

You can modify an existing cluster using the AWS CLI modify-cache-cluster operation. To modify a cluster’s configuration value, specify the cluster’s ID, the parameter to change, and the parameter’s new value.

The --apply-immediately parameter applies only to modifications in the engine version and changing the number of nodes in a cluster. If you want to apply any of these changes immediately, use the --apply-immediately parameter. If you prefer postponing these changes to your next maintenance window, use the --no-apply-immediately parameter. Other modifications, such as changing the maintenance window, are applied immediately.

The following example changes the maintenance window for a cluster named my-cluster and applies the change immediately.

aws elasticache modify-cache-cluster \
    --cache-cluster-id my-cluster \
    --preferred-maintenance-window sun:23:00-mon:02:00

This command returns the following result.

Viewing the elements in a Memcached cluster with AWS CLI

You can view detailed information about one or more clusters using describe-cache-clusters

By default, abbreviated information about the clusters is returned. You can use the optional ShowCacheNodeInfo flag to retrieve detailed information about the cache nodes associated with the clusters. These details include the DNS address and port for the cache node endpoint.

The following code lists the details for my-cluster

aws elasticache describe-cache-clusters --cache-cluster-id my-cluster

This command returns the following result.

Rebooting a Memcached cluster with AWS CLI

Some changes require that the cluster be rebooted for the changes to be applied. For example, for some parameters, changing the parameter value in a parameter group is only applied after a reboot.

When you reboot a cluster, the cluster flushes all its data and restarts its engine. During this process, you cannot access the cluster. Because the cluster flushed all its data, when it is available again, you start with an empty cluster.

To reboot specific nodes in the cluster, use the --cache-node-ids-to-reboot to list the specific clusters to reboot.

To reboot a cluster (AWS CLI), use the reboot-cache-cluster CLI operation.

Run the following command to reboot a cluster.

aws elasticache reboot-cache-cluster --cache-cluster-id my-cluster --cache-node-ids-to-reboot "0001"

This command returns the following result.

Discovering the endpoints of the Memcached cluster with AWS CLI

Your application connects to your cluster using endpoints. An endpoint is a node or cluster’s unique address. Which endpoints to use

• If you use Automatic Discovery, you can use the cluster’s configuration endpoint to configure your Memcached client. This means you must use a client that supports Automatic Discovery.
• If you don’t use Automatic Discovery, you must configure your client to use the individual node endpoints for reads and writes. You must also keep track of them as you add and remove nodes.

You can use the AWS CLI to discover the endpoints for a cluster and its nodes with the describe-cache-clusterscommand. For more information, see the topic describe-cache-clusters.

The following command retrieves the configuration endpoint (ConfigurationEndpoint)

aws elasticache describe-cache-clusters \
    --cache-cluster-id my-cluster --query 'CacheClusters[].ConfigurationEndpoint'

This command returns the following result.

For Memcached clusters, the command returns the configuration endpoint. If you include the optional parameter --show-cache-node-info, the following command retrieves the configuration endpoint (ConfigurationEndpoint) and individual node endpoints (Endpoint) for the Memcached cluster.

aws elasticache describe-cache-clusters \
    --cache-cluster-id my-cluster \
    --show-cache-node-info

Adding nodes to a Memcached cluster with AWS CLI

Adding nodes to a Memcached cluster increases the number of your cluster’s partitions.

To add nodes to a cluster using the AWS CLI, use the AWS CLI operation modify-cache-cluster. For more information, see the AWS CLI topic modify-cache-cluster.

Run the following command to add nodes to a cluster

aws elasticache modify-cache-cluster \
    --cache-cluster-id my-cluster \
    --num-cache-nodes 4 \
    --apply-immediately

This command returns the following result.

Removing nodes from a Memcached cluster with AWS CLI

To remove nodes from a cluster using the command-line interface, use the command modify-cache-cluster with the following parameters:

• --cache-cluster-id The ID of the cache cluster that you want to remove nodes from.
• --num-cache-nodes The --num-cache-nodes parameter specifies the number of nodes that you want in this cluster after the modification is applied.
• --cache-node-ids-to-remove A list of node IDs that you want removed from this cluster.
• --apply-immediately or --no-apply-immediately Specifies whether to remove these nodes immediately or at the next maintenance window.
• --region Specifies the AWS Region of the cluster that you want to remove nodes from.

The following example immediately removes node 0004 from the cluster my-cluster.

aws elasticache modify-cache-cluster \
    --cache-cluster-id my-cluster \
    --num-cache-nodes 3 \
    --cache-node-ids-to-remove 0004 \
    --region us-east-1 \
    --apply-immediately  

This command returns the following result.

Scaling Memcached vertically

To scale a Memcached cache cluster vertically

• Create a new cache cluster with the new node type. 
• In your application, update the endpoints to the new cluster’s endpoints.
• Delete the old cache cluster. 

Deleting a Memcached cluster with AWS CLI

For more information, see the AWS CLI for ElastiCache topic delete-cache-cluster

Run the following command to delete a cluster.

aws elasticache delete-cache-cluster --cache-cluster-id my-cluster

This command returns the following result.

Conclusion

These steps provide an example to manage the Memcached cluster. The specific configuration details may vary depending on your environment and setup. It’s recommended to consult the relevant documentation from AWS for detailed instructions on setting up. I hope will this be helpful. Thank you for reading the DevopsRoles page!

Refer

https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/WhatIs.html

https://docs.aws.amazon.com/lambda/latest/dg/services-elasticache-tutorial.html

Introduction

This tutorial shows you how to create a Redshift cluster resource, connect to Amazon Redshift, load sample data from S3 to Redshift into Redshift, and run queries with data usage command line tools.

You can use SQL Workbench or Amazon Redshift Query Editor v2.0 (web-based analyst workbench). In this tutorial, we choose to load sample data from an Amazon S3 bucket to Amazon Redshift using the PLSQL command-line tool.

psql is a terminal-based front-end to PostgreSQL. It enables you to type in queries interactively, issue them to PostgreSQL, and see the query results. Alternatively, input can be from a file or command line arguments. In addition, psql provides several meta-commands and various shell-like features to facilitate writing scripts and automating a wide variety of tasks.

Prerequisites

Before starting, you should have the following prerequisites configured

• An AWS account
• AWS CLI on your computer

Load data from S3 to Redshift into Redshift example with AWS CLI

• Install PSQL on MacOS
• Creating a data warehouse with Amazon Redshift using AWS CLI
• Connect to the Redshift cluster using PSQL
• Create Redshift cluster tables using PSQL
• Redshift default role setting uses AWS Console manage
• Loading sample data from S3 to Redshift with PSQL
• Delete the sample cluster using AWS CLI

Install PSQL on MacOS

We can choose a version from PostgresSQL page or execute the following command on MacOS

brew install postgresql

Creating a data warehouse with Amazon Redshift using AWS CLI

Before you begin, If you have not installed the AWS CLI, see Setting up the Amazon Redshift CLI. This tutorial uses the us-east-1 region.

Now we’re ready to launch a cluster by using the AWS CLI.

The create-cluster the command has a large number of parameters. For this tutorial, you will use the parameter values that are described in the following table. Before you create a cluster in a production environment, we recommend that you review all the required and optional parameters so that your cluster configuration matches your requirements. For more information, see create-cluster

Parameter name	Parameter value for this exercise
cluster-identifier	examplecluster
master-username	awsuser
master-user-password	Awsuser123
node-type	dc2.large
cluster-type	single-node

Run the following command to create a cluster.

aws redshift create-cluster --cluster-identifier examplecluster --master-username awsuser --master-user-password Awsuser123 --node-type dc2.large --cluster-type single-node

This command returns the following result.

The cluster creation process will take several minutes to complete. To check the status, enter the following command.

aws redshift describe-clusters --cluster-identifier examplecluster | grep ClusterStatus

When the ClusterStatus field changes from creating to available, the cluster is ready for use.

Connect to the Redshift cluster using PSQL

Run the following command to connect to the Redshift cluster.

psql -h examplecluster.ccfmryooawwy.us-east-1.redshift.amazonaws.com -U awsuser -d dev -p 5439

You must explicitly grant inbound access to your client to connect to the cluster. When you created a cluster in the previous step, because you did not specify a security group, you associated the default cluster security group with the cluster.

The default cluster security group contains no rules to authorize any inbound traffic to the cluster. To access the new cluster, you must add rules for inbound traffic, which are called ingress rules, to the cluster security group. If you are accessing your cluster from the Internet, you will need to authorize a Classless Inter-Domain Routing IP (CIDR/IP) address range.

#get VpcSecurityGroupId
aws redshift describe-clusters --cluster-identifier examplecluster | grep VpcSecurityGroupId

Run the following command to enable your computer to connect to your Redshift cluster. Then login into your cluster using psql.

#allow connect to cluster from my computer
aws ec2 authorize-security-group-ingress --group-id sg-083f2ca0560111a3b --protocol tcp --port 5439 --cidr 111.111.111.111/32

This command returns the following result.

Now test the connection by querying the system table

Create Redshift cluster tables using PSQL

In this tutorial, I use sample data from AWS. Run the following command to create Redshift tables.

create table users(
userid integer not null distkey sortkey,
username char(8),
firstname varchar(30),
lastname varchar(30),
city varchar(30),
state char(2),
email varchar(100),
phone char(14),
likesports boolean,
liketheatre boolean,
likeconcerts boolean,
likejazz boolean,
likeclassical boolean,
likeopera boolean,
likerock boolean,
likevegas boolean,
likebroadway boolean,
likemusicals boolean);                        

create table event(
eventid integer not null distkey,
venueid smallint not null,
catid smallint not null,
dateid smallint not null sortkey,
eventname varchar(200),
starttime timestamp);

create table sales(
salesid integer not null,
listid integer not null distkey,
sellerid integer not null,
buyerid integer not null,
eventid integer not null,
dateid smallint not null sortkey,
qtysold smallint not null,
pricepaid decimal(8,2),
commission decimal(8,2),
saletime timestamp);

This command returns the following result.

Test by querying the public.sales table as follows

select * from public.sales;

Redshift default role setting uses AWS Console manage

Before you can load data from Amazon S3, you must first create an IAM role with the necessary permissions and attach it to your cluster. To do this refer to AWS document

Loading sample data from S3 to Redshift with PSQL

Use the COPY command to load large datasets from Amazon S3 into Amazon Redshift. For more information about COPY syntax, see COPY in the Amazon Redshift Database Developer Guide.

Run the following SQL commands in PSQL to load data from S3 to Redshift

COPY users 
FROM 's3://redshift-downloads/tickit/allusers_pipe.txt' 
DELIMITER '|' 
TIMEFORMAT 'YYYY-MM-DD HH:MI:SS'
IGNOREHEADER 1 
REGION 'us-east-1'
IAM_ROLE default;                    
                    
COPY event
FROM 's3://redshift-downloads/tickit/allevents_pipe.txt' 
DELIMITER '|' 
TIMEFORMAT 'YYYY-MM-DD HH:MI:SS'
IGNOREHEADER 1 
REGION 'us-east-1'
IAM_ROLE default;

COPY sales
FROM 's3://redshift-downloads/tickit/sales_tab.txt' 
DELIMITER '\t' 
TIMEFORMAT 'MM/DD/YYYY HH:MI:SS'
IGNOREHEADER 1 
REGION 'us-east-1'
IAM_ROLE default;

After loading data, try some example queries. 

\timing

SELECT firstname, lastname, total_quantity 
FROM   (SELECT buyerid, sum(qtysold) total_quantity
        FROM  sales
        GROUP BY buyerid
        ORDER BY total_quantity desc limit 10) Q, users
WHERE Q.buyerid = userid
ORDER BY Q.total_quantity desc;

Now that you’ve loaded data into Redshift.

Delete the sample cluster using AWS CLI

When you delete a cluster, you must decide whether to create a final snapshot. Because this is an exercise and your test cluster should not have any important data in it, you can skip the final snapshot.

To delete your cluster, enter the following command.

aws redshift delete-cluster –cluster-identifier examplecluster –skip-final-cluster-snapshot

Congratulations! You successfully launched, authorized access to, connected to, and terminated a cluster.

Conclusion

These steps provide an example of loading data from S3 to Redshift into Redshift with the PSQL tool. The specific configuration details may vary depending on your environment and setup. It’s recommended to consult the relevant documentation from AWS for detailed instructions on setting up. I hope will this be helpful. Thank you for reading the DevopsRoles page!

Refer

https://docs.aws.amazon.com/redshift/latest/mgmt/getting-started-cli.html#getting-started-create-sample-db-cli

https://docs.aws.amazon.com/redshift/latest/gsg/new-user-serverless.html