For years, AWS Batch and Amazon EKS (Elastic Kubernetes Service) operated in parallel universes. Batch excelled at queue management and compute provisioning for high-throughput workloads, while Kubernetes won the war for container orchestration. With the introduction of AWS Batch support for EKS, we can finally unify these paradigms.

This convergence allows you to leverage the robust job scheduling of AWS Batch while utilizing the namespace isolation, sidecars, and familiarity of your existing EKS clusters. However, orchestrating this integration via Infrastructure as Code (IaC) is non-trivial. It requires precise IAM trust relationships, Kubernetes RBAC (Role-Based Access Control) configuration, and specific compute environment parameters.

In this guide, we will bypass the GUI entirely. We will architect and deploy a production-ready AWS Batch Terraform EKS solution, focusing on the nuances that trip up even experienced engineers.

GigaCode Pro-Tip:
Unlike standard EC2 compute environments, AWS Batch on EKS does not manage the EC2 instances directly. Instead, it submits Pods to your cluster. This means your EKS Nodes (Node Groups) must already exist and scale appropriately (e.g., using Karpenter or Cluster Autoscaler) to handle the pending Pods injected by Batch.

Architecture: How Batch Talks to Kubernetes

Before writing Terraform, understand the control flow:

1. Job Submission: You submit a job to an AWS Batch Job Queue.
2. Translation: AWS Batch translates the job definition into a Kubernetes PodSpec.
3. API Call: The AWS Batch Service Principal interacts with the EKS Control Plane (API Server) to create the Pod.
4. Execution: The Pod is scheduled on an available node in your EKS cluster.

This flow implies two critical security boundaries we must bridge with Terraform: IAM (AWS permissions) and RBAC (Kubernetes permissions).

Step 1: IAM Roles for Batch Service

AWS Batch needs a specific service-linked role or a custom IAM role to communicate with the EKS cluster. For strict security, we define a custom role.

resource "aws_iam_role" "batch_eks_service_role" {
  name = "aws-batch-eks-service-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "batch.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "batch_eks_policy" {
  role       = aws_iam_role.batch_eks_service_role.name
  policy_arn = "arn:aws:iam::aws:policy/AWSBatchServiceRole"
}

Step 2: Preparing the EKS Cluster (RBAC)

This is the most common failure point for AWS Batch Terraform EKS deployments. Even with the correct IAM role, Batch cannot schedule Pods if the Kubernetes API rejects the request.

We must map the IAM role created in Step 1 to a Kubernetes user, then grant that user permissions via a ClusterRole and ClusterRoleBinding. We can use the HashiCorp Kubernetes Provider for this.

2.1 Define the ClusterRole

resource "kubernetes_cluster_role" "aws_batch_cluster_role" {
  metadata {
    name = "aws-batch-cluster-role"
  }

  rule {
    api_groups = [""]
    resources  = ["namespaces"]
    verbs      = ["get", "list", "watch"]
  }

  rule {
    api_groups = [""]
    resources  = ["nodes"]
    verbs      = ["get", "list", "watch"]
  }

  rule {
    api_groups = [""]
    resources  = ["pods"]
    verbs      = ["get", "list", "watch", "create", "delete", "patch"]
  }

  rule {
    api_groups = ["rbac.authorization.k8s.io"]
    resources  = ["clusterroles", "clusterrolebindings"]
    verbs      = ["get", "list"]
  }
}

2.2 Bind the Role to the IAM User

You must ensure the IAM role ARN matches the user configured in your aws-auth ConfigMap (or EKS Access Entries if using the newer API). Here, we create the binding assuming the user is mapped to aws-batch.

resource "kubernetes_cluster_role_binding" "aws_batch_cluster_role_binding" {
  metadata {
    name = "aws-batch-cluster-role-binding"
  }

  role_ref {
    api_group = "rbac.authorization.k8s.io"
    kind      = "ClusterRole"
    name      = kubernetes_cluster_role.aws_batch_cluster_role.metadata[0].name
  }

  subject {
    kind      = "User"
    name      = "aws-batch" # This must match the username in aws-auth
    api_group = "rbac.authorization.k8s.io"
  }
}

Step 3: The Terraform Compute Environment

Now we define the aws_batch_compute_environment resource. The key differentiator here is the compute_resources block type, which must be set to FARGATE_SPOT, FARGATE, EC2, or SPOT, and strictly linked to the EKS configuration.

resource "aws_batch_compute_environment" "eks_batch_ce" {
  compute_environment_name = "eks-batch-compute-env"
  type                     = "MANAGED"
  service_role             = aws_iam_role.batch_eks_service_role.arn

  eks_configuration {
    eks_cluster_arn      = data.aws_eks_cluster.main.arn
    kubernetes_namespace = "batch-jobs" # Ensure this namespace exists!
  }

  compute_resources {
    type               = "EC2" # Or FARGATE
    max_vcpus          = 256
    min_vcpus          = 0
    
    # Note: For EKS, security_group_ids and subnets might be ignored 
    # if you are relying on existing Node Groups, but are required for validation.
    security_group_ids = [aws_security_group.batch_sg.id]
    subnets            = module.vpc.private_subnets
    
    instance_types = ["c5.large", "m5.large"]
  }

  depends_on = [
    aws_iam_role_policy_attachment.batch_eks_policy,
    kubernetes_cluster_role_binding.aws_batch_cluster_role_binding
  ]
}

Technical Note:
When using EKS, the instance_types and subnets defined in the Batch Compute Environment are primarily used by Batch to calculate scaling requirements. However, the actual Pod placement depends on the Node Groups (or Karpenter provisioners) available in your EKS cluster.

Step 4: Job Queues and Definitions

Finally, we wire up the Job Queue and a basic Job Definition. In the EKS context, the Job Definition looks different—it wraps Kubernetes properties.

resource "aws_batch_job_queue" "eks_batch_jq" {
  name                 = "eks-batch-queue"
  state                = "ENABLED"
  priority             = 10
  compute_environments = [aws_batch_compute_environment.eks_batch_ce.arn]
}

resource "aws_batch_job_definition" "eks_job_def" {
  name        = "eks-job-def"
  type        = "container"
  
  # Crucial: EKS Job Definitions define node properties differently
  eks_properties {
    pod_properties {
      host_network = false
      containers {
        image = "public.ecr.aws/amazonlinux/amazonlinux:latest"
        command = ["/bin/sh", "-c", "echo 'Hello from EKS Batch'; sleep 30"]
        
        resources {
          limits = {
            cpu    = "1.0"
            memory = "1024Mi"
          }
          requests = {
            cpu    = "0.5"
            memory = "512Mi"
          }
        }
      }
    }
  }
}

Best Practices for Production

• Use Karpenter: Standard Cluster Autoscaler can be sluggish with Batch spikes. Karpenter observes the unschedulable Pods created by Batch and provisions nodes in seconds.
• Namespace Isolation: Always isolate Batch workloads in a dedicated Kubernetes namespace (e.g., batch-jobs). Configure ResourceQuotas on this namespace to prevent Batch from starving your microservices.
• Logging: Ensure your EKS nodes have Fluent Bit or similar log forwarders installed. Batch logs in the console are helpful, but aggregating them into CloudWatch or OpenSearch via the node’s daemonset is superior for debugging.

Frequently Asked Questions (FAQ)

Can I use Fargate with AWS Batch on EKS?

Yes. You can specify FARGATE or FARGATE_SPOT in your compute resources. However, you must ensure you have a Fargate Profile in your EKS cluster that matches the namespace and labels defined in your Batch Job Definition.

Why is my Job stuck in RUNNABLE status?

This is the classic “It’s DNS” of Batch. In EKS, RUNNABLE usually means Batch has successfully submitted the Pod to the API Server, but the Pod is Pending. Check your K8s events (kubectl get events -n batch-jobs). You likely lack sufficient capacity (Node Groups not scaling) or have a `Taint/Toleration` mismatch.

How does this compare to standard Batch on EC2?

Standard Batch manages the ASG (Auto Scaling Group) for you. Batch on EKS delegates the infrastructure management to you (or your EKS autoscaler). EKS offers better unification if you already run K8s, but standard Batch is simpler if you just need raw compute without K8s management overhead.

Conclusion

Integrating AWS Batch with Amazon EKS using Terraform provides a powerful, unified compute plane for high-performance computing. By explicitly defining your IAM trust boundaries and Kubernetes RBAC permissions, you eliminate the “black box” magic and gain full control over your batch processing lifecycle.

Start by deploying the IAM roles and RBAC bindings defined above. Once the permissions handshake is verified, layer on the Compute Environment and Job Queues. Your infrastructure is now ready to process petabytes at scale. Thank you for reading the DevopsRoles page!

The Model Context Protocol (MCP) has rapidly become the standard for connecting AI models to your data and tools. However, most initial implementations are strictly local—relying on stdio to pipe data between a local process and your AI client (like Claude Desktop or Cursor). While this works for personal scripts, it doesn’t scale for teams.

To truly unlock the potential of AI agents in the enterprise, you need to decouple the “Brain” (the AI client) from the “Hands” (the tools). This means moving your MCP servers from localhost to robust cloud infrastructure.

This guide details the architectural shift required to run AWS ECS EKS MCP workloads. We will cover how to deploy remote MCP servers using Server-Sent Events (SSE), how to host them on Fargate and Kubernetes, and—most importantly—how to secure them so you aren’t exposing your internal database tools to the open internet.

The Architecture Shift: From Stdio to Remote SSE

In a local setup, the MCP client spawns the server process and communicates via standard input/output. This is secure by default because it’s isolated to your machine. To move this to AWS, we must switch the transport layer.

The MCP specification supports SSE (Server-Sent Events) for remote connections. This changes the communication flow:

• Server-to-Client: Uses a persistent SSE connection to push events (like tool outputs or log messages).
• Client-to-Server: Uses standard HTTP POST requests to send commands (like “call tool X”).

Pro-Tip: Unlike WebSockets, SSE is unidirectional (Server -> Client). This is why the protocol also requires an HTTP POST endpoint for the client to talk back. When deploying to AWS, your Load Balancer must support long-lived HTTP connections for the SSE channel.

Option A: Serverless Simplicity with AWS ECS (Fargate)

For most standalone MCP servers—such as a tool that queries a specific RDS database or interacts with an internal API—AWS ECS Fargate is the ideal host. It removes the overhead of managing EC2 instances while providing native integration with AWS VPCs for security.

1. The Container Image

You need an MCP server that listens on a port (usually via a web framework like FastAPI or Starlette) rather than just running a script. Here is a conceptual Dockerfile for a Python-based remote MCP server:

FROM python:3.11-slim

WORKDIR /app

# Install MCP SDK and a web server (e.g., Starlette/Uvicorn)
RUN pip install mcp[cli] uvicorn starlette

COPY . .

# Expose the port for SSE and HTTP POST
EXPOSE 8080

# Run the server using the SSE transport adapter
CMD ["uvicorn", "server:app", "--host", "0.0.0.0", "--port", "8080"]

2. The Task Definition & ALB

When defining your ECS Service, you must place an Application Load Balancer (ALB) in front of your tasks. The critical configuration here is the Idle Timeout.

• Health Checks: Ensure your container exposes a simple /health endpoint, or the ALB will kill the task during long AI-generation cycles.
• Timeout: Increase the ALB idle timeout to at least 300 seconds. AI models can take time to “think” or process large tool outputs, and you don’t want the SSE connection to drop prematurely.

Option B: Scalable Orchestration with Amazon EKS

If your organization already operates on Kubernetes, deploying AWS ECS EKS MCP servers as standard deployments allows for advanced traffic management. This is particularly useful if you are running a “Mesh” of MCP servers.

The Ingress Challenge

The biggest hurdle on EKS is the Ingress Controller. If you use NGINX Ingress, it defaults to buffering responses, which breaks SSE (the client waits for the buffer to fill before receiving the first event).

You must apply specific annotations to your Ingress resource to disable buffering for the SSE path:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mcp-server-ingress
  annotations:
    # Critical for SSE to work properly
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
spec:
  ingressClassName: nginx
  rules:
    - host: mcp.internal.yourcompany.com
      http:
        paths:
          - path: /sse
            pathType: Prefix
            backend:
              service:
                name: mcp-service
                port:
                  number: 80

Warning: Never expose an MCP server Service as LoadBalancer (public) without strict Security Groups or authentication. An exposed MCP server gives an AI direct execution access to whatever tools you’ve enabled (e.g., “Drop Database”).

Security: The “MCP Proxy” & Auth Patterns

This is the section that separates a “toy” project from a production deployment. How do you let an AI client (running on a developer’s laptop) access a private ECS/EKS service securely?

1. The VPN / Tailscale Approach

The simplest method is network isolation. Keep the MCP server in a private subnet. Developers must be on the corporate VPN or use a mesh overlay like Tailscale to reach the `http://internal-mcp:8080/sse` endpoint. This requires zero code changes to the MCP server.

2. The AWS SigV4 / Auth Proxy Approach

For a more cloud-native approach, AWS recently introduced the concept of an MCP Proxy. This involves:

1. Placing your MCP Server behind an ALB with AWS IAM Authentication or Cognito.
2. Running a small local proxy on the client machine (the developer’s laptop).
3. The developer configures their AI client to talk to localhost:proxy-port.
4. The local proxy signs requests with the developer’s AWS credentials (SigV4) and forwards them to the remote ECS/EKS endpoint.

This ensures that only users with the correct IAM Policy (e.g., AllowInvokeMcpServer) can access your tools.

Frequently Asked Questions (FAQ)

Can I use the official Amazon EKS MCP Server remotely?

Yes, but it’s important to distinguish between hosting a server and using a tool. AWS provides an open-source Amazon EKS MCP Server. This is a tool you run (locally or remotely) that gives your AI the ability to run kubectl commands and inspect your cluster. You can host this inside your cluster to give an AI agent “SRE superpowers” over that specific environment.

Why does my remote MCP connection drop after 60 seconds?

This is almost always a Load Balancer or Reverse Proxy timeout. SSE requires a persistent connection. Check your AWS ALB “Idle Timeout” settings or your Nginx proxy_read_timeout. Ensure they are set to a value higher than your longest expected idle time (e.g., 5-10 minutes).

Should I use ECS or Lambda for MCP?

While Lambda is cheaper for sporadic use, MCP is a stateful protocol (via SSE). Running SSE on Lambda requires using Function URLs with response streaming, which has a 15-minute hard limit and can be tricky to debug. ECS Fargate is generally preferred for the stability of the long-lived connection required by the protocol.

Conclusion

Moving your Model Context Protocol infrastructure from local scripts to AWS ECS and EKS is a pivotal step in maturing your AI operations. By leveraging Fargate for simplicity or EKS for mesh-scale orchestration, you provide your AI agents with a stable, high-performance environment to operate in.

Remember, “Powering Up” isn’t just about connectivity; it’s about security. Whether you choose a VPN-based approach or the robust AWS SigV4 proxy pattern, ensuring your AI tools are authenticated is non-negotiable in a production environment.

Next Step: Audit your current local MCP tools. Identify one “heavy” tool (like a database inspector or a large-context retriever) and containerize it using the Dockerfile pattern above to deploy your first remote MCP service on Fargate. Thank you for reading the DevopsRoles page!

For years, the gold standard in cloud security has been defined by deterministic automation. We detect an anomaly in Amazon GuardDuty, trigger a CloudWatch Event (now EventBridge), and fire a Lambda function to execute a hard-coded remediation script. While effective for known threats, this approach is brittle. It lacks context, reasoning, and adaptability.

Enter Agentic AI. By integrating Large Language Models (LLMs) via services like Amazon Bedrock into your security stack, we are moving from static “Runbooks” to dynamic “Reasoning Engines.” AWS Security Incident Response is no longer just about automation; it is about autonomy. This guide explores how to architect Agentic workflows that can analyze forensics, reason through containment strategies, and execute remediation with human-level nuance at machine speed.

The Evolution: From SOAR to Agentic Security

Traditional Security Orchestration, Automation, and Response (SOAR) platforms rely on linear logic: If X, then Y. This works for blocking an IP address, but it fails when the threat requires investigation. For example, if an IAM role is exfiltrating data, a standard script might revoke keys immediately—potentially breaking production applications—whereas a human analyst would first check if the activity aligns with a scheduled maintenance window.

Agentic AI introduces the ReAct (Reasoning + Acting) pattern to AWS Security Incident Response. Instead of blindly firing scripts, the AI Agent:

1. Observes the finding (e.g., “S3 Bucket Public Access Enabled”).
2. Reasons about the context (Queries CloudTrail: “Who did this? Was it authorized?”).
3. Acts using defined tools (Calls boto3 functions to correct the policy).
4. Evaluates the result (Verifies the bucket is private).

GigaCode Pro-Tip:
Don’t confuse “Generative AI” with “Agentic AI.” Generative AI writes a report about the hack. Agentic AI logs into the console (via API) and fixes the hack. The differentiator is the Action Group.

Architecture: Building a Bedrock Security Agent

To modernize your AWS Security Incident Response, we leverage Amazon Bedrock Agents. This managed service orchestrates the interaction between the LLM (reasoning), the knowledge base (RAG for company policies), and the action groups (Lambda functions).

1. The Foundation: Knowledge Bases

Your agent needs context. Using Retrieval-Augmented Generation (RAG), you can index your internal Wiki, incident response playbooks, and architecture diagrams into an Amazon OpenSearch Serverless vector store connected to Bedrock. When a finding occurs, the agent first queries this base: “What is the protocol for a compromised EC2 instance in the Production VPC?”

2. Action Groups (The Hands)

Action groups map OpenAPI schemas to AWS Lambda functions. This allows the LLM to “call” Python code. Below is an example of a remediation tool that an agent might decide to use during an active incident.

Code Implementation: The Isolation Tool

This Lambda function serves as a “tool” that the Bedrock Agent can invoke when it decides an instance must be quarantined.

import boto3
import json
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

ec2 = boto3.client('ec2')

def lambda_handler(event, context):
    """
    Tool for Bedrock Agent: Isolates an EC2 instance by attaching a forensic SG.
    Input: {'instance_id': 'i-xxxx', 'vpc_id': 'vpc-xxxx'}
    """
    agent_params = event.get('parameters', [])
    instance_id = next((p['value'] for p in agent_params if p['name'] == 'instance_id'), None)
    
    if not instance_id:
        return {"response": "Error: Instance ID is required for isolation."}

    try:
        # Logic to find or create a 'Forensic-No-Ingress' Security Group
        logger.info(f"Agent requested isolation for {instance_id}")
        
        # 1. Get current SG for rollback context (Forensics)
        current_attr = ec2.describe_instance_attribute(
            InstanceId=instance_id, Attribute='groupSet'
        )
        
        # 2. Attach Isolation SG (Assuming sg-isolation-id is pre-provisioned)
        isolation_sg = "sg-0123456789abcdef0" 
        
        ec2.modify_instance_attribute(
            InstanceId=instance_id,
            Groups=[isolation_sg]
        )
        
        return {
            "response": f"SUCCESS: Instance {instance_id} has been isolated. Previous SGs logged for analysis."
        }
        
    except Exception as e:
        logger.error(f"Failed to isolate: {str(e)}")
        return {"response": f"FAILED: Could not isolate instance. Reason: {str(e)}"}

Implementing the Workflow

Deploying this requires an Event-Driven Architecture. Here is the lifecycle of an Agentic AWS Security Incident Response:

• Detection: GuardDuty detects UnauthorizedAccess:EC2/TorIPCaller.
• Ingestion: EventBridge captures the finding and pushes it to an SQS queue (for throttling/buffering).
• Invocation: A Lambda “Controller” picks up the finding and invokes the Bedrock Agent Alias using the invoke_agent API.
• Reasoning Loop:
  • The Agent receives the finding details.
  • It checks the “Knowledge Base” and sees that Tor connections are strictly prohibited.
  • It decides to call the GetInstanceDetails tool to check tags.
  • It sees the tag Environment: Production.
  • It decides to call the IsolateInstance tool (code above).
• Resolution: The Agent updates AWS Security Hub with the workflow status, marks the finding as RESOLVED, and emails the SOC team a summary of its actions.

Human-in-the-Loop (HITL) and Guardrails

For expert practitioners, the fear of “hallucinating” agents deleting production databases is real. To mitigate this in AWS Security Incident Response, we implement Guardrails for Amazon Bedrock.

Guardrails allow you to define denied topics and content filters. Furthermore, for high-impact actions (like terminating instances), you should design the Agent to request approval rather than execute immediately. The Agent can send an SNS notification with a standard “Approve/Deny” link. The Agent pauses execution until the approval signal is received via a callback webhook.

Pro-Tip: Use CloudTrail Lake to audit your Agents. Every API call made by the Agent (via the assumed IAM role) is logged. Create a QuickSight dashboard to visualize “Agent Remediation Success Rates” vs. “Human Intervention Required.”

Frequently Asked Questions (FAQ)

How does Agentic AI differ from AWS Lambda automation?

Lambda automation is deterministic (scripted steps). Agentic AI is probabilistic and reasoning-based. It can handle ambiguity, such as deciding not to act if a threat looks like a false positive based on cross-referencing logs, whereas a script would execute blindly.

Is it safe to let AI modify security groups automatically?

It is safe if scoped correctly using IAM Roles. The Agent’s role should adhere to the Principle of Least Privilege. Start with “Read-Only” agents that only perform forensics and suggest remediation, then graduate to “Active” agents for low-risk environments.

Which AWS services are required for this architecture?

At a minimum: Amazon Bedrock (Agents & Knowledge Bases), AWS Lambda (Action Groups), Amazon EventBridge (Triggers), Amazon GuardDuty (Detection), and AWS Security Hub (Centralized Management).

Conclusion

The landscape of AWS Security Incident Response is shifting. By adopting Agentic AI, organizations can reduce Mean Time to Respond (MTTR) from hours to seconds. However, this is not a “set and forget” solution. It requires rigorous engineering of prompts, action schemas, and IAM boundaries.

Start small: Build an agent that purely performs automated forensics—gathering logs, querying configurations, and summarizing the blast radius—before letting it touch your infrastructure. The future of cloud security is autonomous, and the architects who master these agents today will define the standards of tomorrow.

For deeper reading on configuring Bedrock Agents, consult the official AWS Bedrock User Guide or review the AWS Security Incident Response Guide.