Maximizing Kubernetes Storage Efficiency: A Comprehensive Guide to Amazon EBS Integration with EKS

Introduction:

In the realm of Kubernetes management, efficient storage allocation and management are pivotal for ensuring seamless operations and optimal resource utilization. Amazon Elastic Kubernetes Service (EKS) offers a robust platform for deploying, managing, and scaling containerized applications, while Amazon Elastic Block Store (EBS) provides reliable and scalable block storage solutions in the AWS cloud. Integrating EBS with EKS using the EBS CSI Driver empowers users to leverage the full potential of Kubernetes storage orchestration capabilities.

Use Cases:

1. Stateful Applications: EBS integration with EKS enables the deployment of stateful applications such as databases, message queues, and caching systems. By persisting data to EBS volumes, these applications can maintain state across pod restarts and scale dynamically in response to changing workloads.

2. Data Analysis Workloads: Organizations running data analysis workloads on Kubernetes can benefit from EBS storage integration to store large datasets and intermediate results. EBS volumes offer high-performance storage options suitable for processing and analyzing vast amounts of data efficiently.

3. Continuous Integration/Continuous Deployment (CI/CD) Pipelines: Integrating EBS with EKS facilitates the storage of artifacts, logs, and configuration files used in CI/CD pipelines. PersistentVolumeClaims (PVCs) backed by EBS volumes ensure that critical pipeline data persists across pipeline runs and can be accessed by multiple pods simultaneously.

4. Machine Learning Model Training: Kubernetes-based machine learning workflows often require persistent storage for storing training data, model checkpoints, and experiment results. EBS integration with EKS simplifies the management of storage resources for machine learning pipelines, enabling seamless model training and experimentation at scale.

Prerequisites:

• Setting up an AWS account with appropriate permissions.

• Installing eksctl and kubectl for managing EKS clusters.

• Configuring IAM roles and policies to grant necessary permissions for managing EBS volumes.

• Familiarity with Kubernetes concepts such as StorageClass, PersistentVolume, and PersistentVolumeClaim.

Step-by-Step Guide:

1. Installeksctl and Associate IAM OIDC Provider

2. Create IAM Policy for EBS

3. Deploy Amazon EBS CSI Driver

4. Configure StorageClass and PersistentVolumeClaim (PVC)

5. Deploying Applications with EBS Storage

Step 1: Installeksctl and Associate IAM OIDC Provider:

• Provide direct commands for installing eksctl and verifying the installation.

  Refer: https://eksctl.io/installation/

•   eksctl utils associate-iam-oidc-provider --cluster devcluster --approve
  

  The command you provided is used to associate an IAM OIDC (OpenID Connect) provider with your Amazon EKS cluster. This association is necessary to enable IAM roles for service accounts (IRSA) within your Kubernetes cluster.

  Let's break down the command:

  • eksctl: This is the command-line utility for Amazon EKS management.

  • utils: This subcommand allows you to execute utility tasks related to your EKS cluster.

  • associate-iam-oidc-provider: This specific task associates an IAM OIDC provider with your EKS cluster.

  • --cluster devcluster: This flag specifies the name of your EKS cluster, in this case, devcluster.

  • --approve: This flag indicates that you approve the association without further confirmation.

When you run this command, eksctl automatically configures the IAM OIDC provider for your EKS cluster, enabling Kubernetes service accounts to leverage IAM roles for AWS service access. This integration enhances security and simplifies access control for applications running in your Kubernetes environment.

Step 2: Create IAM Policy for EBS:

• Walk users through the process of navigating to the IAM console, creating a new policy, and entering the provided JSON document.

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": [
            "ec2:AttachVolume",
            "ec2:CreateSnapshot",
            "ec2:CreateTags",
            "ec2:CreateVolume",
            "ec2:DeleteSnapshot",
            "ec2:DeleteTags",
            "ec2:DeleteVolume",
            "ec2:DescribeInstances",
            "ec2:DescribeSnapshots",
            "ec2:DescribeTags",
            "ec2:DescribeVolumes",
            "ec2:DetachVolume"
          ],
          "Resource": "*"
        }
      ]
    }
  

• Clarify each permission and its significance for managing EBS volumes within Kubernetes clusters.

• Save policy with the name "Amazon_EBS_CSI_Driver"

Step 3: Get Worker node IAM Role AR

• Use the kubectl command to describe the aws-auth ConfigMap in the kube-system namespace. This ConfigMap is responsible for mapping Kubernetes users and roles to IAM roles in AWS.

    kubectl -n kube-system describe configmap aws-auth
  

• The output of the command will include a section that lists the IAM roles associated with the EKS cluster, specifically the rolearn.

    rolearn: arn:aws:iam::180789647333:role/eksctl-eksdemo1-nodegroup-eksdemo-NodeInstanceRole-IJN07ZKXAWNN
  

1. Associate the Policy with the IAM Role:

   • Access the AWS Management Console and navigate to the IAM service.

   • Locate the IAM role corresponding to the worker nodes. In this case, the role name is eksctl-eksdemo1-nodegroup.

   • Click on the role to open its details.

   • Switch to the "Permissions" tab.

   • Click on "Attach Policies" to associate a new policy with the role.

   • Search for the policy named Amazon_EBS_CSI_Driver.

   • Select the policy from the list of available policies.

   • Click on "Attach Policy" to finalize the association.

By following these steps, you ensure that the IAM role associated with the worker nodes in your EKS cluster has the necessary permissions granted by the Amazon_EBS_CSI_Driver policy. This allows the worker nodes to perform operations related to managing EBS volumes, such as attaching, detaching, creating, and deleting volumes, as specified in the policy's permissions.

Step 3: Deploy Amazon EBS CSI Driver:

• Provide additional context on the compatibility requirements between kubectl versions and the EBS CSI Driver.

    kubectl apply -k "github.com/kubernetes-sigs/aws-ebs-csi-driver/deploy/kubernetes/overlays/stable/?ref=master"
  

    # Verify ebs-csi pods running
    kubectl get pods -n kube-system
  

Step 4: Create a StorageClass:

• Open your favorite text editor and create a file named storageclass.yaml.

• Copy and paste the following YAML content into storageclass.yaml:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: datastore
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
reclaimPolicy: Retain
allowVolumeExpansion: true
mountOptions:
  - debug
volumeBindingMode: Immediate

• Save the file.

Step 5: Apply the StorageClass:

• Run the following command to apply the StorageClass configuration:

kubectl apply -f storageclass.yaml

Step 6: Define a PersistentVolumeClaim (PVC):

• Open your text editor again and create a file named persistentvolumeclaim.yaml.

• Copy and paste the following YAML content into persistentvolumeclaim.yaml:

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: devpvc
      namespace: devproject
    spec:
      accessModes:
        - ReadWriteOnce
      storageClassName: datastore
      resources:
        requests:
          storage: 5Gi
  

• Save the file.

Step 7: Apply the PersistentVolumeClaim (PVC):

• Run the following command to apply the PersistentVolumeClaim configuration:

kubectl apply -f persistentvolumeclaim.yaml

Step 8: Deploy applications:

• Use my Github repo for manifest files Examples: https://github.com/NavyaDeveloper/StorageClass_eks_CSI

• Break down the components of a StorageClass and a PersistentVolumeClaim, highlighting their roles in Kubernetes storage management.

• Explain each parameter in the provided sample YAML manifest for creating a StorageClass and PersistentVolumeClaim, including provisioner, parameters, reclaimPolicy, allowVolumeExpansion, and volumeBindingMode.

• Provide examples of how users can customize the StorageClass and PersistentVolumeClaim configurations based on their specific requirements.

Conclusion:

• Summarize the key takeaways from the blog post, emphasizing the importance of effectively managing storage in Kubernetes clusters.

• Encourage readers to explore advanced features of EBS storage in EKS environments, such as volume snapshots, encryption, and data lifecycle management.

• Invite feedback and suggestions for future topics or improvements to the blog post content.