🌟 Setting Resource Constraints in Kubernetes Across Namespaces 🌟

---

Introduction 🚀

As companies transition from monolithic applications to microservices, Kubernetes has become the go-to platform for managing these systems. Its powerful orchestration capabilities facilitate deployment, scaling, and management of applications. However, managing resources effectively within a Kubernetes cluster is crucial to ensure fair usage and optimal performance.

---

The Issue 🧐

Generally, to divide applications, stacks, services, and users working on different applications in Kubernetes, namespaces are utilized. Namespaces help isolate resources and manage access control. However, without proper resource management, one namespace might consume more resources than intended, potentially impacting the performance of other namespaces.

---

Solution 💡

To manage resource usage across namespaces effectively, Kubernetes provides ResourceQuotas and LimitRanges. These mechanisms help ensure that resources are used fairly and efficiently across namespaces

---

Example Scenario

Consider a Kubernetes cluster with below namespaces:-

• backend: Application Backend.

• db-service: Databases.

• cache-service: Caching systems.

Without ResourceQuotas and LimitRanges, a service in the backend namespace could use more resources than allocated, leading to performance issues or even outages for other services.

ResourceQuotas 💡

ResourceQuotas set limits on the total amount of resources that can be consumed by all pods and containers within a namespace. This ensures that no single namespace can consume more resources than allocated, helping to maintain a balanced distribution of cluster resources.

ResourceQuota Configuration

Example YAML for a ResourceQuota in the backend Namespace:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: backend-quota
  namespace: backend
spec:
  hard:
    # Maximum amount of CPU resources that can be requested by all pods in this namespace
    requests.cpu: "2"  # Total CPU requested by all pods cannot exceed 2 CPUs
    # Maximum amount of memory resources that can be requested by all pods in this namespace
    requests.memory: "4Gi"  # Total memory requested by all pods cannot exceed 4 GiB
    # Maximum amount of CPU resources that can be limited by all pods in this namespace
    limits.cpu: "4"  # Total CPU limit for all pods cannot exceed 4 CPUs
    # Maximum amount of memory resources that can be limited by all pods in this namespace
    limits.memory: "8Gi"  # Total memory limit for all pods cannot exceed 8 GiB
    # Maximum number of pods that can be created in this namespace
    pods: "4"  # We can only have at most 4 pods in this namespace
    # Maximum number of services that can be created in this namespace
    services: "5"  # We can only have at most 5 services in this namespace

Deployment Example That Exceeds ResourceQuota

Example YAML for a Pod:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: high-resource-deployment
  namespace: backend
spec:
  replicas: 6  # Increased replicas to demonstrate quota limits
  selector:
    matchLabels:
      app: high-resource-app
  template:
    metadata:
      labels:
        app: high-resource-app
    spec:
      containers:
      - name: high-resource-container
        image: busybox
        resources:
          # Resource requests and limits for this container
          requests:
            memory: "1Gi"  # Requested memory for each container
            cpu: "500m"  # Requested CPU for each container
          limits:
            memory: "2Gi"  # Limit memory for each container
            cpu: "1"  # Limit CPU for each container

What Happens:

• Total Resources Requests: With 6 replicas, each container requesting 1Gi memory and 500m CPU:

  • Memory: 6 replicas * 1Gi = 6Gi (Exceeds the quota limit of 4Gi)

  • CPU: 6 replicas * 500m = 3000m (3 CPUs ,Exceeds the quota limit of 2CPUs)

• Total Resource Limits: With 6 replicas, each container limited to 2Gi memory and 1 CPU:

  • Memory: 6 replicas * 2Gi = 12Gi (Exceeds the quota limit of 8Gi)

  • CPU: 6 replicas * 1 = 6 CPUs (Exceeds the quota limit of 4 CPUs)

• No.of Pods:- Pod count: 6 (Exceeds the quota limit 4)

Sample Error Message:
Error from server (Forbidden): error when creating "deployment.yaml": deployments "high-resource-deployment" is forbidden: violated resource quota

Explanation:

The deployment exceeds the ResourceQuota limits for both memory and CPU. Kubernetes prevents the creation of the deployment to enforce resource constraints and ensure fair usage.

Solution:

To resolve this, adjust the deployment’s resource requests and limits to fit within the bounds defined by the ResourceQuota:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: high-resource-deployment
  namespace: backend
spec:
  replicas: 3    # Reduce replicas to fit within quota limits
  selector:
    matchLabels:
      app: high-resource-app
  template:
    metadata:
      labels:
        app: high-resource-app
    spec:
      containers:
      - name: high-resource-container
        image: busybox
        resources:
          requests:
            memory: "500Mi"  # Adjust requests to fit within the quota
            cpu: "200m"
          limits:
            memory: "1Gi"  # Adjust limits to fit within the quota
            cpu: "500m"

ResourceQuota Configuration Options:

• requests.cpu: Total CPU requested by all pods in the namespace.

• requests.memory: Total memory requested by all pods in the namespace.

• limits.cpu: Total CPU limit for all pods in the namespace.

• limits.memory: Total memory limit for all pods in the namespace.

• pods: Maximum number of pods that can be created in the namespace.

• services: Maximum number of services that can be created in the namespace.

• persistentvolumeclaims: Maximum number of persistent volume claims that can be created in the namespace.
  ...

LimitRanges

LimitRanges define default and maximum resource limits for containers within a namespace. They ensure that all containers within the namespace adhere to minimum and maximum resource requirements.

LimitRange Configuration

Example YAML for a LimitRange in the backend Namespace:

apiVersion: v1
kind: LimitRange
metadata:
  name: backend-limit-range
  namespace: backend
spec:
  limits:
  - type: Container
    # Maximum resource limits for each container
    max:
      memory: "2Gi"  # Maximum memory limit per container is 2 GiB
      cpu: "1"  # Maximum CPU limit per container is 1 CPU
    # Minimum resource requests for each container
    min:
      memory: "128Mi"  # Minimum memory request per container is 128 MiB
      cpu: "50m"  # Minimum CPU request per container is 50 milli-CPU
    # Default resource limits if not specified by the user
    default:
      memory: "512Mi"  # Default memory limit per container is 512 MiB
      cpu: "250m"  # Default CPU limit per container is 250 milli-CPU
    # Default resource requests if not specified by the user
    defaultRequest:
      memory: "256Mi"  # Default memory request per container is 256 MiB
      cpu: "100m"  # Default CPU request per container is 100 milli-CPU

Deployment Example That Exceeds LimitRange

Example YAML for a Pod:

apiVersion: v1
kind: Pod
metadata:
  name: over-limit-pod
  namespace: backend
spec:
  containers:
  - name: over-limit-container
    image: busybox
    resources:
      # Resource requests and limits for this container
      requests:
        memory: "3Gi"  
        cpu: "2"  
      limits:
        memory: "3Gi"  # Requested memory exceeds the maximum allowed limit
        cpu: "1"  # Requested CPU exceeds the maximum allowed limit

What Happens:

If the LimitRange specifies a maximum memory of 2Gi and a maximum CPU of 1, this pod will fail to start because it exceeds these limits. Kubernetes will prevent the pod from being created, and you will receive an error message.

Explanation:

The deployment exceeds the limits defined in the LimitRange. Kubernetes prevents the creation of the deployment to enforce resource constraints and ensure that all containers adhere to the specified limits.

Solution:

To resolve this, adjust the deployment’s resource requests and limits to fit within the bounds defined by the LimitRange:

apiVersion: v1
kind: Pod
metadata:
  name: over-limit-pod
  namespace: backend
spec:
  containers:
  - name: over-limit-container
    image: busybox
    resources:
      requests:
        memory: "512Mi"  
        cpu: "500m"  
      limits:
        memory: "1Gi"  # Limit within the allowed memory limits
        cpu: "750m"  # Limit within the allowed CPU limits

LimitRange Configuration Options:

• max.cpu: Maximum CPU limit for containers.

• max.memory: Maximum memory limit for containers.

• min.cpu: Minimum CPU request for containers.

• min.memory: Minimum memory request for containers.

• default.cpu: Default CPU request for containers when not specified.

• default.memory: Default memory request for containers when not specified.

• defaultRequest.cpu: Default CPU request for containers when not specified

• defaultRequest.memory: Default memory request for containers when not specified

• maxLimitRequestRatio: represents the max burst for the named resource

...

---

Closing ✨

By implementing ResourceQuotas and LimitRanges, you ensure balanced and fair resource distribution within your Kubernetes namespace. Proper management of these quotas and limits helps maintain application performance and cluster stability.

Have you faced challenges with Kubernetes quotas or limits? Share your experiences or solutions in the comments! If you found this guide helpful, please like, comment, and share to help others in the community. Your feedback is always appreciated!

Sharing knowledge is like optimizing a pipeline—every insight accelerates collective progress and drives innovation forward.

#cloudengineering #cloudcomputing #devops #kubernetes #googlecloud #containers #cloud-native #azure #sre