Let me begin with some context on why I thought of writing about this. Recently I attended a meetup on “Securing a Kubernetes Cluster”. One of the key takeaways from this meetup is blocking access to one namespace from another(Using Network Policies). Some of you may wonder why is this a topic worth exploring. To answer this question let’s look at a common attack vector observed in the hacking world.

Let’s say we have one Kubernetes cluster hosting multiple environments. Multiple environments in the cluster; Separated by namespaces. As in many other cases, our production application is bug-free and thoroughly tested to be free from vulnerabilities as well. But our development environment(dev) sees many changes rapidly. Because of this, the Dev server is not always bug-free and may contain vulnerabilities. Attackers may use this to their advantage. Since they cannot directly access the production environment, they can initially gain access to the Dev environment. After that they can try to penetrate in to the production environment through that. Or stay in Dev and do their nefarious activities from there.

What next? The attackers can either actively attack the production environment. Or they can remain undetected for as long as they can and eavesdrop on the production environment. This depends on their motive. So, for this reason we need to create network policies. Let’s take a closer look on this with a demo.

Expand the sections below to learn more on this.

Setting up the demo

So for this demo I am using a 2 node Kubernetes cluster. One master node and one worker node. I have created 2 namespaces with labels.

kubectl create namespace dev-namespace --dry-run=client -o yaml > dev-ns.yml

Next, I added some labels to the namespaces.

#Editing the file dev-ns.yml
apiVersion: v1
kind: Namespace
metadata:
  creationTimestamp: null
  name: dev-space
  labels:
    env:  dev
spec: {}
status: {}

Then, I created the namespace for Dev.

kubectl apply -f dev-ns.yml

Similarly, I created another namespace for production.

Now we need to create some deployments in those namespaces that we created. We can easily do that using the below commands. Here, I am using nginxdemos/hello:plain-text as the image. Basically, it prints some information about the host. You can find more information about this container image here.

kubectl create deployment devapp --image=nginxdemos/hello:plain-text --namespace=dev-space
kubectl create deployment prodapp --image=nginxdemos/hello:plain-text --namespace=prod-space

Lastly, in this section lets see if the above deployment works. For this we just need to do a curl.

Attacking production from dev

Here, we are trying to do things to prod from dev. This is just a demo so we are not doing anything fancy. I will explain few things attacker can do later in this section. To simulate execution of commands run against the production we are making use of the ‘kubectl exec’ command. But attacker may a different approach to reach the same outcome shown here. For example, attacker may run commands against prod from dev using a hacking tool/script.

As pictured above, we are using the existing pod in the dev deployment to execute various commands against the production server. What can the attacker do here? Attacker may use commands to download malicious scripts from elsewhere in to the dev pod and run them. Here, the attacker has more visibility over accessing from externally. Therefore, attacker has access to ports that are inaccessible from outside. He/she can easily identify them by probing them from the Dev as we did up there 🙂

Network policies to the rescue

First of all, we need to acknowledge the fact that attackers should not have access to the dev environment in the first place. We need to ensure they are kept out of our perimeter and out of our cluster. But it is always nice to have defense in depth. So as part of our contingency plans, we can add a network policy.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: ns-netpol
  namespace: prod-space
spec:
  podSelector:
    matchLabels:
      app: prodapp
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          env: prod

Finally, once you have the above typed out, create a network policy using it. This will ensure that only any object within the prod namespace can reach the deployment in question. Above is just an example and this can be customized as per our requirement.

Let’s see what happens when we try to probe the production environment from the dev environment now.

Success! Now we have blocked all requests from namespaces other than prod going in to prod.

I hope that the above content has given you some insight about network policies being used on namespaces. Questions? Comments? post them down below. If you found this interesting please check out my featured blogs here.