Container security from A to Z

Today, some form of virtualization or containerization can be found in almost all large IT solutions. Containers provide a host of benefits during system development, installation, maintenance, and use. They promote faster development, cost savings, and conservation of other resources. At the same time, many security solutions that work on physical and virtual servers are not directly applicable to containers. What risks should companies consider when implementing containerization, and what measures are needed to protect container infrastructure?

Benefits of containerization in development and operation

A container is an isolated environment for running a single application created by OS kernel-level tools. The container image includes both the application and required settings and auxiliary components, making it very convenient for developers to pack everything they need into the container. Those using such a container find it much easier to operate than old-fashioned infrastructure. What’s more, isolation greatly reduces the influence of containerized applications on each other. In a container infrastructure, therefore, there are fewer causes for failures, while at the same there’s more controllability for administrators.

Containerization is a lighter technology than virtualization: containers don’t emulate hardware, and there’s no need to supply the entire contents of the virtual machine — in particular the guest OS. In many cases, containerized workloads are easier to scale.

Without a doubt, the most common tool for creating and storing container images is Docker, while container workload orchestration is most often implemented with Kubernetes, Docker Swarm, or Red Hat OpenShift.

Containerization has become a key part of modern IT development approaches. Many applications are developed in a microservice architecture: individual features of a large application are allocated to microservices that communicate with other parts of the application through APIs. An example is a video player within a social network or an online store’s payment process. These microservices are often delivered as containers, allowing developers to have their own development and delivery cycle.

Containers dovetail perfectly with the CI/CD (continuous integration/continuous delivery) modern methodology, so application updates get released more quickly and with reduced quantities of bugs. This approach envisages a short development cycle, teams working in parallel on the same code, and automation of routine actions. Containerization in a CI/CD pipeline also improves the efficiency of the pipeline: the CI/CD system uses container images as templates and delivers the build as a ready-to-deploy image. The key point is that updates are delivered in the form of new images — rather than deployed inside an existing and operational container. This speeds up the preparation and debugging of the release, lessens the requirements for the infrastructure of both the developer and customer, improves operational stability, and makes the application easier to scale.

By properly integrating container security requirements into development and build processes, a company takes a big stride toward full implementation of DevSecOps.

Core threats in container infrastructure

The host system, containerization environments, and containerized applications are all susceptible to most of the typical information security risks, such as vulnerabilities in components, insecure settings and the like.

Malicious actors are already actively exploiting all of the above. For example, 1650 container images with malware were found in the public Docker Hub repository. In a similar case, malicious images went undetected for around a year. There are known malicious campaigns that use the Docker API to create malicious containers on targeted systems, disable monitoring systems, and engage in mining. In another attack, threat actors went after Kubernetes clusters with misconfigured PostgreSQL. Another common problem is that outdated container images harboring known vulnerabilities like Log4shell can be stored in repositories for quite some time. Also, developers regularly leave behind API keys and other secrets in containers.

Systematizing the threats to each element in the containerization system, we get this somewhat simplified scheme:

Containers and protection using traditional security tools

Many defenses that have worked well for virtual machines cannot be applied to container security. It’s usually not possible to run an EDR agent inside a container, as in a virtual machine. Moreover, what happens in the container is not fully available for analysis by conventional security systems on the host system. Therefore, detecting, for example, vulnerable and malicious software inside the container is problematic, as is applying protection tools such as WAF in containerized applications. Traffic between containers is often carried over a virtual network at the orchestrator level and might not be accessible to network security tools.

Even on the host OS, an unadapted protection agent can lead to degradation of the performance or stability of deployed containerized applications. Cluster security must be provided at the host level in line with the particular orchestration environment and the nature of the container workloads.

There are also specific issues that must be addressed for container environments — like preventing untrusted containers from running, searching for secrets in containers, and restricting network traffic for each specific container based on its functions. All this is only available in specialized solutions such as Kaspersky Container Security.

What about protection with native tools?

All key containerization vendors appear to be working hard to improve the security of their products. Native Kubernetes tools, for example, can be used to configure resource quotas and logging policies, as well as implement RBAC (role-based access control) with the least-privilege principle. All the same, there are entire classes of information security tasks that cannot be solved with native tools — such as monitoring processes inside a running container, vulnerability analysis, checking compliance with information security policies and best practices, and much more.

But above all, a mature and full-fledged container security system needs to ensure protection at the early stages of containerization: development, delivery, and storage. To achieve this, containerization security has to be built into the development process and integrated with developer tools.

How container protection becomes part of DevSecOps

The DevOps approach has evolved into DevSecOps due to the ever-increasing demands for application reliability and security. To make security an organic part of development, core information security requirements are automatically checked at all phases of application preparation and delivery wherever possible. Container environments facilitate this.

Planning phase: securing VCS and registry operations. Early in the development cycle, software developers select the components, including containerized ones, to be deployed in the application. The security system must scan registry images for up-to-dateness, and analyze configuration files (IaC — in particular, Dockerfile) for errors and insecure settings. Base images used in development need to be scanned for vulnerabilities, malware, secrets, and the like. By doing so, developers significantly reduce the risks of supply-chain compromise.

Build and test phase: securing continuous integration operations. In this phase, it’s necessary to ensure that no secrets, vulnerable versions of libraries, or malware have gotten into the image, and that all information security aspects that can be analyzed comply with the requirements of regulators and the company itself. An application build cannot be completed successfully if there are violated policies. This is done by integrating the container security system with a CI/CD platform, be it Jenkins, Gitlab, or CircleCI. Along with static and dynamic testing of application security (AppSec), this measure is what distinguishes DevSecOps from other development approaches.

Delivery and deployment phase: security at the Continuous Delivery level. Images made operational need to be scanned for both integrity and full compliance with adopted policies. If the situation warrants an exception (for example, a vulnerability is published but not yet patched), it must always be documented and time-limited.

Operation phase: protecting the orchestrator and running containers. Startup and operation control of containers. This phase minimizes the risks associated with vulnerabilities in the runtime environment or its misconfiguration. More importantly, only here is it possible to detect various anomalies in application operation, such as excessive computational load or unexpected communications with other containers and the network as a whole. This step also monitors the secure customization of the orchestrator itself, and also access to it. For container security, native operation with the Kubernetes or OpenShift orchestrator is critical here. At the same time, the host OS itself must not be left unprotected.

To operate at these stages, the container security system itself must be multi-component. The illustration shows the core elements of Kaspersky Container Security and their relationship with the containerization platform and the CI/CD platform.

What protection measures to take for each container environment component?

Let’s look at a more detailed list of protection measures that must be applied to each component in the containerization system to describe its security as comprehensive.

The central element of the security system is the in-depth scanning of images. The security system needs to integrate with key registries (such as DockerHub, GitLab Registry, or JFrog Artifactory), both public and corporate, and regularly scan used images in accordance with company policies. Each scan on the list is important in itself, but the risk profile and specifics of applications vary from company to company, so it may be possible, for example, to allow the use of images with low-criticality vulnerabilities. Also, depending on the security policies in place, CIS Kubernetes recommendations or various vulnerability databases, for instance, may be key.

Container images that fail scanning are either simply flagged for administrators, or blocked in later development and deployment phases.

The second, equally important and specific, group of protection tools operates at the container deployment and startup stage. First of all, containers that do not comply with policies and are not included in the trusted lists are prevented from running.

Runtime environment protection is incomplete without inspecting the orchestrator itself. This helps identify configuration errors, non-compliance with security policies, and unauthorized attempts to modify the configuration. Once the containers are running, monitoring orchestrator activity makes it possible to detect and halt suspicious activity both within and between clusters.

Some tasks from the matrix cannot be delegated to a security solution of any kind at all. These include the initial choice of a secure and minimalist OS build specially adapted for running container workloads, plus the crucial task of container grouping. For proper layered protection and convenient management, running containers need to be grouped on hosts so that information with certain security requirements is processed separately from information with lower security requirements. The implementation here depends on the orchestrator in use, but in any case, it’s primarily an exercise in risk assessment and threat modeling.

Generally, there are numerous container protection tasks, and trying to solve each of them in isolation, with one’s own tools or a manual configuration, would cause costs to soar. Hence, medium and large container environments require holistic security solutions that are deeply integrated with the containerization platform, the CI/CD pipeline, and the information security tools used in the company.

The job of information security experts is simplified by: integration with SIEM and channels for notifying about issues detected; regular automatic scanning of all images against an updated vulnerability database (such as NVD); functionality for temporary acceptance of information security risks; and detailed logging of administrative events in the containerization environment-protection system.

How Kaspersky Container Security implements protection

Our comprehensive solution protects container infrastructure by design: its components secure the entire lifecycle of containerized applications — from development to day-to-day operation. The dedicated scanner works with container images and provides static protection; the KCS agent running as a separate container under orchestrator control protects hosts in the runtime environment and the orchestration environment as a whole. Concurrently, the central component of Kaspersky Container Security integrates these parts and provides a management interface.

The high-performance platform offers robust protection for K8s clusters with hundreds of nodes.

The first version of Kaspersky Container Security, which implements core protection for container environments, is already available. And we are committed to developing the product and extending its functionality going forward.