The Power of Lazy Loading in Container Images
INTRODUCTION
Containerization has revolutionized the way we develop, deploy, and manage applications. However, as containers gain popularity, the need for efficient image management and faster container startup times becomes crucial. Let’s explore the concept of lazy loading in container images.
Let’s have a look at the container image inside:
The process of creating a Docker image begins with a Dockerfile. This file serves as a blueprint that contains instructions on how to build the image. It includes directives such as defining the base image, copying files, running commands, configuring the environment, and more.
Dockerfile
FROM python 3.8
WORKDIR /app
COPY deps. Txt deps.txt
RUN pip3 install -r deps. Txt
COPY ..
CMD ["python3", "=m", "flask", "run", " - host=0.0.0.0"]The first layer of a Docker image is the base image. It is specified in the Dockerfile using the FROM directive. The base image serves as the starting point for the subsequent layers. Docker provides a wide range of official base images for different operating systems and programming languages, such as Ubuntu, Alpine, Python, Node.js, etc.
Following the base image, each instruction in the Dockerfile represents a new layer in the image. Common instructions include COPY, ADD, RUN, WORKDIR, EXPOSE, ENV, and more. Let’s look at a few key instructions and their effects on the layers:
- COPY and ADD: These instructions copy files and directories from the host machine into the image. Each COPY or ADD command creates a new layer for the copied files.
- RUN: The RUN instruction executes commands within the image during the build process. Each RUN command creates a new layer with the changes made by the command.
- ENV: The ENV instruction sets environment variables within the image.
- EXPOSE: The EXPOSE instruction specifies the ports that should be exposed by the container at runtime.
A container image is built of layers, referred to as docker images or OCI images. OCI is Open Container Initiative, it’s essentially the open-source version of the docker format.
Layer Caching:
Docker employs layer caching to optimize the build process. When a Dockerfile is built, each instruction is processed sequentially, and Docker checks if any previous layers with the same instructions exist in the local cache. If a layer is already present, Docker reuses it, avoiding the need to rebuild that particular layer. This caching mechanism speeds up subsequent builds by only rebuilding the layers that have changed.
Intermediate Containers:
During the build process, Docker creates intermediate containers for each instruction in the Dockerfile. These containers are temporary and are used to execute instructions and capture the changes made in each layer. Once an instruction is processed, the changes are committed, and a new layer is added to the image.
Compressed Tarball:
Once all the instructions in the Dockerfile are executed, the resulting layers are combined into a single compressed tarball. This tarball represents the final Docker image. It contains all the layers, along with the metadata and configuration information.
It’s important to note that each layer in a Docker image is read-only, meaning it cannot be modified. When a container is created from an image, a new read-write layer, known as the container layer, is added on top of the image layers. This container layer allows changes and modifications specific to the running container, such as writing files, installing packages, and storing runtime data.
Container build-time
The snapshotter also plays an important role of container runtime.
Linux Overlay Filesystem
The Linux Overlay Filesystem is essential for the functioning of container layering schemes. It enables the merging of disparate directories that already exist on the disk into a unified filesystem. Instead of duplicating the directories, the Overlay FS creates a new filesystem that is backed by the original files. This new filesystem possesses all the capabilities of a typical filesystem, such as being mountable in specific locations. For example, it can be mounted as the root filesystem inside a container.
Image Repository
The build process we have discussed usually takes place within a CI/CD pipeline, which incorporates integration tests, vulnerability scans, cryptographically secure image signing, and other related tasks. The final stage of the pipeline involves deploying the image to an image registry.
An image registry serves as a repository for storing collections of images. Images are pushed to the repository so that the runtime system can later pull them into a production environment. Typically, alongside the image, the image signatures are also pushed to the repository. These signatures can be used by the runtime system to verify that the image has not been altered or tampered with.
Now, let’s delve into the specific layers of the Dockerfile. Our example Dockerfile results in an image consisting of 11 layers. Out of these layers, 8 are inherited from the Python base image, while the remaining three are derived from the “copy” and “pip” commands that were executed during the build process.
Let’s shift our focus from build time to runtime.
While Docker images provide many benefits for application deployment, there are also some drawbacks to consider when aiming for fast deployment of applications. These drawbacks include:
- Image Size: Docker images can become quite large, especially when using base images that include unnecessary dependencies or libraries. The larger the image, the longer it takes to transfer over the network, which can impact deployment speed.
- Image Pulling: When deploying containers, Docker needs to pull the required images from a registry. This process involves downloading and transferring multiple layers, which can take time, particularly if the network connection is slow or if there are many images to pull.
- Startup Latency: Starting a container involves several steps, such as image pulling, untarring layers, and setting up the container environment. These operations can introduce latency, especially for large images with many layers. This latency can impact the responsiveness of the application during the startup phase.
- Build Time: Building a Docker image often involves multiple steps, such as installing dependencies, copying files, and configuring the environment. If the image build process is time-consuming, it can slow down the overall deployment process, especially when frequent builds are required.
- Image Versioning: Docker images are versioned based on tags or digests. When deploying containers, specifying a specific image version ensures consistency. However, managing image versions and ensuring that the correct version is used for deployment can be challenging, particularly in complex deployment scenarios.
- Image Updates: Keeping Docker images up to date with security patches and application updates is crucial. However, updating images can require pulling the latest version, which may involve transferring a significant amount of data. This process can add extra time to the deployment process, especially if updates are frequent.
- Image Compatibility: Docker images are platform-specific. An image built for one platform may not work on another, requiring separate images for different environments. Managing and deploying multiple images for different platforms can increase complexity and potentially slow down the deployment process.
Lets look at some potential solutions to speed up the container deployment:
1. Caching: If the primary cause of latency is the network hop to the remote repository (which it often is), reducing the latency can be achieved by utilizing caching. However, setting up and fine-tuning the cache can be complex and potentially costly. Additionally, caching only helps when there is a cache hit, so it does not alleviate the latency during initial container launches or infrequent launches. Moreover, caching does not eliminate the overhead of uncompressing and untarring the data on the runtime host.
2. Distro-less (FROM scratch): Another approach that gained popularity for a while was going “distro-less.” Base images tend to be large, such as the Python base image being over 900MB. Since popular base images are shared by numerous containers, they often contain unnecessary components that may not be required by a specific application. Going distro-less involves rewriting the Dockerfile to exclude any base image altogether and manually adding each dependency explicitly. This approach ensures that the resulting image only contains the essential components needed for the application. However, it introduces a reliability risk as it is easy to forget a dependency in subsequent releases. While distro-less can slim down the container image to its core essentials, it makes the overall system more fragile and requires ongoing maintenance.
3. Minimal base images: A more cautious approach is to use minimal base images. Instead of starting with a bulky Ubuntu base image, which can exceed 100MB, you can opt for something like Alpine, which weighs in at a mere 5MB or exactly Alpine. Minimal base images offer advantages and are often preferred for various reasons. However, they share some of the risks associated with the distro-less approach, and availability of a minimal base image may vary depending on your specific environment.
All the methods discussed thus far have shifted the workload to the customer. However, there is another concept in the container world that has gained significant attention: lazily loading the image instead of pulling the entire image upfront. Returning to the concept of snapshots, this approach aims to minimize the process of pulling, uncompressing, and untarring the image.
The concept of lazy loading involves launching the container immediately without fetching the entire data upfront. Instead, the data is retrieved on-demand when and if requested by the application.
Several lazy loaders exist for containers, and they all encounter two common challenges. It is widely accepted that tarballs cannot be randomly accessed, and Gzip files lack seekability. So, the question arises: how can we lazily load data from a format that inherently requires sequential reading from start to end? The existing solutions address this by converting the standard container image into a more easily lazily loadable format. Some approaches flatten the layers into a single filesystem image, while others convert the image into a block format. There are also some solutions that establish an entirely new container ecosystem rather than just modifying the snapshot format. However, all of these methods involve a conversion step during the build process.
Among these lazy loaders, StarGZ stands out as the most popular and widely discussed choice in the container community today. (https://github.com/containerd/stargz-snapshotter)
At build time, StarGZ uncompresses the tarball and then recompresses each file individually.
The process involves packaging the individual files into tar format again, but with the addition of an index at the end. This index contains the offsets of each compressed file within the container. During container launch, the snapshotter reads only the index. When the application requests access to a specific file, it retrieves that file from the remote repository, uncompresses it, and proceeds with its operations. Remarkably, the repacked file maintains backward compatibility. The GZIP format allows for concatenating multiple GZIP files together, which aligns with the approach employed by StarGZ. Consequently, the resulting file remains a valid GZIP file. This means that the StarGZ layer can still function as a traditional Docker layer, even in container runtimes that lack knowledge about lazy loading. These runtimes may perceive an additional file (the index), but the application can simply disregard it. Optionally, StarGZ can analyze container execution to identify which files are accessed during startup, allowing for profiling and optimization purposes.
As part of the repacking process, the files are gathered together to enable convenient and efficient prefetching through a single GET request.
Challenges:
· Size: Compression algorithms typically perform better on larger files, but the majority of files in existence are small. It is necessary to address the loss of compression efficiency when dealing with small files.
· Build-time: Customers prefer to avoid modifying their existing build processes.
· Security: Image conversion poses a challenge to image signing. As containers increasingly become the standard for application distribution and deployment, security is a significant and growing concern. The image signing process relies on the cryptographic hash of the entire image to uniquely identify it. Altering a single byte in a single layer result in a change in the hash of the entire image. Converting the image alters its hash, effectively invalidating all the associated signatures.
· Repo Management: Maintaining two copies of each image proves to be a challenging task.
· Prefetching: Determining what files to prefetch is dependent on the workload, rather than the layer itself. While the StarGZ approach of moving the prefetch files to the front of the layer is clever, it does not necessarily align with how customers utilize containers.
Amazon introduces SOCI Snapshotter (Seekable OCI (Open Container Initiative) to mitigate the above challenges. https://github.com/awslabs/soci-snapshotter
SOCI addresses these issues by loading from the original, unmodified OCI image. Instead of converting the image, it builds a separate index artifact (the “SOCI index”), which lives in the remote registry, right next to the image itself. At container launch time, SOCI Snapshotter queries the registry for the presence of the SOCI index
Goals of SOCI:
· Use the OCI image without modification
· Eliminate build friction
· Optimize the workload, not the image
Conclusion:
Lazy loading of container images offers an effective solution to mitigate startup latency and optimize resource utilization. By selectively retrieving data on-demand, containerized applications can achieve faster startup times and enhanced scalability. StarGZ and lazy loading paved the way for future advancements in container image management, promoting a more efficient and streamlined containerization ecosystem.
