A small group of companies (HP, IBM, Ingadi Flower Farm, Seeed, SoftServe Inc.) came together late in 2020 to demonstrate how edge computing solutions could improve agriculture. One of the published core beliefs of the group is "with sufficient sensors and data, you should be able to apply the optimum resources to grow crops to their maximum yield". This Special Interest Group (SIG) is hosted by the Open Horizon open-source software project that lives within LF Edge, part of the Linux Foundation. All meetings are open to the public, and meeting recordings and presentation materials are published on the SIG's wiki page.
The initial effort of the group will be to grow a variety of hot-weather crops in two hoop houses on a plot at the Ingadi Flower Farm in Chelsea, Alabama. Hoop houses are a row of curved support hoops covered with heavy plastic that can extend the growing season of crops in a location by weeks or months. As an experiment, the same crops will be grown in the same layout in adjacent 10' x 20' hoop houses. Both houses will contain identical sensors to collect soil and air data at one-minute intervals, and will be configured to send alerts when the soil moisture exceeds pre-configured thresholds. One will be watered based on the farmer's best judgement, and the other will be watered based on the collected data. When the crops are harvested, the yields will be compared to see which technique or approach was superior.
We intend to publish a Bill of Materials (BOM) for the hardware used, software developed, and data collected so that anyone can replicate the results should they wish to do so. Additionally, the data can be analyzed to see what additional learnings can be derived as well as what changes should be made to data collection for future growing plans. We also plan to publish detailed blog posts at the IBM Developer Blog for Open Horizon that will give step-by-step instructions on how to set up your own Smart Agriculture solution based on what we learn.
Please watch this space weekly for updates. And post any comments and questions below.
Introduction
This blog post is written for software developers who understand basic concepts about containerized applications (hereinafter referred to as containers), and are new to using Open Horizon to deliver edge computing services.
On a host machine, some tasks can only be performed by an account with root access. This means that the account you are currently logged in as is either the root account itself (generally not a good idea), or your account has acquired root-level privileges through `sudo`. Likewise, containers generally do not need privileged mode on the host: to be run as the root user or to have root-level access on the host computer.
In Open Horizon and all commercial distributions based on it, you have the ability to specify that a service should be deployed with privileged process execution enabled. By default, it is disabled. You must explicitly enable it in the respective Service Definition file for each container that needs to run in this mode. And further, any node on which you want to deploy that service must also explicitly allow privileged mode containers.
The reason for requiring the node policy file to explicitly enable privileged mode is because the node owner gets a say/vote in what runs on the node. This is the whole purpose of the node policy, to give the node owner agency in the decision about what runs there. If the service definition or one of its dependencies requires privileged mode, the node policy must also allow privileged mode, or else services will not be deployed to the node.
How does privileged process execution impact security?
A major security principle the Open Horizon project follows is: "All parties are untrusted by default." As a result, Node Policies and Service Definitions do not allow privileged process execution by default. You must explicitly enable it in both the node and the service if you want to deploy and run a service that requires it.
However, a privileged container is a powerful and potentially dangerous tool and should not be used without considering alternatives. If you run a container with privileged access, it can access all resources on the host system as the root user. If a privileged container can be hacked by a third party, that third party could then gain access to all resources on the host computer.
Therefore, try not to use privileged containers. If you must, use the following guidelines to ensure that privileged containers:
- are thoroughly and continuously vetted for vulnerabilities
- have a narrow scope for their duties ... meaning they should only perform a specific task
- only mount necessary host directories and devices (Specified in the Service Definition file. See the example at the bottom of this blog post.)
Why do we use it (what is it good for)?
With all of the potential drawbacks, what situations require a container to run as privileged?
- Does your code require direct access to host hardware? For example, you may need to use a microphone in order to record and analyze sound waves. You might need to use the host GPU for model (re)training. You could potentially need to access a video stream directly from an attached camera. In these situations, you should first try to bind mount the device to see if that approach is sufficient. Another approach is to use `cap-add` to add only the kernel capabilities that you specifically need. By way of contrast, privileged mode adds all of the kernel's `CAP_*` capabilities.
- Does your service need to spawn other containers? This is a common task in CI/CD pipelines. You may be able to spawn containers by bind mounting the docker daemon socket (normally /var/run/docker.sock) in systems that use docker. In systems that use podman, which has no daemon process, you may need elevated privileges. In any case, if your container can spawn other containers without restrictions then it can effectively run any code as root on the host.
When should you not use it?
If you need to access a file on the host with elevated permissions, try mounting the file into the container and ensuring that the container is running as a user that belongs to the same group as the file owner, with group read privileges enabled. Do not run the container as privileged just to read from or write to a file.
Can you show me an example?
The `audio2text` example service requires access to the microphone to record audio, and thus needs privileged access to the host hardware. View the Service Definition's deployment section to see how it is enabled.