Motivation

EVE system has been built with security at the core of its design. One of its SECURITY principles is that, EVE should be trustworthy, and it should provide a deterministic way to measure its software layer, right from the firmware, all through bootloader, kernel and user-space applications.  It should also provide a mechanism to report these measurements to a third-party for attestation. This is to provide a verified software environment to launch user applications, i.e. the Edge Container Objects. Since any software can be potentially modified, such measurement architectures typically use a hardware based root of trust (HRoT). Here we present a solution based on Trusted Platform Module as the Hardware Root of Trust.

Trusted Platform Module as the Silicon RoT

Trusted Platform Module(TPM) supports many crypto functions, and notably the “PCR Extend” and “Seal” operations are used to build this solution. 

TPM can be asked to perform a “PCR Extend” command,  where a particular hash value would be added to the existing hash value in a Platform Configuration Register(PCR), and the resultant hash value can be stored back in the same PCR. i.e.

PCR_Content = PCR_Content || New_Measurement

One can only extend the current value in the PCR with a new hash value, and the existing contents can not be overwritten. This provides us these key capabilities:

1. The order of measurements - Final value will be the same if and only if the measurements are done in the same order
2. Final PCR value captures the whole history of measurements - useful in quick validation of final states against expected state
3. Deterministic - If one repeats the same history of measurements, he will end up in the same final PCR value - Useful in validation of a change in one of the input sequences

TPM can seal a given secret information against the current set of PCR values, through a TPM command called “Seal”.  Once sealed, the information can be read back only through an unseal command, which will succeed only if those PCRs hold the same set of values as they were during the sealing operation. In other words, if those PCR values aren’t the same, the secret can not be recovered. 

Using these two TPM capabilities, one can build powerful solutions to measure and validate the software state of a given system. The measuring process is called Measured Boot, and the method of getting the measurements verified and attested through a third-party is called Remote Attestation 

Measured Boot is a method where each of the software layers in the booting sequence of the EVE node, measures the layer above that, and extends the value in a designated PCR. e.g. BIOS measures various components of Bootloader and stores these values in PCRs 0-7. Likewise, bootloaders can measure the Linux kernel and store the measurements in PCRs 8-15. The Linux kernel has a feature called Integrity Measurement Architecture (IMA), where various kernel executables/drivers can be measured and stored in PCR 10.

During these extend operations, the extend operations are recorded by BIOS and Bootloader, in a special firmware table, called the TPM Eventlog table, and this table is handed over to the operating system during OS takeover. By playing the same sequence of extend operations recorded in a given TPM Event Log, one can check if the final PCR values match, and if so, then the Event Log(and hence the software layers) can be trusted. 

Based on the above constructs, we present a solution to measure and attest software integrity of EVE node. Just for recap, EVE is the open-source software from LF-Edge for Edge Virtualization, running on IoT Edge gateways. EVC is the controller for managing these EVE instances.  ZedCloud is a commercial offering of one such EVC implementation from ZEDEDA.  In this context, the EVC is the attesting authority and EVE reports its measurements for attestation. 

Fig1. Role of ZedCloud as the Remote Attestation Server

Solution Overview

1. A new device state,  “Unknown Update Detected” (Abbreviated as UUD) will be introduced in EVC, to indicate to in EVC that software running on a given device is not one of the intended/certified images
2. An optional feature, "Location-Lock" may be introduced in EVC, to additionally check the Geo-Location reported by device, and flag if the location has changed since its last-seen location. This feature may be critical in some deployments, where a given EVE node may be mounted permanently, and any change in its physical location should be flagged, and optionally considered along with software measurements to conclude if the EVE node is trusted. In this regard, another new device flag, “Unknown Movement Detected” (Abbreviated as UMD) will be introduced in EVC, to indicate that a change in the location of the device has been detected.  
3. EVC maintains a central database of all the supported EVE software images, and their hash values, indexed with the EVE image version tag
4. EVC also maintains a central database of mostly used BIOS firmware images, their signatures, and the certificate provided by the BIOS vendor for validating the signatures, indexed by a combined tag of BIOS version string + Manufacturer 
5. On every reboot, EVE sends PCR quote to EVC, along with optional Geo-Location information, along with its EVE image version and BIOS firmware version.
6. After normal reboot operation of EVE, EVC checks PCR Quote against the baseline value. If there is a change, the edge-node is marked as “UUD”.  If "Location-Lock" feature is enabled and the location of the EVE node has changed, EVE node is marked "UMD"
7. After EVE software upgrade, it is expected that the baseline value will change for the PCR values. However, after comparing the reported values with the expected values(since EVC knows about the new image version and its hash values), EVC makes a decision: If the reported values match, the baseline for the EVE node is updated. If they don’t, it is a UUD detection.  Optionally, if administrator enables “Location-Lock” feature,  Geo-Location of the device will also be included in the checks, before sending the latest configuration.
8. If edge-node is in “UUD” or "UMD" state, only the “last-seen” EVE configuration, along with the latest baseOS configuration is sent as config request response, all other new configuration items will be omitted in config response. This is to protect any new sensitive  images/credentials from getting exposed to the compromised device. Here "last-seen" refers to the configuration last received by EVE from EVC.
9. If EVE node is not in “UUD" or "UMD" state, all the latest configuration items are included. 
10. Frequency of attestation: EVE node will be required to periodically attest itself via attestation requests. The interval is fixed to be every 4 hours, or whenever the device connects back to the cloud again, or after the next reboot, whichever is earlier. This 4 hours is a tentative interval, and can be changed based on deployment requirements.
11. If EVE reboots with a different software image from the configured version, EVC should be able to detect as quickly as possible and force attestation. To this effect a new token is introduced, called "Integrity Token". This token is a random nonce that will be stored in EVE under an encrypted folder. The master-key to decrypt this folder will be sealed against the TPM PCRs.  Therefore, if EVE reboots with a different software, unsealing of this key will fail, and hence the new software will not be able to recover the Integrity Token inside it.
12. This token is sent during attestation requests, and if the attestation is successful, the provided token value is locked against the EVE node. 
13. Every configuration request from EVE will include this Integrity Token. If there is a token mismatch, HTTP code 401 - Unauthorized will be sent to the device as the response, indicating that device should do re-attestation. 

Fig 2. Remote Attestation Flow in EVC

Protecting App Instances using TPM Sealing

All the app images and container volumes would be inside the encrypted folder (/persist/vault), with the master key sealed inside TPM, against a set of PCR values. Without access to this master-key, the encrypted folders can not be accessed.

If EVE or BIOS image was changed during boot, then unsealing will fail, which will implicitly disable all the apps from getting launched, since all the app images and container volumes would be inside the encrypted folder (/persist/vault), with the master key sealed inside TPM. 

To provide some flexibility to the EVE node administrator, when deploying app instances on the EVE platform, there will be 3  security modes for app deployment:

1. No-Lock Mode - app images will not be in encrypted folder (not recommended, unless there is a compelling need to run any app instance unconditionally e.g. service apps like SD-WAN, DHCP Server)
2. Software Lock Mode - app images will be in encrypted folder - with key from TPM, sealed with PCRs (the default option)
3. Software Lock + Location Lock Mode  - app images will be encrypted folder - with key from TPM, sealed with PCRS + a key provided after Geo-Location check from the EVC.  This will work only when device is connected to EVC

Offline vs. Online checks

With the approach to seal the key(s) needed for fscrypt under the TPM, this means that if the PCR values have not changed (e.g., a normal reboot, or a reboot due to a power outage) the EVE node will be able to unseal those keys and start the app instances without any controller connectivity.

After a BIOS and/or EVE update, the PCR values will presumably change so the unsealing will fail. In that case the EVE node needs to perform remote attestation with the controller (with automatic and/or user acceptance of the new measurements) before it can retrieve the key from the controller. Note that as part of EVE update we already require this connectivity to verify that the update is fully working. Once retrieved the EVE node will again seal that key under the current PCR values.

This means that the controller will need to store the fscrypt key(s) so it can hand it back to the EVE node. But we should be able to encrypt them under the EVE node private key so that even if the controller is compromised the fscrypt keys can not be extracted from it.

The offline approach means that if there is something else wrong, such as IP geolocation doesn’t match, the applications are likely to start before the controller can raise a flag and potentially disable the applications.

And since this is measured boot it means that if some firmware or EVE component has been compromised, EVE will still boot and attempt remote attestation. Such a compromised device might be running some root-kit which can access the adapters on the EVE node, but applications will not start. Thus this implicitly assumes that the security of connected systems is based on credentials stored in the application instances and not merely by having the physical connectivity to some serial port or Ethernet network.

Establishment of Baseline

1. To establish a baseline, the following is proposed:
   EVE device is on-boarded in a trusted environment, usually in a manufacturing/supply chain setup (or Intel SDO)
2. When EVE  is on-boarded for the first time, during device-certificate registration, the PCR values coming out of the EVE node will be taken as the baseline for that device. Any change in the PCR value (other than during an EVC-driven upgrade, explained below)  after will be flagged as UUD, and unless the user marks it as legal/accepted, the EVE node will remain in the UUD state.
3.  If the enterprise account is a manufacturing account, then geo-location will be exempted in calculating the baseline.
4. When the EVE node comes online for the first time in a non-manufacturing enterprise, geo-location of the EVE node will be taken and will be frozen as the baseline value for geo-location. 
5. In case of mobile gateways, there will be an option to turn-off Geo-Fencing(i.e. Locking of Geo location)

Securely Upgrading EVE Software

When EVE goes through a software upgrade, the PCR values might change depending on what is new in the upgraded version. However, during the upgrade window, an attacker might boot a different software as well, and try to present the compromised software as the upgraded version of EVE. Therefore, it is important that we measure the boot sequence after the upgrade and validate that the new software is indeed the new image that was pushed by the EVC. This is done through the following steps:

1. EVE encrypts the master-key of the vault using a TPM-based key, and sends it to EVC. EVE can send this information when the master-key is created for the first time, and also can include this in the periodic info message. Sending this in periodic info message also provides additional facility of rotating the key if required.
2. After the upgrade, the EVE software presents the new Event Log, along with the PCR Quote. As mentioned earlier, the PCR Extend operations are recorded in a table called TPM Event Log Table.  This is sometimes also called the Boot Log.
3. First, EVC repeats the transactions mentioned in the Event log, and the final PCR as computed from the Event Log should match the PCR Quote. If they don’t match, then Event Log can’t be trusted(say it was manipulated), and the EVE node is marked as UUD.  Please note: Event Log is stored in system memory and PCR Quote is generated by TPM. So PCR quote is the source of truth, pinned to HRoT.
4. Secondly, if PCR Quote matches the Event Log, then Event Log entries are compared between old Event Log and the new Event Log, and the differing entries are extracted
5. EVC maintains a central database of all the supported EVE software images, and their hash values, indexed with the EVE image version tag
6. EVC also maintains a central database of all the BIOS firmware images, their signatures, and the certificate provided by the BIOS vendor for validating the signatures, indexed by a combined tag of BIOS version string + Manufacturer 
7. The differing values are compared against acceptable values stored against the given image version. Additionally, If admin has enabled “Location-Lock” feature, additionally the geo-location reported by the device is checked as well. The Geo-location reported by the device can be trusted if the software state (established through PCR quote) can be trusted.
8. If the PCR quote and the optional Geo-Location check pass, the encrypted master-key is given back to the EVE node. EVE decrypts the master-key, and unlocks the vault with the master-key. The master-key is now sealed into the TPM, against the current PCR values.
9. User will have an option to override the attestation decision taken by EVC, through a configuration in EVC portal, against the EVE node

Fig 3.  Flow of events during Secure Software Update

Managing Firmware Upgrades

While the workflow is almost the same for securely upgrading firmware on EVE, the following additional steps are required:

1. BIOS/Firmware updates can be done either manually (say a person inserts USB drive with new firmware and updates it out-of-band, and reboots the device with new firmware) or can be done remotely (updates driven through remote management framework, without a physical access to the EVE node). In both the cases, the new firmware version needs to be validated by EVC, before approving the new software state.
2. In case of remote updates, firmware Upgrades will be driven through EVC. This is to make sure EVC has the access to the firmware image, the measurement values to expect from EVE node for this firmware version etc.
3. To this effect, EVC would maintain a central database of all the BIOS firmware images, their signatures, and the certificate provided by the BIOS vendor for validating the signatures, indexed by a combined tag of BIOS version string + Manufacturer 
4. EVC will communicate the upgrade to a dedicated agent for this purpose on EVE.
5. The Firmware upgrade agent can either be a EVE service or can be a service app deployed on EVE
6. The agent will have a platform abstraction layer and a platform dependent layer. 
7. The platform abstraction layer drives the overall logic of downloading the firmware image, validating the image using manufacturer certificates, and reporting the status back to EVC
8. The platform dependent layer will be responsible for the actual firmware upgrade, by talking to any special component on the EVE hardware, like BMC
9. To address out-of-band update of firmware, there will be an option provided in the EVC portal, where the admin (after the upgrade is done), can configure EVC to accept the new firmware as trusted. 
10. If this is a general update across many devices, admin can also configure EVC database to feed in the new hash values to expect from the new firmware version. Please note, this is almost the same as remote update workflow w.r.t. validating the new version, except that the image update is done out-of-band

Fig 4.  Firmware Upgrade Management