...

1. A new device state,  “Unknown Update Detected” (Abbreviated as UUD) will be introduced in EVC, to notify that measurements of software running on a given device didn't match expected measurements.
2. EVC maintains a central database of all the supported EVE software images, and their hash values, indexed with the EVE image version tag
3. EVC also maintains a central database of mostly used BIOS firmware images, their signatures, and the certificate provided by the BIOS vendor for validating the signatures, indexed by a combined tag of BIOS version string + Manufacturer
4. On receiving attestation request EVE,  attestation service module checks PCR Quote against the baseline value. If there is a change, the EVE node is marked as “UUD”.  When there is no software change, the PCR quote is expected to be the same across reboots. However, after an EVE software upgrade, it is expected that the baseline value will change for the PCR values. However, after comparing the reported values with the expected values(since EVC knows about the new image version and its hash values), EVC makes a decision: If the reported values match, the baseline for the EVE node is updated. If they don’t, EVE node is marked as UUD. 
5. An optional feature, "Location-Lock" may be introduced in EVC, to additionally check the Geo-Location reported by device, and flag if the location has changed since its last-seen location. This feature may be critical in some deployments, where a given EVE node may be mounted permanently, and any change in its physical location should be flagged, and optionally considered along with software measurements to conclude if the EVE node is trusted. In this regard, another new device flag, “Unknown Movement Detected” (Abbreviated as UMD) will be introduced in EVC, to indicate that a change in the location of the device has been detected.  
6. If EVE node is not in “UUD" or "UMD" state, EVC includes all the latest configuration in reply to config request from EVE.
7. If edge-node is in “UUD” or "UMD" state, any config request from EVE will be responded with response code 403 - Forbidden. This is to protect any new sensitive  images/credentials from getting exposed to the compromised device. The response from EVC in such cases will carry an error code to indicate that there was an attestation failure, and hence partial configuration is being sent.  EVE, upon receiving such error codes MUST schedule re-attestation immediately.
8. If EVE reboots with a different software image from the configured version, EVC should be able to detect as quickly as possible and force attestation. To this effect a new token is introduced, called "Integrity Token". This token is a random nonce that will be stored in EVE under an encrypted folder. the Volume Storage Key to decrypt this folder will be sealed against the TPM PCRsin process memory, which is valid only for the current boot session.  Thus the key property of this Integrity Token is that it clears up automatically after every reboot.  Therefore, if EVE reboots with a different software, unsealing of this key will fail, and hence the new software will not be able to recover the Integrity Token inside it. , it will automatically cause re-attestation, since there will not be any Integrity-Token to present to the Controller. This token is sent during attestation requests, and if the attestation is successful, the provided token value is sealed against PCR values. This is usually implemented by storing the token inside an encrypted folder, with the Volume Storage Key protected by TPM with Seal operation.
9. Every configuration request from EVE will include this Integrity Token. If there is a token mismatch, HTTP code 403 - Forbidden will be sent to the device as the response, indicating that device should do re-attestation. 
10. Frequency of attestation: EVE node will be required to periodically attest itself via attestation requests. It is up to the EVC to schedule the frequency of attestation. Whenever EVC replies with Error 403, EVE MUST re-attest.  EVC can trigger this either periodically (say every few hours) or when it sees any discrepancy in the Integrity-Token.

...

Updated EVE Startup State Transition

Image RemovedImage Added

Fig 2. State Diagram for EVE Startup, with Attestation enabled

...

1. Fetches certificates used by EVC. e.g. X.509 certificate chain used in signing AuthContainers
2. Tries to retrieve UUID provisioned in the EVC for this EVE instance
3. Attempts to unseal the key in TPM. If unsuccessful, retries in regular intervals
4. Sends Attest request periodic intervals, containing new Integrity-Token value, PCR Quote, Event Log and Image version
5. Whenever attestation response is successful, opens the vault and reseals the key with new PCR values, and populates the Integrity-Token in a memory mapped file (not on the disk)
6. Configuration is requested in periodic intervals, with the Integrity-Token(or without that).
7. If config response contains attestation error, triggers attestation request immediately.

...

Module Level Interaction - Startup Sequence (Reboot with no change) - No EVC connectivity required 

Image AddedImage Removed

Fig 3. Module Level Interaction in EVE - Startup Sequence (Reboot with no change; works without EVC connectivity)

...

Module Level Interaction - EVE Startup Sequence (Reboot with a change)

Image RemovedImage Added

Fig 4. Module Level Interaction in EVE - Startup Sequence (Reboot with differing PCR values)

1. Device-steps starts client.go (the provisioning client) which will check and do the following:
   1. If certificates from EVC are not yet fetched,  fetches them 
   2. Retrieves UUID from EVC
2. Device-steps starts Vault Mgr
   1. Vault Mgr tries to retrieve the master decryption key from TPM with Unseal operation, and Unseal operation fails since the PCR values have changed
   2. Publishes vault status (as "locked")
   3. Waits for Integrity-Token and/or key for the encrypted volume Waits for Volume Storage Key from EVC - it will block here forever
3. Device-steps starts TPM mgr
   1. TPM manger retrieves the attestation certificate and publishes to Zedagent
   2. Waits for Quote requests on pubsub channel from Zedagent
4. Device-steps starts Zedagent (and Zedagent starts 3 concurrent tasks: attest, info and config)
5. Device-steps starts Edge App microservices - e.g. Domain Mgr 
   
   1. Attest task requests for a nonce from EVC (to prepare PCR quote)
   2. Attest task sends the nonce back to TPM Mgr and waits for PCR quote
   3. Attest task, once notified about the quote readiness, creates a random nonce value for Integrity-Token
   4. Attest task sends { Quote, Location, Event Log, Integrity-Token, Image Version } to EVC
   5. EVC, since the quote is different, tries to compare EventLog entries with its known hashes against the version reported. Since the PCR values are different, there will not be match, unless Admin has uploaded measurements for this new version. Assuming that Admin is yet to upload, attestation request will fail here.
6. EVC sends an error back to EVE, to retry attestation.
7. In the mean time, configuration task keeps requesting config from EVC (expected to fail till attestation goes through).
   1. EVC replies to configuration request with HTTP Error Code 403 - Forbidden. Indicates attestation failure (due to no or invalid Integrity Token)
   2. Config task communicates to Attest Task to re-trigger attestation
8. Admin uploads measurements for the new image version
9. The next attestation request is matched successfully against the uploaded measurements
10. EVC approves the attestation request, and responds back with the Volume Storage Key (encrypted with TPM key) for the EVE node, and also saves the caches Integrity-Token supplied in the attestation request in a memory mapped file
11. Vault Mgr 
    1. picks up the Volume Storage Key
    2. unlocks vault
    3. populates Integrity-Token approved by EVC inside /persist/vault/itokena memory mapped file
    4. Publishes new vault status (unlocked) to all the microservices
12. The next configuration request picks up the new Integrity Token, and sends configuration request along with this Integrity-Token
13. EVC verifies that Integrity-Token matches with its copy, and approves configuration request and responds with the latest configuration.
14. Domain Mgr notices the new vault status (unlocked) from Vault Mgr , and latest configuration from ZedAgent, and starts the Edge Apps
15. Info task reports the encrypted key for the encrypted volume back to EVC, for backup purposes

Module Level Interaction - EVE Startup Sequence (Initial Onboarding)

Image RemovedImage Added

Fig 5. Module Level Interaction in EVE - Startup Sequence (Onboarding)

1. Device-steps starts client.go (the provisioning client) which will check and do the following:
   1. First fetches EVC Certificates
   2. Registers the device certificate
   3. Retrieves UUID from EVC
2. Device-steps starts Vault Mgr
   1. Vault Mgr creates new vault: /persist/vault
   2. Seals the Volume Storage Key into TPM with PCR values
   3. Publishes vault status (unlocked)
   4. Waits for Integrity-Token from zedAgent
   5. Once received, copies the new Integrity-Token to /persist/vault/itoken file
3. Device-steps starts TPM mgr
   1. TPM manger retrieves the attestation certificate and publishes to Zedagent
   2. Waits for Quote requests on pubsub channel from Zedagent
4. Device-steps starts Zedagent (and Zedagent starts 3 concurrent tasks: attest, info and config)
5. Attest task,
   1. picks up the attestation certificate and publishes to EVC
   2. requests for a nonce from EVC (to prepare PCR quote)
   3. sends the nonce back to TPM Mgr and waits for PCR quote
   4. Once notified about the quote readiness, creates a random nonce value for Integrity-Token
   5. Sends { Quote, Location, Event Log, Integrity-Token, Image Version } to EVC
6. EVC, if key for the encrypted volume is not found for the EVE, approves the PCR quote as the baseline
1. If Location-Lock is enabled, location is also approved as baseline (provided the EVE device is not in manufacturing account)
7. In the mean time, configuration task keeps requesting config from EVC (expected to fail till attestation goes through)
8. Attest task, once attestation request is approved, gets the Volume Storage Key and Integrity-Token and passes it to Vault Mgr
9. Attest task, also caches the Integrity-Token acknowledged by EVC in a memory mapped file
10. Config task now picks up the right Integrity-Token and Config request is now accepted by EVC, and full configuration is sent back
11. Info task reports the encrypted key for the encrypted volume back to EVC, for backup purposes

...

1. EVE encrypts the Volume Storage Key using a TPM-based key, and sends it to EVC. EVE can send this information when the Volume Storage Key is created for the first time, and also can include this in the periodic info message. Sending this in periodic info message also provides additional facility of rotating the key if required.
2. After the upgrade, the EVE software presents the new Event Log, along with the PCR Quote. As mentioned earlier, the PCR Extend operations are recorded in a table called TPM Event Log Table.  This is sometimes also called the Boot Log.
3. First, EVC repeats the transactions mentioned in the Event log, and the final PCR as computed from the Event Log should match the PCR Quote. If they don’t match, then Event Log can’t be trusted(say it was manipulated), and the EVE node is marked as UUD.  Please note: Event Log is stored in system memory and PCR Quote is generated by TPM. So PCR quote is the source of truth, pinned to HRoT.
4. Secondly, if PCR Quote matches the Event Log, then Event Log entries are compared between old Event Log and the new Event Log, and the differing entries are extracted
5. EVC maintains a central database of all the supported EVE software images, and their hash values, indexed with the EVE image version tag
6. EVC also maintains a central database of all the BIOS firmware images, their signatures, and the certificate provided by the BIOS vendor for validating the signatures, indexed by a combined tag of BIOS version string + Manufacturer 
7. The differing values are compared against acceptable values stored against the given image version. Additionally, If admin has enabled “Location-Lock” feature, additionally the geo-location reported by the device is checked as well. The Geo-location reported by the device can be trusted if the software state (established through PCR quote) can be trusted.
8. If the PCR quote and the optional Geo-Location check pass, the encrypted key for the encrypted volume is given back to the EVE node. EVE decrypts the Volume Storage Key, and unlocks the vault with the Volume Storage Key. the Volume Storage Key is now sealed into the TPM, against the current PCR values.
9. User will have an option to override the attestation decision taken by EVC, through a configuration in EVC portal, against the EVE node

Image RemovedImage Added

Fig 76.  Overview of flow of events during Secure Software Update

...

1. BIOS/Firmware updates can be done either manually (say a person inserts USB drive with new firmware and updates it out-of-band, and reboots the device with new firmware) or can be done remotely (updates driven through remote management framework, without a physical access to the EVE node). In both the cases, the new firmware version needs to be validated by EVC, before approving the new software state.
2. In case of remote updates, firmware Upgrades will be driven through EVC. This is to make sure EVC has the access to the firmware image, the measurement values to expect from EVE node for this firmware version etc.
3. To this effect, EVC would maintain a central database of all the BIOS firmware images, their signatures, and the certificate provided by the BIOS vendor for validating the signatures, indexed by a combined tag of BIOS version string + Manufacturer 
4. EVC will communicate the upgrade to a dedicated agent for this purpose on EVE.
5. The Firmware upgrade agent can either be a EVE service or can be a service app deployed on EVE
6. The agent will have a platform abstraction layer and a platform dependent layer. 
7. The platform abstraction layer drives the overall logic of downloading the firmware image, validating the image using manufacturer certificates, and reporting the status back to EVC
8. The platform dependent layer will be responsible for the actual firmware upgrade, by talking to any special component on the EVE hardware, like BMC
9. To address out-of-band update of firmware, there will be an option provided in the EVC portal, where the admin (after the upgrade is done), can configure EVC to accept the new firmware as trusted. 
10. If this is a general update across many devices, admin can also configure EVC database to feed in the new hash values to expect from the new firmware version. Please note, this is almost the same as remote update workflow w.r.t. validating the new version, except that the image update is done out-of-band

Fig 87.  Firmware Upgrade Management

...