Versions Compared

Old Version 31

changes.mady.by.user Hari C S

Saved on

compared with

New Version 32

changes.mady.by.user Hari C S

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Fig 1. Role of EVC as the Remote Attestation Server

...

Introduction to fundamental constructs used

  1. A new device state,  “Unknown Update Detected” (Abbreviated as UUD) will be introduced in EVC, to indicate to in EVC that notify that measurements of software running on a given device is not one of the intended/certified images
  2. An optional feature, "Location-Lock" may be introduced in EVC, to additionally check the Geo-Location reported by device, and flag if the location has changed since its last-seen location. This feature may be critical in some deployments, where a given EVE node may be mounted permanently, and any change in its physical location should be flagged, and optionally considered along with software measurements to conclude if the EVE node is trusted. In this regard, another new device flag, “Unknown Movement Detected” (Abbreviated as UMD) will be introduced in EVC, to indicate that a change in the location of the device has been detected.  
  3. EVC maintains a central database of all the supported EVE software images, and their hash values, indexed with the EVE image version tag
  4. EVC also maintains a central database of mostly used BIOS firmware images, their signatures, and the certificate provided by the BIOS vendor for validating the signatures, indexed by a combined tag of BIOS version string + Manufacturer 
  5. On every reboot, EVE sends PCR quote to EVC, along with optional Geo-Location information, along with its EVE image version and BIOS firmware version.
  6. After normal reboot operation of EVE, EVC checks PCR Quote against the baseline value. If there is a change, the edge-node is marked as “UUD”.  If "Location-Lock" feature is enabled and the location of the EVE node has changed, EVE node is marked "UMD"
  7. After EVE software upgrade, it is expected that the baseline value will change for the PCR values. However, after comparing the reported values with the expected values(since EVC knows about the new image version and its hash values), EVC makes a decision: If the reported values match, the baseline for the EVE node is updated. If they don’t, it is a UUD detection.  Optionally, if administrator enables “Location-Lock” feature,  Geo-Location of the device will also be included in the checks, before sending the latest configuration.
  8. If edge-node is in “UUD” or "UMD" state, any config request from EVE will be responded with response code 403 - Forbidden. This is to protect any new sensitive  images/credentials from getting exposed to the compromised device. The response from EVC in such cases will carry an error code to indicate that there was an attestation failure, and hence partial configuration is being sent.  EVE, upon receiving such error codes will schedule re-attestation immediately (instead of waiting for the next attestation timer to expire)
  9. If EVE node is not in “UUD" or "UMD" state, all the latest configuration items are included. 
  10. didn't match expected measurements.
  11. EVC maintains a central database of all the supported EVE software images, and their hash values, indexed with the EVE image version tag
  12. EVC also maintains a central database of mostly used BIOS firmware images, their signatures, and the certificate provided by the BIOS vendor for validating the signatures, indexed by a combined tag of BIOS version string + Manufacturer
  13. On receiving attestation request EVE,  attestation service module checks PCR Quote against the baseline value. If there is a change, the EVE node is marked as “UUD”.  When there is no software change, the PCR quote is expected to be the same across reboots. However, after an EVE software upgrade, it is expected that the baseline value will change for the PCR values. However, after comparing the reported values with the expected values(since EVC knows about the new image version and its hash values), EVC makes a decision: If the reported values match, the baseline for the EVE node is updated. If they don’t, EVE node is marked as UUD. 
  14. An optional feature, "Location-Lock" may be introduced in EVC, to additionally check the Geo-Location reported by device, and flag if the location has changed since its last-seen location. This feature may be critical in some deployments, where a given EVE node may be mounted permanently, and any change in its physical location should be flagged, and optionally considered along with software measurements to conclude if the EVE node is trusted. In this regard, another new device flag, “Unknown Movement Detected” (Abbreviated as UMD) will be introduced in EVC, to indicate that a change in the location of the device has been detected.  
  15. If EVE node is not in “UUD" or "UMD" state, EVC includes all the latest configuration in reply to config request from EVE.
  16. If edge-node is in “UUD” or "UMD" state, any config request from EVE will be responded with response code 403 - Forbidden. This is to protect any new sensitive  images/credentials from getting exposed to the compromised device. The response from EVC in such cases will carry an error code to indicate that there was an attestation failure, and hence partial configuration is being sent.  EVE, upon receiving such error codes MUST schedule re-attestation immediately.
  17. If EVE reboots with a different software image from the configured version, EVC should be able to detect as quickly as possible and force attestation. To this effect a new token is introduced, called "Integrity Token". This token is a random nonce that will be stored in EVE under an encrypted folder. The master-key to decrypt this folder will be sealed against the TPM PCRs.  Therefore, if EVE reboots with a different software, unsealing of this key will fail, and hence the new software will not be able to recover the Integrity Token inside it. This token is sent during attestation requests, and if the attestation is successful, the provided token value is sealed against PCR values. This is usually implemented by storing the token inside an encrypted folder, with the master-key protected by TPM with Seal operation.
  18. Every configuration request from EVE will include this Integrity Token. If there is a token mismatch, HTTP code 403 - Forbidden will be sent to the device as the response, indicating that device should do re-attestation. 
  19. Frequency of attestation: EVE node will be Frequency of attestation: EVE node will be required to periodically attest itself via attestation requests. The interval It is fixed to be every 4 hours, or whenever the device connects back to the cloud again, or after the next reboot, whichever is earlier. This 4 hours is a tentative interval, and can be changed based on deployment requirements.
  20. If EVE reboots with a different software image from the configured version, EVC should be able to detect as quickly as possible and force attestation. To this effect a new token is introduced, called "Integrity Token". This token is a random nonce that will be stored in EVE under an encrypted folder. The master-key to decrypt this folder will be sealed against the TPM PCRs.  Therefore, if EVE reboots with a different software, unsealing of this key will fail, and hence the new software will not be able to recover the Integrity Token inside it.
  21. This token is sent during attestation requests, and if the attestation is successful, the provided token value is locked against the EVE node. 
  22. Every configuration request from EVE will include this Integrity Token. If there is a token mismatch, HTTP code 403 - Forbidden will be sent to the device as the response, indicating that device should do re-attestation. 

Image Removed

Fig 2. Overview of Remote Attestation Flow in EVC

Protecting App Instances With Encrypted Folder

All the app images and container volumes would be inside the encrypted folder (/persist/vault), with the master key sealed inside TPM, against a set of PCR values. Without access to this master-key, the encrypted folders can not be accessed.

If EVE or BIOS image was changed during boot, then unsealing will fail, which will implicitly disable all the apps from getting launched, since all the app images and container volumes would be inside the encrypted folder (/persist/vault), with the master key sealed inside TPM. 

To provide some flexibility to the EVE node administrator, when deploying app instances on the EVE platform, there will be 3  security modes for app deployment:

  1. No-Lock Mode - app images will not be in encrypted folder (not recommended, unless there is a compelling need to run any app instance unconditionally e.g. service apps like SD-WAN, DHCP Server)
  2. Software Lock Mode - app images will be in encrypted folder - with key from TPM, sealed with PCRs (the default option)
  3. Software Lock + Location Lock Mode  - app images will be encrypted folder - with key from TPM, sealed with PCRS + a key provided after Geo-Location check from the EVC.  This will work only when device is connected to EVC

Offline vs. Online checks

With the approach to seal the key(s) needed for fscrypt under the TPM, this means that if the PCR values have not changed (e.g., a normal reboot, or a reboot due to a power outage) the EVE node will be able to unseal those keys and start the app instances without any controller connectivity.

After a BIOS and/or EVE update, the PCR values will presumably change so the unsealing will fail. In that case the EVE node needs to perform remote attestation with the controller (with automatic and/or user acceptance of the new measurements) before it can retrieve the key from the controller. Note that as part of EVE update we already require this connectivity to verify that the update is fully working. Once retrieved the EVE node will again seal that key under the current PCR values.

This means that the controller will need to store the fscrypt key(s) so it can hand it back to the EVE node. But we should be able to encrypt them under the EVE node private key so that even if the controller is compromised the fscrypt keys can not be extracted from it.

The offline approach means that if there is something else wrong, such as IP geolocation doesn’t match, the applications are likely to start before the controller can raise a flag and potentially disable the applications.

And since this is measured boot it means that if some firmware or EVE component has been compromised, EVE will still boot and attempt remote attestation. Such a compromised device might be running some root-kit which can access the adapters on the EVE node, but applications will not start. Thus this implicitly assumes that the security of connected systems is based on credentials stored in the application instances and not merely by having the physical connectivity to some serial port or Ethernet network.

Establishing a Reference for Verification

...

  1. up to the EVC to schedule the frequency of attestation. Whenever EVC replies with Error 403, EVE MUST re-attest.  EVC can trigger this either periodically (say every few hours) or when it sees any discrepancy in the Integrity-Token.

Protecting App Instances With Encrypted Folder

All the app images and container volumes would be inside the encrypted folder (/persist/vault), with the master key sealed inside TPM, against a set of PCR values. Without access to this master-key, the encrypted folders can not be accessed.

If EVE or BIOS image was changed during boot, then unsealing will fail, which will implicitly disable all the apps from getting launched, since all the app images and container volumes would be inside the encrypted folder (/persist/vault), with the master key sealed inside TPM. 

To provide some flexibility to the EVE node administrator, when deploying app instances on the EVE platform, there will be 3  security modes for app deployment:

  1. No-Lock Mode - app images will not be in encrypted folder (not recommended, unless there is a compelling need to run any app instance unconditionally e.g. service apps like SD-WAN, DHCP Server)
  2. Software Lock Mode - app images will be in encrypted folder - with key from TPM, sealed with PCRs (the default option)
  3. Software Lock + Location Lock Mode  - app images will be encrypted folder - with key from TPM, sealed with PCRS + a key provided after Geo-Location check from the EVC.  This will work only when device is connected to EVC

Offline vs. Online checks

With the approach to seal the key(s) needed for fscrypt under the TPM, this means that if the PCR values have not changed (e.g., a normal reboot, or a reboot due to a power outage) the EVE node will be able to unseal those keys and start the app instances without any controller connectivity.

After a BIOS and/or EVE update, the PCR values will presumably change so the unsealing will fail. In that case the EVE node needs to perform remote attestation with the controller (with automatic and/or user acceptance of the new measurements) before it can retrieve the key from the controller. Note that as part of EVE update we already require this connectivity to verify that the update is fully working. Once retrieved the EVE node will again seal that key under the current PCR values.

This means that the controller will need to store the fscrypt key(s) so it can hand it back to the EVE node. But we should be able to encrypt them under the EVE node private key so that even if the controller is compromised the fscrypt keys can not be extracted from it.

The offline approach means that if there is something else wrong, such as IP geolocation doesn’t match, the applications are likely to start before the controller can raise a flag and potentially disable the applications.

And since this is measured boot it means that if some firmware or EVE component has been compromised, EVE will still boot and attempt remote attestation. Such a compromised device might be running some root-kit which can access the adapters on the EVE node, but applications will not start. Thus this implicitly assumes that the security of connected systems is based on credentials stored in the application instances and not merely by having the physical connectivity to some serial port or Ethernet network.

...

Updated EVE Startup State Transition

...

The next section presents the same steps, but at more functional level, with interaction among various EVE microservices.

Module Level Interaction - EVE Startup Sequence (

...

Initial Onboarding)

Image RemovedImage Added

Fig. Module  Module Level Interaction in EVE - Startup Sequence (Post Onboarding)

  1. Device-steps starts client.go (the provisioning client) which will check and do the following:
      If certificates from EVC are not yet fetched,  fetches them 
      1. First fetches EVC Certificates
      2. Registers the device certificate
      3. Retrieves UUID from EVC
    2. Device-steps starts Vault Mgr
      1. Vault Mgr
      3. retrieves the master decryption key from TPM with Unseal operationUnlocks
      1. creates new vault: /persist/vault
      4. with
      2. Seals the master-
      5. key
      1. key into TPM with PCR values
      2. Publishes vault status (unlocked)
      3. Waits for Integrity-Token
      6. and/or master-key from EVC. 
      1. from zedAgent
      2. Once received,
      7. rotates the master-key if requested, and also
      1. copies the new Integrity-Token to /persist/vault/itoken file
    8. Device-steps starts TPM mgr
      1. TPM manger retrieves the attestation certificate and publishes to Zedagent
      2. Waits for Quote requests on pubsub channel from Zedagent
    9. Device-steps starts Zedagent (and Zedagent starts 3 concurrent tasks: attest, info and config)
      1. Attest task picks up the attestation certificate and publishes to EVC
      2. Attest task requests for a nonce from EVC (to prepare PCR quote)
      3. Attest task sends the nonce back to TPM Mgr and waits for PCR quote
      4. Attest task, once notified about the quote readiness, creates a random nonce value for Integrity-Token
      5. Attest task sends { Quote, Location, Event Log, Integrity-Token, Image Version } to EVC
      6. EVC, if
      10. quote is the same as previous quote
      1. master-key is not found for the EVE, approves the
      11. attestation request
      1. PCR quote as the baseline
        1. If Location-Lock is enabled, location is
      12. checked in addition to QuoteAlong with attestation approval, the encrypted master-key is sent back to EVE
        1. also approved as baseline (provided the EVE device is not in manufacturing account)
      2. In the mean time, configuration task keeps requesting config from EVC (expected to fail till attestation goes through)
      3. Attest task, once attestation request is approved, gets the master-key and Integrity-Token and passes it to Vault Mgr
      4. Config task now picks up the right Integrity-Token and Config request is now accepted by EVC, and full configuration is sent back
      5. Info task reports the encrypted master-key back to EVC, for backup purposes

    Module Level Interaction -

    ...

    Startup Sequence (

    ...

    Reboot with no change)

    Image RemovedImage Added

    Fig.  Module Module Level Interaction in EVE - Startup Sequence (Post Onboarding)

    1. Device-steps starts client.go (the provisioning client) which will check and do the following:
      1. First fetches EVC Certificates
        2. Registers the device certificate
        1. If certificates from EVC are not yet fetched,  fetches them 
        2. Retrieves UUID from EVC
      3. Device-steps starts Vault Mgr
        1. Vault Mgr
        4. creates new vault:
        1. retrieves the master decryption key from TPM with Unseal operation
        2. Unlocks /persist/vault
        5. Seals
        1. with the master-key
        6. into TPM with PCR values
        2. Publishes vault status (unlocked)
        3. Waits for Integrity-Token
        7. from zedAgent
        1. and/or master-key from EVC. 
        2. Once received, rotates the master-key if requested, and also copies the new Integrity-Token to /persist/vault/itoken file
      8. Device-steps starts TPM mgr
        1. TPM manger retrieves the attestation certificate and publishes to Zedagent
        2. Waits for Quote requests on pubsub channel from Zedagent
      9. Device-steps starts Zedagent (and Zedagent starts 3 concurrent tasks: attest, info and config)
        1. Attest task picks up the attestation certificate and publishes to EVC
        2. Attest task requests for a nonce from EVC (to prepare PCR quote)
        3. Attest task sends the nonce back to TPM Mgr and waits for PCR quote
        4. Attest task, once notified about the quote readiness, creates a random nonce value for Integrity-Token
        5. Attest task sends { Quote, Location, Event Log, Integrity-Token, Image Version } to EVC
        6. EVC, if
        10. master-key is not found for the EVE
        1. quote is the same as previous quote, approves the
        11. PCR quote as the baseline
        1. attestation request
          1. If Location-Lock is enabled, location is
        12. also approved as baseline (provided the EVE device is not in manufacturing account)
          1. checked in addition to Quote
        2. Along with attestation approval, the encrypted master-key is sent back to EVE
        3. In the mean time, configuration task keeps requesting config from EVC (expected to fail till attestation goes through)
        4. Attest task, once attestation request is approved, gets the master-key and Integrity-Token and passes it to Vault Mgr
        5. Config task now picks up the right Integrity-Token and Config request is now accepted by EVC, and full configuration is sent back
        6. Info task reports the encrypted master-key back to EVC, for backup purposes

      Module Level Interaction - EVE Startup Sequence (

      ...

      Reboot with a change)

      Fig. Module Level Interaction in EVE - Startup Sequence (Running Unknown Software)

      ...