...

1. Nodes in an HA group MAY have different node policies.
2. Adding a node to an HA Group MUST NOT terminate/restart running services.
3. Nodes MAY be placed into an HA Group after node registration.
4. A node MUST be in 0 or 1 HA Groups. A node MUST NOT be in more than 1 HA Group.
5. A node specifies the other nodes in it's HA Group by Id. All nodes in an HA Group MUST specify all the other nodes in the group. Can we get the exchange to enforce this part of the model?
6. A user MUST have permission to modify all the node's (resources) in an HA Group in order to form the group.
7. A service that is deployed to all the nodes in an HA Group MUST be upgraded in a rolling restart in order to avoid a complete outage of the service.
8. The node agent on nodes in an HA Group MUST be upgraded in a rolling restart in order to avoid a complete outage of the service and model deployment capability.

The principles 

Agbot

The Agbot is responsible for ensuring that all deployment policies have been checked for compatibility against all nodes in the system, and making adjustments to the deployed state of services as node policy, deployment policy and service policy change over time. The Agbot already has support for ensuring that service upgrades are performed in a rolling fashion across an HA Group. The existing support will have to be augmented in the following ways:

1. HA Group membership is obtained from the node's hagroup resource in the Exchange. The existing support obtains this info from an internal representation of the node (in the code it's called the producer policy). The HA Group definition should be removed from this internal representation.
2. The existing support assumes that ALL services running on a node in an HA Group are supposed to be running on ALL nodes in the group. This assumption is no longer true in this design. The Agbot needs to perform some additional checking (before attempting a rolling upgrade) that the service being upgraded is intended to be running on all nodes in the group.

User Experience

<Describe which user roles are related to the problem AND the solution, e.g. admin, deployer, node owner, etc. If you need to define a new role in your design, make that very clear. Remember this is about what a user is thinking when interacting with the system before and after this design change. This section is not about a UI, it's more abstract than that. This section should explain all the aspects of the proposed feature that will surface to users.>

...