...

<Describe any changes to the hzn CLI, including before and after command examples for clarity. Include which users will use the changed CLI. This section should flow very naturally from the User Experience section.>

hzn exchange node hagroup create --nodeId node1 --nodeId node2 --nodeId node3 --force

hzn exchange node hagroup remove --nodeId node1

hzn exchange node hagroup list

hzn deploycheck --checkHA

External Components

<Describe any new or changed interactions with components that are not the agent or the management hub.>

None

Affected Components

<List all of the internal components (agent, MMS, Exchange, etc) which need to be updated to support the proposed feature. Include a link to the github epic for this feature (and the epic should contain the github issues for each component).>

Agent - Awareness of HAGroup membership for agent upgrade procedure

Agbot - Awareness of a node's HAGroup membership for making agreements

CLI - To list, add, remove nodes from an HA Group

Exchange - To hold the new HAGroup membership APIs

Security

<Describe any related security aspects of the solution. Think about security of components interacting with each other, users interacting with the system, components interacting with external systems, permissions of users or components>

None

APIs

<Describe and new/changed/deprecated APIs, including before and after snippets for clarity. Include which components or users will use the APIs.>

The HAGroup object schema

members: [

   {

        "node":"node1234"

    },

]

Exchange APIs:

Create a new node group. The caller must have permission to modify all the nodes listed in the body (shown above). The Exchange will set this same object onto all the nodes listed in the body. The Exchange will return an error (409) if one of the nodes is already in an hagroup. If force=true is specified, the Exchange will set this membership onto all listed nodes and will remove listed nodes from any group they are already in.

POST /org/<org>/node/<node-id>/hagroup?force=true

Modify the group membership of an existing group. All the desired members of the group MUST be listed in the body. This API behaves like a full replace. The force=true parameter has the same behavior as on POST.

PUT /org/<org>/node/<node-id>/hagroup?force=true

List all the members in an hagroup. This API returns the exact same results when called on any member in an ha group.

GET /org/<org>/node/<node-id>/hagroup

Build, Install, Packaging

<Describe any changes to the way any component of the system is built (e.g. agent packages, containers, etc), installed (operators, manual install, batch install, SDO), configured, and deployed (consider the hub and edge nodes).>

None

Documentation Notes

<Describe the aspects of documentation that will be new/changed/updated. Be sure to indicate if this is new or changed doc, the impacted artifacts (e.g. technical doc, website, etc) and links to the related doc issue(s) in github.>

Need an Overview of how HA Groups work on the OH doc site.

Need a new article describing how to use HA Groups, this would be focused toward the administrator and node owners. It could be the same article for both roles (but we might change our mind on this AFTER we have tried to write it).

Update/remove HA doc in the anax repo for the HA /attribute API. This is being removed.

CLI commands

Test

<Summarize new automated tests that need to be added in support of this feature, and describe any special test requirements that you can foresee.>

1. Edge Clusters are out of scope for this support. Edge clusters already natively support HA and CA, and therefore don't need any special assistance from OpenHorizon.
2. Test service lifecycles with services deployed to HA groups that are homogeneous (all members have the same services) and heterogeneous (there is at least 1 service common to all members, but some services are only running on a subset of members).
3. Performance test of service upgrades with and without HA groups in the system.