Motivation

Setting up a communication channel from the app instance to the controller meaning that EVE won't save anything. EVE will pass on the information to the controller whatever app instance will send. Downloading kubeconfig from the K3S cluster instance is the first use case of the proposal. Kubeconfig is used to configure access to a K3S cluster instance when used in conjunction with the kubectl command-line tool or any other client.

EVE and Controller Communication Proposal

EVE will use the existing device info API for sending app instance metadata information to the controller. We will introduce a new structure for app instance metadata which will have application UUID, data, and its type.

// AppInstMetaData sends metadata of an application instance
// e.g kubeconfig, etc
// Size of app metadata <= 32KB
message ZInfoAppInstMetaData {
  string uuid = 1;
  AppInstanceMetaData type = 2;
  bytes data = 3;
}
// Different types of app instance metadata
enum AppInstMetaDataType {
  APP_INST_META_DATA_TYPE_NONE = 0;
  APP_INST_META_DATA_TYPE_KUBE_CONFIG = 1;
}

App Instance and EVE Communication Proposal

Need a mechanism in EVE to enable file exchange between EVE and the application instance. Zedrouter in EVE currently implements some API endpoints at http://169.254.169.254/ where the application instances can fetch some data. We could add some new EVE-specific POST APIs under http://169.254.169.254/eve/v1.

For downloading kubeconfig, we will add POST API http://169.254.169.254/eve/v1/kubeconfig on which seed server application instance of K3S cluster will publish its kubeconfig. In zedrouter, we will get the kubeconfig of the K3S cluster and we will forward it to zedagent and will send it to the controller from the zedagent in the device info message. However, this only works if the size of the data is limited to less than about 32 KB.

Appendix

Sample K3S Cluster KubeConfig File

apiVersion: v1
kind: Config
clusters:
- name: "test-cluster"
cluster:
server: "https://104.211.222.233/k8s/clusters/c-6n45m"
certificate-authority-data: "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJpRENDQ\
VM2Z0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQTdNUnd3R2dZRFZRUUtFeE5rZVc1aGJXbGoKY\
kdsemRHVnVaWEl0YjNKbk1Sc3dHUVlEVlFRREV4SmtlVzVoYldsamJHbHpkR1Z1WlhJdFkyRXdIa\
GNOTWpBeApNREV5TURjMU5UQXpXaGNOTXpBeE1ERXdNRGMxTlRBeldqQTdNUnd3R2dZRFZRUUtFe\
E5rZVc1aGJXbGpiR2x6CmRHVnVaWEl0YjNKbk1Sc3dHUVlEVlFRREV4SmtlVzVoYldsamJHbHpkR\
1Z1WlhJdFkyRXdXVEFUQmdjcWhrak8KUFFJQkJnZ3Foa2pPUFFNQkJ3TkNBQVJza1A3cjNCU3VYd\
1I2d3pIQ0N1NVovVzNaZGxlQlpZSDN5cW1vVHBrNgoxLzhGSkdiMVhNSE01d3JxSUU0WVJZYTJmd\
3FPdkFjM2VKL2xJSGxCd0RZVm95TXdJVEFPQmdOVkhROEJBZjhFCkJBTUNBcVF3RHdZRFZSMFRBU\
UgvQkFVd0F3RUIvekFLQmdncWhrak9QUVFEQWdOSUFEQkZBaUF5YnRRSEpINEsKZVJucW9MajduM\
WdTSEZ0aFZDOURxSm1DeUtrUzduSE9RZ0loQU9uNCtpbElXd0hyVXBxMFp2bFhIc1BLaENRawpnM\
GVYaGkwOS9zSlQ0V1E2Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0="

users:
- name: "test-cluster"
user:

token: "kubeconfig-user-5vmrh:858wtpjhnkstqv5c9wgbjgjw5scxx4l5hqdfwprrkpcpsvbzws6qlz"

contexts:
- name: "test-cluster"
context:
user: "test-cluster"
cluster: "test-cluster"

current-context: "test-cluster"

Sample request body sent from application instance to EVE

{
    "kind": "Config",
    "apiVersion": "v1",
    "preferences": {},
    "clusters": [
        {
            "name": "multi-device-cluster-seedserver",
            "cluster": {
                "server": "https://192.168.254.180:6443",
                "certificate-authority-data": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJkakNDQVIyZ0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQWpNU0V3SHdZRFZRUUREQmhyTTNNdGMyVnkKZG1WeUxXTmhRREUyTWpZd09EUTNNREF3SGhjTk1qRXdOekV5TVRBeE1UUXdXaGNOTXpFd056RXdNVEF4TVRRdwpXakFqTVNFd0h3WURWUVFEREJock0zTXRjMlZ5ZG1WeUxXTmhRREUyTWpZd09EUTNNREF3V1RBVEJnY3Foa2pPClBRSUJCZ2dxaGtqT1BRTUJCd05DQUFTa2FadjI4ekcyK21HNkJBTnFFdEx6ZFR0c2F4a1RnMmpmTm82eStLaXMKakdqUWlUWDN3TjcyUjdFRWdYZ1AvK0JwTnZER09ZZmRzSm1nVnFHcUMvbytvMEl3UURBT0JnTlZIUThCQWY4RQpCQU1DQXFRd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVVR2V0JXTWtyUzlEeXdJMzBhU3Q4CktLU0xucVl3Q2dZSUtvWkl6ajBFQXdJRFJ3QXdSQUlnTGs2SmdEanNVRGtOeHlvMUMrWlpCek43aWtGUnJhT3IKbXozREs3R1NNeG9DSUZjSVk0TzlvVnN5UTBreDlpMU4wWnYzeThTSzI3cENxSmt4RW9IQXhNUTgKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="
            }
        }
    ],
    "users": [
        {
            "name": "multi-device-cluster-seedserver",
            "user": {
                "token": "eyJhbGciOiJSUzI1NiIsImtpZCI6Imk1aWxkNXBMVktkYnVpZTFFS0xrX3JpRTZOcE05RFpmM2xjbnBzZTJXSlkifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtdWx0aS1kZXZpY2UtY2x1c3Rlci1zZWVkc2VydmVyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6Im11bHRpLWRldmljZS1jbHVzdGVyLXNlZWRzZXJ2ZXItdG9rZW4tZG1wbnoiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoibXVsdGktZGV2aWNlLWNsdXN0ZXItc2VlZHNlcnZlciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImIxNDM3NzZhLTRmMzQtNGI5MS04NDAwLThjNzliYjc4MzEyNCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDptdWx0aS1kZXZpY2UtY2x1c3Rlci1zZWVkc2VydmVyOm11bHRpLWRldmljZS1jbHVzdGVyLXNlZWRzZXJ2ZXIifQ.rYN1Rmutaw1AAyx-3pIOv_hdaiXGnpbEQI2nCiS9sGEjK49ZqSmNasx9mHM62YBF1IBkmJOez95rkjdouiSOU_08DpBoxOEGzd2r_01kk-96LsEUXb7D0gkwWN5Fvr-CKIk2dRK3k0v40U9j4Vsx8Va3rGjjRm0C9n0LO7PPkbEk1Ox0S8sWe9qsOpCt3cVwFF57LwAb6hQbZR8LvJUNcw25_fgF1qBCitGB4WccB1z5_DKNJ2B1bPISJQyPWxMAnQvrJNVX--IVrXelb7DBl-J47S6rEXWq9GetAv2mmgUte6jvsB5j7yLWzcN1xDk_rBLZsOhOghf6ckZp8bLBbg"
            }
        }
    ],
    "contexts": [
        {
            "name": "multi-device-cluster-seedserver",
            "context": {
                "cluster": "multi-device-cluster-seedserver",
                "user": "multi-device-cluster-seedserver"
            }
        }
    ],
    "current-context": "multi-device-cluster-seedserver"
}