Description
Is your feature request related to a problem? Please describe.
Operator permissions and the ability for operators to affect cluster scoped resources are a problem within the operator space. At the moment, operator bundles include both APIs (which are always a cluster scoped resource) and Controllers (which are potentially namespace scoped), but the problem of running the same operator in different namespaces, especially at different versions, is hard.
Lets assume we have a Foo operator with two versions:
- 1.0.0 includes the v1beta1 API
- 2.0.0 includes the v1beta1 API and the v1 API, and a conversion web hook
- 3.0.0 includes the v1 and v2 API, and a conversion web hook
If version 3.0.0 is installed in one namespace, followed by 2.0.0 in another namespace, what happens to the installed CRD and web hook? How does OLM decide which one is correct? If 3.0.0 is removed, what does OLM do? Remove the CRD? Replace the CRD with an older version and webhook? How is this managed?
Operator developers also have a hard job to do in managing API versions and compatibility. This complexity is somewhat glossed over by CRDs being included within the same bundle as the controller, as there is the implication that the CRD can be changed easily along with the controller, when in reality the API interface should be managed very carefully and with as much stability as possible to ensure that 3rd parties can use the API.
Describe the solution you'd like
Decouple APIs and Controllers by introducing two new bundle types (potentially with an annotation such as operators.operatorframework.io.bundle.mediatype.v1) that OLM can manage in a decoupled way.
API Bundles
API bundles would always be cluster scoped. Controller bundles would depend on them by depending on a specific API version that it will then manage.
OLM responsibilities:
- OLM would know how to install the cluster scoped API bundle as a dependency of a namespace scoped controller bundle.
- OLM would need to ensure that to install a controller bundle, an API bundle defining the API is installed.
- OLM would need to ensure that an API bundle cannot be removed or upgraded if that would remove an API version required by a controller bundle.
The API bundle would include the conversion webhook, which can be configured by OLM.
Additional restrictions could be added to API bundles:
- Only ever cluster scoped
- Run in a specific namespace for APIs, segregated from other operator workload
- OLM can ensure the conversion webhook has minimal cluster access, in theory, it would only need to run one service with no rolebindings at all and a network policy that only allowed ingress by the cluster API server and no egress. OLM may need to manage scaling
Controller bundles
Controller bundles would be similar to existing operator bundles with the following changes:
- Controller bundles would not include CRDs.
- To enable OLM to separate controllers that manage APIs, controller bundles would need to declare what APIs they manage (ie owned CRDs), so that overlapping controllers can be avoided.