Yesterday I had a terrible doubt about kubernetes operator deployments. Briefly, I wanted to be sure when it is 100% safe for a custom resource to be created and what possible race-conditions should be taken into account between operator controller instances and its corresponding ‘watched’ custom resources.

I dag on google through post and specs but this chatGTP pretty much summarises my conclusions. I did it following the same questions I did to myself when googling and then used the chat to validate them (and play a bit with the bot), hope it is helpful for you too:

me>> what happens if a kubernetes custom resource is created before the corresponding operator controller is running?

If a Kubernetes custom resource is created before the corresponding operator controller is running, the controller will not be able to process the resource until it is started. Depending on the implementation of the operator and the custom resource, the resource may not be fully functional or may not be monitored for updates until the controller is running. It is recommended to start the operator controller before creating any custom resources to ensure proper functionality.

me>> why is it recommended to start the operator controller before creating any custom resources ?

It is recommended to start the operator controller before creating any custom resources because the controller is responsible for managing the lifecycle of the custom resources. This includes creating, updating, and deleting resources based on the desired state specified in the custom resource.

If the controller is not running when a custom resource is created, the controller will not be able to process the resource and ensure that it is in the desired state. Additionally, the controller may not be able to monitor the resource for updates, which can lead to the resource becoming out of sync with the desired state.

By starting the operator controller before creating any custom resources, you can ensure that the controller is able to immediately process and manage the resources as they are created, which helps to maintain a consistent and desired state of resources, which is the main objective of using kubernetes operator.

me>> does kubernetes operator controller use webhooks?

Yes, Kubernetes operator controllers can use webhooks as a way to perform certain actions on custom resources.

Webhooks are a way to trigger an external service to perform an action when certain events occur in the Kubernetes cluster. For example, a webhook could be configured to validate a custom resource before it is created, or to mutate a custom resource before it is updated.

Operators can use webhooks to perform specific actions on custom resources, such as validating the resource’s configuration, mutating the resource’s specification, or performing a pre- or post-hook.

Webhooks are typically implemented using a Kubernetes admission controller and a separate service that handles the webhook requests. The admission controller intercepts the request to create or update a resource, and sends it to the webhook service, which performs the necessary action before allowing the request to continue.

me>> which alternative to webhooks can be used in kubernetes operator controller?

An alternative to webhooks that can be used in a Kubernetes operator controller is to use a Kubernetes controller pattern.

A controller pattern is a way to implement a specific behavior in a Kubernetes operator. It is a specific type of controller that runs in the same process as the operator, and watches for changes to custom resources. It then takes the appropriate actions to bring the custom resource to the desired state.

Controller patterns are typically implemented using the Go client library for Kubernetes (client-go) and the controller-runtime library. This allows the operator to watch for changes to custom resources and take appropriate actions using the Kubernetes API.

Controller pattern is an alternative to webhooks, as it allows the operator to perform actions on custom resources directly, rather than relying on an external service to handle the webhook request. This can be more efficient and easier to implement, but also might require more knowledge and expertise of kubernetes API.

me>> if an operator controller uses controller pattern, can it process existing resources already present when operator starts running?

If an operator controller uses the controller pattern, it can process existing resources that are already present in the cluster when the operator starts running.

When the operator starts, it will typically register a watch on the custom resources it manages. The watch will trigger a callback function for each existing resource in the cluster, allowing the operator to process these resources.

Depending on the implementation of the operator, it may take different actions when processing existing resources. The operator could, for example, check the current state of the resource and ensure it matches the desired state, or it could update the resource to the desired state.

It is important to note that if the resources were created before the operator and not managed by it, they may not match the desired state that the operator is expecting and may require additional logic or steps to reconcile them.

Also, if the resources were created with some properties which are not recognized by the operator, it may not be able to reconcile them and may require manual intervention or updates to the operator’s code to handle those cases.

me>> but what happens if webhook is used instead of controller pattern?

If a webhook is used instead of a controller pattern in an operator, the behavior when the operator starts and processes existing resources will be different.

When the operator starts, it will register a webhook with the Kubernetes API server. The webhook will be triggered for any new resources that are created or updated, but it will not be triggered for existing resources that were already present in the cluster when the operator starts.

This means that the operator will not be able to process existing resources that were already present in the cluster when it starts. The operator will only be able to process resources that are created or updated after it starts.

To reconcile the existing resources and bring them to the desired state, the operator would have to have additional logic that periodically scans the cluster for resources that it is supposed to manage, and if it finds any, it would need to invoke the webhook manually.

It is worth noting that using a webhook can be useful in situations where you want to perform a specific action on a resource before it is created or updated, such as validating its configuration or mutating its specification. But it is less efficient and might require more complexity to reconcile the existing resources, when compared to using a controller pattern.

from https://chat.openai.com/chat

This explains why the results I got changed from one operator to another. Some of them can handle ‘existing’ custom resources created after CRDs are deployed but before the controller is up and running while others require to check the running state of the controller before creating any custom resource.