Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ConfigMapList's resourceVersion for watching (w/ PropertySource) #300

Commits on May 25, 2021

  1. Use ConfigMapList's resourceVersion for watching

    Rationale: avoid `ERROR` notifications during rolling upgrades.
    
    Using the greatest `resourceVersion` of all the `ConfigMap`s returned
    within the `ConfigMapList` works as expected for *fresh* deployments.
    But, when performing a *rolling upgrade* (and depending on the upgrade
    strategy), the watcher happens to frequently stop after having received
    an `ERROR` notification:
    
    > [ERROR] [KubernetesConfigMapWatcher] [] Kubernetes API returned an
    error for a ConfigMap watch event: ConfigMapWatchEvent{type=ERROR,
    object=ConfigMap{metadata=Metadata{name='null', namespace='null',
    uid='null', labels={}, resourceVersion=null}, data={}}}
    
    What's actually streamed in that case is a `Status` object such as:
    ```json
    {
      "type": "ERROR",
      "object": {
        "kind": "Status",
        "apiVersion": "v1",
        "metadata": {},
        "status": "Expired",
        "message": "too old resource version: 123 (456)",
        "reason": "Gone",
        "code": 410
      }
    }
    ```
    
    A few references:
    * ManageIQ/kubeclient#452
    * https://www.baeldung.com/java-kubernetes-watch#1-resource-versions
    
    It's possible to recover by adding some logic to reinstall the watcher
    starting with the newly advertised `resourceVersion`, but this may be
    avoided at all by starting the initial watch at the `resourceVersion`
    of the `ConfigMapList` itself: this one won't expire.
    
    The proposed implementation consists in storing last received
    `resourceVersion` as an additional `PropertySource` (through a dedicated
    name and version key) and later using it when installing the watcher.
    
    Co-authored-by: Regis Desgroppes <rdesgroppes@users.noreply.github.com>
    Co-authored-by: Pavol Gressa <pavol.gressa@oracle.com>
    3 people committed May 25, 2021
    Configuration menu
    Copy the full SHA
    01c2ea4 View commit details
    Browse the repository at this point in the history