Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not stable issue: Sometimes records from two different GoogleSheets table Sources aren't recorded to Destination #43

Open
SerhiiSpivak opened this issue Nov 15, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@SerhiiSpivak
Copy link

Bug description

The user creates two different GoogleSheets Source connectors and the Source Destination (for example NATS PubSub); then the user starts the pipeline -> sometimes data isn't transferred to the Destination connector

Not stable issue

Additional info: data from two different GoogleSheets Source connectors are recorded after restarting the pipeline

Steps to reproduce

  1. Create Pipeline
  2. Create different GoogleSheets Source connectors and NATS PubSub Destination (see below config):
version: 1.0                  

pipelines:                    
  pipeline1:                   
    status: stopped             
    name: pipeline1             
    description: desc           
    connectors:                 
      con1:                 
        type: source            
        plugin: standalone:google-sheets  
        name: googlesheet-source-sheet1            
        settings:              
          credentialsFile: <path://to/credential/file>
          tokensFile: <path://to/token/file>
          sheetsURL: <URL of the google spreadsheet#1>
          dateTimeRenderOption:
          valueRenderOption:
          pollingPeriod:
      con2:                 
        type: source            
        plugin: standalone:google-sheets  
        name: googlesheet-source-sheet2            
        settings:              
          credentialsFile: <path://to/credential/file>
          tokensFile: <path://to/token/file>
          sheetsURL: <URL of the google spreadsheet#2>
          dateTimeRenderOption:
          valueRenderOption:
          pollingPeriod:
      con3:
        type: destination
        plugin: standalone:nats-pubsub
        name: nats-pubsub-DESTINATION
        settings:
           urls: <NATS connection URL> 
           subject: sbj
           connectionName:
           nkeyPath:
           credentialsFilePath:
           tls.clientCertPath:
           tls.clientPrivateKeyPath:
           tls.rootCACertPath:
           maxReconnects:
           reconnectWait:
  1. Start the pipeline
  2. sometimes data isn't transferred to the Destination connector

Additional info: data from two different GoogleSheets Source connectors are recorded after restarting the pipeline

Version

v0.3.0-nightly.20220927

@SerhiiSpivak SerhiiSpivak added the bug Something isn't working label Nov 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant