Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

additional goodies csv sftp perf sql and more #855

Merged
merged 13 commits into from
Dec 22, 2023
Merged

additional goodies csv sftp perf sql and more #855

merged 13 commits into from
Dec 22, 2023

Commits on Dec 19, 2023

  1. feat: add connector SQL step

    @keevan
    keevan committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    ac2398c View commit details
    Browse the repository at this point in the history

  2. fix: allow values to be corrected in bad states 

    Instead of allowing the state to be not recoverable, this will instead fail to render the available variables section only.
    @keevan
    keevan committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    ccda9cd View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2023

  1. feat: add set multiple variables step type 

    Similar to set_variable but this allows multiple values to be set on an existing or new path.
This simplifies initialisation of multiple variables per instance run, such as counters used for reporting.
    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    12318d1 View commit details
    Browse the repository at this point in the history

  2. fix: warning logs now display and are more legible 

    - The background with white text was difficult to read (low contrast)
- The warning regex previously in place was incorrect
    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    a703315 View commit details
    Browse the repository at this point in the history

  3. fix: add copy file error handling for directory creation

    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    3e2d24f View commit details
    Browse the repository at this point in the history

  4. feat: run name and id are now available as expressable vars 

    Also tweaked comment on string format of run based output logs.
    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    7600e48 View commit details
    Browse the repository at this point in the history

  5. fix: SQL trait handling of integer values and improve error message

    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    5467d78 View commit details
    Browse the repository at this point in the history

  6. fix: update email flow step to support it as a final step

    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    7f95f1c View commit details
    Browse the repository at this point in the history

  7. fix: reader csv error to be more correct for header/fields count alig… 

    …nment

It is more correct to expect the number of fields to match the number of headers, not the other way around necessarily.
    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    b760edd View commit details
    Browse the repository at this point in the history

  8. fix: include row number in csv reader error

    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    27290a3 View commit details
    Browse the repository at this point in the history

  9. perf: cache sftp object per identity to speed up subsequent actions

    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    1c3948d View commit details
    Browse the repository at this point in the history

  10. feat: add an option to continue processing the csv file on known pars… 

    …e issues

The only one is currently a count mismatch on header and the row fields
    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    7ffa407 View commit details
    Browse the repository at this point in the history

  11. build: bump version

    @keevan
    keevan committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    f7485ef View commit details
    Browse the repository at this point in the history