Skip to content
This repository has been archived by the owner on Sep 10, 2020. It is now read-only.

Schema violation: [2018-11-20T10:25:58] is not a valid date-time. #2

Open
wub opened this issue Nov 23, 2018 · 1 comment
Open

Schema violation: [2018-11-20T10:25:58] is not a valid date-time. #2

wub opened this issue Nov 23, 2018 · 1 comment

Comments

@wub
Copy link

wub commented Nov 23, 2018

Hey guys, seems to be an issue with the date-time format when loading into Stitch. Any ideas? Using the catalog.json generated by v0.1.0. The website is running in the "Auckland" timezone.


Versions:

tap-woocommerce 0.1.0
target-stitch 1.8.1
Python: 3.7.1
WooCommerce 3.5.1
Windows 10

Command:

$ ~/.virtualenvs/tap-woocommerce/Scripts/tap-woocommerce --config tap-config.json --catalog catalog.json | ~/.virtualenvs/target-stitch/Scripts/target-stitch --config target-config.json

Output:

INFO Starting sync. Will sync these streams: ['orders']
INFO Syncing orders
INFO Only syncing orders updated since 2018-11-20T00:00:00
INFO GET https://www.[website].com/wp-json/wc/v2/orders?after=2018-11-20T00%3A00%3A00&orderby=date&order=asc&per_page=100&page=1
INFO Using Stitch import URL https://api.stitchdata.com/v2/import/batch
INFO Sending version information to stitchdata.com. To disable sending anonymous usage data, set the config parameter "disable_collection" to true
{"currently_syncing": "orders"}
INFO METRIC: {"type": "timer", "metric": "http_request_duration", "value": 4.188087463378906, "tags": {"endpoint": "orders", "http_status_code": 200, "status": "succeeded"}}
INFO GET https://www.[website].com/wp-json/wc/v2/orders?after=2018-11-20T00%3A00%3A00&orderby=date&order=asc&per_page=100&page=2
INFO METRIC: {"type": "timer", "metric": "http_request_duration", "value": 2.826814889907837, "tags": {"endpoint": "orders", "http_status_code": 200, "status": "succeeded"}}
INFO METRIC: {"type": "counter", "metric": "record_count", "value": 105, "tags": {"endpoint": "orders"}}
INFO Completed Orders Sync
INFO Sending batch with 105 messages for table orders to https://api.stitchdata.com/v2/import/batch
CRITICAL Error persisting data for table "orders": Record 0 did not conform to schema:
CRITICAL #: 4 schema violations found
CRITICAL #/date_modified: #: no subschema matched out of the total 2 subschemas
CRITICAL #/date_modified: expected: null, found: String
CRITICAL #/date_modified: [2018-11-20T10:25:58] is not a valid date-time. Expected [yyyy-MM-dd'T'HH:mm:ssZ, yyyy-MM-dd'T'HH:mm:ss.[0-9]{1,9}Z]
CRITICAL #/date_created: #: no subschema matched out of the total 2 subschemas
CRITICAL #/date_created: expected: null, found: String
CRITICAL #/date_created: [2018-11-20T00:06:08] is not a valid date-time. Expected [yyyy-MM-dd'T'HH:mm:ssZ, yyyy-MM-dd'T'HH:mm:ss.[0-9]{1,9}Z]
@dmosorast
Copy link
Contributor

HI @wub, thanks for posting this issue! What's happening here seems to be that the tap isn't formatting the datetime value properly. Stitch expects all datetimes to be in UTC format. The way of representing that the timezone offset in string format is to append a Z to the end.

It looks like the tap is handling its own datetimes judging by the usages of parser and tzinfo, but singer-python has many functions to help with parsing and formatting that ensure the values get converted to UTC and formatted properly as strings in the JSON. You can check out singer-python:utils.py, specifically the strptime_to_utc and strftime functions, and see if those help you move forward.

Please let me know if I can clarify at all. If you get it working, feel free to submit a PR and I can review it for merging.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants