Skip to content

Commit

Permalink
FIX: Update ALL_RT_fields to fix UTC timezone adjustement on seconds …
Browse files Browse the repository at this point in the history
…fields (#198)

Fix for following feedback from Heather Davis on LAMP_ALL_RT_fields Hyper file extract:

> In the all RT fields view, the stop arrival sec and previous stop departure sec fields are not calculated properly -- they seem to be in GMT rather than EST/EDT. Pre-11/5/23, they are 14,400 seconds or 4 hours too large, and post-11/5/23 they are 18,000 seconds or 5 hours too large. The datetime fields are in the correct time zone and do account for DST.

> The stop departure datetime field does not have an associated field in seconds after midnight (i.e. stop departure sec). Can you please add this field?
  • Loading branch information
rymarczy authored Dec 11, 2023
1 parent 5b2e27f commit 911283d
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions python_src/src/lamp_py/tableau/jobs/rt_rail.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,9 @@ def __init__(self) -> None:
" , TIMEZONE('America/New_York', TO_TIMESTAMP(ve.vp_move_timestamp)) as previous_stop_departure_datetime"
" , TIMEZONE('America/New_York', TO_TIMESTAMP(COALESCE(ve.vp_stop_timestamp, ve.tu_stop_timestamp))) as stop_arrival_datetime"
" , TIMEZONE('America/New_York', TO_TIMESTAMP(COALESCE(ve.vp_stop_timestamp, ve.tu_stop_timestamp) + ve.dwell_time_seconds)) as stop_departure_datetime"
" , (ve.vp_move_timestamp - extract(epoch FROM date(vt.service_date::text)))::int as previous_stop_departure_sec"
" , (ve.vp_move_timestamp - extract(epoch FROM date(vt.service_date::text)) + ve.travel_time_seconds)::int as stop_arrival_sec"
" , (ve.vp_move_timestamp - extract(epoch FROM date(vt.service_date::text) AT TIME ZONE 'UTC'))::int as previous_stop_departure_sec"
" , (ve.vp_move_timestamp - extract(epoch FROM date(vt.service_date::text) AT TIME ZONE 'UTC') + ve.travel_time_seconds)::int as stop_arrival_sec"
" , (ve.vp_move_timestamp - extract(epoch FROM date(vt.service_date::text) AT TIME ZONE 'UTC') + ve.travel_time_seconds + ve.dwell_time_seconds)::int as stop_departure_sec"
" , vt.direction_id::int"
" , vt.route_id"
" , vt.branch_route_id"
Expand Down Expand Up @@ -118,6 +119,7 @@ def parquet_schema(self) -> pyarrow.schema:
("stop_departure_datetime", pyarrow.timestamp("us")),
("previous_stop_departure_sec", pyarrow.int64()),
("stop_arrival_sec", pyarrow.int64()),
("stop_departure_sec", pyarrow.int64()),
("direction_id", pyarrow.int8()),
("route_id", pyarrow.string()),
("branch_route_id", pyarrow.string()),
Expand Down

0 comments on commit 911283d

Please sign in to comment.