Skip to content

docs: document the data_as_of response envelope - #1

Merged
tomaszdobrowolski1 merged 2 commits into
mainfrom
docs/data-as-of-provenance
Aug 26, 2026
Merged

docs: document the data_as_of response envelope#1
tomaszdobrowolski1 merged 2 commits into
mainfrom
docs/data-as-of-provenance

Conversation

@tomaszdobrowolski1

@tomaszdobrowolski1 tomaszdobrowolski1 commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Every FlashAlpha API response carries data_as_of, reporting when each upstream feed last delivered to the node that answered, plus endpoint_version identifying the deployment.

"endpoint_version": "2026.08.25",
"data_as_of": {
  "node": "fa2",
  "equity_feed": "2026-08-25T18:48:58.204Z",
  "equity_options_feed": "2026-08-25T18:48:57.900Z",
  "index_feed": null,
  "flow_feed": "2026-08-25T18:48:55.100Z",
  "oi_feed": "2026-08-22T20:00:00.000Z",
  "macro_feed": "2026-08-25T18:45:00.000Z"
}

Nine feeds reported separately - equity and index spot, their option chains, futures and futures options, the classified tape, settled OI, macro - because they arrive over different pipes and fail independently. An index chain can be current while the index level behind it is not, and one timestamp cannot express that.

How it should be read

The section covers how to read it, not just what it is. Each feed against its own cadence, not against as_of:

  • oi_feed at the previous session's close is correct - settled OI is published once per session, so on a Monday the newest figure that exists is Friday's. Three days behind as_of over a weekend is right, not stale.
  • An options feed an hour behind during the regular session is not correct.
  • null means that node has not seen that feed, not that it is broken.

The limit is stated alongside the claim: the field evidences that a feed delivered recently, not that every contract in a chain is equally current. An illiquid strike may not have quoted for hours while its feed is healthy.

Scope

Documentation only. No code changes.

Reference: https://flashalpha.com/docs/lab-api-overview#response-envelope


Review round 3: three blockers fixed

  1. Impossible OI timestamp. The sample showed 2026-08-22T20:00:00Z as the prior session close for an 2026-08-25 response - but 2026-08-22 is a Saturday. The prior close for Tuesday the 25th is Monday the 24th. An impossible timestamp in a provenance example is worse than none, because the field exists to teach correct market-calendar behaviour. Fixed in 15 files, docs and test fixtures.
  2. Import named a non-existent package. Live is flashalpha, replay is flashalpha-historical - not @flashalpha/sdk / @flashalpha/historical. Both were wrong; only one had been reported.
  3. "Every successful response" was an overclaim through the SDK. Bare-array endpoints put provenance in X-Data-As-Of / X-Endpoint-Version headers, and these clients return the parsed body only - so the envelope is not reachable through those methods. The claim is now scoped to JSON-object responses and the array case is documented as a limitation. On the replay service every endpoint returns an object, so the unscoped claim holds there and says so.

Plus the agent-facing gap: AGENTS.md - the file an agent actually reads as operating rules - mentioned none of this. All 11 now map each call type to the feeds that answer it, state that timestamps are UTC instants to compare not parse, judge staleness against cadence rather than the wall clock, and say the missing part outright: a null on a feed you depend on means freshness is unknown - qualify or decline, never present as current. Also: node can change between calls so timestamps must not be diffed across them, and endpoint_version is opaque deployment metadata, not a semver.

Four .csproj.lscache build artefacts committed by an over-broad git add are now untracked and ignored.

Suites green: .NET 79, Java 188, Go clean, Python 170, historical-Python 50.

Every FlashAlpha API response carries data_as_of, reporting when each
upstream feed last delivered to the node that answered, plus
endpoint_version identifying the deployment.

Nine feeds are reported separately - equity and index spot, their option
chains, futures and futures options, the classified trade tape, settled
open interest, and the macro series - because they arrive over different
pipes and fail independently. An index chain can be current while the
index level behind it is not, and one timestamp cannot express that.

The section covers how to read it, not just what it is: each feed against
its OWN cadence rather than against as_of. Settled open interest dated to
the previous session's close is correct, because it is published once per
session - on a Monday the newest figure that exists is Friday's. An
options feed an hour behind during the regular session is not. A null
means that node has not seen that feed, not that it is broken.

The limit is stated alongside the claim: the field evidences that a feed
delivered recently, not that every contract in a chain is equally current.

Documentation only. No code changes.
Review found three defects that were teaching the wrong thing.

The sample OI timestamp was 2026-08-22T20:00:00Z presented as the prior
session close for a 2026-08-25 response. 2026-08-22 is a Saturday, so no
session closed then; the prior close for Tuesday the 25th is Monday the
24th. An impossible timestamp in a provenance example is worse than no
example, because the whole point of the field is teaching people - and
models - what a correct market calendar looks like. Corrected in 15
files, docs and test fixtures alike.

The TypeScript import named a package that does not exist. The live
package is `flashalpha` and the replay package is `flashalpha-historical`,
not `@flashalpha/sdk` and `@flashalpha/historical`. Both were wrong; only
one had been reported.

"Every successful response carries data_as_of" was true of the HTTP API
but not of this client. A few endpoints return a bare JSON array, where
the API puts the envelope in the X-Data-As-Of and X-Endpoint-Version
headers instead - and these clients return the parsed body only, so the
envelope is not reachable through those methods at all. The claim is now
scoped to JSON-object responses and the array case is documented as the
limitation it is rather than implied to work. On the replay service every
endpoint returns an object, so the unscoped claim holds there and says so.

AGENTS.md gains a freshness section, which is the file an agent actually
reads as operating rules and mentioned none of this. It maps each kind of
call to the feeds that answer it, so a null on an unrelated feed is not
treated as a problem; states that timestamps are UTC instants to compare
rather than parse; judges staleness against each feed's cadence rather
than the wall clock; and - the part that was missing entirely - says that
a null on a feed you depend on means freshness is UNKNOWN, so the answer
must be qualified or declined rather than presented as current. It also
warns that node can change between calls, so timestamps must not be
diffed across calls, and that endpoint_version is opaque deployment
metadata rather than a semver to parse.

Four .csproj.lscache files were committed by an over-broad git add in the
first commit. They declare themselves disposable; now untracked and
ignored.
@tomaszdobrowolski1
tomaszdobrowolski1 merged commit d1b8da3 into main Aug 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants