feat: expose the data_as_of response envelope - #1
Merged
Conversation
The API now returns provenance on every successful response: data_as_of, reporting when each upstream feed last delivered to the node that answered, and endpoint_version identifying the deployment. Nine feeds are reported separately - equity and index spot, their option chains, futures and futures options, the classified trade tape, settled open interest, and the macro series - because they arrive over different pipes and fail independently. An index chain can be current while the index level behind it is not, and one timestamp cannot express that. Responses are dicts at runtime, so the fields were already reachable; what was missing was the typing and the documentation. DataAsOf is now a TypedDict, exported from the package root, and added to all 84 *Response types, so the envelope has editor completion and type checking. Purely additive - existing code is unaffected. README and llms.txt gain a provenance section covering how to read it: each feed against its OWN cadence rather than against as_of. Settled open interest dated to the previous session's close is correct, because it is published once per session - on a Monday the newest figure that exists is Friday's. An options feed an hour behind during the regular session is not correct. A null means that node has not seen that feed, not that it is broken. The limit is stated alongside the claim: the field evidences that a feed delivered recently, not that every contract in a chain is equally current. An illiquid strike may not have quoted for hours while its feed is healthy. Version 1.3.0.
The four statically-typed SDKs each got a sweep guard; Python and JS did not, purely because TypedDicts and interfaces are erased at runtime and there was no object to reflect over. That left the largest sweep in the fleet - 84 types in the live Python SDK - resting on a regex having touched every one, with nothing to catch a type added later. Reading the declarations from source removes the excuse. The guard walks the declared annotations rather than any instance, so it checks the source has not drifted rather than any runtime behaviour. It asserts the scan found response types at all, so it cannot pass vacuously if a module is renamed or the scan breaks, and pins the nine feed names: spot and options being reported separately is a contract shared with the live API and the other SDKs, and collapsing any pair would lose the distinction the field exists to make. Mutation-checked by removing the envelope from VexResponse.
Second-pass review against the API source found three claims that were wrong rather than merely loose. NDX was listed as an example of index_feed. It is not in IndexSymbols, so SymbolClassifier falls it through to equity - an NDX request ticks equity_feed, not index_feed. The other services in the API do treat NDX as an index, which is what made the claim look safe. Now names only roots that are actually classified as indexes. data_as_of on the replay service was described as "all null". Every feed is null, but node is always populated - it identifies which node answered. "All null" would have a caller testing the wrong thing. The historical SDKs repeated the live line about bare JSON arrays carrying the envelope in X-Data-As-Of and X-Endpoint-Version headers. That does not hold there: the header path never calls Additional(), so archive_as_of - the only provenance that means anything on a replay host - is not emitted at all, and the historical service has no bare-array endpoints in the first place. The claim is dropped rather than reworded, since the case does not arise. The nine feed names and their order were verified against FeedClock and ReplayClock in all ten SDKs: every one matches, order included.
Review found three defects that were teaching the wrong thing. The sample OI timestamp was 2026-08-22T20:00:00Z presented as the prior session close for a 2026-08-25 response. 2026-08-22 is a Saturday, so no session closed then; the prior close for Tuesday the 25th is Monday the 24th. An impossible timestamp in a provenance example is worse than no example, because the whole point of the field is teaching people - and models - what a correct market calendar looks like. Corrected in 15 files, docs and test fixtures alike. The TypeScript import named a package that does not exist. The live package is `flashalpha` and the replay package is `flashalpha-historical`, not `@flashalpha/sdk` and `@flashalpha/historical`. Both were wrong; only one had been reported. "Every successful response carries data_as_of" was true of the HTTP API but not of this client. A few endpoints return a bare JSON array, where the API puts the envelope in the X-Data-As-Of and X-Endpoint-Version headers instead - and these clients return the parsed body only, so the envelope is not reachable through those methods at all. The claim is now scoped to JSON-object responses and the array case is documented as the limitation it is rather than implied to work. On the replay service every endpoint returns an object, so the unscoped claim holds there and says so. AGENTS.md gains a freshness section, which is the file an agent actually reads as operating rules and mentioned none of this. It maps each kind of call to the feeds that answer it, so a null on an unrelated feed is not treated as a problem; states that timestamps are UTC instants to compare rather than parse; judges staleness against each feed's cadence rather than the wall clock; and - the part that was missing entirely - says that a null on a feed you depend on means freshness is UNKNOWN, so the answer must be qualified or declined rather than presented as current. It also warns that node can change between calls, so timestamps must not be diffed across calls, and that endpoint_version is opaque deployment metadata rather than a semver to parse. Four .csproj.lscache files were committed by an over-broad git add in the first commit. They declare themselves disposable; now untracked and ignored.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The API now returns provenance on every successful response. This SDK exposes it.
What changed
DataAsOfTypedDict exported from the package root, added to all 84*Responsetypes.The field
Nine feeds reported separately - equity and index spot, their option chains, futures and futures options, the classified tape, settled OI, macro - because they arrive over different pipes and fail independently. An index chain can be current while the index level behind it is not, and one timestamp cannot express that.
Compatibility
Responses are dicts at runtime, so these fields were already reachable and unknown keys were never rejected. This adds typing and docs only. No breaking change.
How it should be read
Each feed against its own cadence, not against
as_of:oi_feedat the previous session's close is correct - settled OI is published once per session, so on a Monday the newest figure that exists is Friday's. Three days behindas_ofover a weekend is right, not stale.nullmeans that node has not seen that feed, not that it is broken.The limit is documented alongside the claim: the field evidences that a feed delivered recently, not that every contract in a chain is equally current.
Reference: https://flashalpha.com/docs/lab-api-overview#response-envelope
Follow-up from self-review
Second-pass review: three factual corrections
Verified the envelope against
FeedClock/ReplayClockin the API. All ten SDKs declare the nine feed names in the correct order - no field defects. Three documentation claims were wrong:NDXwas listed as an example ofindex_feed. It is not inIndexSymbols, soSymbolClassifierfalls it through to equity - an NDX request ticksequity_feed. Other services in the API do treat NDX as an index, which is what made the claim look safe.data_as_ofon the replay service was described as "all null". Every feed is null, butnodeis always populated. "All null" would have a caller testing the wrong thing.X-Data-As-Of/X-Endpoint-Versionheaders. That does not hold there: the header path never callsAdditional(), soarchive_as_ofis not emitted at all - and the historical service has no bare-array endpoints anyway. Claim dropped rather than reworded.Suites re-run and green at baseline after the edits.
Review round 3: three blockers fixed
2026-08-22T20:00:00Zas the prior session close for an2026-08-25response - but 2026-08-22 is a Saturday. The prior close for Tuesday the 25th is Monday the 24th. An impossible timestamp in a provenance example is worse than none, because the field exists to teach correct market-calendar behaviour. Fixed in 15 files, docs and test fixtures.flashalpha, replay isflashalpha-historical- not@flashalpha/sdk/@flashalpha/historical. Both were wrong; only one had been reported.X-Data-As-Of/X-Endpoint-Versionheaders, and these clients return the parsed body only - so the envelope is not reachable through those methods. The claim is now scoped to JSON-object responses and the array case is documented as a limitation. On the replay service every endpoint returns an object, so the unscoped claim holds there and says so.Plus the agent-facing gap:
AGENTS.md- the file an agent actually reads as operating rules - mentioned none of this. All 11 now map each call type to the feeds that answer it, state that timestamps are UTC instants to compare not parse, judge staleness against cadence rather than the wall clock, and say the missing part outright: a null on a feed you depend on means freshness is unknown - qualify or decline, never present as current. Also:nodecan change between calls so timestamps must not be diffed across them, andendpoint_versionis opaque deployment metadata, not a semver.Four
.csproj.lscachebuild artefacts committed by an over-broadgit addare now untracked and ignored.Suites green: .NET 79, Java 188, Go clean, Python 170, historical-Python 50.