Skip to content

feat: import verified Origin OPJ worksheets - #5

Merged
nmrtist merged 2 commits into
nmrtist:mainfrom
Limdongcheng:codex/origin-project-import
Aug 1, 2026
Merged

feat: import verified Origin OPJ worksheets#5
nmrtist merged 2 commits into
nmrtist:mainfrom
Limdongcheng:codex/origin-project-import

Conversation

@Limdongcheng

@Limdongcheng Limdongcheng commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add experimental, content-detected Origin OPJ worksheet import without installing, launching, or calling Origin
  • support the fixture-proven Origin 7.0552 profile and the exact Origin 9.51 build 195 W64 numeric worksheet profile verified on two user-owned local projects
  • import supported worksheet values, nulls, column names, workbook/window names, and profile metadata through PlotX's existing table preview flow
  • recognize OPJU projects and reject them with a clear unsupported-format error without creating partial output
  • document the tested compatibility boundary in English and Simplified Chinese

This PR intentionally focuses on correct worksheet import. It does not reconstruct Origin graphs. The generic table-import workflow may create a blank PlotX canvas; that canvas is not an imported Origin graph.

Technical approach

  • plotx-io owns content probing, checked OPJ framing and record parsing, metadata association, cell decoding, resource budgets, and the engine-neutral project model
  • plotx-core preflights and converts supported worksheets into typed table snapshots without adding an execution-engine dependency to plotx-data
  • the desktop application probes by content, opens the selected path once, verifies the handle is a regular file, reuses that handle for classification and import, and presents every imported table in the existing preview/confirm flow
  • Origin 9.51 records are accepted only when the complete observed type tuple matches; unobserved combinations fail closed instead of being inferred from individually known fields
  • the parser is an idiomatic Rust implementation informed by the MIT-licensed OpenOPJ record descriptions; GPL liborigin source was used only as an independent behavioral reference and was not copied, translated, linked, or shipped

Actual support boundary

OPJ import

Verified Origin 7 profile:

  • classic CPYA 4.2673 552# framing with embedded Origin version 7.0552
  • f64, f32, signed i32, signed i16, fixed-width ASCII text, mixed numeric/text cells, nulls, and nonzero row offsets
  • validated workbook/window names, column names, project parameters, and notes

Verified Origin 9 profile:

  • exact CPYA 4.3268 195 W64 # framing with embedded version 9.510195
  • numeric f64 values and nulls in the exact observed 10-byte cell layout
  • validated empty columns, workbook/window names, dataset-derived column names, source profile, and producer version metadata
  • compatibility checked against two user-owned local projects and independent liborigin/CSV exports; those private files and their scientific data are not committed

Other OPJ producer versions and unobserved record layouts are recognized or rejected explicitly, not treated as compatible based only on the .opj extension.

OPJU recognition

CPYUA OPJU files are recognized by content, but no OPJU variant is imported in this PR. PlotX returns a clear unsupported message and creates no partial table or recent-file entry.

Known limitations

  • no Origin graph reconstruction, plot styling, formulas, scripts, analysis recomputation, executable saved analyses, matrices, embedded objects, preview images, OLE/XML extraction, or protected/encrypted projects
  • Origin 9.51 text cells, dates, categorical values, notes, parameters, attachments, presentation properties, and other producer builds are not supported
  • Origin 7 support does not claim non-ASCII code pages, dates, categorical values, units, comments, column designations, long-name variants, or other OPJ versions
  • unsupported independently framed columns may be skipped only after their boundaries validate, with a visible warning; ambiguous grouping, corrupt framing, unsupported versions, and files with no safely importable worksheet fail closed
  • compatibility claims apply to the exact tested profiles, not every file carrying an .opj or .opju suffix

Safety and error handling

  • signature and initial-structure detection instead of extension-only dispatch
  • 128 MiB default input cap, checked one-byte oversize sentinel, fixed-size reads, fallible reservation, and bounded retained/allocation accounting
  • checked offsets, lengths, counts, arithmetic, row/cell/string/metadata limits, and bounded diagnostic work
  • same-handle reads, regular-file verification, and nonblocking Unix opens reduce path-replacement and special-file blocking risks
  • no archive extraction, decompression, filesystem path extraction, script execution, or embedded object execution
  • truncation, corruption, invalid lengths and limits, unsupported versions/content, encoding failures, and allocation failures become structured errors or user-visible warnings rather than panics or silent data changes

Fixtures and licenses

  • test-origin-7.0552.opj: OpenOPJ test.opj, pinned source commit 42ddcf1eb3a490744c54fca0a4ed6fe7a5e723ca, MIT, redistributed unchanged; source, attribution, byte length, and SHA-256 are recorded beside the fixture
  • RawData_Locust_Revision1_TIS_Mechanism.opju: Figshare DOI 10.6084/m9.figshare.28535426.v1, CC BY 4.0, redistributed unchanged; source, attribution, byte length, and SHA-256 are recorded beside the fixture
  • Origin 9.51 regression fixtures are synthetic framed bytes generated in tests; the two real acceptance projects are user-owned, private, and remain outside the repository

Verification

  • cargo pr-check passed all 7 stages on final HEAD: formatting, 800-line source limit, dependency licenses/advisories, default frontends, Clippy with warnings denied, all-feature/DataFusion tests, and reference-backend tests
  • cargo test -p plotx-io --locked origin: 124 matching unit/integration tests passed
  • cargo test -p plotx-core --locked origin: 29 matching tests passed
  • cargo test -p plotx --locked ui::file_dialogs: 45 matching desktop import tests passed
  • manual desktop acceptance imported worksheets from a local Origin 9.51 project; the displayed blank canvas is the generic destination canvas, not a claim of graph import
  • npm run build from docs/: 69 English and Simplified Chinese pages built with Pagefind and sitemap
  • final independent review found no remaining Critical, Important, or Minor issues

npm ci reported five advisories in the existing documentation dependency tree (one moderate and four high). This PR does not change the documentation lockfile or apply a potentially breaking audit fix.

Follow-up work

  • support more OPJ versions only after obtaining legally redistributable fixtures and independent value/metadata verification
  • implement a complete bounded OPJU outer-container profile before enabling OPJU import
  • consider automatic PlotX chart creation as a separate product feature; it is deliberately outside this worksheet-import PR

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@Limdongcheng

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

@Limdongcheng

Copy link
Copy Markdown
Contributor Author

recheck

@Limdongcheng Limdongcheng changed the title feat: import Origin 7.0552 OPJ worksheets feat: import verified Origin OPJ worksheets Jul 30, 2026
@nmrtist
nmrtist enabled auto-merge (squash) July 30, 2026 16:07
@nmrtist
nmrtist disabled auto-merge July 30, 2026 16:07
@Limdongcheng
Limdongcheng force-pushed the codex/origin-project-import branch 2 times, most recently from 0bb06ca to 5a06517 Compare August 1, 2026 06:24
@nmrtist
nmrtist merged commit 0f69905 into nmrtist:main Aug 1, 2026
11 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 1, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants