Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transform individual splats #217

Merged
merged 26 commits into from
Oct 17, 2024
Merged

Conversation

slimbuck
Copy link
Member

This PR add the ability to move/rotate/scale individual splats using transform gizmo and transform panel.

Screen.Recording.2024-10-15.at.11.07.54.mov

The code changes can be summarised as follows:

  • the transform panel and gizmos now interact with a Pivot object
  • two new TransformHandler classes respond to pivot updates by applying the changes to the scene and updating the undo/redo history
  • in order to transform large scenes in realtime we introduce a Transform Palette
    • the palette stores 3x4 transform matrices in an RGBA32F texture
    • each splat has a uint32 transform palette index assigned. 0 means no transform.
    • the transform palette is managed and updated during edits
  • since splat positions are dynamic, some JS (cpu) processing has to be moved to GLSL (gpu), including:
    • intersection testing for mask/rect/sphere selections
    • bounding box calculations
    • splat centers updates (used for render sorting of splats)
    • these are all implemented in data-processor.ts

Issues/Concerns

  • this change results in higher memory usage (primarily for transform palette indices per splat) and slightly slower rendering due to transform palette
  • the transform palette supports 4096 matrices initially and just doubles in size every time more space is needed
  • the SPLAT DATA panel doesn't reference the transformed splat positions. this will be fixed in a followup PR.
  • Value field manipulation with mouse doesn't work #212 is still present, will be addressed in a followup PR

Testing notes

Full regression testing is needed, specifically:

  • PLY and compressed PLY loads correctly
  • PLY, compressed PLY and .splat formats save correctly with arbitrary edits
  • gizmos work correctly under position/rotation/scale
  • the transform panel works correctly under position/rotation/scale changes
  • undo/redo history works correctly

@slimbuck slimbuck added the enhancement New feature label Oct 16, 2024
@slimbuck slimbuck requested a review from a team October 16, 2024 10:57
@slimbuck slimbuck self-assigned this Oct 16, 2024
src/camera.ts Outdated Show resolved Hide resolved
src/data-processor.ts Outdated Show resolved Hide resolved
@simonbethke
Copy link
Contributor

simonbethke commented Oct 16, 2024

Saving is currently broken:

TypeError: Cannot read properties of null (reading '0')
    at uw.getMat (splat-convert.ts:74:40)
    at mw (splat-convert.ts:188:40)
    at file-handler.ts:333:28
    at file-handler.ts:330:22
    at async Sy.<anonymous> (file-handler.ts:225:13)

@slimbuck
Copy link
Member Author

slimbuck commented Oct 16, 2024

Saving is currently broken:

Thanks for this @simonbethke - fixed.

I hadn't tested not changing anything then trying to save yet :)

@simonbethke
Copy link
Contributor

The "Reset Splat" function is not resetting the TransformsPalette


// reallocate the storage texture and copy over old data
const realloc = (width: number, height: number) => {
const newTexture = new Texture(device, {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a note that when you move this to the engine v2 you can use texture.resize()

@slimbuck slimbuck merged commit 94d0043 into playcanvas:main Oct 17, 2024
2 checks passed
@slimbuck slimbuck deleted the splat-move-dev branch October 17, 2024 09:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants