Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to KTX-Software v4.3.2 #1

Merged
merged 207 commits into from
Apr 14, 2024
Merged

Upgrade to KTX-Software v4.3.2 #1

merged 207 commits into from
Apr 14, 2024

Conversation

kring
Copy link
Member

@kring kring commented Apr 14, 2024

The version we were using previously had a bug where a failure to git describe would result in uncompilable code in writer2.c. git describe doesn't work with a git checkout on GitHub Actions with default options, because it avoids downloading the necessary information in order to save time.

sidsethupathi and others added 30 commits January 24, 2022 14:21
…#522)

by making BASISU_SUPPORT_SSE a CMAKE_DEPENDENT_OPTION.

Fixes KhronosGroup#499.
Fixes KhronosGroup#523. The features are KTX_FEATURE_KTX1, KTX_FEATURE_KTX2,
KTX_FEATURE_VULKAN and KTX_FEATURE_GL_UPLOAD.
And fix asserts in DeflateZstd so they fire as intended.

Fixes KhronosGroup#512.
Both require bash so change to #! /usr/bin/env bash.

Fixes KhronosGroup#500.
subrepo:
  subdir:   "lib/dfdutils"
  merged:   "58e23c17"
upstream:
  origin:   "https://github.com/KhronosGroup/dfdutils.git"
  branch:   "master"
  commit:   "58e23c17"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/MarkCallow/git-subrepo.git"
  commit:   "c1f1132"
* Validate BasisU transcoding.

   In order to keep supporting stdin, this now uses std::istream for input
   which can be from a std::ifstream, std::stringstream or std::cin. If seek
   fails on std:cin the entire file is read into a stringstream. A streambufStream
   class that wraps a std::streambuf in a ktxStream has been added as the
   way to pass the data to libktx for transcode validation.

   Fixes KhronosGroup#374.

* Fix handling of unicode filenames.

* Add tests of transcode validation and reading from stdin.

* Fix: return with non-zero code when --quiet and errors.

  The process  of adding the new tests revealed a bug in ctest that it does
  not check exit codes when a {PASS,FAIL}_REGULAR_EXPRESSION is
  specified. That bug was hiding this and another bug in ktx2check.

   Fixes KhronosGroup#531.

* Workaround ctest bug of not checking exit code
  when a {PASS,FAIL}_REGULAR_EXPRESSION is specified. Done by
  essentially duplicating certain tests to separately check exit code
  and output.

  Also tighten RE for matching --version output.

* Ensure correct RE match result on non-nul-terminated metadata value.
  Windows RE only matches when nul-terminated.

  Add more exit-code tests.
subrepo:
  subdir:   "lib/dfdutils"
  merged:   "e742e8e2"
upstream:
  origin:   "https://github.com/KhronosGroup/dfdutils.git"
  branch:   "master"
  commit:   "e742e8e2"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/MarkCallow/git-subrepo.git"
  commit:   "c1f1132"
normalMap in ktxAstcParams and ktxBasisParams (`--normal_mode` in `toktx`), causes the
encoders to split the R & G components of the input into the RGB & alpha channels of the
encoded texture and to apply encoder-specific settings that improve quality for normal
maps.

Add `--normalize`  option to `toktx.

Add tests for normalMap and normalize functionality.

Deprecate `separateRGToRGB_A` and remove related code. Fixes KhronosGroup#455.

Co-authored-by: Wasim Abbas <abbas.wasim@gmail.com>
Co-authored-by: Wasim Abbas <abbas.wasim@gmail.com>
* git subrepo pull (merge) lib/astc-encoder

subrepo:
  subdir:   "lib/astc-encoder"
  merged:   "adc637b0"
upstream:
  origin:   "https://github.com/ARM-software/astc-encoder.git"
  branch:   "main"
  commit:   "42a8f6ee"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

* Update gold images

Co-authored-by: Wasim Abbas <abbas.wasim@gmail.com>
* git subrepo pull (merge) lib/astc-encoder

subrepo:
  subdir:   "lib/astc-encoder"
  merged:   "7a0d929e"
upstream:
  origin:   "https://github.com/ARM-software/astc-encoder.git"
  branch:   "main"
  commit:   "a33dbb44"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/ingydotnet/git-subrepo"
  commit:   "2f68596"

Co-authored-by: Wasim Abbas <abbas.wasim@gmail.com>
* git subrepo pull (merge) lib/basisu

    subrepo:
      subdir:   "lib/basisu"
      merged:   "5dc6149f"
    upstream:
      origin:   "https://github.com/BinomialLLC/basis_universal.git"
      branch:   "master"
      commit:   "44e1bcd3"
    git-subrepo:
      version:  "0.4.3"
      origin:   "https://github.com/MarkCallow/git-subrepo.git"
      commit:   "c1f1132"

    Get everything working with BasisU 1.16.1.

    - Remove deleted files from and add new files to project.
    - Remove obsolete global_selector references.
    - Fix compile warnings in basisu_fronten.cppd and basisu_transcoder.cpp.
    - ifdef problematic printfs in basisu_opencl.
    - Remove error condition when failing to find OpenCL GPU device.
    - Move lodepng to toktx source folder.
    - Regenerate reference files.

    Incorporating this version of Basis triggered a major reworking of the
    build matrix on Appveyor to incorporate test building more configs
    including the OpenCL config. This in turn triggered moving the install
    and build phase code to scripts.

   Incidentally fixes KhronosGroup#404.
* Remove image.hpp dependency

* Fix memset warning

Co-authored-by: Wasim Abbas <abbas.wasim@gmail.com>
* git subrepo commit (merge) lib/basisu

subrepo:
  subdir:   "lib/basisu"
  merged:   "7e156388"
upstream:
  origin:   "https://github.com/BinomialLLC/basis_universal.git"
  branch:   "master"
  commit:   "1f4b564a"
git-subrepo:
  version:  "0.4.3"
  origin:   "https://github.com/MarkCallow/git-subrepo.git"
  commit:   "c1f1132"

* Fix syntax errors in on_failure.ps1.
Co-authored-by: Wasim Abbas <abbas.wasim@gmail.com>
)

Expose the main CMake config knobs as environment variables to tailor build configurations & jobs.

Replace macOS Universal Binary builds with Apple Silicon builds while retaining x86_64 builds.

Add new configs/jobs for testing OpenCL and SSE combos.

Cleanup CI files.
…Group#548)

* Add explicit static_cast to convert from ktx_size_t to jsize in JNI library

* Add bounds check for array allocation

* Remove separateRGToRGB_A from KtxBasisParams, and add perceptual to KtxAstcParams
* Fix collision between our global 'm' and one in gtest.

* Suppress gcc unused result warning in  etcdec.cxx.

* Fix warnings newly added in gcc 11.

  The warning in lodepng.cc is bogus and has been suppressed.

* (Unrelated) avoid exit in on_failure.ps1.
Enable building load tests for Apple Silicon.

* Update the included macOS libraries to universal binaries which
  resulted in updating libassimp to v5.2.2 and SDL to 2.0.20. The
  latter was for Apple Silicon support.

* Make symbol visibility of the load tests apps and associated libraries
  compatible with libassimp i.e. hidden.

* A minimum of Vulkan SDK 1.2.189 is now required for Apple Silicon
  support in the form of a universal binary.

* Make macOS load test app bundles self-contained.

Other notable changes included here:

* Polish the documentation.

  - Add information about Apple Silicon support.

  - Add note that bash is needed on Windows for mkversion.

* Delete no-longer used SDL binaries: the SDL framework for macOS
  and all binaries for GNU/Linux.

* Properly handle the ramifications of exceptions in sample construction.

* Fix descriptions of some samples.

* Remove old commented-out code from glloadtest.cmake.

* Clean-up git lfs pulls so the minimum data is pulled from LFS for each build.

* Switch to latest Xcode/macOS (Xcode 13.2/macOS 12.2) in Travis CI builds.
These are set-but-not-used warnings new in Xcode 13.3.

* Rationalize suppressions of old and new warnings

* Incidentally, fix build_wasm_docker.sh to use EXIT trap to ensure terminating the docker container.
Raffaello and others added 25 commits January 2, 2024 09:37
…KhronosGroup#840)

The option defaults to OFF. Only project developers need to regenerate
these files.

Fixes KhronosGroup#834.

Other included related fixes:
* Modify mkvkformatfiles to write CRLF on Windows.
* Stop using unix2dos in mkvk custom commands and remove near
  duplicate WIN32 custom commands. Developers must ensure they
  use a Windows native Perl to get the correct line endings when
  generating dfdutils' `*.inl` files.
* Generate dfdutils .inl files from vulkan_core not vkformat_enum
  to remove dependency on the latter's generation.
* Add CI jobs to test generation on all platforms since main builds
  will no longer be building those targets.

Unrelated fixes:
* Set `HOMEBREW_NO_AUTO_UPDATE` for macOS Travis CI jobs
  to finally stop the auto update and shave more than 1 hour off the
  total build time.
* Bracket the whole `script` section of `.travis.yml` with `set -e` and
`set +e`
  instead of piecemeal in various `if` clauses.
Fixes KhronosGroup#838.

Contains 2 minor unrelated fixes due to overhead of PRs:
* Move include of CMakePrintHelpers to root CMakeLists.txt so it is
  available everywhere.
* Don't set Vulkan_INCLUDE_DIR in mkvk.cmake.
  - Fixes vkloadtests build when KTX_GENERATE_VK_FILES is ON.
This is needed since we don't have a workflow to build manylinux wheels
at moment, and [it's non-trivial to implement
that](https://github.com/pypa/manylinux?tab=readme-ov-file#docker-images)
…up#849)

Fix manual publishing of pyktx to PyPI.org.

Update `checkout`, `setup-java`, `setup-python` and `upload-artifact`
actions to avoid warnings about deprecation of
older versions of node.js.

---------

Co-authored-by: Shukant Pal <security@shukantpal.com>
)

A change we made to the the subrepo'ed BasisU code as part of the
mass warning fixes in PR KhronosGroup#687 broke initialization of the KTX2 header
when BasisU is creating a .ktx2 file. This code is not used by libktx.
The fix is being committed to avoid inadvertently pushing the broken
code upstream.

The user who encountered this was doing something unsupported by libktx.
Fixes KhronosGroup#857. It also fixes the part of KhronosGroup#853 that is related to the
changes about `mPACKn` rules.

Both of these affected the `16X4` and `10X6` formats, so they were
addressed together.
of correct result for all valid VkFormats.

Modify mkvkformatfiles to generate the list of valid format names
to populate the table that drives the new test.

Fix dfdutils Doxygen config file to include queries.c in the input.
Fixes issue KhronosGroup#860.

Updates the switches used by the the GL loader to those generated by
`generate_format_switches.rb` in the KTX-Specification repo as a way to
get them updated for these and some other recently added Vulkan formats.
The glloader now loads the newly allowed formats.

VkFormat declarations for Java and Python are now generated by
`mkvkformatfiles` which has been moved to ci_scripts since it now
generates files outside the `lib` directory. The unattractive alternative
was to hand edit these files to add the newly allowed formats.

CI testing of vkformat-related file generation (`mkvk`) has been moved
to a dedicated GitHub workflow and removed from the main CI builds. This
is because testing now requires a clone of the KTX-Specification repo due
to the use of `generate_format_switches.rb`.

The change to `createdfd.c` is to remove a duplicated line of code and
is unrelated to the purpose of this PR.

KhronosGroup/KTX-Software-CTS#19 removes the
tests for these as prohibited and adds new tests for the formats.
when not found. Fix comments in FindBash.cmake.
This includes a new round-trip test of vk2dfd and dfd2vk for all Vulkan
formats except multi-plane and modifications to both functions, or
rather their generators, to fix support for some formats, including
VK_FORMAT_R16G16_S10_5_NV and the YUV 4:2:2 single plane formats. Those
listed required changes to`interpretDFD`.

Fixes KhronosGroup#859.

Co-authored-by: Andrew Garrard <fluppeteer@users.noreply.github.com>
@kring kring merged commit 6a231bd into cesium-native Apr 14, 2024
15 checks passed
@kring kring deleted the cesium-native-upgrade branch April 14, 2024 23:58
yunusyurtturk added a commit to yunusyurtturk/osgearth that referenced this pull request Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.