Skip to content

Fix CPU dequantize_4bit output shape for 1-D inputs - #2048

Open
2sumtech wants to merge 1 commit into
bitsandbytes-foundation:mainfrom
2sumtech:fix/dequantize-4bit-1d-shape
Open

Fix CPU dequantize_4bit output shape for 1-D inputs#2048
2sumtech wants to merge 1 commit into
bitsandbytes-foundation:mainfrom
2sumtech:fix/dequantize-4bit-1d-shape

Conversation

@2sumtech

Copy link
Copy Markdown

Fixes #2047

The native CPU dequantize_4bit mutated a 1-D shape to (1, n) before allocating the output, so even-length 1-D inputs dequantized to 2-D. The (1, n) view is only needed for the C++ kernel's 2-D indexing, and m = prod(shape[:-1]) is already 1 for 1-D inputs — this PR drops the mutation so the output keeps the caller-provided shape, matching the CUDA backend, the default/MPS backends, the registered fake kernel, and the CPU backend's own odd-length fallback path.

  • Output values are bit-for-bit identical before and after over 36 configurations (fp32/fp16/bf16 × nf4/fp4 × blocksize 64/128 × n 128/256/4096); only the shape changes, (1, n) -> (n,).
  • New regression test test_dequantize_4bit_1d_shape (24 params): fails 12/12 on CPU without the fix, passes 24/24 (cpu+mps) with it.
  • Test4bitBlockwiseQuantOps: 536 passed, 24 skipped, 60 xfailed. test_functional.py -k 4bit: 1958 passed. Full pre-commit suite passes on touched files.
  • Verified on macOS arm64 (NEON path) + MPS; the fix is backend-dispatch Python only, no csrc change. I could not run the CUDA/AVX-512 paths on this machine, but they are untouched.

Disclosure: prepared with AI assistance (Claude Code); I reviewed the change and take responsibility for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CPU dequantize_4bit returns shape (1, n) for even-length 1-D inputs; all other backends return (n,)

1 participant