Test failures on Debian i386

[plugwash]

I recently updated the hdrhistogram package in Debian, and as part of that I resolved the issues that were blocking the tests from running on Debian's test infrastructure.

Unfortunately when running the CI tests on i386 two tests failed, I can also reproduce this locally.

failures:

---- iter_quantiles_saturated_count_before_max_value stdout ----
thread 'iter_quantiles_saturated_count_before_max_value' panicked at 'capacity overflow', library/alloc/src/raw_vec.rs:518:5
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.59.0/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.59.0/library/core/src/panicking.rs:116:14
   2: core::panicking::panic
             at /usr/src/rustc-1.59.0/library/core/src/panicking.rs:48:5
   3: alloc::raw_vec::capacity_overflow
             at /usr/src/rustc-1.59.0/library/alloc/src/raw_vec.rs:518:5
   4: alloc::raw_vec::handle_reserve
             at /usr/src/rustc-1.59.0/library/alloc/src/raw_vec.rs:489:34
   5: alloc::raw_vec::RawVec<T,A>::reserve::do_reserve_and_handle
             at /usr/src/rustc-1.59.0/library/alloc/src/raw_vec.rs:287:13
   6: alloc::raw_vec::RawVec<T,A>::reserve
             at /usr/src/rustc-1.59.0/library/alloc/src/raw_vec.rs:291:13
   7: alloc::vec::Vec<T,A>::reserve
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/mod.rs:809:9
   8: alloc::vec::Vec<T,A>::extend_desugared
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/mod.rs:2642:17
   9: <alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<T,I>>::spec_extend
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/spec_extend.rs:18:9
  10: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/spec_from_iter_nested.rs:37:9
  11: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/spec_from_iter.rs:33:9
  12: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/mod.rs:2541:9
  13: core::iter::traits::iterator::Iterator::collect
             at /usr/src/rustc-1.59.0/library/core/src/iter/traits/iterator.rs:1745:9
  14: iterators::iter_quantiles_saturated_count_before_max_value
             at ./tests/iterators.rs:569:55
  15: iterators::iter_quantiles_saturated_count_before_max_value::{{closure}}
             at ./tests/iterators.rs:561:1
  16: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.59.0/library/core/src/ops/function.rs:227:5
  17: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.59.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- iter_quantiles_iterates_to_quantile_10_as_it_reaches_last_bucket stdout ----
thread 'iter_quantiles_iterates_to_quantile_10_as_it_reaches_last_bucket' panicked at 'capacity overflow', library/alloc/src/raw_vec.rs:518:5
stack backtrace:
   0: rust_begin_unwind
             at /usr/src/rustc-1.59.0/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at /usr/src/rustc-1.59.0/library/core/src/panicking.rs:116:14
   2: core::panicking::panic
             at /usr/src/rustc-1.59.0/library/core/src/panicking.rs:48:5
   3: alloc::raw_vec::capacity_overflow
             at /usr/src/rustc-1.59.0/library/alloc/src/raw_vec.rs:518:5
   4: alloc::raw_vec::handle_reserve
             at /usr/src/rustc-1.59.0/library/alloc/src/raw_vec.rs:489:34
   5: alloc::raw_vec::RawVec<T,A>::reserve::do_reserve_and_handle
             at /usr/src/rustc-1.59.0/library/alloc/src/raw_vec.rs:287:13
   6: alloc::raw_vec::RawVec<T,A>::reserve
             at /usr/src/rustc-1.59.0/library/alloc/src/raw_vec.rs:291:13
   7: alloc::vec::Vec<T,A>::reserve
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/mod.rs:809:9
   8: alloc::vec::Vec<T,A>::extend_desugared
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/mod.rs:2642:17
   9: <alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<T,I>>::spec_extend
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/spec_extend.rs:18:9
  10: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/spec_from_iter_nested.rs:37:9
  11: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/spec_from_iter.rs:33:9
  12: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /usr/src/rustc-1.59.0/library/alloc/src/vec/mod.rs:2541:9
  13: core::iter::traits::iterator::Iterator::collect
             at /usr/src/rustc-1.59.0/library/core/src/iter/traits/iterator.rs:1745:9
  14: iterators::iter_quantiles_iterates_to_quantile_10_as_it_reaches_last_bucket
             at ./tests/iterators.rs:717:55
  15: iterators::iter_quantiles_iterates_to_quantile_10_as_it_reaches_last_bucket::{{closure}}
             at ./tests/iterators.rs:703:1
  16: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.59.0/library/core/src/ops/function.rs:227:5
  17: core::ops::function::FnOnce::call_once
             at /usr/src/rustc-1.59.0/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    iter_quantiles_iterates_to_quantile_10_as_it_reaches_last_bucket
    iter_quantiles_saturated_count_before_max_value

test result: FAILED. 18 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 39.15s

Debian i386 uses the x87 FPU and I strongly suspect it's quirky floating point is involved in these failures, but I have no idea how to debug this further.

[jonhoo]

Hmm, very weird. It seems likely that you're right and the FPU doing something weird is the culprit here. Specifically, that first test attempts to put a high count in the first bucket, and then checks that iterating over the quantiles yields exactly the expected steps. But the panic you're seeing suggests that the quantiles iterator yields u32::MAX elements, which it definitely shouldn't. You can see the setup (which relies on some very large f64s and some casts) here:

HdrHistogram_rust/tests/iterators.rs

Lines 709 to 751 in 5854b2e

	let total = 10_000_000_000_000_000_u64;
	let quantile = 0.9999999999999998_f64;
	let first_bucket = (quantile * total as f64) as u64;
	assert_eq!(9999999999999998, first_bucket);
	h.record_n(1, first_bucket).unwrap();
	// and now the leftovers to reach the total
	h.record_n(2, 2).unwrap();
	let iter_values: Vec<(u64, u64, u64, f64, f64)> = h
	.iter_quantiles(2)
	.map(|v| {
	(
	v.value_iterated_to(),
	v.count_since_last_iteration(),
	v.count_at_value(),
	v.quantile(),
	v.quantile_iterated_to(),
	)
	})
	.collect();
	let expected = vec![
	(1, first_bucket, first_bucket, quantile, 0.0),
	(1, 0, first_bucket, quantile, 0.25),
	(1, 0, first_bucket, quantile, 0.5),
	(1, 0, first_bucket, quantile, 0.625),
	(1, 0, first_bucket, quantile, 0.75),
	(1, 0, first_bucket, quantile, 0.8125),
	(1, 0, first_bucket, quantile, 0.875),
	(1, 0, first_bucket, quantile, 0.90625),
	(1, 0, first_bucket, quantile, 0.9375),
	(1, 0, first_bucket, quantile, 0.953125),
	(1, 0, first_bucket, quantile, 0.96875),
	(1, 0, first_bucket, quantile, 0.9765625),
	(1, 0, first_bucket, quantile, 0.984375),
	(1, 0, first_bucket, quantile, 0.98828125),
	(1, 0, first_bucket, quantile, 0.9921875),
	(1, 0, first_bucket, quantile, 0.994140625),
	(1, 0, first_bucket, quantile, 0.99609375),
	(1, 0, first_bucket, quantile, 0.9970703125),
	(1, 0, first_bucket, quantile, 0.998046875),
	(1, 0, first_bucket, quantile, 0.99853515625),
	(1, 0, first_bucket, quantile, 0.9990234375),

I think the way I'd go about debugging this is to print out the items yielded by the quantile iterator to try and see what exactly goes wrong. For example, it could be that it gets "stuck" on a particular quantile, or it underflows/underflows somewhere and so basically ends up counting to u64::MAX or some such. You should be able to just inject a .map(|v| {println!("{:?}", v); v}) after

HdrHistogram_rust/tests/iterators.rs

Line 727 in 5854b2e

})

[jonhoo]

Also cc @marshallpierce who may find this interesting.

[marshallpierce]

I think we should be able to rule out under/overflow at least because in tests an overflow would panic, right?

[jonhoo]

Does that hold for casts too? I thought it would only panic for arithmetic over/underflow, but could be wrong.

Though having said that I guess there should be very few casts of relevance involved here.

[jamessan]

For example, it could be that it gets "stuck" on a particular quantile,

It does get stuck, on (1, 0, 9999999999999998, 0.9999999999999998, 0.9999999999999998)