nist_freq_block does not err when returning NaN

[chrysn]

To be fair, I have little clue of what I'm doing here (wrapping an embedded OS's RNG into the rand_core traits, writing tests for that, and I'd to at least catch if the underlying generator is XKCD style), but I hope this is valuable anyway:

nist_freq_block returns Ok(NaN) to me, even for any input I've tried (up to 1000000 bits produced from what is supposed to be a CSRNG, tried block_len values 2, 8, 32, 128 and 1024, some of them even with 1Gbit input).

Unless Ok(NaN) is really a sensible result (I guess not, given the if p < 0.01 { return Err(Error::BadPValue(p)); } check), I think that NaNs should be reported as some form of error; a !(p >= 0.01) check would minimally do that.

[ryankurte]

I'd to at least catch if the underlying generator is XKCD style

that's the goal! turns out embedded RNGs aren't always trustworthy / configured correctly and it's nice to know before you do anything, although the one thing i have discovered is that given enough runs sometimes an RNG will return all 4s... it'd probably make sense to wrap the suite of tests in something that takes an RNG and configures tests / does retries.

nist_freq_block returns Ok(NaN) to me, even for any input I've tried (up to 1000000 bits produced from what is supposed to be a CSRNG, tried block_len values 2, 8, 32, 128 and 1024, some of them even with 1Gbit input).

that seems cursed, you're right this should probably report an error but also there's got to be an underlying bug to get to the NaN in the first place... either a block_n ending up as zero in a divide, or something propagating through the igamma function, you should be able to see this appear with a reasonably small block size and a debugger if you were interested?

there is a constraint on that particular test 800-22 2.2.7 that the block size should be >= 20 (though they use 10 in the example) and there should be > 10 tests, have you tried say a 1000 bit test stream with 100 bit blocks? (also it occurs to me that cases where data_len % block_len != 0 may not be well handled.)

an example that's working for me:

// Fetch random bytes for tests
let mut a = [0xFF; 100];
rng.fill_bytes(&mut a);

// Check we filled -something- before attempting more in-depth tests
if &a[..2] == &[0xFF; 2] && &a[a.len() - 2..] == &[0xFF; 2] {
    return Err(rngcheck::Error::RngFailed);
}

// Run NIST frequency checks
nist_freq_monobit(BitIter::new(&a))?;
nist_freq_block(BitIter::new(&a), 10)?;

[chrysn]

Hm, even your example also gives me NaN, both with std and my embedded RNG, no matter the divisibility (the ? hides that there is a NaN -- so without a closer look, that example indeed appears to work but doesn't):

//! ```cargo
//! [dependencies]
//! rand_core = { version = "0.6", features = [ "getrandom" ] }
//! rngcheck = "0.1.1"
//! ```

use rand_core::*;
use rngcheck::{nist::*, helpers::*};

fn main() {
    let mut rng = rand_core::OsRng;

    // Fetch random bytes for tests
    let mut a = [0xFF; 100];
    rng.fill_bytes(&mut a);

    // Check we filled -something- before attempting more in-depth tests
    if &a[..2] == &[0xFF; 2] && &a[a.len() - 2..] == &[0xFF; 2] {
        panic!("RNG no-oped?");
    }

    // Run NIST frequency checks
    dbg!(nist_freq_monobit(BitIter::new(&a)));
    dbg!(nist_freq_block(BitIter::new(&a), 10));
}

$ cargo +nightly -Zscript ./test.rs
[...]
     Running `/home/chrysn/.cache/cargo-target/debug/test-`
[test.rs:23] nist_freq_monobit(BitIter::new(&a)) = Ok(
    0.257899,
)
[test.rs:24] nist_freq_block(BitIter::new(&a), 10) = Ok(
    NaN,
)

I'm having a closer look, gotta be somewhere...

[chrysn]

The culprits for the NaN is dbg!(libm::powf(x, a)) * libm::expf(-x) / dbg!(libm::tgammaf(a + 1.0)) being inf/inf; for values of x = 37.99999something and a = 40.0.

The calculations work out for this example when using f64 internally. Still NaNs out on tests with somewhere between 1kbit and 10kbit (block size 16), but it's a start. Instead, I've dug around and found the (no_std) "special" crate (yes that's the name, b/c gamma is one of the so-called special functions). It's no_std, passes the internal tests, so I guess that's fine?