-
-
Notifications
You must be signed in to change notification settings - Fork 385
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Utilize NEW_TOKEN frames #1912
base: main
Are you sure you want to change the base?
Utilize NEW_TOKEN frames #1912
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall this looks pretty good, and seems well motivated. Thanks!
Thanks also for your patience while I got around to this; day job has been very busy lately.
@@ -806,16 +817,22 @@ pub struct ServerConfig { | |||
|
|||
impl ServerConfig { | |||
/// Create a default config with a particular handshake token key | |||
/// | |||
/// Setting `token_reuse_preventer` to `None` makes the server ignore all NEW_HANDSHAKE tokens. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's a NEW_HANDSHAKE token?
pub fn retry_token_lifetime(&mut self, value: Duration) -> &mut Self { | ||
self.retry_token_lifetime = value; | ||
self | ||
} | ||
|
||
/// Duration after a NEW_TOKEN frame token was issued for which it's considered valid | ||
pub fn new_token_lifetime(&mut self, value: Duration) -> &mut Self { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be clearer to refer to these as "address validation tokens"? "new token" doesn't mean anything to someone who hasn't studied the RFC.
let new_tokens_to_send = server_config | ||
.as_ref() | ||
.map(|sc| sc.new_tokens_sent_upon_validation) | ||
.unwrap_or(0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we assert that we don't have both a server config and a token store? Or maybe pass them in an enum?
if space_id == SpaceId::Data { | ||
// NEW_TOKEN | ||
while self.path.new_tokens_to_send > 0 { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Relying on PathData
like this will cause lost NEW_TOKEN
frames to not be retransmitted. Consider also adding a field to Retransmits
and coordinating it with path changes.
}); | ||
|
||
if buf.len() + new_token.size() >= max_size { | ||
self.path.pending_new_token = Some(new_token); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These look like stateless tokens. Why bother saving them if we can't fit one?
impl Default for BloomTokenReusePreventer { | ||
fn default() -> Self { | ||
// 10 MiB per bloom filter, totalling 20 MiB | ||
// k=55 is optimal for a 10 MiB bloom filter and one million hits |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there's a specific optimal k
for each size, why take it as a separate parameter rather than deriving it from size?
/// Address validation token from a NEW_TOKEN frame | ||
pub(crate) struct NewTokenToken { | ||
/// Randomly generated unique value | ||
pub(crate) rand: u128, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tokens are cryptographically authenticated, right? If so, we don't need to guard against people guessing tokens; the only concern should just be having enough space that simultaneously live tokens are unlikely to collide. Should this be a u64 instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, UUIDs are 128 bits, which implies at least some norm of 128 bits being considered a good number to guarantee non-collision.
By birthday paradox, you'd expect 64-bit tokens to collide after 2^32 uses. In a default expiration period of 2 weeks, that would mean 3 token usages per millisecond would result in 50% chance of collision. Which I guess is a bit much, especially considering that the consequences are soft.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I'll convert it to u64
, especially since that'll let the reuse preventer hash set store twice as many before converting to bloom.
match address.ip() { | ||
impl NewTokenToken { | ||
pub(crate) fn encode(&self, key: &dyn HandshakeTokenKey, address: &IpAddr) -> Vec<u8> { | ||
let aead_key = key.aead_from_hkdf(&[]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will yield the same key for every token. That means that we must use a unique nonce for each token, but AeadKey::seal
assumes a zero nonce is permissible. That only works for retry tokens because we derive a unique key for each one. As written, this is vulnerable.
/// > (Section 19.7) need to be valid for longer but SHOULD NOT be accepted multiple times. | ||
/// > Servers are encouraged to allow tokens to be used only once, if possible; tokens MAY include | ||
/// > additional information about clients to further narrow applicability or reuse. | ||
pub trait TokenReusePreventer: Send + Sync { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This name seems kind of awkward. Maybe TokenLog
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Renaming it sounds ok, maybe TokenLog
but I feel like that's a significant decrease in accuracy
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Certainly open to other ideas. Just a bikeshed, anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess there's TokenReuseMitigator
. I guess I'm not entirely sure I'm perceiving what you're perceiving about it being awkward--that might be more awkward rather than less.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/// > Servers are encouraged to allow tokens to be used only once, if possible; tokens MAY include | ||
/// > additional information about clients to further narrow applicability or reuse. | ||
pub trait TokenReusePreventer: Send + Sync { | ||
/// Called when a client uses a token from a NEW_TOKEN frame |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should document what the function should do, not (just) where it's used.
/// moment, for each of the two periods currently non-expired tokens could expire in. As such, | ||
/// turns over filters as time goes on to avoid bloom filter false positive rate increasing | ||
/// infinitely over time. | ||
pub struct BloomTokenReusePreventer { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we pull in a mature third-party bloom filter rather than rolling out own? A few moments' search turns up e.g. https://github.com/tomtomwombat/fastbloom/.
This is currently a draft PR so I can get some feedback on whether the overall design is good. If the overall design is good, I will address some remaining TODO points and polish it up some more before marking it as ready for review.
Goal and motivation:
The server now sends the client NEW_TOKEN frames, and the client now stores and utilizes them.
The main motivation is that this allows 0.5-RTT data to not be subject to anti-amplification limits. This is a scenario likely to occur in HTTP/3 requests, as one example: a client makes a 0-RTT GET request for something like a jpeg, such that the response will be much bigger than the request, and so unless NEW_TOKEN frames are used, the response may begin to be transmitted but then hit the anti-amplification limit and have to pause until the full 1-RTT handshake completes.
For example, here's some experimental data that should be similar in the relevant ways:
sudo tc qdisc add dev lo root netem delay 100ms
(and undone withsudo tc qdisc del dev lo root netem
)For responses in a certain size range, avoiding the anti-amplification limits by using NEW_TOKEN frames made the request/response complete in 1 RTT on this branch versus 2 RTT on main.
Reproducible experimental setup
newtoken.rs
can be placed intoquinn/examples/
:science.py
crates the data:graph_it.py
graphs the data, after you've manually renamed the files:Here's a nix-shell for the Python graphing:
Other motivations may include:
.retry()
, this means that requests take a minimum of 3 round trips to complete even for 1-RTT data, and makes 0-RTT impossible. If NEW_TOKENs are used, however, 1-RTT requests can once more be done in only 2 round trips, and 0-RTT requests become possible again.TokenReusePreventer
has false negatives, which may range from "sometimes" to "never," in contrast to the current situation of "always."Code change:
Key points:
NewToken
variant'saead_from_hkdf
key derivation is based on an empty byte slice&[]
rather than theretry_src_cid
.NewToken
variant's encrypted data consists of: randomly generated 128 bits, IP address (not including port), issued timestamp.ServerConfig.new_tokens_sent_upon_validation
)ClientConfig.new_token_store: Option<Arc<dyn NewTokenStore>>
object stores NEW_TOKEN tokens received by client, and dispenses them for one-time use when connecting to sameserver_name
againInMemNewTokenStore
stores 2 newest unused tokens for up to 256 servers with LRU eviction policy of server names, so as to pair well withrustls::client::ClientSessionMemoryCache
ServerConfig.token_reuse_preventer: Option<Arc<Mutex<Box<dyn TokenReusePreventer>>>>
object is responsible for mitigating reuse of NEW_TOKEN tokensDefault implementation
BloomTokenReusePreventer
:Divides all time into periods of length
new_token_lifetime
starting at unix epoch. Always maintains two "filters" which track used tokens which expires in that period. Turning over filters as time passes prevents infinite accumulation of tracked tokens.Filters start out as FxHashSets. This achieves the desirable property of linear-ish memory usage: if few NEW_TOKEN tokens are actually being used, the server's bloom token reuse preventer uses negligible memory.
Once a hash set filter would exceed a configurable maximum memory consumption, it's converted to a bloom filter. This achieves the property that an upper bound is set on the number of bytes allocated by the reuse preventer. Instead, as more tokens are added to the bloom filter, the false positive rate (tokens not actually reused but considered to be reused and thus ignored anyways) increases.
ServerConfig.new_token_lifetime
is different fromServerConfig.retry_token_lifetime
and defaults to 2 weeks.TODO: