-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow Longer keys upto 600 chars #25
Conversation
What's the math for 360 characters? I'm getting a significantly less number (112), e.g., |
We guarantee a max length of 120 chars for each of namespace, sub-namespace, and key (cf. https://github.com/lightningdevkit/rust-lightning/blob/ce7463486ee1ae61e9af439c3d34d00244248ee9/lightning/src/util/persist.rs#L35). |
Yes each of namespace subnamespace key can be up to 120 char each. But I don't think anyone else will use keys longer than ldk. So i will be fine with tighter bound as well. Current number is from 120*5 = 600 (considering some buffer and if some other application wants to use longer keys.) We could do around 400, as well, shouldn't matter much. |
Ah, thanks for explaining. Couple questions:
|
Answering one of my own questions, https://www.postgresql.org/docs/current/sql-altertable.html#SQL-ALTERTABLE-DESC-SET-DATA-TYPE I don't have a strong preference on |
I think it is basic sql that text is for unlimited size strings, and varchar is for limiting size of column. If we were using text, we owe an explanation. by default everything should be varchar if possible. |
FWIW, I'm seeing opposite advice but it doesn't touch on primary keys, FWIW. https://wiki.postgresql.org/wiki/Don't_Do_This#Text_storage Could you update the commit message to say why 600 was chosen? |
Motivation: Earlier we were conservative about our key-size since it is easier to increase but difficult to increase in backward compatible fashion. Now with introduction of sub_namespace and directly allowing upto 360char keys in LDK, we need support for longer key lengths. * We use varchar over text as it is sql standard and can be indexed in all sql-implementations while limiting key size. * LDK can have keys of length up to 362(120*3+2), we use 600 as a safe limit with some buffer and considering applications other than LDK.
Added explanation for both in description. Mainly:
For example MySQL doesn't allow indexes on text, postgres i will have to check. Hence we shouldn't use them as primary keys. Varchar and a text with constraint have similar characterstics apart from storage, indexing and sql-standardness. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if it makes sense to use a power of two for any efficiency gains, but otherwise LGTM.
Acc. to stackoverflow it doesnt matter. In reality, even much longer string lengths have comparable performance. https://stackoverflow.com/questions/593364/varchar-fields-is-a-power-of-two-more-efficient |
This is to reflect changes from lightningdevkit#25 in tests.
Motivation: Earlier we were conservative about our key-size since it is easier to increase but difficult to increase in backward compatible fashion. Now with introduction of sub_namespace and directly allowing upto 360char keys in LDK, we need support for longer key lengths.
Earlier it was 120 based on uuid length.