fix(core): read emphasis all-caps as words in AnA - #4180
Conversation
Closes Automattic#3921. `AnA` treated every all-caps token as an initialism, so "a FREE event" was flagged in favour of "an FREE event". The likely-acronym check only looks at the vowel-ness of the first two characters, which cannot tell an ordinary word written for emphasis from a letter-by-letter abbreviation. Use the orthography the dictionary already records instead. An all-caps token whose entry licenses a lowercase spelling but no all-caps one is a word being shouted, so read it as the lowercase word. Entries that do license all-caps (`LAN`, `NAT`, `MA`, `FBI`) keep the initialism handling, and Roman numerals are excluded because they are numbers rather than words. That guard is only as good as the all-caps coverage in the dictionary, so add the entries for the abbreviations it would otherwise misread: FPS frames per second; first-person shooter FTP file transfer protocol FTPS FTP over TLS MPG miles per gallon SSH secure shell Verified on 999 GitHub READMEs and 15 Wikipedia articles, 1.9M words: `AnA` goes from 178 lints to 170. Nine removed were false positives ("a FREE provider", "a FLASH based filing system", "a SYN flood attack"), and the one added is correct ("an SYN data packet" -> "a SYN", since SYN is read "sin"). `cargo test -p harper-core` passes except for the pre-existing `curated_default_config_lists_every_registered_rule` drift, which fails identically on `master`. `cargo fmt` is clean and `cargo clippy -p harper-core --all-targets -- -D warnings` reports the same 17 errors as `master`.
|
Answering the casing question with numbers, since I still had the corpus set up. First, it doesn't block this PR. The guard needs Where it does land is I built the branch twice, once as-is and once with +76 Capitalization lints, 0 removed. On your frequency evidence — you're right, and I don't think it settles it. I counted the same corpus with code stripped and all-caps wins there too: But dominance isn't quite the question being asked. It tells us the all-caps spelling has to be in the dictionary, which is what this PR adds. It doesn't tell us the lowercase spelling should be rejected, and rejecting is what all-caps-only actually does. Sampling the 76:
The first two are plainly wrong to flag. Collins lists "FTP in British English or ftp" and Oxford Learner's carries a lowercase One knock-on worth flagging:
So my vote is both spellings: all-caps is the canonical form, which is what the new entries record, but lowercase is still attested and costs nothing to accept. If you'd rather go all-caps-only on |


Closes #3921.
The problem
AnAtreats every all-caps token as an initialism, soa FREE eventgets flagged in favour ofan FREE event. As @jbt found in #3921, the likely-acronym check only looks at the vowel-ness of the first two characters, which cannot tell an ordinary word written for emphasis from a letter-by-letter abbreviation.The approach
Use the orthography the dictionary already records, rather than adding a new property. An all-caps token whose entry licenses a lowercase spelling but no all-caps one is a word being shouted, so read it as the lowercase word:
freeandspectreareLOWERCASEonly, soFREEandSPECTREare emphasis caps.LAN,NAT,MA,FBIandNASAcarryALLCAPS, because the all-caps form is a real spelling of the entry, so they keep the initialism handling.an XVIkeeps whatevermasterdoes today.This needs no acronym/initialism property and no commitment on what to call these things, which was @hippietrail's objection to my first framing in the issue thread.
The dictionary lines
The guard is only as good as the all-caps coverage in the dictionary, so five abbreviations it would otherwise misread need their all-caps spelling recorded:
FPS/NFTP/Ngftpandsshare wrong in the dictionary and should both be all-caps"FTPS/NgSaffix onFTPproducesFTPs, notFTPS, so the plural does not inherit the all-caps spellingMPG/NSSH/Ngmlis deliberately not touched. It survives today only because M+L parses as 1050, so the Roman-numeral exclusion catches it by accident. It wants a real entry once the millilitre/megalitre/machine-learning question is settled, which is @hippietrail's to call.SYNneeds no entry either: it is read "sin", so the word reading is the correct one.Verification
Corpus of 999 GitHub READMEs and 15 Wikipedia articles, 1.9M words.
AnAgoes from 178 lints to 170:Nine false positives removed, one correct new flag, no regressions.
an SSH(12 uses),an FTP(6) andan ML(1) all occur in the corpus and stay clean, anda SSH and Telnet connectoris still flagged as it should be.Tests:
cargo test -p harper-core an_a— 67 passed, 0 failed (two new tests cover both directions)cargo test -p harper-core— passes exceptcurated_default_config_lists_every_registered_rule, which fails identically onmaster(AWholeOther,AllWellAndGood,UnderneathOfdrift indefault_config.json, unrelated to this change)cargo fmtcleancargo clippy -p harper-core --all-targets -- -D warnings— same 17 errors asmaster, none inan_a.rsNot in this PR
Sweeping the dictionary, 9,726 entries have an article the guard changes; 143 of those actually occur in the corpus. The frequent end is all shouting and reads correctly now. The tail has about a dozen two-and-three-letter abbreviations I would not decide alone (
RPS,RT,RM,LB,HMM,SQ,MFR,MPH,FT,LTD,SH). Those belong in a dictionary-curation pass, not here — happy to send the full list.