Skip to content

fix(core): read emphasis all-caps as words in AnA - #4180

Open
mauropereiira wants to merge 2 commits into
Automattic:masterfrom
mauropereiira:fix/ana-emphasis-caps-v2
Open

fix(core): read emphasis all-caps as words in AnA#4180
mauropereiira wants to merge 2 commits into
Automattic:masterfrom
mauropereiira:fix/ana-emphasis-caps-v2

Conversation

@mauropereiira

Copy link
Copy Markdown
Contributor

Disclaimer, per AGENT_POLICY.md: this patch was produced by an agent (Claude Opus 5) operated by me. The design came out of the discussion in #3921 with @hippietrail, and every number below was measured rather than asserted, but the code and the dictionary lines were written by the model and I reviewed them.

Closes #3921.

The problem

AnA treats every all-caps token as an initialism, so a FREE event gets flagged in favour of an FREE event. As @jbt found in #3921, the likely-acronym check only looks at the vowel-ness of the first two characters, which cannot tell an ordinary word written for emphasis from a letter-by-letter abbreviation.

The approach

Use the orthography the dictionary already records, rather than adding a new property. An all-caps token whose entry licenses a lowercase spelling but no all-caps one is a word being shouted, so read it as the lowercase word:

  • free and spectre are LOWERCASE only, so FREE and SPECTRE are emphasis caps.
  • LAN, NAT, MA, FBI and NASA carry ALLCAPS, because the all-caps form is a real spelling of the entry, so they keep the initialism handling.
  • Roman numerals are excluded. They are numbers rather than words, so an XVI keeps whatever master does today.

This needs no acronym/initialism property and no commitment on what to call these things, which was @hippietrail's objection to my first framing in the issue thread.

The dictionary lines

The guard is only as good as the all-caps coverage in the dictionary, so five abbreviations it would otherwise misread need their all-caps spelling recorded:

Entry Why
FPS/N frames per second; first-person shooter — both read letter-by-letter
FTP/Ng @hippietrail's call in #3921: "ftp and ssh are wrong in the dictionary and should both be all-caps"
FTPS/Ng the S affix on FTP produces FTPs, not FTPS, so the plural does not inherit the all-caps spelling
MPG/N miles per gallon
SSH/Ng secure shell

ml is deliberately not touched. It survives today only because M+L parses as 1050, so the Roman-numeral exclusion catches it by accident. It wants a real entry once the millilitre/megalitre/machine-learning question is settled, which is @hippietrail's to call.

SYN needs no entry either: it is read "sin", so the word reading is the correct one.

Verification

Corpus of 999 GitHub READMEs and 15 Wikipedia articles, 1.9M words. AnA goes from 178 lints to 170:

fixed     a FREE provider (no signup needed)     x2
          a FREE graphics generator
          a **FREE System Design Interview
          a FLASH based filing system
          a [SPONSOR](./SPONSORS.md)!
          a SYN flood attack / a SYN to the server / a SYN-ACK

newly     an SYN data packet over an IP network
flagged   (correct — SYN is read "sin")

Nine false positives removed, one correct new flag, no regressions. an SSH (12 uses), an FTP (6) and an ML (1) all occur in the corpus and stay clean, and a SSH and Telnet connector is still flagged as it should be.

Tests:

  • cargo test -p harper-core an_a — 67 passed, 0 failed (two new tests cover both directions)
  • cargo test -p harper-core — passes except curated_default_config_lists_every_registered_rule, which fails identically on master (AWholeOther, AllWellAndGood, UnderneathOf drift in default_config.json, unrelated to this change)
  • cargo fmt clean
  • cargo clippy -p harper-core --all-targets -- -D warnings — same 17 errors as master, none in an_a.rs

Not in this PR

Sweeping the dictionary, 9,726 entries have an article the guard changes; 143 of those actually occur in the corpus. The frequent end is all shouting and reads correctly now. The tail has about a dozen two-and-three-letter abbreviations I would not decide alone (RPS, RT, RM, LB, HMM, SQ, MFR, MPH, FT, LTD, SH). Those belong in a dictionary-curation pass, not here — happy to send the full list.

Closes Automattic#3921.

`AnA` treated every all-caps token as an initialism, so "a FREE event" was
flagged in favour of "an FREE event". The likely-acronym check only looks at
the vowel-ness of the first two characters, which cannot tell an ordinary word
written for emphasis from a letter-by-letter abbreviation.

Use the orthography the dictionary already records instead. An all-caps token
whose entry licenses a lowercase spelling but no all-caps one is a word being
shouted, so read it as the lowercase word. Entries that do license all-caps
(`LAN`, `NAT`, `MA`, `FBI`) keep the initialism handling, and Roman numerals
are excluded because they are numbers rather than words.

That guard is only as good as the all-caps coverage in the dictionary, so add
the entries for the abbreviations it would otherwise misread:

    FPS   frames per second; first-person shooter
    FTP   file transfer protocol
    FTPS  FTP over TLS
    MPG   miles per gallon
    SSH   secure shell

Verified on 999 GitHub READMEs and 15 Wikipedia articles, 1.9M words: `AnA`
goes from 178 lints to 170. Nine removed were false positives ("a FREE
provider", "a FLASH based filing system", "a SYN flood attack"), and the one
added is correct ("an SYN data packet" -> "a SYN", since SYN is read "sin").

`cargo test -p harper-core` passes except for the pre-existing
`curated_default_config_lists_every_registered_rule` drift, which fails
identically on `master`. `cargo fmt` is clean and `cargo clippy -p harper-core
--all-targets -- -D warnings` reports the same 17 errors as `master`.
@hippietrail

hippietrail commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

This is looking pretty good. I've put it in my inbox to test with my own corpora. I've also requested Elijah to review it. I'm still a bit in two minds for some of them whether to include only all caps or both all caps and all lowercase. Professionally edited dictionaries seem to prefer all-caps for most of them
image
Google Ngram Viewer indicates a couple hugly prefer the all-caps versions while others are 50/50 all-caps vs lowercase
image

@mauropereiira

Copy link
Copy Markdown
Contributor Author

Answering the casing question with numbers, since I still had the corpus set up.

First, it doesn't block this PR. The guard needs LOWERCASE present and ALLCAPS absent. Both spellings give LOWERCASE | ALLCAPS, so it's skipped. All-caps only gives ALLCAPS, so it's also skipped. an SSH, an FTP and an FPS stay clean either way, and a SSH and Telnet connector stays flagged either way. So this is a dictionary decision you can take independently of merging this.

Where it does land is OrthographicConsistency. orthographic_consistency.rs:71 fires when an entry is all-caps with no lowercase spelling, and suggests uppercasing every other use — the nasaNASA behaviour. So dropping the lowercase entries turns every lowercase ftp and ssh in prose into a Capitalization lint.

I built the branch twice, once as-is and once with fps/~N, ftp/~JNOSVGZ, mpg/ and ssh/~VO deleted, then ran --only OrthographicConsistency over the corpus (959 READMEs and 144 Wikipedia articles). Harper skips fenced and inline code, so this is prose only:

+76 Capitalization lints, 0 removed. ssh 33, fps 20, ftp 17, ftps 5, mpg 1.

On your frequency evidence — you're right, and I don't think it settles it. I counted the same corpus with code stripped and all-caps wins there too: SSH 210 / ssh 55, FTP 33 / ftp 27, FPS 17 / fps 9. That surprised me; I'd expected READMEs to swing lowercase. So Ngram and technical prose agree.

But dominance isn't quite the question being asked. It tells us the all-caps spelling has to be in the dictionary, which is what this PR adds. It doesn't tell us the lowercase spelling should be rejected, and rejecting is what all-caps-only actually does. Sampling the 76:

  • .mpeg/.mpg格式的视频文件 — a file extension
  • 3921:ftp, 3922:sftp, ..., 3990:ftps — a port list
  • Jcode can render at over a thousand fps
  • open sftp tab for ssh connection

The first two are plainly wrong to flag. Collins lists "FTP in British English or ftp" and Oxford Learner's carries a lowercase fps entry, so the professional dictionaries aren't unanimous on rejecting it either.

One knock-on worth flagging: ftps isn't an entry, it's the S affix on ftp/~JNOSVGZ. Dropping lowercase ftp takes lowercase ftps with it, which is where 5 of the 76 come from. Moving that flag set onto FTP instead would generate FTPING and FTPERS, which is why I gave FTPS its own entry rather than an affix.

mpg is the one I can't argue from data. Zero prose occurrences in either case in this corpus, and the single lint is that file extension. Your call there.

So my vote is both spellings: all-caps is the canonical form, which is what the new entries record, but lowercase is still attested and costs nothing to accept. If you'd rather go all-caps-only on SSH and FTP specifically, the two Ngram is clearest on, that's a two-line change and I'll re-run the count for just those.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Indefinite article a/an inconsistently false-positive'd when followed by all-caps dictionary word

2 participants