Implement simple example using swash to render to png #54

nicoburns · 2024-05-21T12:33:05Z

This is basically working. But it could definitely do with some actual review and cleanup before merging.

Output

Notes

~~Emoji rendering is not working~~
Font fallback for arabic text does seem to be working (note: I pulled the arabic text from cosmic text's sample files. Google translate translate the included arabic text as "arabic", but I cannot otherwise vouch for it's content).
Subpixel rendering is not enabled (does this make sense when rendering to image?)
The rendered text doesn't look fantastic. Possibly my blending code (used to apply the Swash alpha mask to an RGBA ImageBuffer from the image buffer is incorrect / too naive?). Cosmic-text pulls in tiny-skia in it's examples. I was hoping to avoid that here, but perhaps it's necessary?
Converting from a parley::Font to a swash::FontRef is very awkward, specifically when the font file is a collection. The Parley Font stores the index of the font within the collection, but Swash's FontRef expect the byte offset of the font within the collection. I could not find this exposed anywhere in parley/fontique/skrifa/read-fonts so I'm currently doing pointer arithmetic to get it.
~~I'm currently writing to a file called output.png in the users current working directory. Possibly we could do better than that (some kind of gitignored "data" directory in the parley repo?)~~
~~I've added an impl of Brush for peniko::Color, but possibly we ought to just blanket impl for all types implementing the requisite super traits?~~
~~I'm not using the NormalizedCoords as they always seemed to be empty in my example (what are they for?)~~

examples/simple.rs

dfrg · 2024-05-21T21:39:06Z

Emoji rendering is not working

We lost this in the switch to fontique. I can submit a small patch to fix it.

Subpixel rendering is not enabled (does this make sense when rendering to image?)

Probably not unless we’re just doing it for effect. Might be nice to add an option at some point.

The rendered text doesn't look fantastic.

Black on white without gamma correction is going to look “dirty”. Something simple like alpha.pow(1.0/2.2) would improve this a bit.

I'm not using the NormalizedCoords as they always seemed to be empty in my example (what are they for?)

These are the parameters for controlling interpolation in variable fonts. You should be able to feed the slice directly into ScalerBuilder::normalized_coords.

nicoburns · 2024-05-21T22:44:39Z

We lost this in the switch to fontique. I can submit a small patch to fix it.

Ah. Yes, it seems important!

Black on white without gamma correction is going to look “dirty”. Something simple like alpha.pow(1.0/2.2) would improve this a bit.

Perhaps using these functions? https://observablehq.com/@toja/color-blending-with-gamma-correction.

If I'm understanding correctly the correct process is:

Convert to float
Convert to linear rgb
Perform computation
Convert srgb
Convert to u8

These are the parameters for controlling interpolation in variable fonts. You should be able to feed the slice directly into ScalerBuilder::normalized_coords.

I see. Does this also need to be plugged in with skrifa?

dfrg · 2024-05-21T22:55:18Z

See #56 for a patch that might get emoji working with this example

dfrg · 2024-05-21T23:07:28Z

Perhaps using these functions? https://observablehq.com/@toja/color-blending-with-gamma-correction.

If I'm understanding correctly the correct process is:

Convert to float

Convert to linear rgb

Perform computation

Convert srgb

Convert to u8

We can do full sRGB if you'd like but I think just applying the power function to the alpha component will give us reasonably good results for the example.

These are the parameters for controlling interpolation in variable fonts. You should be able to feed the slice directly into ScalerBuilder::normalized_coords.

I see. Does this also need to be plugged in with skrifa?

It does. We use the LocationRef type there which is really just a wrapper for &[NormalizedCoord]. This unfortunately requires a type conversion and allocation as is done in xilem/masonry. We should probably change parley to use this type in the future.

nicoburns · 2024-05-21T23:18:52Z

We can do full sRGB if you'd like but I think just applying the power function to the alpha component will give us reasonably good results for the example.

The trouble I was having with apply pow to the alpha channel is that it's a u8 and I'm unable to apply a fractional power to a u8.

I have implemented an attempt at sRGB <-> Linear RGB conversion. Not sure if I've done it correctly (it's interesting that you suggest applying the curve to the alpha channel as the solution I've ended up with applies it to everything but the alpha channel!) as the result isn't obviously better (but it's good enough that it looks plausibly correct to me). I've left both functions in the code for the time being if you want to inspect them.

Naive:

Gamma corrected:

It does. We use the LocationRef type there which is really just a wrapper for &[NormalizedCoord]. This unfortunately requires a type conversion and allocation as is done in xilem/masonry. We should probably change parley to use this type in the future.

Ah, gotcha. I'll have a look at plugging this in tomorrow.

dfrg · 2024-05-21T23:19:18Z

Assuming this is using system-ui on macOS, setting the normalized coordinates should also get you the requested bold on the first four characters since San Francisco is a variable font.

nicoburns · 2024-05-21T23:22:50Z

Assuming this is using system-ui on macOS, setting the normalized coordinates should also get you the requested bold on the first four characters since San Francisco is a variable font.

Ah, that would be good. Bold was working with Helvetica, but it generally looked pretty bad with Helvetica so I switched it out. And I'd like to leave it on system-ui as that will presumably pick something sensible cross-platform. I guess maybe I could also use sans-serif?

nicoburns · 2024-05-21T23:26:25Z

Pretty sure my gamma correction function is wrong as the serif font looks much better with the naive blending.

examples/simple.rs

dfrg · 2024-05-22T14:28:53Z

Pretty sure my gamma correction function is wrong as the serif font looks much better with the naive blending.

Yeah, this is exceptionally tricky to get right. I’ll play with it later this evening.

waywardmonkeys

I'm assuming that once the other (tiny-skia) lands, this will move to a crate as well?

nicoburns · 2024-05-22T15:36:30Z

I'm assuming that once the other (tiny-skia) lands, this will move to a crate as well?

Yes. Hoping to land the tiny-skia one (#55) first. Will update this one once we've settled on a design there.

nicoburns · 2024-05-22T15:53:28Z

Pretty sure my gamma correction function is wrong as the serif font looks much better with the naive blending.

Yeah, this is exceptionally tricky to get right. I’ll play with it later this evening.

Hmm... the image crate (that we're already using here) has a blend function (https://docs.rs/image/latest/image/struct.Rgba.html#method.blend). Perhaps we could just use that. Looks the same as my "naive" version to me, but it certainly makes the example shorter.

EDIT: I've update this PR to use image's built in blend function. Old code is still available in an older commit if you want to play with it.

nicoburns · 2024-05-22T16:01:50Z

These are the parameters for controlling interpolation in variable fonts. You should be able to feed the slice directly into ScalerBuilder::normalized_coords.

@dfrg Hmm.. I've added this, but I still don't seem to get bold text.

dfrg · 2024-05-22T16:52:49Z

These are the parameters for controlling interpolation in variable fonts. You should be able to feed the slice directly into ScalerBuilder::normalized_coords.

@dfrg Hmm.. I've added this, but I still don't seem to get bold text.

Was just working on this :) Found a bug in swash's avar table. Update to swash 0.1.16 should fix it.

dfrg · 2024-05-22T16:55:27Z

Overall, this is looking pretty good to me now. I'm happy to punt on the gamma issue, get the tiny-skia version landed and then merge this one after making the crate extraction changes.

This example uses `skrifa` and `tiny-skia` rather than `swash` and `image` as in #54. ## Output ![tiny_skia_render](https://github.com/linebender/parley/assets/1007307/74cbf3b6-380c-4dff-91e4-976612982fac) ## Notes - This has the same issue as #54 around emoji rendering not working / being implemented - There is an issue with the centers of closed letter forms incorrectly being filled - The code for this is IMO much nicer than the swash version (at the cost of the heavier tiny-skia dependency) - This example does not have hinting enabled

nicoburns · 2024-05-23T21:37:25Z

examples/swash_render/src/main.rs

+                    }
+                }
+            }
+            Content::SubpixelMask => unimplemented!(),


@dfrg What are the semantics of the SubpixelMask? (/ how would one apply that to an RGBA image?). I reckon it would be good to demonstrate that even if we don't actually enable subpixel mode in the example.

This is an rgba image with an alpha component per color channel (the alpha in the mask is ignored). This document (https://github.com/servo/webrender/blob/main/webrender/doc/text-rendering.md#subpixel-text-blending) has a good explanation of the math.

examples/swash_render/src/main.rs

nicoburns · 2024-05-23T21:48:48Z

I've rebased this on top of latest main and made it fit the new directory structure. It is ready for another round of review.

dfrg

LGTM with a couple suggestions and some confusion about the code duplication.

examples/simple.rs

examples/swash_render/src/main.rs

Requested changes addressed

nicoburns requested review from dfrg and xorgy May 21, 2024 12:33

nicoburns force-pushed the swash-example branch 3 times, most recently from 4ea250c to 4cc54f5 Compare May 21, 2024 14:40

nicoburns mentioned this pull request May 21, 2024

Implement Tiny-Skia example #55

Merged

waywardmonkeys reviewed May 21, 2024

View reviewed changes

examples/simple.rs Outdated Show resolved Hide resolved

dfrg reviewed May 21, 2024

View reviewed changes

examples/simple.rs Outdated Show resolved Hide resolved

nicoburns force-pushed the swash-example branch from 5132322 to cab2dfb Compare May 21, 2024 22:40

DJMcNab mentioned this pull request May 22, 2024

add emoji family for emoji clusters #56

Merged

dfrg reviewed May 22, 2024

View reviewed changes

examples/simple.rs Outdated Show resolved Hide resolved

waywardmonkeys previously requested changes May 22, 2024

View reviewed changes

nicoburns force-pushed the swash-example branch from 2d5b9b7 to 48e5128 Compare May 22, 2024 15:56

nicoburns force-pushed the swash-example branch from a995f5f to 535a119 Compare May 23, 2024 14:24

nicoburns marked this pull request as draft May 23, 2024 14:32

nicoburns commented May 23, 2024

View reviewed changes

examples/swash_render/src/main.rs Outdated Show resolved Hide resolved

nicoburns marked this pull request as ready for review May 23, 2024 21:45

nicoburns requested review from dfrg and waywardmonkeys May 23, 2024 21:45

nicoburns added 12 commits May 23, 2024 22:46

Implement simple example using swash to render to png

98cb2cc

Simplify swash FontRef creation

c47fa50

Gamma correction WIP

8cef302

Implement gamma correction

cea8aa8

Fix bitmap emoji rasterisation

c78fbcd

Use blend function from the image crate

3a8438b

Pass normalized_coords through to swash

389b288

Make hinting always-on to reduce arg count to make clippy happy

3a88a7a

Apply image based blending

a73d530

Reapply changes lost in rebase

a5c5ef7

Fix clippy lints

399a447

Simplify background fill

a671f69

nicoburns force-pushed the swash-example branch from 836af29 to a671f69 Compare May 23, 2024 21:46

dfrg approved these changes May 23, 2024

View reviewed changes

examples/simple.rs Outdated Show resolved Hide resolved

examples/simple.rs Outdated Show resolved Hide resolved

examples/swash_render/src/main.rs Show resolved Hide resolved

nicoburns added 3 commits May 23, 2024 23:09

Remove duplicate file (rebase conflict)

bcd2358

Only build one scaler per glyph run

3e45a9a

Move render_glyph back into it's own function

4b42666

nicoburns added this pull request to the merge queue May 23, 2024

Merged via the queue into linebender:main with commit 8e10403 May 23, 2024
19 checks passed

nicoburns deleted the swash-example branch May 23, 2024 22:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement simple example using swash to render to png #54

Implement simple example using swash to render to png #54

nicoburns commented May 21, 2024 •

edited

Loading

dfrg commented May 21, 2024

nicoburns commented May 21, 2024

dfrg commented May 21, 2024

dfrg commented May 21, 2024

nicoburns commented May 21, 2024 •

edited

Loading

dfrg commented May 21, 2024

nicoburns commented May 21, 2024

nicoburns commented May 21, 2024

dfrg commented May 22, 2024

waywardmonkeys left a comment

nicoburns commented May 22, 2024

nicoburns commented May 22, 2024 •

edited

Loading

nicoburns commented May 22, 2024

dfrg commented May 22, 2024

dfrg commented May 22, 2024

nicoburns May 23, 2024

dfrg May 23, 2024

nicoburns commented May 23, 2024

dfrg left a comment

Implement simple example using swash to render to png #54

Implement simple example using swash to render to png #54

Conversation

nicoburns commented May 21, 2024 • edited Loading

Output

Notes

dfrg commented May 21, 2024

nicoburns commented May 21, 2024

dfrg commented May 21, 2024

dfrg commented May 21, 2024

nicoburns commented May 21, 2024 • edited Loading

dfrg commented May 21, 2024

nicoburns commented May 21, 2024

nicoburns commented May 21, 2024

dfrg commented May 22, 2024

waywardmonkeys left a comment

Choose a reason for hiding this comment

nicoburns commented May 22, 2024

nicoburns commented May 22, 2024 • edited Loading

nicoburns commented May 22, 2024

dfrg commented May 22, 2024

dfrg commented May 22, 2024

nicoburns May 23, 2024

Choose a reason for hiding this comment

dfrg May 23, 2024

Choose a reason for hiding this comment

nicoburns commented May 23, 2024

dfrg left a comment

Choose a reason for hiding this comment

nicoburns commented May 21, 2024 •

edited

Loading

nicoburns commented May 21, 2024 •

edited

Loading

nicoburns commented May 22, 2024 •

edited

Loading