Windows (and ClearType) vs. Truetype Fonts with Embedded Bitmaps

Round 1: FIGHT!

While I was working on the TrueType versions of the BigBlue Terminal font (and on my upcoming pack of oldschool PC font remakes), a stubborn annoyance cropped up in the form of ClearType.  Microsoft's subpixel text rasterizer (optimized for flat-panel displays) doesn't play nice at all with TrueType pixelfonts -- that is, scalable fonts, whose outlines are designed to snap to the pixel grid at a particular size.  Depending on your choice of foreground and background colors, you may see smudged colorful fringes that clearly have no business being there:

Without ClearType, the TrueType outlines obediently stick to the pixel grid: the rendering is identical to the bitmap version of the font.  With ClearType, the subpixel rasterization somehow causes the red and blue color components to be shifted away in opposite directions, "overflowing" the grid.  This looks laughably similar to the color fringing artifacts that marred text on composite monitors decades ago, even if the causes are entirely different. 

The 'smearing' is usually not very noticeable at typical resolutions, and personally it doesn't bother me that much - but in certain cases, it can get ugly.  Now, whether ClearType itself is good or bad is not the issue: current versions of Windows have it enabled by default, most people don't disable it, and Microsoft didn't provide any user control to toggle it per application (or per font), as it damn well should have. Naturally, ClearType wasn't designed with pixelfonts in mind, because they're hardly a typical use case. 

This only affects scalable TrueType fonts, of course.  Why not just use bitmaps, then? A bitmap grid is the natural format for a pixelfont anyway.  Alas, there's no cross-platform format for bitmap fonts -- and the .FON format used by Windows can only handle 8-bit encodings, so full Unicode support is out.  While .FON can be useful (and I've ideed included such a version), we're still gonna need TrueType. 

Bitmap strikes for 2 damage!

The TrueType format has a trick up its sleeve that we could use to our advantage: on top of the scalable vector glyphs, the .TTF font can contain embedded bitmap strikes.  "Strikes" are complete bitmap versions of the scalable font, each tuned for a particular pixel size: whenever a bitmap strike is available for the current size, the text renderer is supposed to use that instead of rasterizing the outlines from scratch. 

Apparently, Windows supports this: "ClearType antialiasing is disabled [...] if the font has tuned embedded bitmaps, only for those font sizes that contain the embedded bitmaps".  So there - just embed a bitmap strike of the pixelfont's "native" size, and you'll artfully dodge the SmearType issue.  Right? Wrong.  I tried doing exactly that, but Windows was not impressed - it cheerfully ignored my crisp bitmaps and rendered the outlines through ClearType anyway. 

I checked, double-checked and ensured that the TTF and embedded bitmap data were up to spec, but no luck.  That ticked me off, because I knew it was supposed to work: several TrueType fonts that ship with Windows have embedded bitmaps which display just fine, even on Windows 10.  These are all CJK (East Asian) fonts - MS Mincho, MS Gothic and so on. 

After much peeking under the hood, snooping around on the web, wailing and gnashing of teeth, the real problem emerged: current versions of ClearType -- ever since DirectWrite -- have unceremoniously dropped support for embedded bitmap strikes, with the explicit exception of East Asian fonts! MS seems to feel that CJK glyphs are still more readable in bitmap form, but cannot conceive that the same feature would be useful anywhere else, so they simply ditched the option for all other fonts. 

Unleashed in the East

The question then becomes, of course, what causes Windows (or DirectWrite, or WPF) to identify a TrueType font as "East Asian" for rasterization purposes.  The answer was pieced together from some clues and then confirmed experimentally:

1. At least one CJK code page must be specified in the font's 'OS/2' table (e.g.  Chinese Traditional, or JIS/Japan); and
2. A few seemingly-random East Asian glyphs actually have to exist in the font.
   
   

This finally worked, and produced an outline TTF with usable embedded bitmaps on Windows.  Well then, there's our solution...  right?

Wrong again, bucko! As it turns out, doing #1 above breaks several other things while fixing the bitmap issue.  Most annoyingly,

1. The "DOS/OEM" encoding option disappears.  The codepage bit field is designed to accommodate several encoding options, but for some nonsensical reason, codepage 437 wants to be exclusive.  You can specify it all you want - as soon as you add another one alongside it (such as a CJK codepage), Windows will simply excise "DOS/OEM" from the list of detected encodings.
2. The font can no longer be used in console windows.  The culprit here is yet another set of nonsensical restrictions from Microsoft, which again treat East Asian fonts as a special case (in a poorly-worded, and probably misleading, way).  As if there wasn't enough brain-damage in forcing the user to perform unsupported registry hacks just to add *any* TrueType font to the console.
   
   

Since both of these things are significant for my purposes, I give up on the embedded bitmap idea - you can only do so much when faced with such a silly combo of arbitrary limitations.  If anyone figures out a functional way to sidestep SmearType for TTF pixelfonts, I'm all ears!