diff options
Diffstat (limited to 'doc/context/sources/general/manuals/fonts/fonts-features.tex')
| -rw-r--r-- | doc/context/sources/general/manuals/fonts/fonts-features.tex | 2742 |
1 files changed, 2742 insertions, 0 deletions
diff --git a/doc/context/sources/general/manuals/fonts/fonts-features.tex b/doc/context/sources/general/manuals/fonts/fonts-features.tex new file mode 100644 index 000000000..bf9f39385 --- /dev/null +++ b/doc/context/sources/general/manuals/fonts/fonts-features.tex @@ -0,0 +1,2742 @@ +% language=uk + +\startcomponent fonts-features + +\environment fonts-environment + +% windows: seguiemj.ttf (windows 10) +% public : emojionecolor-svginot.ttf (https://github.com/eosrei/emojione-color-font) + +\startMPextensions + vardef MyRectangle(expr n, w, h, x, y, c) = + image ( + fill unitsquare xyscaled (w,h) shifted (x,y) withcolor c ; + draw textext("\tttf " & decimal n) xsized (1/2) shifted (w/2,h/2) shifted (x,y) withcolor white ; + ) + enddef ; + vardef MyDot(expr x, y) = + image ( + draw (x,y) withpen pencircle scaled (2/3) withcolor white ; + draw (x,y) withpen pencircle scaled (1/2) withcolor black ; + ) + enddef ; +\stopMPextensions + +\startchapter[title=Features][color=darkmagenta] + +\startsection[title=Introduction] + +If you look into fonts, it is hard not to bump into kerns (spacing between +characters) and ligatures (combining multiple shapes into one) and apart from +monospaced fonts most \TYPEONE\ fonts have them. In the \OPENTYPE\ universe we +call these properties features and in such a font there can be many such +features. + +For those who grew up with \TEX\ or still remember the times of eight bit fonts, +it is no secret that \TEX\ macro packages did some magic to get most out of a +font: replacing missing glyphs, fixing metrics, using commands to access shapes +that had a weird code point, to mention a few. As there is absolutely no +guarantee that an \OPENTYPE\ font does better, there is a good reason to continue +messing around with fonts. After all, it's what \TEX\ users seem to like: +control. + +So, when we started writing support for \OPENTYPE\ quite soon a mechanism has +been created that permits adding our own features to the repertoire that comes +with a font. Because \OPENTYPE\ features demand a configuration and control +mechanism, it made sense to generalize that and provide a single interface. + +This means that when we talk about font features, we don't limit ourselves to +those provided by the font, but also those provided by \CONTEXT. As with font +features, they are enabled per font. + +Some of the extra features are sort of generic, others are very font specific and +their properties are somewhat bound to a font. Such features are defined in a +font goodie files. Consider these goodies a font extension mechanism. + +Some features need information that only the engine can provide. This is why we +have analyzers. Some are generic, others are bound to scripts. They come in +action before features are applied. Rather special is applying features in +combination with paragraph building. This is something very specific to \CONTEXT\ +but it depends on properties of the font. It falls into the category \quote +{optimizing}. + +It is clear that when we talk of features many aspects of a font play a role. In +this chapter we will discuss all the mentioned aspects. There is quite a bit of +\LUA\ code shown in this chapter, but don't worry, users will seldom need to +tweak fonts this way. On the other hand it's good to see what is possible. + +\stopsection + +\startsection[title=Regulars] + +\startsubsection[title=Introduction] + +The \OPENTYPE\ specification, which can be found on the \MICROSOFT\ website +is no easy reading. Some of the concepts are easy to understand, like relative +positioning (that we call kerning in \TEX) or ligature substitution (as we +have ligatures in \TEX\ too). It makes no sense to discuss the bitwise composition +of an \OPENTYPE\ or \TRUETYPE\ file here. First of all, all we get to see is +a \LUA\ table, and in \CONTEXT\ even that one gets sanitized and optimized +into a more useable table. However, as the data that comes with a font is +a good indication of what a font is capable of, we will discuss some of it in +an appendix. In this section we will discuss the basic principles and categories +of features. + +\stopsubsection + +\startsubsection[title=Feature sets] + +Because in the next examples we will demonstrate features, we need to know how +we can tell \CONTEXT\ what features to use. Although you can add explicit +feature definitions to a font specification, I strongly advice you not to do this +but use the more abstract mechanism of feature sets. These are defined as follows: + +\starttyping +\definefontfeature + [MyFeatureSet] + [alpha=yes, + beta=no, + gamma=123] +\stoptyping + +Such a set is bound to a font with the \type {*} specifier, as in: + +\starttyping +\definefont + [MyFontInstance] + [MyNiceFont*MyFeatureSet at 12pt] +\stoptyping + +In most cases the already defined \type {default} feature set will suffice. It often +makes sense to use that one as base for new definitions: + +\starttyping +\definefontfeature + [MyFeatureSet] + [default] + [alpha=yes, + beta=no, + gamma=123] +\stoptyping + +The second argument can be a list, as in: + +\starttyping +\definefontfeature + [MyFeatureSet] + [MyFirstSet,MySecondSet] + [alpha=yes, + beta=no, + gamma=123] +\stoptyping + +Of course you need to know what features a font support, and one way to find +out is: + +\starttyping +mtxrun --script font --list --info --pattern=pagella +\stoptyping + +Don't be too surprised if different fonts show different features and even similar +features can be implemented differently. Sometimes you really need to know the font, +but fortunately many fonts come with examples. + +There are many features and there values are kept with the font when it gets +defined. This means that when you change a featureset, it will not affect already +defined fonts. Because fonts are often defined on demand, you need to be aware of +the fact that a redefinition of a featureset can have consequences for already +defined fonts. For instance, a bodyfont switch only sets up the fonts and delays +defining them. + +Although features are a sort of abstractions it can be interesting to see what features +and values are actually used: + +\starttyping +\usemodule[fonts-features] \showusedfeatures +\stoptyping + +You will notice that we have more features than \OPENTYPE\ fonts can offer. This +is because in \CONTEXT\ features is a more general concept. + +\showusedfeatures + +\stopsubsection + +\startsubsection[title=Main categories] + +There are two (but potentially more) main groups of features: those that deal +with substitution, and those that lead to positioning. It is not really needed +to know the gory details, but it helps to know at least a bit about them as +it can help to track down issues with fonts. + +There are several substitutions possible: + +\startitemize[packed] +\startitem a single substitution replaces one glyph by another \stopitem +\startitem a multiple substitution replaces one glyph by one or more \stopitem +\startitem a ligature substitution replaces multiple glyphs by one glyph \stopitem +\startitem an alternate substitution replaces one glyph by one out of a set \stopitem +\stopitemize + +Like it or not, but these categories are not always used as intended: they just +are a way of replacing one or more glyphs by one or more other glyphs. This means +that when for instance \type {ij} gets replaced by one glyph (given that the font +supports it) a ligature substitution is used, even when in fact we have to do +with a diftong that can be represented by one character. + +No matter what features you will use, keep in mind that they are nothing more +than a combination of substitutions and positioning directives. So, the de facto +standard ligature building feature \type {liga} indeed uses a ligature +substitution, but other features with names that resemble no ligatures might use +this substitution as well. + +An example of a single substitution is an oldstyle (\type {onum}) although it can +as well be implemented as a choice out of alternate glyphs. Another example is +smallcaps (\type {smcp}). Nowdays these are more or less standard features for a +grown up font, while in the past they came as separate fonts. So, instead of loading +an extra font, one sticks to one and selects the feature that does the +substitution. + +A second category concerns relative positioning. Again we have several variants: + +\startitemize[packed] +\startitem a single positioning moves a glyph over one of two axis and can change the width and|/|or height \stopitem +\startitem a pair positioning also moved glyphs but concerns two adjacent glyphs \stopitem +\startitem a cursive positioning operates on a range of glyphs and is used to visually connect them \stopitem +\stopitemize + +In addition there are three ways to anchor marks onto glyphs: + +\startitemize[packed] +\startitem a mark can be anchored on a base glyph \stopitem +\startitem a mark can be anchored on a specific (visual) component of a ligature \stopitem +\startitem a mark can be anchored on another mark \stopitem +\stopitemize + +In base mode the single, alternate and ligature substitutions can rather easily +be mapped onto the traditional \TEX\ font handling mechanism and this is what +happens in base mode. A single substitution is just another instance of a glyph +so there we just replace the original index into the glyph table by another one. +In the case of an alternate we change the default index into one of several +possible replacements in the alternate set. Ligatures can be mapped onto \TEX s +ligature mechanism. The single positioning maps nicely on \TEX s kerning +mechanism and pairwise positioning is not applicable in base mode. In node mode +we don't do any remapping at loading time but delegate that to \LUA\ when +processing the node lists. + +Marks are special in the sense that they normally only occur in scripts that also +use substitution and positioning which in turn means that some more housekeeping +is involved. After all, we need to keep track to what a mark applies. Of course a +font can provide regular latin accents as marks but that is ill practice because +cut and paste might not work out as expected. A proper font will support composed +characters and provide glyphs that have the accents built in. Marks are not dealt +with in base mode. + +Talking of complex scripts, the above set of operations is far from enough. Take +for instance Arabic, where a sequence of 5~characters with 3~marks can easily +become two glyphs glued together with two marks only. In the process we can have +single substitutions, ligatures being built, marks being anchored and glyphs +being cursively positioned. But, in order to do this well, some contextual +analysis has to be done as well. Again we have several variants of this: + +\startitemize[packed] +\startitem with contextual substitution a replacement takes place depending on a matching sequence of glyphs, +optionally preceded or followed by matches \stopitem +\startitem with contextual positioning shifting and anchoring happens based on a matching sequence of glyphs, +optionally preceded or followed by matches \stopitem +\startitem multiple contextual substitutions or positionings can be chained together \stopitem +\startitem this can also happen in the reverse order (for right|-|to|-|left scripts) \stopitem +\stopitemize + +In practice there is no fundamental difference between these and we can collapse +them all in a sequence of lookups resulting in a sequence of whatever other +manipulation is wanted. + +Given this, what is a feature? It's mostly a sequence of actions expressed in the +above. And although there is a whole repertoire of semi|-|standardized features +like \type {liga} and \type {onum}, there is no real hard coded support for them +in \CONTEXT. Instead we have a generic feature processor that deals with all of +them. A feature, say \type {abcd}, has a definition that boils down to a sequence +of lookups. A lookup is just a name that is associated to one of the mentioned +actions. So, \type {abcd} can do a decomposition (multiple substitution), then a +replacement (single substitution) based on neighbouring glyphs, then do some +ligature building (ligature substitution) and finally position the resulting +glyphs relative to each other (like cursive positioning and anchoring marks). + +Imagine that we start out with 5 characters in the input. Instead of real glyphs +we represent them by rectangles. The third one is a mark. + +\startlinecorrection +\startMPcode + draw MyRectangle(1,2,6, 0,0,.5red ) ; + draw MyRectangle(2,2,4, 3,0,.5green ) ; draw MyDot(4,4.25) ; + draw MyRectangle(3,2,1, 6,5,.5blue ) ; draw MyDot(7,4.75) ; + draw MyRectangle(4,2,5, 9,0,.5yellow ) ; + draw MyRectangle(5,2,5,12,0,.5magenta) ; + currentpicture := currentpicture ysized(4cm) ; +\stopMPcode +\stoplinecorrection + +In the next variant we see that four and five have been replaced by +number six. This is a ligature replacement. + +\startlinecorrection +\startMPcode + draw MyRectangle(1,2,6,0,0,.5red ) ; + draw MyRectangle(2,2,4,3,0,.5green) ; draw MyDot(4,4.25) ; + draw MyRectangle(3,2,1,6,5,.5blue ) ; draw MyDot(7,4.75) ; + draw MyRectangle(6,3,5,9,0,.5cyan ) ; + currentpicture := currentpicture ysized(4cm) ; +\stopMPcode +\stoplinecorrection + +The mark is an independent entity. Sometimes it has a width, sometimes it hasn't. +In both cases we can position it. Here we move the shape left and down. There are +two ways to do this: simple pairwise kerning but better is to use anchors. Here +we have one anchor per shape but there can be many. + +\startlinecorrection +\startMPcode + draw MyRectangle(1,2,6,0,0 ,.5red ) ; + draw MyRectangle(2,2,4,3,0 ,.5green) ; + draw MyRectangle(3,2,1,3,4.5,.5blue ) ; draw MyDot(4,4.25) ; + draw MyRectangle(6,3,5,6,0 ,.5cyan ) ; + currentpicture := currentpicture ysized(4cm) ; +\stopMPcode +\stoplinecorrection + +Next we apply some kerning. Of course the anchored marks need to move as well. + +\startlinecorrection +\startMPcode + draw MyRectangle(1,2,6,0, 0 ,.5red ) ; + draw MyRectangle(2,2,4,2.5,0 ,.5green) ; + draw MyRectangle(3,2,1,2.5,4.5,.5blue ) ; draw MyDot(3.5,4.25) ; + draw MyRectangle(6,3,5,5, 0 ,.5cyan ) ; + currentpicture := currentpicture ysized(4cm) ; +\stopMPcode +\stoplinecorrection + +Alternatively we can connect the shapes in a cursive way. The name cursive is +somewhat misleading as it just boils down to shifting. The cursive indicates that +the shifts accumulate within a word. + +\startlinecorrection +\startMPcode + draw MyRectangle(1,2,6,0,0 ,.5red ) ; + draw MyRectangle(2,2,4,2,0.5,.5green) ; + draw MyRectangle(3,2,1,2,5 ,.5blue ) ; draw MyDot(3,4.75) ; + draw MyRectangle(6,3,5,4,1 ,.5cyan ) ; + currentpicture := currentpicture ysized(4cm) ; +\stopMPcode +\stoplinecorrection + +\stopsubsection + +\startsubsection[title={Single substitution}] + +Single substitutions are probably the most used ones. For instance, when you +ask for small caps, a lot of glyphs get replaced. When using oldstyle numerals +only digits get replaced but even then each glyph has to be checked. This can be +demonstrated with the Latin Modern fonts. + +\startlinecorrection +\scale + [height=1cm] + {\strut + {\definedfont[lmroman10-bold*default]\$123.45}% + \quad + {\definedfont[lmroman10-bold*oldstyle]\$123.45}} +\stoplinecorrection + +As you can see here, Latin Modern has an oldstyle dollar sign. If you don't like +that one, you're in troubles as it comes with the rest of the oldstyles. The only +way out is to apply the oldstyle numerals to digits only which involves more tagging +than you might be willing to add. So, whenever you choose a substitution, be aware +that you have not that much control over what gets substituted: it's the font that +drives it. Here are some examples: + +\starttyping +\definefontfeature[capsandold][smallcaps,oldstyle] + +\showotfcomposition{dejavu-serif*capsandold at 24pt}{}{It's 2013!} +\showotfcomposition{cambria*capsandold at 24pt}{}{It's 2013!} +\showotfcomposition{lmroman10regular*capsandold at 24pt}{}{It's 2013!} +\showotfcomposition{texgyrepagellaregular*capsandold at 24pt}{}{It's 2013!} +\stoptyping + +\definefontfeature[capsandold][smallcaps,oldstyle] + +\blank \showotfcomposition{dejavu-serif*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank +\blank \showotfcomposition{cambria*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank +\blank \showotfcomposition{lmroman10regular*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank +\blank \showotfcomposition{texgyrepagellaregular*capsandold at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{It's 2013!}} \blank + +\stopsubsection + +\startsubsection[title={Multiple substitution}] + +In a multiple substitution a sequence of characters (glyphs) gets replaced by +another sequence. In fact, you might wonder why one||to||one, multiple||to||one +and multiple||to||multiple are not all generalized into this variant. Efficiency +is probably the main reason. \footnote {Isn't it strange that complex mechanisms +are designed to save a few bytes while at the same time we produce ridiculous +large pictures with cameras.} For instance the many||to||one is often used for +ligatures (\type {liga}) and as a consequence \type {liga} is often misused also +for non||ligatures. + +One usage of multiple replacements is to avoid and or undo other replacements. In +the next example we see a language dependent \type {fi} ligature. Take the dutch +\type {ij} and \type {ie} diftongs. Here we need to prevent the \type {i} +becoming combined with the \type {f} as it would look weird. Among the solutions +for this are: context dependent ligatures (which involves a lot of rules), or +multiple to multiple replacements (looking at the \type {fij} sequence). + +\startbuffer[definitions] +\definefontfeature[default-fijn-en][default][language=eng,script=latn] +\definefontfeature[default-fijn-nl][default][language=nld,script=latn] +\stopbuffer + +\getbuffer[definitions] \typebuffer[definitions] + +\starttyping +\definedfont[lmroman10-regular*default-fijn-en]\en effe fijn fietsen +\definedfont[lmroman10-regular*default-fijn-nl]\nl effe fijn fietsen +\stoptyping + +This gives: + +\startlinecorrection[blank] +\scale [width=\textwidth] \bgroup + \framed [offset=overlay,frame=off,foregroundcolor=maincolor,align=normal,strut=no] \bgroup + \definedfont[lmroman10-regular*default-fijn-en]\en effe fijn fietsen\vskip-1ex + \definedfont[lmroman10-regular*default-fijn-nl]\nl effe fijn fietsen\par + \egroup +\egroup +\stoplinecorrection + +Of course from this result one cannot see what (combination of) substitution(s) +was used, but it's a nice exercise to work out a solution. + +Multiple substitutions are mostly used for scripts more complex than latin or +special fonts like Zapfino where advanced contextual analysis happens. + +\stopsubsection + +\startsubsection[title={Alternate substitution}] + +Alternates are simple one||to||one substitutions. Popular examples are small +capitials and oldstyle numerals. + +A nice application of alternates is the punk font. This font is a Knuth original. +As part of experimenting with the \METAPOST\ library in the early days of +\LUATEX\ and \MKIV, runtime randomization was implemented. However, that variant +used virtual fonts and was somewhat resource hungry. So, in a later stage Khaled +Hosny made an \OPENTYPE\ version using \METAPOST\ output. Randomization is +implemented through the \type {rand} feature. + +In \MKIV\ the \type {rand} feature is not really special and behaves just like +any other (stylistic) alternate. The only difference is that for this feature a +value of \type {yes} equals \type {random}. This also means that any feature that +uses alternates use them randomly. + +\startbuffer +\definefontfeature[punknova-first] [mode=node,kern=yes,rand=first] +\definefontfeature[punknova-2] [mode=node,kern=yes,rand=2] +\definefontfeature[punknova-yes] [mode=node,kern=yes,rand=yes] +\definefontfeature[punknova-random][mode=node,kern=yes,rand=random] +\stopbuffer + +\typebuffer \getbuffer + +We use this is: + +\startbuffer[sample] +The original punk font is designed by Don Knuth: xxxxxxxxxxxx +\stopbuffer + +\startbuffer +\definedfont[punknova-regular at 15pt] \getbuffer[sample] +\definedfont[punknova-regular*punknova-first at 15pt] \getbuffer[sample] +\definedfont[punknova-regular*punknova-2 at 15pt] \getbuffer[sample] +\definedfont[punknova-regular*punknova-yes at 15pt] \getbuffer[sample] +\definedfont[punknova-regular*punknova-random at 15pt] \getbuffer[sample] +\stopbuffer + +\typebuffer[sample] + +\typebuffer + +In order to illustrate the variants we show a sequence of \type {x}'s. There are +upto ten different variants per characters. + +\startlines[color=maincolor] \getbuffer \stoplines + +There is one pitfall with random alternates: if each run leads to a different +outcome, we can end up in oscillation: different shapes give different paragraphs +and we can get more pages or cross references etc.\ that can end up differently +so this is why \CONTEXT\ always uses the same random seed (which gets reset when +you purge the auxiliary files. + +\stopsubsection + +\startsubsection[title={Ligature substitution}] + +A ligature is traditionally a combination of several characters into one. Popular +ligatures are \quote {fi}, \quote {fl}, \quote {ffi} and , \quote {ffl}. +Occasionally we see \quote {\ae}, \quote {\oe} and some more. Often ligatures are +language dependant. For instance in languages like Dutch and German there can be +compound words where one part ends with an \type {f} and the next part starts with +an \type {f} and that looks bad or at least not intuitive. To some extent one +can wonder if this tradition of ligatures is a good one. It definitely made +sense ages ago, but I wouldn't be surprised if they are often added to fonts +because the encoding vectors have them. After all, nothing prevents to go ahead +and come up with way more ligatures. + +There can be many ligature features in a font. Although we support arbitrary +features, that is: those not registered as being official one way or the other, +the following are known by description: + +\startluacode +context.starttabulate { "|lTCT{maincolor}|l|" } +for k, v in table.sortedhash(fonts.handlers.otf.tables.features) do + if string.find(v,"ligature") then + context.NC() + context(k) + context.NC() + context(v) + context.NR() + end +end +context.stoptabulate() +\stopluacode + +The \type {default} feature set has type {liga} as wel as the \TEX\ specific \type {tlig} +that replaces successive hyphen signs into en- and emdashes. The \type {arabic} feature +set also has \type {rlig} enabled. + +Now, there is one thing you should realize when we discuss specific features and +the underlaying mechanisms: there is no real relationship between the features's +name and the mechanisms used: any feature can use any underlying mechanism or +combination. This is why deep down we see that what is internally called ligature +gets used for any purpose where multiple||to||one replacements happen, and why the +\type {liga} feature can use single substitutions or alternates to swap in +another rendering so that the dot of the \type {i} stays free of the preceding +\type {f}. And for some fonts relative positioning can be used to achieve a +ligature effect. + +The next examples demonstrate how the \type {liga} feature deals with \type {ffi}. +Possible solutions are: replace all three at once, replace the first two first and +in a next step, combine a ligature and following character, replace one or more +components by variants that have no interference with the dot of the~\quote{i}. + +\starttyping +\showotfcomposition{dejavu-serif*default at 48pt}{}{ffi} +\showotfcomposition{cambria*default at 48pt}{}{ffi} +\showotfcomposition{lmroman10regular*default at 48pt}{}{ffi} +\showotfcomposition{texgyrepagellaregular*default at 48pt}{}{ffi} +\stoptyping + +\blank \showotfcomposition{dejavu-serif*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank +\blank \showotfcomposition{cambria*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank +\blank \showotfcomposition{lmroman10regular*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank +\blank \showotfcomposition{texgyrepagellaregular*default at 48pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{ffi}} \blank + +\stopsubsection + +\startsubsection[title={Single positioning}] + +Single positioning is also known as kerning, moving characters closer together so +that we get a more uniformly spaced sequence of glyphs. It is a mistake to think +that kerning is always needed! There are fonts that have hardly any kerns or no +kerns at all and still look good. + +\start + \showfontkerns \blank + \definedfont[dejavu-serif*default at 8pt]Dejavu Serif: \input tufte (E.R. Tufte)\blank + \definedfont[cambria*default at 9pt]Cambria: \input tufte (E.R. Tufte)\blank + \definedfont[lmroman10regular*default at 10pt]Latin Roman Regular: \input tufte (E.R. Tufte)\blank + \definedfont[lucidabrightot*default at 8pt]Lucida Bright: \input tufte (E.R. Tufte)\blank + \definedfont[texgyrepagellaregular*default at 9pt]Pagella Regular: \input tufte (E.R. Tufte)\blank +\stop + +The next couple of examples show the action for a few words: + +\blank \showotfcomposition{dejavu-serif*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank +\blank \showotfcomposition{cambria*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank +\blank \showotfcomposition{lmroman10regular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank +\blank \showotfcomposition{lucidabrightot*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank +\blank \showotfcomposition{texgyrepagellaregular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{We thrive}} \blank + +\stopsubsection + +\startsubsection[title={Pairwise positioning}] + +This variant of positioning involved the first, second or both glyphs of a glyph +pair. The specification can influence the horizontal and vertical positions we +well as the widths of the positioned glyphs. + +\startnotabene + We need an example here. +\stopnotabene + +\stopsubsection + +\startsubsection[title={Mark positioning}] + +Marks are (often) small symbols that represent accents (in latin) or vowels (in +arabic) that get attached to base glyphs. In the input stream they come after the +character that they apply to. Many fonts come with precomposed latin characters +which means that an \type {à} in the input is mapped directly onto its +corresponding shape. When the input contains an \type {a} followed by a \type{̀ } +input normalization will normally turn this into an \type {à}. But, when this +doesn't happen, the font machinery has to make sure that the mark gets positioned +right onto the base character. In traditional \TYPEONE\ fonts that more or less +happened automatically by overlaying the shapes. In \OPENTYPE\ (single) +positioning is used to place the mark right. + +\startnarrowtyping +\showotfcomposition{dejavu-serif*default at 24pt}{}{à a\utfchar{"0300} à} +\showotfcomposition{cambria*default at 24pt}{}{à a\utfchar{"0300} à} +\showotfcomposition{lmroman10regular*default at 24pt}{}{à a\utfchar{"0300} à} +\showotfcomposition{lucidabrightot*default at 24pt}{}{à a\utfchar{"0300} à} +\showotfcomposition{texgyrepagellaregular*default at 24pt}{}{à a\utfchar{"0300} à} +\stopnarrowtyping + +Of course a font can contain logic that replaces a sequence of base and mark into +precomposed characters with the right \UNICODE\ entry. + +\blank \showotfcomposition{dejavu-serif*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank +\blank \showotfcomposition{cambria*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank +\blank \showotfcomposition{lmroman10regular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank +\blank \showotfcomposition{lucidabrightot*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank +\blank \showotfcomposition{texgyrepagellaregular*default at 24pt}{}{\disabletrackers[otf.analyzing]\color[maincolor]{à a\utfchar{"0300} à}} \blank + +You can imagine that when marks are bound to characters that have become +ligatures the anchoring is more complex as the font machinery has to keep track +of onto which component the mark goes. For this purpose marks as well as base +characters and base ligatures have anchors and feature lookups can explicitly +refer to them. + +\stopsubsection + +\startsubsection[title={Contextual analysis}] + +What actually happens when turning a list of characters into a list of glyphs can +range from real simple to pretty complex. For instance the \type {smcp} feature +only has to run over the list and relate characters to a smallcaps shape. A +slightly more complex feature might also demand some positioning. One step further +is the use of contextual analysis, i.e. looking at previous, current and following +characters (or glyphs). Because features can range from simple to complex the actual +processing is not per feature! A font comes with a sequence of so called lookups that +relate to a feature, script and language. Also, each feature can use one||to||one, +multiple||to||one and many||to||many replacements as well as relative positioning. + +So, what actually happens is not that a feature is processed, but that all features +are dealt with at the same time, in the order that the font prescribes. Enabling a +specific feature means that a step is executed, while a disabled feature skips all +steps that are tagged as belonging to that feature. And, as each feature can use +contextual analysis, you can imagine that the effective sequence of actions can be +a complex mix. + +A nice example of a contextual substitution is the centered period character in +catalan in \type {ebgaramond}: + +\startbuffer +\definefontfeature + [example] + [default] + [locl=yes,script=latn,language=cat] + +\definedfont[file:ebgaramond12-regular.otf*default at 40pt]l·l\quad +\definedfont[file:ebgaramond12-regular.otf*example at 40pt]l·l +\stopbuffer + +\typebuffer + +We show the boundingbox of the glyphs. The centered period between two l's is +is replaced by a raised variant with no width. + +\blank \start \showglyphs \maincolor \midaligned{\getbuffer} \stop \blank + +It will be clear that in order to use such features you need to know what the font +provides. For some fonts you need to explicitly enable the latin script (while others +use default). Such a feature can be part of localized support but that is no rule. +In that respect \OPENTYPE\ features are a rather unpredictable mess. For instance, +nothing prevents such a feature to be a ligature, and in case you find that strange, +especially ligature features are often abused for any purpose. + +\stopsubsection + +\startsubsection[reference=ligatures:hyphenation,title=Ligatures and hyphenation] + +In this section we will say a few words on how hyphenation interferes with +(especially) ligature building. For this you need to know that: + +\starttyping +effe +\stoptyping + +But when hyphenation is permitted between the two \type{s}'s we actually have +internally: + +\starttyping +ef{-}{}{}fe +\stoptyping + +The first snippet comes at the end of a line, the second at the beginning of a +the next line and the last snippet is used when no hyphenation is needed. Such +triplets need to be taken into account when we do replacements and positioning +and also when we do contextual lookups. + +An \OPENTYPE\ font is just a container that collects the following: + +\startitemize[packed] +\startitem + graphic representations of characters and symbols +\stopitem +\startitem + information about what characters the shapes represent +\stopitem +\startitem + rules about converting (sequences of) characters into one or more + representations +\stopitem +\startitem + rules about positioning representations relative to each other +\stopitem +\stopitemize + +Although the way this information is stored is standardized, the rules are not. +You can imagine that there would be some standard way to turn an \type {f} and +\type {i} into an \quote{fi} but we already saw that this is not the case. Here +are some possibilities: + +\startitemize[packed] +\startitem + The two characters get their own standard glyph, maybe with some kerning. +\stopitem +\startitem + The two characters are combined into one shape. +\stopitem +\startitem + The \type{f} gets a narrow representation and is kept close to the standard + \type{i}. +\stopitem +\startitem + A standard \type {f} is kerned with a dotless \type{i} (not to be confused + with the \UNICODE\ character). +\stopitem +\startitem + A special \type {f} is combined with a special \type {i}. +\stopitem +\stopitemize + +% maybe mark lig components when separate chars so that we can do spacing + +If the two characters are represented by their own shape, some contextual +analysis takes place. Again there are several approaches to this: + +\startitemize[packed] +\startitem + When an \type{f} is seen in the input, the next character is checked and one + or both gets replaced. +\stopitem +\startitem + When an \type{i} is seen in the input, the previous character is checked and + the \type {i} gets replaced. +\stopitem +\startitem + When an \type{f} several following characters are checked, for instance to + see if we need to take \type {ij} into account. +\stopitem +\stopitemize + +Traditionally the \type {f} followed by an \type{f}, \type{l} and \type{i} get a +treatment, but some fonts also combine the \type {f} with \type {k}, \type {j}, +\type {b}, \type {t} and more. + +The \MKIV\ font handler is rather generic in the sense that it support what the +font requires. However, a complication is that the scripts (languages) that use +these diverse methods also expect hyphenation within such a ligature. Script like +Arabic that are more demanding don't hyphenate so there interference with +hyphenation is not a problem. + +Some ligatures are sensitive for languages. In languages that have compound words +it might be undesirable to have a ligature at a word boundary, or in the Dutch +word \type {fijn} we like to have a nice glyph (or combinations) for \type {ij} +but no \type {fi} ligature. In a similar way hyphenation patterns can have +rules and it will be no surprise that the hyphenation mechanism can compete with +the ligature building for the best solution. This gets complicated by the fact +that there is no real way to recognize in the font handler if we really are +dealing with ligature building. Not only is the \type {liga} feature (and deep +down the ligature gsub handling) not bound to ligatures (but simply a +many|-|to|-|one mapper), some of the mentioned pseudo ligature builders use simple +substitution and kerning and there is no way to recognize that as a ligature. + +Although it is possible to come up with a solution that is acceptable for many cases, +there is no way to predict what kind of tricks font designers will use. A hyphenation +point can be seen as follows: + +\starttabulate[||||] +\NC \type{effe} \NC \type{ef-fe} \NC \type{e{f-}{f}{ff}e} \NC \NR +\NC \type{efficient} \NC \type{ef-fi-cient} \NC \type{e{ffi-}{}{ffi}cient} \NC \NR +\stoptabulate + +In the second case the larger ligatures has replaced the previous one. We could +have kept the first one because there are ways to manage two|-|step bounding +ligatures but it's not worth the trouble (read: way more complex code and +increased runtime for the whole mechanism). Here the \type {{ff}} and \type +{{ffi}} can be individual shapes or just one shape. + +The three components of a hyphenation point: the pre, post and replacement text +need to be looked at independently so that we get the proper kerning with the +preceding and following characters. Also, in more complex (chained) lookups we +need to compare each element with its surrounding. A fully expanded solution tree +is too time consuming so we take some shortcuts and limits the checks to the +level that it has no big impact on performance. The occasionally needed +backtracking and inspection of components is currently quite reasonable. We need +to trade quality with convenience: the result should look okay but processing +speed should also be as high as possible. There is no need to let other scripts +or regular fonts suffer too much from excessive script demands of fonts that +could have be done better. + +The complication is that we not only need to check and replace but also need to +check the kerning with preceding and following characters. We also need to take +the hyphen into account (here one, but there can also be one after the break. + +It is for this reason that in \MKIV\ we have a (we think) acceptable mix of +heuristics around hyphenation points that deal with single and multiple +substitution as well as kerning. It will never be 100\% pertect but we consider +it better to drop an occasional hyphenation in favor of proper font handling. In +practice \TEX\ is clever enough to break a paragraph in lines within these +restrictions. + +In \CONTEXT\ we have the traditional \TEX\ hyphenator but also provide an +extensible \LUA\ reimplementation. That one might become the default in future +versions. In traditional \TEX\ there are several low level hyphenation +representations: simple hyphen only points, injected by the hyphenator, +explicitly injected by the user or originating from a hyphen character. Then +there is the generic (pre, post, replace) discretionary that can be explicitly +injected by the user (or a macro). In \MKIV\ all hyphenation points get +normalized to this generic discretionary. There is no need for old|-|time +optimizations and a consistent (expanded) representation is easier to deal with +in other extensions. However, because the font handler is supposed to also work +outside \CONTEXT\ we need to deal with traditional cases too. But \unknown\ the +results might differ a bit. + +\stopsubsection + + \startsubsection[title=Color] + + % TODO: use emojionecolor-svginot-archived.ttf + + A recent new (and evolving) addition to \OPENTYPE\ is colored glyphs. One variant + (by \MICROSOFT) uses overlays and this method is quite efficient. + + \startbuffer + \definefontfeature[colored][colr=yes] + \definefontsynonym[Emoji][file:seguiemj.ttf*default,colored] + + \definesymbol[bug][\getglyphdirect{Emoji}{\char"1F41B}] + \definesymbol[ant][\getglyphdirect{Emoji}{\char"1F41C}] + \definesymbol[bee][\getglyphdirect{Emoji}{\char"1F41D}] + \stopbuffer + + \typebuffer \getbuffer + + Here we see a \symbol[bug], \symbol[ant] and \symbol[bee], and they come in + color! Once \UNICODE\ started adding such symbols (and more get added) the + distinction between characters and symbols get even fuzzier. Of course one + can argue that we communicate in pictograms but even then, given that + mankind lasts a while, the \UNICODE\ repertoire will explode. + + \startplacefigure[title={A few emojis from \type {seguiemj.ttf}}] + \startcombination [3*1] + {\scale[width=.3\textwidth]{\symbol[bug]}} {\type{U+1F41B}: bug} + {\scale[width=.3\textwidth]{\symbol[ant]}} {\type{U+1F41C}: ant} + {\scale[width=.3\textwidth]{\symbol[bee]}} {\type{U+1F41D}: bee} + \stopcombination + \stopplacefigure + + Here we use \type {seguiemj.ttf}, a font that comes with \MSWINDOWS. Colors are + achieved by combining glyphs rendered in different colors. A variant that uses + \SVG\ instead of overlays is \type {emojionecolor-svginot.ttf}: + + \startbuffer + \definefontfeature[svg][svg=yes] + \definefontsynonym[Emoji][file:emojionecolor-svginot.ttf*default,svg] + \stopbuffer + + \typebuffer \getbuffer + + This time we get \symbol[bug], \symbol[ant] and \symbol[bee] and they look + quite different. Both fonts also have ligatures and you can wonder what sense + that makes. It makes it impossible to swap fonts and as there is no standard + one never knows what to expect. + + \startplacefigure[title={A few emojis from \type {emojionecolor-svginot.ttf}}] + \startcombination [3*1] + {\scale[width=.3\textwidth]{\symbol[bug]}} {\type{U+1F41B}: bug} + {\scale[width=.3\textwidth]{\symbol[ant]}} {\type{U+1F41C}: ant} + {\scale[width=.3\textwidth]{\symbol[bee]}} {\type{U+1F41D}: bee} + \stopcombination + \stopplacefigure + + \definefont[emoji][file:emojionecolor-svginot.ttf*default,svg] + + \def\FourFaces{\char128104\zwj\char128105\zwj\char128102\zwj\char128102\relax} + + \def\Man {\char"1F468\relax} + \def\Woman{\char"1F469\relax} + \def\Boy {\char"1F466\relax} + \def\Girl {\char"1F467\relax} + + How do we know what faces add up to the ligature {\emoji\Man \zwj \Woman \zwj + \Girl \zwj \Boy} and how are we supposed to know that there should {\darkgray + \type {zwj}} in between? When we input four faces separated by zero width + joiners, we get a four face symbol instead. The reason for having the joiners in + between is probably to avoid unexpected ligatures. The sequence \type {man}, + \type {woman}, \type {boy}, \type {boy} gives \type {family}: + % + {\emoji\Man} + {\darkgray \type {zwj}} + {\emoji\Woman} + {\darkgray \type {zwj}} + {\emoji\Boy} + {\darkgray \type {zwj}} + {\emoji\Boy} = {\emoji\Man \zwj \Woman \zwj \Boy \zwj \Boy}, + % + but two girls also work: + % + {\emoji\Man} + {\darkgray \type {zwj}} + {\emoji\Woman} + {\darkgray \type {zwj}} + {\emoji\Girl} + {\darkgray \type {zwj}} + {\emoji\Girl} = {\emoji\Man \zwj \Woman \zwj \Girl \zwj \Girl}, + % + so does a mixture of kids: + % + {\emoji\Man} + {\darkgray \type {zwj}} + {\emoji\Woman} + {\darkgray \type {zwj}} + {\emoji\Girl} + {\darkgray \type {zwj}} + {\emoji\Boy} = {\emoji\Man \zwj \Woman \zwj \Girl \zwj \Boy}, + % + although (at least currently): + % + {\emoji\Man} + {\darkgray \type {zwj}} + {\emoji\Woman} + {\darkgray \type {zwj}} + {\emoji\Boy} + {\darkgray \type {zwj}} + {\emoji\Girl} = {\emoji\Man \zwj \Woman \zwj \Boy \zwj \Girl}, + % + gives twin boys. Of course the real family emoj is {\emoji\char"1F46A}. + + In our times for sure many combinations are possible, so: + % + {\emoji\Man} + {\darkgray \type {zwj}} + {\emoji\Man} + {\darkgray \type {zwj}} + {\emoji\Girl} + {\darkgray \type {zwj}} + {\emoji\Girl} = {\emoji\Man \zwj \Man \zwj \Girl \zwj \Girl}, + % + indeed gives a family, but I wonder at what point cultural bias will creep into + font design. One can even wonder how clothing and haircut will demand frequent + font updates: {\emoji\char"1F46B}, {\emoji\char"1F46C}, {\emoji\char"1F46D}. + + In the math alphabets we have a couple of annoying holes because some characters + were already present in \UNICODE. The bad thing here is that we now always have + to deal with these exceptions. But not so with emojis because here eventually all + variants will show up. Where a character \type {A} in red or blue uses the same + code point, a white telephone {\emoji\char"1F57E} and black telephone + {\emoji\char"1F57F} have their own. And because obsolete scripts are already + supported in \UNICODE\ and more get added, we can expect old artifacts also + showing up at some time. Soon the joystick {\emoji\char"1F579} will be an unknown + item to most of us, while the \MICROSOFT\ hololens migth get its slot. + + \startplacefigure[title={Will all animals come in stages of development?}] + \startcombination [3*1] + {\scale[width=.3\textwidth]{\emoji\char"1F423}} {\type{U+1F423}: hatching chick} + {\scale[width=.3\textwidth]{\emoji\char"1F424}} {\type{U+1F424}: baby chick} + {\scale[width=.3\textwidth]{\emoji\char"1F425}} {\type{U+1F425}: front-facing baby chick} + \stopcombination + \stopplacefigure + + For sure these mechanisms will evolve and to what extent we support them depends + on what users want. At least we have the basics implemented. + + \stopsubsection + + \stopsection + +\startsection[title=Extras] + +\startnotabene + Todo. +\stopnotabene + +\stopsection + +\startsection[reference=goodies,title=Goodies] + +Goodies range from simple to complex. They share that they are defined in files +and loaded at runtime. There is a good change that when you read this, that there +are already more goodies than mentioned here. Here we will just mention a couple +of goodies. More details can be found in the files that ship with \CONTEXT\ and +have suffix \type {lfg}. + +A goodie file is a regular \LUA\ file and is supposed to return a table. This +table collects data that is used for implementing the goodie or relates to a +regular feature. It can also provide information that is used for patching a +font. An example of a simple goodie file is the ones that accompanies the first +release of the \OPENTYPE\ Lucida fonts. + +\starttyping +return { + name = "lucida-opentype-math", + version = "1.00", + comment = "Goodies that complement lucida opentype.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + alternates = { + italic = { + feature = 'ss01', + value = 1, + comment = "Mathematical Alternative Italic" + }, + } + } +} +\stoptyping + +This goodie file is only providing information about the meaning of a stylistic +alternate. These have abstract tags like \type {ss01} and in this case this +category collects alternative italic (calligraphic) shapes. Because math does +not follow the same rules as text, this feature is enabled explicitly. + +In the goodie file of Xits math the alternates table has more entries: + +\startnarrowtyping +alternates = { + cal = { ... comment = "Mathematical Calligraphic Alphabet" }, + greekssup = { ... comment = "Mathematical Greek Sans Serif Alphabet" }, + greekssit = { ... comment = "Mathematical Italic Sans Serif Digits" }, + monobfnum = { ... comment = "Mathematical Bold Monospace Digits" }, + mathbbbf = { ... comment = "Mathematical Bold Double-Struck Alphabet" }, + mathbbit = { ... comment = "Mathematical Italic Double-Struck Alphabet" }, + mathbbbi = { ... comment = "Mathematical Bold Italic Double-Struck Alphabet" }, + upint = { ... comment = "Upright Integrals" }, + vertnot = { ... comment = "Negated Symbols With Vertical Stroke" }, +} +\stopnarrowtyping + +An alternate is triggered at the \TEX\ end with: + +\starttyping +$ABC$ $\cal ABC$ $\mathalternate{cal}\cal ABC$ +\stoptyping + +This is an example of a dynamic feature that gets applied when enabled at a +specific location in the input. The \type {cal} is only recognized when it +is defined in a goodies file, where the value is defined (in all of the above cases +the value is~\type {1}). + +The Xits math fonts has a goodie files that starts with: + +\starttyping +return { + name = "xits-math", + version = "1.00", + comment = "Goodies that complement xits (by Khaled Hosny).", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + italics = { + ["xits-math"] = italics, + }, + alternates = { +\stoptyping + +Here the \type {italics} variable is a table defined before the \type {return} +that looks as follows: + +\starttyping +local italics = { + defaultfactor = 0.025, + disableengine = true, + corrections = { + -- [0x1D44E] = 0.99, -- a (fraction of quad) + -- [0x1D44F] = 100, -- b (font points) + [0x1D453] = -0.0375, -- f + } +} +\stoptyping + +This rather specific table tells \CONTEXT\ that (when enabled) it has to apply +italic correction. It disables support built into the \TEX\ engine (which in the +case of \LUATEX\ is close to absent anyway). It will apply a default italic +correction of \type {0.025} but for some shapes a different value is used. Again +we have some commands at the \TEX\ end: + +\starttyping +\setupmathematics[italics=1] % apply italics with we have an italic font +\setupmathematics[italics=2] % apply italics anyway +\setupmathematics[italics=3] % apply italics only when italic or bolditalic shapes +\setupmathematics[italics=4] % combination of 1 and 3 +\stoptyping + +An alternative is this: + +\starttyping +\definefontfeature[mathextra][mathextra][collapseitalics=yes] +\stoptyping + +This extends the \type {mathextra} feature to move the italic correction into +the character's width. Often this works out ok. + +Because (definitely at the start of the \LUATEX\ project) we had no +proper \OPENTYPE\ math fonts, but at the same time wanted to move on +to \OPENTYPE\ and \UNICODE\ math and no longer struggle with all +those math families and definitions. The way out of this problem +is to define a virtual math font. The code for doing this is built +into the \MKIV\ core but is controlled by a goodie definition. Take +for instance Antykwa Math: + +\startnarrowtyping +return { + name = "antykwa-math", + version = "1.00", + comment = "Goodies that complement antykwa math.", + author = "Hans, Mojca, Aditya", + copyright = "ConTeXt development team", + mathematics = { + mapfiles = { + "antt-rm.map", + "antt-mi.map", + "antt-sy.map", + "antt-ex.map", + "mkiv-base.map", + }, + virtuals = { + ["antykwa-math"] = { + { name = "file:AntykwaTorunska-Regular", features = "virtualmath", main = true }, + { name = "mi-anttri.tfm", vector = "tex-mi", skewchar=0x7F }, + { name = "mi-anttri.tfm", vector = "tex-it", skewchar=0x7F }, + { name = "sy-anttrz.tfm", vector = "tex-sy", skewchar=0x30, parameters = true } , + { name = "ex-anttr.tfm", vector = "tex-ex", extension = true } , + { name = "msam10.tfm", vector = "tex-ma" }, + { name = "msbm10.tfm", vector = "tex-mb" }, + }, +\stopnarrowtyping + +Normally users will not define such tables but the keys give an indication of +what is involved. The same is true for the previously shown tables: they are just +provided in the \CONTEXT\ distribution. + +Text fonts also can have goodies. We start with a rather dumb one and there +will be not that many of those. This one is needed to turn a \TYPEONE\ font +with a rather special encoding into a \UNICODE\ font. The next mapping is +possible because the dingbats are part of \UNICODE. + +\starttyping +return { + name = "dingbats", + version = "1.00", + comment = "Goodies that complement dingbats (funny names).", + author = "Hans Hagen", + copyright = "ConTeXt development team", + remapping = { + tounicode = true, + unicodes = { + a1 = 0x2701, + a10 = 0x2721, + a100 = 0x275E, + a101 = 0x2761, + a102 = 0x2762, +\stoptyping + +Applying this encoding happens in two steps. Because goodies like this are just +features, we need to define a proper font feature set: + +\starttyping +\definefontfeature + [dingbats] + [mode=base, + goodies=dingbats, + unicoding=yes] +\stoptyping + +We have a base mode font, so no special processing takes place. The \type {goodies} +key is used to communicate the goodies file. The \type {unicoding} key is used +to apply the encoding. Of course this only works because the remapper code is present +in the core and is hooked in to the font initialization code. The \type {dingbats} +feature set is predefined, just as the font definition: + +\starttyping +\definefontsynonym [ZapfDingbats] [file:uzdr] [features=dingbats] +\stoptyping + +Here is a goodie file that I made a while ago: + +\starttyping +return { + name = "oxoniensis", + version = "1.00", + comment = "Oxoniensis test file for Thomas Schmitz.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + features = { + lunatesigma = { + type = "substitution", + data = { + sigma = 0x03F2, + sigma1 = 0x03F2, + Sigma = 0x03F9, + phi = phi1, + }, + } + }, +} +\stoptyping + +There is not that much to say about this, apart from that it's a sort of fake +feature that gets enabled as regular one: + +\starttyping +\definefontfeature[test] + [mode=node, + kern=yes, + lunatesigma=yes, + goodies=oxoniensis] + +\definefont[somefont][file:oxoniensis*test] +\stoptyping + +A complete different kind of goodie is the following. At one of the \CONTEXT\ meetings +Mojca Miklavec discussed the possibility to have an additional mechanism for +defining combinations of fonts. Often fonts come in a set of four (regular, italic, +bold and bold italic). In \MKII\ the complexity of typescripts depends on the amount of +encodings that need to be supported but in \MKIV\ things are easier. For a set of four fonts +a typescript looks as follows: + +\starttyping +\starttypescript [sans] [somesansfont] [name] + \setups[font:fallback:sans] + \definefontsynonym [Sans] [file:somesans] [features=default] + \definefontsynonym [SansBold] [file:somesansb] [features=default] + \definefontsynonym [SansItalic] [file:somesansi] [features=default] + \definefontsynonym [SansBoldItalic] [file:somesansz] [features=default] +\stoptypescript +\stoptyping + +We still have the abstract notion of a \type {Sans} font so that we can refer to +the regular shape without knowing the real name but the number of lines needed +is small. Such a definition can then be referred to using: + +\starttyping +\starttypescript[somefontset] + \definetypeface [somefontset] [rm] [serif] [someserif] [default] + \definetypeface [somefontset] [ss] [sans] [somesans] [default] + \definetypeface [somefontset] [tt] [mono] [somemono] [default] + \definetypeface [somefontset] [mm] [math] [somemath] [default] +\stoptypescript +\stoptyping + +So far things look simple. Given that many fonts follow a similar naming scheme +Wolfgang made a module that avoids such definitions altogether. However, being +involved in the development of the Antykwa fonts, Mojca ran into the situation +that not just four fonts were part of the set but many more. There are several +weight (think of light and heavy variants) as well as condensed variants and of +course the whole set is not per se a multiple of four. + +In the meantime, in addition to the \type {file:} and \type {name:} accessors, +\CONTEXT\ had an additional one tagged \type {spec:} where a string made out of +weight, style, width etc.\ is turned into a (best guessed) font name. Therefore +the most natural way to deal with the many|-|fonts|-|in|-|a|-|set dilemma was to +provide an additional interface between this specification and the font set and +the most robust method was to define all in a goodie file. + +In this case the goodies are loaded independent of the font, that is: not +as a feature. For instance: + +\starttyping +\loadfontgoodies[antykwapoltawskiego] +\stoptyping + +This file maps obscure fontnames onto the \type {spec:} interface so that +we can access them in a robust way. + +\starttyping +\definefont + [MyFontA] + [file:Iwona-Regular*smallcaps] +\definefont + [MyFontB] + [file:AntykwaTorunska-Regular*smallcaps] +\definefont + [MyFontC] + [file:antpoltltcond-regular*smallcaps] +\definefont + [MyFontD] + [spec:antykwapoltawskiego-bold-italic-condensed-normal*smallcaps] +\definefont + [MyFontE] + [spec:antykwapoltawskiego-bold-italic-normal] +\stoptyping + +The goodies file looks as follows: + +\starttyping +return { + name = "antykwa-poltawskiego", + version = "1.00", + comment = "Goodies that complement Antykwa Poltawskiego", + author = "Hans & Mojca", + copyright = "ConTeXt development team", + files = { + name = "antykwapoltawskiego", -- shared + list = { + ["AntPoltLtCond-Regular.otf"] = { + weight = "light", + style = "regular", + width = "condensed", + }, + ... + ["AntPoltExpd-BoldItalic.otf"] = { + weight = "bold", + style = "italic", + width = "expanded", + }, + }, + }, + typefaces = { + ["antykwapoltawskiego-light"] = { + shortcut = "rm", + shape = "serif", + fontname = "antykwapoltawskiego", + normalweight = "light", + boldweight = "medium", + width = "normal", + size = "default", + features = "default", + }, + ... + }, +} +\stoptyping + +In addition to the files|-|to|-|specification mapping, there is +also a typeface specification table. This permits the definition +of a typeface in the following way: + +\starttyping +\definetypeface + [name=mojcasfavourite, + preset=antykwapoltawskiego, + normalweight=light, + boldweight=bold, + width=expanded] + +\setupbodyfont + [mojcasfavourite] +\stoptyping + +When resolving the definition, the best possible match will be taken from the +typeface table in the goodie file. Of course this is not something that we expect +the average user to deliver and deal with. + +As the Antykwa font is somewhat atypical and not used in everyday typesetting, +you might wonder if all this overhead makes sense. However, there are type +foundries that do ship their fonts in many weights and for those using a \LUA\ +goodie file instead of many typescripts in \TEX\ coding makes sense. Take for +instance TheMix: + +\starttyping +\loadfontgoodies + [themix] + +\definetypeface + [name=themix, + preset=themix-light] + +\definetypeface + [name=themix, + preset=themixmono-light] + +\setupbodyfont + [themix] +\stoptyping + +In this case the goodie file can serve as a template for more such fonts. +In order to be efficient this goodie file uses a couple of local +tables (we could have used metatables instead). + +\starttyping +local themix = { + name = "themix", + shortcut = "ss", + shape = "sans", + fontname = "themix", + width = "normal", + size = "default", + features = "default", +} + +local themixmono = { + name = "themixmono", + shortcut = "tt", + shape = "mono", + fontname = "themixmono", + width = "normal", + size = "default", + features = "default", +} +\stoptyping + +The main goodie table defines a lot of weights: + +\startnarrowtyping +return { + name = "themix", + version = "1.00", + comment = "Goodies that complement TheMix (by and for sale at www.lucasfonts.com).", + author = "Hans Hagen", + copyright = "ConTeXt development team", + files = { + list = { + ["TheMixOsF-ExtraLight"] = { + name = "themix", + weight = "extralight", + style = "regular", + width = "normal" + }, + ["TheMixOsF-ExtraLightItalic"] = { + ... + }, + ... + ["TheMixOsF-Black"] = { + ... + }, + ["TheMixOsF-BlackItalic"] = { + ... + }, + ... + -- + ["TheMixMono-W2ExtraLight"] = { + name = "themixmono", + weight = "extralight", + style = "regular", + width = "normal" + }, + ... + ["TheMixMono-W9BlackItalic"] = { + ... + }, + }, + }, + typefaces = { + ["themix-extralight"] = table.merged(themix, { + normalweight = "extralight", + boldweight = "semilight" + }), + ["themix-light"] = table.merged(themix, { + normalweight = "light", + boldweight = "normal" + }), + ... + ["themixmono-bold"] = table.merged(themixmono, { + normalweight = "bold", + boldweight = "black" + }), + }, +} +\stopnarrowtyping + +It's now time for some generic goodies. In the \CONTEXT\ distribution there +is a goodie file that (at the time of this writing) looks as follows: + +\starttyping +local default = { + analyze = "yes", + mode = "node", + language = "dflt", + script = "dflt", +} + +local smallcaps = { + smcp = "yes", +} + +local function statistics(tfmdata) + commands.showfontparameters(tfmdata) +end + +local function squeeze(tfmdata) + for k, v in next, tfmdata.characters do + v.height = 0.75 * (v.height or 0) + v.depth = 0.75 * (v.depth or 0) + end +end + +return { + name = "demo", + version = "1.01", + comment = "An example of goodies.", + author = "Hans Hagen", + featuresets = { + default = { + default, + }, + smallcaps = { + default, smallcaps, + }, + }, + colorschemes = { + default = { + [1] = { + "one", "three", "five", "seven", "nine", + }, + [2] = { + "two", "four", "six", "eight", "ten", + }, + }, + all = { + [1] = { + "*", + }, + }, + some = { + [1] = { + "0x0030:0x0035", + }, + }, + }, + postprocessors = { + statistics = statistics, + squeeze = squeeze, + }, +} +\stoptyping + +This demo file implements several goodies: featuresets, colors and +postprocessors. Keep in mind that a goodie file can provide whatever information +it wants but of course only known subtables will be dealt with. + +The coloring of glyphs can happen by name, which assumes that glyph names are +used, or by number. Here we use generic glyph names, but for a specific font one +might need to provide a special goodie file. For instance, the color section of +the goodie file for the husayni font has entries like: + +\startnarrowtyping +[3] = { + "Ttaa.waqf", "SsLY.waqf", "QLY.waqf", "Miim.waqf", "LA.waqf", "Jiim.waqf", + "Threedotsabove.waqf", "Siin.waqf", "Ssaad.waqf", "Qaaf.waqf", "SsL.waqf", + "QF.waqf", "SKTH.waqf", "WQFH.waqf", "Kaaf.waqf", "Ayn.ruku", "Miim.nuun_high", + "Siin.Ssaad", "Nuunsmall", "emptydot_low", "emptydot_high", "Sifr.fill", + "Miim.nuun_low", "Nuun.tanwiin", +}, +\stopnarrowtyping + +Of course such a definition can only be made when the internals of the font are +known and in this case it concerns a pretty complex font. + +\startbuffer +\definefontfeature + [demo-colored] + [goodies=demo, + colorscheme=default, + featureset=default] + +\definefontfeature + [demo-colored-all] + [goodies=demo, + colorscheme=all, + featureset=default] + +\definefontfeature + [demo-colored-some] + [goodies=demo, + colorscheme=some, + featureset=default] + +\definefont[DemoFontA][MonoBold*demo-colored at 10pt] +\definefont[DemoFontB][MonoBold*demo-colored-all at 10pt] +\definefont[DemoFontC][MonoBold*demo-colored-some at 10pt] +\stopbuffer + +\typebuffer \getbuffer + +% \definecolor[colorscheme:1:1][s=.75] +% \definecolor[colorscheme:1:2][r=.75] +% \definecolor[colorscheme:1:3][g=.75] +% \definecolor[colorscheme:1:4][b=.75] +% \definecolor[colorscheme:1:5][c=.75] +% \definecolor[colorscheme:1:6][m=.75] +% \definecolor[colorscheme:1:7][y=.75] + +% \definecolor[colorscheme:2:7][s=.75] +% \definecolor[colorscheme:2:6][r=.75] +% \definecolor[colorscheme:2:5][g=.75] +% \definecolor[colorscheme:2:4][b=.75] +% \definecolor[colorscheme:2:3][c=.75] +% \definecolor[colorscheme:2:2][m=.75] +% \definecolor[colorscheme:2:1][y=.75] + +\startbuffer +\starttabulate[||||] +\NC + \DemoFontA \resetfontcolorscheme test 1234567890 \NC + \DemoFontA \setfontcolorscheme [1]test 1234567890 \NC + \DemoFontA \setfontcolorscheme [2]test 1234567890 \NC +\NR +\NC + \DemoFontB \resetfontcolorscheme test 1234567890 \NC + \DemoFontB \setfontcolorscheme [1]test 1234567890 \NC + \DemoFontB \setfontcolorscheme [2]test 1234567890 \NC +\NR +\NC + \DemoFontC \resetfontcolorscheme test 1234567890 \NC + \DemoFontC \setfontcolorscheme [1]test 1234567890 \NC + \DemoFontC \setfontcolorscheme [2]test 1234567890 \NC +\NR +\stoptabulate +\stopbuffer + +\typebuffer \getbuffer + +Here is an example that I made at the TUG 2013 conference in Japan, +after a presentation by Chof. The font (adapted by by Dohyun Kim) can +be downloaded from: \hyphenatedurl {http://ftp.ktug.org/KTUG/hcr-lvt/1.910_nomac/}. + +\startbuffer[korean-demo] +\definefontfeature + [korean-composed] + [goodies=hanbatanglvt, + colorscheme=default, + mode=node, + ljmo=yes, + tjmo=yes, + vjmo=yes, + script=hang, + language=kor] + +\definefont + [KoreanJMO] + [hanbatanglvt*korean-composed] + +\definecolor[colorscheme:100:1][r=.75] +\definecolor[colorscheme:100:2][g=.75] +\definecolor[colorscheme:100:3][b=.75] + +\definecolor[colorscheme:101:1][g=.75,b=.75] +\definecolor[colorscheme:101:2][r=.75,b=.75] +\definecolor[colorscheme:101:3][r=.75,g=.75] +\stopbuffer + +\typebuffer[korean-demo] \getbuffer[korean-demo] + +\startbuffer + % Hunminjeongeum: http://en.wikipedia.org/wiki/Hunminjeongeum + 나랏말ᄊᆞ미中듕國귁에달아문ᄍᆞᆼ와로서르ᄉᆞᄆᆞᆺ디아니ᄒᆞᆯᄊᆡ% + 사ᄅᆞᆷ마다ᄒᆡᅇᅧ수ᄫᅵ니겨나...% ᆯ로ᄡᅮ메便뼌安ᅙᅡᆫ킈ᄒᆞ고져ᄒᆞᇙᄯᆞᄅᆞ미니라 +\stopbuffer + +\startlinecorrection +\startcombination[1*3] + {\framed{\startscript[hangul]\KoreanJMO \getbuffer\stopscript}} {no colorscheme} + {\framed{\startscript[hangul]\KoreanJMO\setfontcolorscheme[100]\getbuffer\stopscript}} {colorscheme 100} + {\framed{\startscript[hangul]\KoreanJMO\setfontcolorscheme[101]\getbuffer\stopscript}} {colorscheme 101} +\stopcombination +\stoplinecorrection + +The goodie definition looks as follows (watch how we use ranges): + +\starttyping +return { + name = "hanbatanglvt", + version = "1.00", + comment = "Goodies that complement the hanbatanglvt fonts.", + author = "Hans Hagen", + colorschemes = { + default = { + { "0x01100:0x0115F" }, -- jamo_initial (r/c) + { "0x01160:0x011A7" }, -- jamo_medial (g/m) + { "0x011A8:0x011FF" }, -- jamo_final (b/y) + } + } +} +\stoptyping + +This is much shorter (and efficent) that defining a whole vector, as in: + +\starttyping +local f_uni_base = string.formatters["uni%04X"] +local f_uni_plus = string.formatters["uni%04X.y%s"] + +local function range(first,last) + local t = { } + for i=first,last do + t[#t+1] = f_uni_base(i) + for j=0,19 do + t[#t+1] = f_uni_plus(i,j) + end + end + return t +end + +return { + name = "hanbatanglvt", + version = "1.00", + comment = "Goodies that complement the hanbatanglvt fonts.", + author = "Hans Hagen", + colorschemes = { + default = { + range(0x01100,0x0115F), -- jamo_initial (r/c) + range(0x01160,0x011A7), -- jamo_medial (g/m) + range(0x011A8,0x011FF), -- jamo_final (b/y) + } + } +} +\stoptyping + +By using names we don't depend on \UNICODE\ which sometimes is needed when glyphs +have ended up in the private space. However, by default, after glyphs have been +mapped to colors, an extra pass will make sure that characters pushed into +private space will get the same mapping as their regular \UNICODE\ has gotten +(given that the number is known). Of course explicitly assigned colors will be +preserved. + +Another generic demo feature is postprocessing. In principle one can +add additional postprocessors but for that the source code needs to +be consulted which in turn assumes some knowledge of fonts and \CONTEXT\ +internals. + +\startbuffer +\definefontfeature + [justademoa] + [default] + [goodies=demo, + postprocessor=squeeze] + +\definefontfeature + [justademob] + [default] + [goodies=demo, + postprocessor=statistics] + +\definefontfeature + [justademoc] + [default] + [goodies=demo, + postprocessor={statistics,squeeze}] +\stopbuffer + +\typebuffer \getbuffer + +The statistics just print some font parameters to the log so that one +is not showing up here. The squeeze looks as follows: + +\startbuffer +\definefont[DemoFontD][Serif*default at 30pt] +\definefont[DemoFontE][Serif*justademoa at 30pt] +\stopbuffer + +\typebuffer \getbuffer + +\startlinecorrection +\hbox\bgroup + \ruledhbox{\color[maincolor]{DemoFontD height \& depth}}\quad + \ruledhbox{\color[maincolor]{DemoFontE height \& depth}} +\egroup +\stoplinecorrection + +The squeezer just makes the height and depth of glyphs a bit smaller and it is +not that hard to imagine other manipulations. The demo goodie file is good +place to start playing with such things. + +Because there is less standardization with respect to features than one might +suspect, goodie files provide a mean to define featuresets. We can use such a set +in another definition: + +\starttyping +\definefontfeature + [demo-smallcaps] + [goodies=demo, + featureset=smallcaps] +\stoptyping + +Of course this only makes sense for more complex combinations. The already mentioned +husayni font comes with many features and most of these work together. + +The basic goodie table looks as follows: + +\startnarrowtyping +return { + name = "husayni", + version = "1.00", + comment = "Goodies that complement the Husayni font by Idris Samawi Hamid.", + author = "Idris Samawi Hamid and Hans Hagen", + featuresets = { }, + solutions = { }, + stylistics = { }, + colorschemes = { }, +} +\stopnarrowtyping + +We already saw the color schemes and now we will fill in the other tables. First +we define a couple of sets: + +\startnarrowtyping +local basics = { + analyze = "yes", + mode = "node", + language = "dflt", + script = "arab", +} + +local analysis = { + ccmp = "yes", + init = "yes", medi = "yes", fina = "yes", +} + +local regular = { + rlig = "yes", calt = "yes", salt = "yes", anum = "yes", + ss01 = "yes", ss03 = "yes", ss07 = "yes", ss10 = "yes", ss12 = "yes", + ss15 = "yes", ss16 = "yes", ss19 = "yes", ss24 = "yes", ss25 = "yes", + ss26 = "yes", ss27 = "yes", ss31 = "yes", ss34 = "yes", ss35 = "yes", + ss36 = "yes", ss37 = "yes", ss38 = "yes", ss41 = "yes", ss42 = "yes", + ss43 = "yes", js16 = "yes", +} + +local positioning = { + kern = "yes", curs = "yes", mark = "yes", mkmk = "yes", +} + +local minimal_stretching = { + js11 = "yes", js03 = "yes", +} + +local medium_stretching = { + js12="yes", js05="yes", +} +local maximal_stretching= { + js13 = "yes", js05 = "yes", js09 = "yes", +} + +local wide_all = { + js11 = "yes", js12 = "yes", js13 = "yes", js05 = "yes", js09 = "yes", +} + +local shrink = { + flts = "yes", js17 = "yes", ss05 = "yes", ss11 = "yes", ss06 = "yes", + ss09 = "yes", +} + +local default = { + basics, analysis, regular, positioning, -- xxxx = "yes", yyyy = 2, +} +\stopnarrowtyping + +Next we define some featuresets and we use the default as starting point: + +\startnarrowtyping + featuresets = { + default = { + default, + }, + minimal_stretching = { + default, js11 = "yes", js03 = "yes", + }, + medium_stretching = { + default, js12="yes", js05="yes", + }, + maximal_stretching= { + default, js13 = "yes", js05 = "yes", js09 = "yes", + }, + wide_all = { + default, js11 = "yes", js12 = "yes", js13 = "yes", js05 = "yes", + js09 = "yes", + }, + shrink = { + default, flts = "yes", js17 = "yes", ss05 = "yes", ss11 = "yes", + ss06 = "yes", ss09 = "yes", + }, + } +\stopnarrowtyping + +When defining the font at the \TEX\ end we can now refer to for instance \type +{wide_all} which saves us some typing. However, it does not stop here. In a later +paragraph we will see how fonts can work in tandem with the parbuilder. For that +purpose the goodie table has a \type {solutions} subtable: + +\startnarrowtyping +solutions = { + experimental = { + less = { + "shrink" + }, + more = { + "minimal_stretching", "medium_stretching", "maximal_stretching", "wide_all" + }, + }, +} +\stopnarrowtyping + +Here we define an experimental solution for optimizing the lines in a paragraph: +we can narrow words or we can widen them according to a specific featureset. In +order to reach the optimal solution the text will be retypeset under a different +feature regime. + +{\em TODO: show how to apply.} + +%D \starttyping +%D \setupfontsolutions[method={random,preroll},criterium=1,randomseed=101] +%D +%D \definefontsolution % actually only the last run needs to be done this way +%D [FancyHusayni] +%D [goodies=husayni, +%D solution=experimental] +%D +%D \definedfont[husayni*husayni-default at 24pt] +%D \setupinterlinespace[line=36pt] +%D \righttoleft +%D \enabletrackers[parbuilders.solutions.splitters.colors] +%D \setfontsolution[FancyHusayni] +%D alb alb alb \par +%D \resetfontsolution +%D \disabletrackers[parbuilders.solutions.splitters.colors] +%D \stoptyping + +Because there are a some 55 stylistic and 21 justification variants the +goodie file also provides a \type {stylistics} table and for tracing purposes +the {colorschemes} table is populated. + +Yet another demonstration of manipulation is the following. Not all fonts come +with all combined glyphs. Although we have an auto|-|compose feature in \CONTEXT\ +it sometimes helps to be specific with respect to some combinations. This is +where the \type {compositions} goodie kicks in: + +\starttyping +local compose = { + [0x1E02] = { + anchored = "top", + }, + [0x1E04] = { + anchored = "bottom", + }, + [0x0042] = { -- B + anchors = { + top = { + x = 300, + y = 700, + }, + bottom = { + x = 300, + y = -30, + }, + }, + }, + [0x0307] = { + anchors = { + top = { + x = -250, + y = 550, + }, + }, + }, + [0x0323] = { + anchors = { + bottom = { + x = -250, + y = -80, + }, + }, + }, +} + +return { + name = "lm-compose-test", + version = "1.00", + comment = "Goodies that demonstrate composition.", + author = "Hans and Mojca", + copyright = "ConTeXt development team", + compositions = { + ["lmroman12-regular"] = compose, + } +} +\stoptyping + +Of course this assumes some knowledge of the font metrics (in base points) and +\UNICODE\ slots, but it might be worth the trouble. After all, one only needs to +figure it out once. But keep in mind that it will always be a kludge. + +A slightly different way to define such compositions is the following: + +\starttyping +local defaultunits = 193 - 30 + +local compose = { + DY = defaultunits, + -- [0x010C] = { DY = defaultunits }, -- Ccaron + -- [0x02C7] = { DY = defaultunits }, -- textcaron +} + +-- fractions relative to delta(X_height - x_height) + +local defaultfraction = 0.85 + +local compose = { + DY = defaultfraction, -- uppercase compensation +} + +return { + name = "lucida-one", + version = "1.00", + comment = "Goodies that complement lucida.", + author = "Hans and Mojca", + copyright = "ConTeXt development team", + compositions = { + ["lbr"] = compose, + ["lbi"] = compose, + ["lbd"] = compose, + ["lbdi"] = compose, + } +} +\stoptyping + +Of course no one really needs this because \OPENTYPE\ Lucida fonts +have replaced the \TYPEONE\ versions. + +The next goodie table is dedicated to the de facto standard \TEX\ font Latin +Modern. There is a bit of history behind this file. When we started writing +\CONTEXT\ there were not that many fonts available and so we ended up with a font +system that was rather well suited for the predecessor of Latin Modern, called +Computer Modern. And because these fonts came in design sizes the font system +was made such that it could cope efficiently with many files in a font set. Although +there is no additional overhead compared to small font sets, apart from more files, +there is some burden in defining them. And, as they are the default fonts, these +definitions slow down the initialization of \CONTEXT\ (which is due to the fact that +the large typescript definitions were loaded and parsed). So, at some point +the decision was made to kick out these definitions and move the burden of figuring +out the right size to \LUA. When Latin Modern is chosen as font the effect is the +same when design sizes are enabled. But, instead of many definitions (one for each +combination of size and style) we now have an option. A non|-|designsize typeface +is defined as follows: + +\startnarrowtyping +\starttypescript [modern,modern-base] + \definetypeface [\typescriptone] [rm] [serif] [modern] [default] + \definetypeface [\typescriptone] [ss] [sans] [modern] [default] + \definetypeface [\typescriptone] [tt] [mono] [modern] [default] + \definetypeface [\typescriptone] [mm] [math] [modern] [default] + \quittypescriptscanning +\stoptypescript +\stopnarrowtyping + +The designsize variant looks like this: + +\startnarrowtyping +\starttypescript [modern-designsize] + \definetypeface [\typescriptone] + [rm] [serif] [latin-modern-designsize] [default] [designsize=auto] + \definetypeface [\typescriptone] + [ss] [sans] [latin-modern-designsize] [default] [designsize=auto] + \definetypeface [\typescriptone] + [tt] [mono] [latin-modern-designsize] [default] [designsize=auto] + \definetypeface [\typescriptone] + [mm] [math] [latin-modern-designsize] [default] [designsize=auto] + \quittypescriptscanning +\stoptypescript +\stopnarrowtyping + +Of course there are accompanying typescripts that map the sans, serif, mono +and math styles onto files. The \type {designsize} magic uses the following +table. We show only part of the file, as it is in the \CONTEXT\ distribution. + +\starttyping +return { + name = "latin modern", + version = "1.00", + comment = "Goodies that complement latin modern.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + tweaks = { + aftercopying = { + mathematics.tweaks.fixbadprime, -- prime is too low + }, + }, + }, + designsizes = { + ["LMMathRoman-Regular"] = { + ["4pt"] = "LMMath5-Regular@lmroman5-math", + ... + ["12pt"] = "LMMath12-Regular@lmroman12-math", + default = "LMMath10-Regular@lmroman10-math" + }, + ["LMMathRoman-Bold"] = { -- not yet ready + ... + }, + ["LMRoman-Regular"] = { + ["4pt"] = "file:lmroman5-regular", + ... + ["12pt"] = "file:lmroman12-regular", + default = "file:lmroman10-regular", + }, + ["LMRoman-Bold"] = { + ... + }, + ["LMRoman-Demi"] = { + default = "file:lmromandemi10-regular", + }, + ["LMRoman-Italic"] = { + ... + }, + ... + ["LMRoman-Unslanted"] = { + default = "file:lmromanunsl10-regular", + }, + ["LMSans-Regular"] = { + ... + }, + ["LMTypewriter-Regular"] = { + ... + }, + ... + ["LMTypewriterVarWd-DarkOblique"] = { + default = "file:lmmonoproplt10-boldoblique", + }, + ... + ["LMTypewriter-CapsOblique"] = { + default = "file:lmmonocaps10-oblique", + }, + } +} +\stoptyping + +The \type {auto} option will choose a best fit compatible to the +\MKII\ implementation. When \type {default} is used instead, the +default filename will be taken. Of course one might wonder if +there will ever be similar goodie files because design sizes +are not that popular nowadays. + +Not all fonts are perfect and of course the \LUATEX\ engine can have flaws as +well. For this reason we can implement patches. Here is another example of a +goodie file that has some more code than just a table: + +\starttyping +local patches = fonts.handlers.otf.enhancers.patches + +local function patch(data,filename,threshold) + local m = data.metadata.math + if m then + local d = m.DisplayOperatorMinHeight or 0 + if d < threshold then + patches.report("DisplayOperatorMinHeight(%s -> %s)",d,threshold) + m.DisplayOperatorMinHeight = threshold + end + end +end + +patches.register("after","analyze math","asana", + function(data,filename) patch(data,filename,1350) end) + +local function less(value,target,original) + return 0.25 * value +end + +local function more(value,target,original) + local o = original.mathparameters.DisplayOperatorMinHeight + if o < 2800 then + return 2800 * target.parameters.factor + else + return value -- already scaled + end +end + +return { + name = "asana-math", + version = "1.00", + comment = "Goodies that complement asana.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + parameters = { + DisplayOperatorMinHeight = more, + StackBottomDisplayStyleShiftDown = less, + StackBottomShiftDown = less, + StackDisplayStyleGapMin = less, + StackGapMin = less, + StackTopDisplayStyleShiftUp = less, + StackTopShiftUp = less, + StretchStackBottomShiftDown = less, + StretchStackGapAboveMin = less, + StretchStackGapBelowMin = less, + StretchStackTopShiftUp = less, + } + } +} +\stoptyping + +In fact, in addition to already mentioned \type {mapfiles} and +\type {virtuals} subtables, we can pass variables and +overload parameters. + +\starttyping +return { + name = "lm-math", + ... + mathematics = { + mapfiles = { + ... + }, + virtuals = { + ... + variables = { + joinrelfactor = 3, -- default anyway + }, + parameters = { -- test values + -- FactorA = 123.456, + -- FactorB = false, + -- FactorC = function(value,target,original) + -- return 7.89 * target.factor + -- end, + -- FactorD = "Hi There!", + }, + } +} +\stoptyping + +This kind of goodie functionality is typical for the development of \LUATEX\ and +experimental math fonts and no user should ever be bothered with it. However, it +demonstrates that we're not stuck with only features built in the fonts. + +% mathdimensions + +It can be that a user is not satisfied by some aspects of a math font design. +There is not much that we can do about the shapes, but we can manipulate for +instance dimensions. + +For this there are two mechanism available: automatically applied dimensional +fixes and a \type {mathdimensions} feature. Both work with the same goody +specification. + +\starttyping +mathematics = { + ... + dimensions = { + }, + ... +} +\stoptyping + +The entries in a dimensions table are tables themselves. There can be many +of them so one can organize dimensional tweaks in groups. The \type {default} +group is always applied, while others are applied on demand. Say that want +to tweak all \type {±} and \type {∓}. \footnote {In fact, this mechanism is a +a response to a mail on the \CONTEXT\ mailing list.} + +\starttyping +mathematics = { + dimensions = { + default = { + [0x00B1] = { -- ± + height = 500, + depth = 0, + }, + [0x2213] = { -- ∓ + height = 500, + depth = 0, + }, + }, + }, +} +\stoptyping + +This will give these two characters a different height and depth. However, this +will not have much effect in rendering (much larger dimensions might have). + +\starttyping +mathematics = { + dimensions = { + default = { + [0x00B1] = { -- ± + yoffset = 100, + }, + [0x2213] = { -- ∓ + yoffset = -100, + }, + }, + }, +} +\stoptyping + +This will raise and lower the glyphs in their bounding boxes and give them +an appearance more close to their ancestors. But defined this way, they are +always applied and that might not be what we want. So, we can do this: + +\starttyping +mathematics = { + dimensions = { + signs = { + [0x00B1] = { -- ± + yoffset = 100, + }, + [0x2213] = { -- ∓ + yoffset = -100, + }, + }, + }, +} +\stoptyping + +This time the application is feature driven. As with all features, setting them +up has to happen {\em before} fonts are loaded. This will do the trick: + +\starttyping +\definefontfeature [lm-math] [mathdimensions=signs] +\stoptyping + +The \type {lm-math} feature is not defined by default but can be used for such +purposes. It {\em is} defined with the fontname: + +\starttyping +\definefontsynonym + [LMMathRoman-Regular] + [file:latinmodern-math-regular.otf] + [features={math\mathsizesuffix,lm-math}, + goodies=lm] +\stoptyping + +A rather specialized goodie is the one that is used to specify math cut|-|ins. A +good quality math font has these kerns already defined but even then you might +want to add or replace some by your own. Here is an example of such a patch. +Normally there are multiple goodies defined in one file but we only show kerns +here: + +\starttyping +local kern_200 = { bottomright = { { kern = -200 } } } +local kern_100 = { bottomright = { { kern = -100 } } } + +return { + name = "pagella-math", + version = "1.00", + comment = "Goodies that complement pagella.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + kerns = { + [0x1D449] = kern_200, -- math italic V + [0x1D44A] = kern_100, -- math italic W + }, + }, +} +\stoptyping + +As with other goodies the file is loaded with: + +\starttyping +\definefontsynonym + [MathRoman] % names used in definitions + [file:texgyrepagella-math.otf] % the file to be loaded + [features=math\mathsizesuffix, % size dependent features + goodies=pagella-math] % the goodie file to be applied +\stoptyping + +This is typically a goodie that is always applied and not driven by a feature. +After all, the values given are passed to the engine (after being scaled). + +Most goodies are bound to fonts of collections of fonts. This is different for +treatments. These ship with the distribution but you can also provide your own. +As this is still somewhat experimental we just mention a few aspects. The entries +are filenames that point to tables. + +\starttyping +return { + name = "treatments", + version = "1.00", + comment = "Goodies that deals with some general issues.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + treatments = { + ["adobeheitistd-regular.otf"] = { + embedded = false, -- not yet used + comment = "this font is part of acrobat", + }, + ["crap.ttf"] = { + ignored = true, + comment = "a text file with suffix ttf", + }, + ["latinmodern-math.otf"] = { + comment = "experimental", + }, + ["rubish-regular.ttf"] = { + comment = "check output for missing à and á", + } + }, +} +\stoptyping + +The comment entry in such a table becomes part of the message at the end +of a run: + +\startnarrowtyping +mkiv lua stats > loaded fonts: 2 files: latinmodern-math.otf (experimental), lmroman12-regular.otf +\stopnarrowtyping + +The ignored flag signals the font name database builder to ignore the file. This +means that the font can still be known as file, but that its (name based) +properties are not collected. As you asked explicitly for a file, the file can +still be loaded. You can use this trick to avoid issues with the database builder +in case of a problematic file, but a real run will still try to load the file. After +all, you get what you ask for. If loading and usage is successful you get at least +the message reported at the end of the run. + +\stopsection + +\startsection[title=Analyzers] + +An \OPENTYPE\ font is kind of special in the sense that it provides some +information on how to turn sequences of characters into sequences of glyphs. In +fact, if all fonts had a reasonable repertoire of glyphs most of the information +that concerns combining, remapping and shuffling the input and|/|or mapping onto +glyphs could as well happen in the renderer. This means that fonts have many of +their internal features tables in common, or more precisely could share many gsub +related issues, if only there had been some predefined sets of substitutional +features. + +So, for most of the time, a feature processor just does what the font demands and +the font provides the information. There are however a few cases where font only +provide part of the logic. Take for instance the \type {init}, \type {medi}, +\type {fina} and \type {isol} features that relate to positions in the word: the +start, the end, in the middle or isolated. For these features to work the engine +has to provide information about the state of a character (glyph) and this is where +analysis kicks in. Just watch this: + +\startbuffer +\definefontfeature + [default-with-analyze] + [default] + [script=latn,mode=node, + init=yes,medi=yes,fina=yes,isol=yes] + +\showotfcomposition + {dejavu-serif*default-with-analyze at 24pt} + {} + {I don't wanna know tha\utfchar{"300}t!} +\stopbuffer + +\typebuffer + +In the tracer the different categories are colored. This kind of information is +especially important for typesetting Arabic. Normally \CONTEXT\ can figure out +itself when this is needed so you don't have to worry too much about this kind of +additional actions. + +\blank \getbuffer \blank + +\stopsection + +\startsection[title=Processors] + + \startnotabene + Todo. + \stopnotabene + +\stopsection + +\startsection[title=Optimizing] + + \startnotabene + Todo. + \stopnotabene + +\stopsection + +\startsection[title=Tracing] + +There are a lot of tracing options in \MKIV, but most will never be seen by users. Most +are enabled using the tracker mechanism. Some have a bit more visibility and have a dedicated +command to trigger them. + +When something is going terribly wrong, you will always get a message but sometimes even an +end|-|user has to request for more information. An example are missing characters. There are +several ways to get them reported: + +\starttyping +\enabletrackers[fonts.missing=replace] +\enabletrackers[fonts.missing=remove] +\enabletrackers[fonts.missing] +\stoptyping + +For historic reasons we also have: + +\starttyping +\checkcharactersinfont +\removemissingcharacters +\replacemissingcharacters +\stoptyping + +which happens automatically when you enable the tracker. There is some extra +overhead involved so you might want to turn on this feature on only if you really +expect characters not to be present. + +Say that we use Latin Modern fonts and ask for some of the rare fractions: + +\startbuffer +\definedfont[lmroman10-regular*default-with-missing at 10pt] +a b c ½ ⅓ ¼ ⅕ ⅙ ⅛ Ɣ ɣ ʤ ʭ ʮ α β γ +\stopbuffer + +\typebuffer + +\enabletrackers[fonts.missing=replace] +We get this: \start \getbuffer \stop +\removeunwantedspaces . \space +In the log file you will find something like this: +\par \disabletrackers[fonts.missing] + +\starttyping +fonts > characters > start missing characters: lmroman10-regular.otf + +missing > U+00194 Ɣ LATIN CAPITAL LETTER GAMMA +missing > U+00263 ɣ LATIN SMALL LETTER GAMMA +missing > U+002A4 ʤ LATIN SMALL LETTER DEZH DIGRAPH +missing > U+002AD ʭ LATIN LETTER BIDENTAL PERCUSSIVE +missing > U+002AE ʮ LATIN SMALL LETTER TURNED H WITH FISHHOOK +missing > U+003B1 α GREEK SMALL LETTER ALPHA +missing > U+003B2 β GREEK SMALL LETTER BETA +missing > U+003B3 γ GREEK SMALL LETTER GAMMA +missing > U+02153 ⅓ VULGAR FRACTION ONE THIRD +missing > U+02155 ⅕ VULGAR FRACTION ONE FIFTH +missing > U+02159 ⅙ VULGAR FRACTION ONE SIXTH +missing > U+0215B ⅛ VULGAR FRACTION ONE EIGHTH + +fonts > characters > stop missing characters +\stoptyping + +If you're lucky your editor will use a font that shows the missing characters (dejavu +monospace is a good candidate). + +The replacement characters can help you to locate the spots where something is missing +so that an alternative can be considered. The replacements resemble the category +of the missing character. + +\showmissingcharacterslegend + +You can call up this legend after loading an extra module: + +\starttyping +\usemodule[s][fonts-missing] + +\showmissingcharacterslegend + +\showmissingcharacters +\stoptyping + +The last command shows a detailed list of missing characters + +\showmissingcharacters + +Here the characters are shown, because we use a monospaced font that happens to +have them. Of course this example uses characters that are rarely used and are +unlikely to show up in future versions of the Latin Modern fonts. + +\startnotabene + Here a few more relevant trackers will be mentioned. +\stopnotabene + +\stopsection + +% \startsection[title=Discretionaries] +% +% \startbuffer +% \definedfont[cambria*default] +% 12\discretionary +% {3} {4} {5}% +% 67\par +% 12{\oldstyle\discretionary +% {3} {4} {5}}% +% 67\par +% 12\discretionary +% {3{\oldstyle3}} {{\oldstyle4}4} {5{\oldstyle5}5}% +% 67\par +% \stopbuffer +% +% The font handler has to do some magick to get features working with and across +% discretionaries. To some extend you can use font switches inside discretionaries +% but for sure border cases are not dealt with. This works: +% +% \startlinecorrection[blank] +% \startcombination[nx=4,ny=1,location=top] +% {\framed[align=normal]{\enabledirectives [otf.alwaysdisc]\setupwhitespace[line]\getbuffer}} {1} +% {\framed[align=normal]{\enabledirectives [otf.alwaysdisc]\hsize1mm\getbuffer}} {2} +% {\framed[align=normal]{\disabledirectives[otf.alwaysdisc]\setupwhitespace[line]\getbuffer}} {3} +% {\framed[align=normal]{\disabledirectives[otf.alwaysdisc]\hsize1mm\getbuffer}} {4} +% \stopcombination +% \stoplinecorrection +% +% The first two examples have \type {otf.alwaysdisk} enabled, the last two have it +% disabled. +% +% \typebuffer +% +% \stopsection + +\startsection[title=Some remarks] + +If you talk about features and fonts it is not difficult to end up speaking +\OPENTYPE . However, in \CONTEXT\ we use the term in a more general way, if only +because we provide more features. In traditional \TEX\ we have a few features: +ligatures, smallcaps and kerns, and to some extent we can see oldstyle numerals +also as feature. It is however important to notice that in \OPENTYPE\ ligatures +are just a synonym for combining multiple characters into on. From the user +interface point of view these operations are grouped into \type {liga}, \type +{dlig}, \type {clig} and \type {rlig} and for \TEX ies we have \type {tlig}. The +distinction is not as clear as one might think: any feature can use the ligature +builder. And as a consequence we see that happen too, for instance some fonts use +\type {ccmp} for constructing mandatory ligatures. + +Some of these interpretations (or maybe even tricks) are side effects of for +instance user interfaces. If one can for instance not turn on or off the \type +{ccmp} feature, but can do that for \type {liga}, then one way to keep some +ligatures in for instance letter spaced text, is to put them into \type {ccmp}, +assuming that this one will always be enabled. Eventually that then becomes a +sort of standard. Personally I don't like such pseudo standards but we have to +live with them. + +Another example of such a standard is the used of non breakable spaces to +influence treatment of some Devanagari characters. Where \UNICODE\ has special +characters to influence mechanisms that combine and replace characters, the lack +of some triggers others to be used and eventually that becomes a standard. +Similar ambiguities arise with math: we have no way to indicate math (while we do +have ways to indicate a change in writing order). + +Talking of math, take \OPENTYPE\ math: at some point there is a draft, that then +gets implemented in one word processor using one font, but omissions or +imperfections that surface (maybe because more fonts and engines are developed) +stay around because the initial implementation is published and frozen, simply +because there are many users that stick to expectations. Where \TEX ies accept a +few years of development, this is not true for commercial applications. \footnote +{Of course \HTML\ is the biggest example of this: we're stuck forever with open +tags without close tags, mixed uppercase and lowercase tags, attributes without +value or values without quotes.} + +So, although there is without doubt progress, some annoyances stay. The \TEX\ +community has always been able to adapt, and this is one reason why a \LUA\ +implementation is nice: it gives room for experiments, extensions, variants, etc. +Of course it also makes a bit more independent, although one may wonder if that +matters any longer in a rapidly changing world. The original idea behind \TEX, +that it should be useable for ages, will survive, but users might see more +changes in a lifetime than foreseen when \TEX\ showed up. + +\stopsection + +\startsection[title=Different spaces] + +The width of the space and its stretch and shrink are taken from the font. The so +called emspace is the reference for much spacing related parameters. It is the +width of character \type {0x2014}. The regular space width is taken from \type +{0x0020}, the space character. When there is no space character, in the case of a +monospaced font we take the emwidth, otherwise we take half the emwidth. As a +last resort we can take the average width of characters. And of even that fails +we take half of the font units. When there is no emwidth that one is set to the +font units. + +In the \CONTEXT\ font loader we use a stretch that is 1/2 of the width of a space +and the shrink is 1/3 the width of a space, so we use values that are quite +similar to what \TEX\ always used. + +You can overload these values when a font is loaded and the overload is +implemented as a feature. The next example demonstrates how this is done: + +\startbuffer +\definefontfeature[morespace][spacing=0.50 plus 0.50 minus 0.250] +\definefontfeature[lessspace][spacing=0.25 plus 0.25 minus 0.125] + +\definedfont[Serif*default] \samplefile{klein}\blank +\definedfont[Serif*default,morespace]\samplefile{klein}\blank +\definedfont[Serif*default,lessspace]\samplefile{klein}\blank +\definedfont[Serif*default] \samplefile{klein}\blank +\stopbuffer + +\typebuffer \blank \getbuffer \blank + +\stopsection + +\startsection[title=Dynamic features] + +We can enable and disable features any time in the input by using the +\type {\feature} command. he following example was posted on the list: + +\startbuffer +\definefont + [WeirdShapes] + [file:libertiner*default] + +\definefontfeature + [hist] + [hlig=yes] + +\definefontfeature + [rare] + [dlig=yes] + +\setupquotation + [style={\feature[+][hist,rare]}] + +\startlines +\WeirdShapes +strict {\feature[+][hist]strict} +wurtzite {\feature[+][rare]wurtzite} +\quotation{strict wurtzite} +\stoplines +\stopbuffer + +\typebuffer + +Or typeset: + +\getbuffer + +The \type {\feature} command takes as first argument a directive of what +do do: + +\starttabulate[|T||] +\NC + more \NC add set to previous set and combine with font set \NC \NR +\NC - less \NC subtract set to previous set and combine with font set \NC \NR +\NC = new \NC replace font set \NC \NR +\NC ! < reset \NC forget sets and revert to font set \NC \NR +\NC > old default \NC make sure the current set is used on top of the font set \NC \NR +\stoptabulate + +\stopsection + +\startsection[title=Spacekerns] + +Some fonts kern glyphs with spaces. Although \TEX\ doesn't really have spaces we do +support this. However, it's implemented as part of kerning so when you define such +kerns you need to hook it into for instance the \type {kern} feature: + +\starttyping +\startluacode + local kern = -50 + local pair = { [32] = kern } + + fonts.handlers.otf.addfeature { + name = "kern", -- spacekerns assume kern + type = "kern", + data = { + A = pair, V = pair, W = pair, + [32] = { + A = kern, + V = kern, + W = kern, + }, + } + } +\stopluacode +\stoptyping + +Of course this depends on font properties so one can wonder how useful this is. + +\stopsection + +\stopchapter + +\stopcomponent |