diff options
Diffstat (limited to 'doc/context/sources/general/fonts/fonts/fonts-formats.tex')
| -rw-r--r-- | doc/context/sources/general/fonts/fonts/fonts-formats.tex | 896 |
1 files changed, 0 insertions, 896 deletions
diff --git a/doc/context/sources/general/fonts/fonts/fonts-formats.tex b/doc/context/sources/general/fonts/fonts/fonts-formats.tex deleted file mode 100644 index 9ad6bc9bd..000000000 --- a/doc/context/sources/general/fonts/fonts/fonts-formats.tex +++ /dev/null @@ -1,896 +0,0 @@ -% language=uk - -\startcomponent fonts-formats - -\environment fonts-environment - -\startchapter[title=Font formats][color=darkred] - -\startsection[title=Introduction] - -In this chapter the font formats as we know them will be introduced. The -descriptions will be rather general but more details can be found in the -appendix. Although in \MKIV\ we do support all these types eventually the focus -will be on \OPENTYPE\ fonts but it does not hurt to see where we are coming from. - -\stopsection - -\startsection[title=Glyphs] - -A typeset text is mostly a sequence of characters turned into glyphs. We talk of -characters when you input the text, but the visualization involves glyphs. When -you copy a part of the screen in an open \PDF\ document or \HTML\ page back to -your editor you end up with characters again. In case you wonder why we make this -distinction between these two states we give an example. - -\startplacefigure [location=here,reference=fig:character-glyph,title=From characters to glyphs.] - \startcombination - {\color[maincolor]{\definedfont[Serif*default at 30pt]affiliation}} {upright} - {\color[maincolor]{\definedfont[SerifItalic*default at 30pt]affiliation}} {italic} - \stopcombination -\stopplacefigure - -We see here that the shape of the \type {a} is different for an upright serif and -an italic. We also see that in \type {ffi} there is no dot on the \type {i}. The -first case is just a stylistic one but the second one, called a ligature, is -actually one shape. The 11 characters are converted into 9 glyphs. Hopefully the -final document format carries some extra information about this transformation so -that a cut and paste will work out well. In \PDF\ files this is normally the -case. In this document we will not be too picky about the distinction as in most -cases the glyph is rather related to the character as one knows it. - -So, a font contains glyphs and it also carries some information about -replacements. In addition to that there needs to be at least some information -about the dimensions of them. Actually, a typesetting engine does not have to -know anything about the actual shape at all. - -\startplacefigure [location=here,reference=fig:glyph-dimension-normal,title=The boundingbox of some normal glyphs.] - \startcombination[9*1] - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt]a}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt]b}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt]g}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt]l}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt]q}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt].}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt];}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt]?}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[Serif*default at 30pt]ffi}}} {} - \stopcombination -\stopplacefigure - -\startplacefigure [location=here,reference=fig:glyph-dimension-italic,title=The boundingbox of some italic glyphs.] - \startcombination[9*1] - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt]a}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt]b}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt]g}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt]l}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt]q}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt].}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt];}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt]?}}} {} - {\ruledhbox{\color[maincolor]{\definedfont[SerifItalic*default at 30pt]ffi}}} {} - \stopcombination -\stopplacefigure - -The rectangles around the shapes \in {figure} [fig:glyph-dimension-normal] and \in -{figure} [fig:glyph-dimension-italic] are called boundingbox. The dashed line -reflects the baseline where they eventually are aligned onto next to each other. -The amount above the baseline is called height, and below is called depth. The -piece of the shape above the baseline is the ascender and the bit below the -descender. The width of the bounding box is not by definition the width of the -glyph. In \TYPEONE\ and \OPENTYPE\ fonts each shape has a so called advance width -and that is the one that will be used. - -\usemodule[fonts-kerns] - -\startplacefigure [location=here,reference=fig:glyph-kerns,title={Kerning in Latin Roman, Cambria, Pagella and Dejavu.}] - \scale[width=\textwidth]{\startcombination[1*4] - {\color[maincolor]{\definedfont[name:lmroman10-regular*default sa 1]\ShowKernedHBox{Very often glyphs get very small spaces inserted horizontally.}}} {} - {\color[maincolor]{\definedfont[name:cambria*default sa 1]\ShowKernedHBox{Very often glyphs get very small spaces inserted horizontally.}}} {} - {\color[maincolor]{\definedfont[name:texgyrepagellaregular*default sa 1]\ShowKernedHBox{Very often glyphs get very small spaces inserted horizontally.}}} {} - {\color[maincolor]{\definedfont[name:dejavuserif*default sa 0.9]\ShowKernedHBox{Very often glyphs get very small spaces inserted horizontally.}}} {} - \stopcombination} -\stopplacefigure - -Another traditional property of a font is kerning. In \in {figure} -[fig:glyph-kerns] you see this in action. These examples -demonstrate that not all fonts need (or provide) the same kerns -(in points). - -So, as a start, we have now met a couple of properties of a font. -They can be summarized as follows: - -\starttabulate[|l|p|] -\NC mapping to glyphs \EQ characters are represented by a shapes that have recognizable - properties so that readers know what they mean \NC \NR -\NC ligature building \EQ a sequence of characters gets mapped onto one glyph \NC \NR -\NC dimensions \EQ each glyph has a width, height and depth \NC \NR -\NC inter-glyph kerning \EQ optionally a bit of positive or negative space has to be inserted between glyphs \NC \NR -%NC italic correction \EQ a correction is applied between an oblique shape and what follows \NC \NR -\stoptabulate - -Regular font kerning is hardly noticeable and improves the overall look of the -page. Typesetting applications sometimes are capable of inserting additional -spaces between shapes. This more excessive kerning is not that much related to -the font and is used for special purposes, like making a snippet of text stand -out. In \CONTEXT\ this kind of kerning is available but it is a font independent -feature. Keep in mind that when applying that kind of rather visible kerning -you'd better not have ligatures and fancy replacements enabled as \CONTEXT\ -already tries to deal with that as good as possible. - -\stopsection - -\startsection[title=The basic process] - -In \TEX\ a font is an abstraction: the engine only needs to know about the -mapping from characters to glyphs, what the width, height and depth is, what -sequences need to be translated into ligatures and when kerning has to be -applied. If for the moment we forget about math, these are all the properties -that matter and this is what the \TEX\ font metric files that we see in the next -section provide. - -Because one of the principles behind \LUATEX\ is that the core engine (the -binary) stays small and that new functionality is provided in \LUA\ code, the -font subsystem largely looks like it always has been. As users will normally use -a macro package most of the loading will be hidden from the user. It is however -good to give a quick overview of how for instance \PDFTEX\ deals with fonts using -traditional metric files. - -\startFLOWchart[pdftex] - \startFLOWcell - \name {source} - \location {1,1} - \shape {action} - \text {input} - \connection [rl] {parser} - \stopFLOWcell - \startFLOWcell - \name {parser} - \location {2,1} - \shape {action} - \text {characters} - \connection [rl] {builder} - \stopFLOWcell - \startFLOWcell - \name {builder} - \location {3,1} - \shape {action} - \text {glyphs} - \connection [rl] {backend} - \stopFLOWcell - \startFLOWcell - \name {backend} - \location {4,1} - \shape {action} - \text {subset} - \stopFLOWcell -\stopFLOWchart - -\startplacefigure [location=here,reference=fig:tfm-pdftex,title={Several translation steps in a traditonal \TEX\ flow.}] - \FLOWchart[pdftex] -\stopplacefigure - -The input (bytes) gets translated into characters by the input parser. Normally -this is a one|-|to|-|one translation but there are examples of some translation -taking place. You can for instance make characters active and give them a -meaning. So, the eight bit represention of an editors code page \type {ë} can -become something else internally, for instance a regular \type {e} with an \type -{¨} overlayed. It can also become another character, which in the code page -would be shown as \type {á} but the user will not know this as by then this byte -is already tokenized. Another example is multibyte translation, for instance -\UTF\ sequences can get remapped to something that is known internally as being a -character of some kind. The \LUATEX\ engine expects \UTF\ so a macro package has -to make sure that translation to this encoding happens beforehand, for instance -using a callback that intercepts the input from file. \footnote {In \CONTEXT\ we -talk of input regimes and these can be mixed, although in practice most users -will stick to \UTF\ and never use regimes.} - -So, the input character (sequence) becomes tokens representing a character. From -these tokens \TEX\ will start building a (linked) node list where each character -becomes a node. In this node there is a reference to the current font. If you -know \TEX\ you will understand that a list can have more than characters: there -can be skips, kerns, rules, references to images, boxes, etc. - -At some point \TEX\ will handle this list over to a routine that will turn them -into something that resembles a paragraph or otherwise snippet of text. In that -stage hyphenation kicks in, ligatures get built and kerning is added. Character -references become glyph indices. This list can finally be broken into lines. - -It is no secret that \TEX\ can box and unbox material and that after unboxing -some new formatting has to happen. The traditional engine has some optimizations -that demand a partial reconstruction of the original list but in \LUATEX\ we -removed this kind of optimization so there the process is somewhat simpler. We -will see more of that later. - -When \TEX\ ships out a page, the backend will load the real font data and merge -that into the final output. It will now take the glyph index and build the right -data structures and references to the real font. As a font gets subset only the -used glyphs end up in the final output. - -There is one tricky aspect involved here: re|-|encoding. In so called map files -one can map a specific metric filename onto a real font name. One can also -specify an encoding vector that tells what a given index really refers to. This -makes it possible to use fonts that have more than 256 glyphs and refer to any of -them. This is also the trick that makes it possible to use \TRUETYPE\ fonts in -\PDFTEX: the backend code filters the right glyphs from the font, remapping -\TEX's glyph indices onto real entries in the font happens via the encoding -vector. In \in {figure} [fig:tfm-bytes] we show a possible route for input byte -68. - -\startFLOWchart[bytes] - \startFLOWcell - \name {source} - \location {1,1} - \shape {action} - \text {bytes (68)} - \connection [rl] {parser} - \stopFLOWcell - \startFLOWcell - \name {parser} - \location {2,1} - \shape {action} - \text {bytes (31)} - \connection [rl] {builder} - \stopFLOWcell - \startFLOWcell - \name {builder} - \location {3,1} - \shape {action} - \text {index (31)} - \connection [rl] {backend} - \stopFLOWcell - \startFLOWcell - \name {backend} - \location {4,1} - \shape {action} - \text {index (88)} - \stopFLOWcell -\stopFLOWchart - -\startplacefigure [location=here,reference=fig:tfm-bytes,title={From bytes to indices.}] - \FLOWchart[bytes] -\stopplacefigure - -As \LUATEX\ carries much of the bagage of older engines, you can still do it this -way but in \CONTEXT\ \MKIV\ we have made our live much simpler: we use unicode as -much as possible. This means that we effectively have removed two steps (see \in -{figure} [fig:tfm-luatex]). - -\startFLOWchart[luatex] - \startFLOWcell - \name {source} - \location {1,1} - \shape {action} - \text {input} - \connection [rl] {builder} - \stopFLOWcell - \startFLOWcell - \name {builder} - \location {2,1} - \shape {action} - \text {glyphs} - \stopFLOWcell -\stopFLOWchart - -\startplacefigure [location=here,reference=fig:tfm-luatex,title={Simplified mapping in \LUATEX.}] - \FLOWchart[luatex] -\stopplacefigure - -There is of course still some work to do for the backend, like subsetting, but -the nasty dependency on the input encoding, font encoding (that itself relates to -hyphenation) and backend re|-|encoding is gone. But keep in mind that the -internal data structure of the font is still quite traditional. - -Before we move on to font formats I like to point out that there is no space in -\TEX. Spaces in the input are converted into glue, either or not with some -stretch and|/|or shrink. This also means that accessing character 32 in -traditional \TEX\ will not end up as space in the output. - -\stopsection - -\startsection[title=\TEX\ metrics] - -\appendixdata{\in[fontdata:tfm]} -\appendixdata{\in[fontdata:vf]} - -Traditional font metrics are packaged in a binary format. Due to the limitations -of that time a font has at most 256 characters. In books dedicated to \TEX\ you -will often find tables that show what glyphs are in a font, so we will not repeat -that here as after all we got rid of that limitation in \LUATEX. - -Because 256 is not that much, especially when you mix many scripts and need lots -of symbols from the same font, there are quite some encodings used in traditional -\TEX, like \type {texnansi}, \type {ec} and \type {qx}. When you use \LUATEX\ -exclusively you can do with way less font files. This is easier for users, -especially because most of those files were never used anyway. It's interesting -to notice that some of the encodings contain symbols that are never used or used -only once in a document, like the copyright or registered symbols. They are often -accessed by symbolic names and therefore easily could have been omitted and -collected in a dedicated symbol font thereby freeing slots for more useful -characters anyway. The lack of coverage is probably one of the reasons why new -encodings kept popping up. In the next table you see how many files are involved -in Latin Modern which comes in a couple of design sizes. \footnote {The original -Computer Modern fonts have \METAFONT\ source files and (runtime) generated bitmap -files in whatever resolutions are needed for previewing and printing. The -\TYPEONE\ follow|-|up came in several sets, organized by language support. The -Latin Modern fonts have a few more weights and variants than Computer Modern.} - -\starttabulate[|l|c|r|r|r|] -\HL -\NC \bf font format \NC \bf type \NC \bf \# files \NC \bf size in bytes \NC \bf \CONTEXT \NC \NR -\HL -\NC type 1 \NC tfm \NC 380 \NC 3.841.708 \NC \NC \NR -\NC \NC afm \NC 25 \NC 2.697.583 \NC \NC \NR -\NC \NC pfb \NC 92 \NC 9.193.082 \NC \NC \NR -\NC \NC enc \NC 15 \NC 37.605 \NC \NC \NR -\NC \NC map \NC 9 \NC 42.040 \NC \NC \NR -\HL[darkgray] -\NC \NC \NC 521 \NC 15.812.018 \NC mkii \NC \NR -\HL -\NC opentype \NC otf \NC 73 \NC 8.224.100 \NC mkiv \NC \NR -\HL -\stoptabulate - -A \TFM\ file can contain so called italic corrections. This is an additional kern -that can be added after a character in order to get better spacing between an -italic shape and an upright one. As this is manual work, it's a not that advanced -mechanism, but in addition to width, height, depth, kerns and ligatures it is -nevertheless a useful piece of information. But, it's a rather \TEX\ specific -quantity. - -Since \TEX\ showed up many fonts have been added. In addition support for -commercial fonts was provided. In fact, for that to happen, one only needs -accompanying metric files for \TEX\ itself and map files and encoding vectors -for the backend. Because a metric file also has some general information, like -spacing (including stretch and shrink), the ex|-|height and em|-|width, this -means that sometimes guesses must be made when the original font does not come -with such parameters. - -At some point virtual fonts were introduced. In a virtual font a \TFM\ file has -an accompanying \VF\ file. In that file each glyph has a specification that tells -where to find the real glyph. It is even possible to construct glyphs from other -glyphs. In traditional \TEX\ this only concerns the backend, which in \PDFTEX\ is -built in. In \LUATEX\ this mechanism is integrated into the frontend which means -that users can construct such virtual fonts themselves. We will see more of that -later, but for now it's enough to know that when we talk about the representation -of font (the \TFM\ table) in \LUATEX, this includes virtual functionality. - -An important limitation of \TFM\ files cq.\ traditional \TEX\ is that the number -of depths and heights is limited to 16 each. Although this results in somewhat -inaccurate dimensions in practice this gets unnoticed, if only because many -designs have some consistency in this. On the other hand, it is a limitation when -we start thinking of accents or even multiple accents which lead to many more -distinctive heights and depths. - -Concerning ligatures we can remark that there are quite some substitutions -possible although in practice only the multiple to one replacement has been used. - -Some fonts that are used in \TEX\ started out as bitmaps but rather soon -\TYPEONE\ outline fonts became the fashion. These are supported using the map -files that we will discuss later. First we look into \TYPEONE\ fonts. - -\stopsection - -\startsection[title=\TYPEONE] - -\appendixdata{\in[fontdata:afm]} -\appendixdata{\in[fontdata:enc]} -\appendixdata{\in[fontdata:map]} - -For a long time \TYPEONE\ fonts have dominated the scene. These are \POSTSCRIPT\ -fonts that can have more that 256 glyphs in the file that defines the shapes, but -only 256 of them can be used at one time. Of course there can be multiple subsets -active in one document. - -In traditional \TEX\ a \TYPEONE\ font is used by making a \TFM\ file from a so -called Adobe metric file (\AFM) that come with such a font. There are several -tool chains for doing this and \CONTEXT\ \MKII\ ships with one that can be of -help when you need to support commercial fonts. Projects like the Latin Modern -Fonts and \TEX\ Gyre have normalized a whole lot of fonts that came in several -more or less complete encodings into a consistent package of \TYPEONE\ fonts. -This already simplified live a lot but still users had to choose a suitable input -and font encoding for their language and|/|or script. As \TEX\ only cares about -metrics and not about the rendering, it doesn't consider \TYPEONE\ fonts as -something special. Also, as \TEX\ and \POSTSCRIPT\ were developed about the same -time support for \TYPEONE\ fonts is rather present in \TEX\ distributions. - -You can still follow this route but for \CONTEXT\ \MKIV\ this is no longer the -recommended way because there we have changed the whole subsystem to use -\UNICODE. As a result we no longer use \TFM\ files derived from \AFM\ files, but -directly interpret the \AFM\ data. This not only removes the 256 limitation, but -also brings more resolution in height and depth as we no longer have at most 16 -alternatives. There can also be more kerns. Of course we need some heuristics to -determine for instance the spacing but that is not different from former times. - -Because most \TEX\ users don't use commercial fonts, they will not notice that -\CONTEXT\ \MKIV\ treats \TYPEONE\ fonts this way. One reason is that the free -fonts also come as wide fonts in \OPENTYPE\ format and whenever possible -\CONTEXT\ prefers \OPENTYPE\ over \TYPEONE\ over \TFM. - -In the beginning \LUATEX\ only could load a \TFM\ file, which is why loading -\AFM\ files is implemented in \LUA. Later, when the \OPENTYPE\ loaded was added, -loading \PFB\ and \AFM\ files also became possible but it's slower and we see no -reason to rewrite the current code in \CONTEXT. We also do a couple of extra -things when loading such a file. As more \TYPEONE\ fonts move on to \OPENTYPE\ we -don't expect that much usage anyway. - -\stopsection - -\startsection[title=\OPENTYPE] - -\appendixdata{\in[fontdata:otf]} - -When an engine can deal with \UNICODE\ directly it also means that internally it -uses pretty large numbers for storing characters and glyph indices. The first -\TEX\ descendent that went wide was \OMEGA, later replaced by \ALEPH. However, this -engine never took off and still used its own extended \TFM\ format: \OFM. In fact, -as \LUATEX\ uses some of the \ALEPH\ code, it can also use these extended metric -files but I don't think that there are any useful fonts around so we can forget -about this. - -We use the term \OPENTYPE\ for a couple of font formats that share the same -principles: \OPENTYPE\ (\OTF), \TRUETYPE\ (\TTF) and \TRUETYPE\ containers -(\TTC). The \LUATEX\ font reader presents them in a similar format. In the case -of a \TRUETYPE\ container, one does not load the whole font but selects an -instance from it. Internally an \OPENTYPE\ font can have the glyphs organized in -subfonts. - -The first \TEX\ descendent to really go wide from front to back is \XETEX. This -engine can use \OPENTYPE\ fonts directly and for a whole category of users this -opened up a new world. Hoever, it is still mostly a traditional engine. The -transition from characters to glyphs is accomplished by external libraries, while -in \LUATEX\ we code in \LUA. This has the disadvantage that it is slower -(although that depends on the job) but the advantage is that we have much more -control and can extend the font handler as we like. - -An \OPENTYPE\ font is much more complex than a \TYPEONE\ one. Unless it is a -quick and dirty converted existing font, it will have more glyphs to start with. -Quite likely it will have kerns and ligatures too and of course there are -dimensions. However, there is no concept of a depth and height. These need to be -deduced from the bounding box instead. There is an advance width. This means that -we can start right away using such fonts if we map those properties onto the -\TFM\ table that \LUATEX\ expects. - -But there is more, take ligatures. In a traditional font the sequence \type {ffi} -always becomes a ligature, given that the font has such a glyph. In \LUATEX\ -there is a way to disable this mechanism, which is sometimes handy when dealing -with mono|-|spaced fonts in verbatim. It's pretty hard to disable that. For -instance one option is to insert kerns manually. In an \OPENTYPE\ font ligatures -are collected in a so called feature. There can be many such features and even -kerning is a feature. Other examples are old style numerals, fractions, -superiors, inferiors, historic ligatures and stylistic alternates. - -\starttabulate[|lT|l|l|l|l|] -\NC \type{onum} \NC \ruledhbox{\maincolor\DemoOnumLM\char45 1} - \NC \ruledhbox{\maincolor\DemoOnumLM1234567890} - \NC \ruledhbox{\maincolor\DemoOnumLM\char"A2} - \NC \ruledhbox{\maincolor\DemoOnumLM\char"24} \NC \NR -%NC \type{lnum} \NC \ruledhbox{\maincolor\DemoLnumLM\char45 1} -% \NC \ruledhbox{\maincolor\DemoLnumLM1234567890} -% \NC \ruledhbox{\maincolor\DemoLnumLM\char"A2} -% \NC \ruledhbox{\maincolor\DemoLnumLM\char"24} \NC \NR -\NC \type{tnum} \NC \ruledhbox{\maincolor\DemoTnumLM\char45 1} - \NC \ruledhbox{\maincolor\DemoTnumLM1234567890} - \NC \ruledhbox{\maincolor\DemoTnumLM\char"A2} - \NC \ruledhbox{\maincolor\DemoTnumLM\char"24} \NC \NR -\NC \type{pnum} \NC \ruledhbox{\maincolor\DemoPnumLM\char45 1} - \NC \ruledhbox{\maincolor\DemoPnumLM1234567890} - \NC \ruledhbox{\maincolor\DemoPnumLM\char"A2} - \NC \ruledhbox{\maincolor\DemoPnumLM\char"24} \NC \NR -\stoptabulate - -To this all you need to add that features operate in two dimensions: languages -and scripts. This means that when ligatures are enabled for Dutch the \type {ij} -sequence becomes a single glyph but for German it gets mapped onto two glyphs. -And, to make it even more complex, a substitution can depend on circumstances, -which means that for Dutch \type {fijn} becomes \type {f ij n} but \type {fiets} -becomes \type {fi ets}. It will be no surprise that not all \OPENTYPE\ fonts come -with a complete and rich repertoire of rules. To make things worse, there can be -rules that turn \type {1/2} into one glyph, or transfer the numbers into superior -and inferior alternatives, but leaves us with an unacceptable rendered \type -{1/a}, given that the \type {frac} features is enabled. It looks like features -like this are to be applied to a manually selected range of characters. - -The fact that an \OPENTYPE\ font can contain many features and rules to apply -them makes it possible to typeset scripts like Arabic. And this is where it gets -vague. A generic \OPENTYPE\ sub|-|engine can do clever things using these rules, -but if you read the specification for some scripts additional intelligence has to -be provided by the typesetting engine. - -While users no longer have to care about encodings, map files and back|-|end -issues, they do have to carry knowledge about the possibilities and limitations -of features. Even worse, he or she needs to be aware that fonts can have bugs. -Also, as font vendors have no tradition of providing updates this is something -that we might need to take care of ourselves by tweaking the engine. - -One of the problems with the transition from \TYPEONE\ to \OPENTYPE\ is that font -designers can take an existing design and start from that basic repertoire of -shapes. If such a design had oldstyle figures only, there is a good chance that -this will be the case in the \OPENTYPE\ variant too. However, such a default -interferes with the fact that the \type {onum} feature is one that we explicitly -have to enable. This means that writing a generic style where a font is later -plugged in becomes somewhat messy if it assumes that features need to be turned -on. - -\TEX\ users expect more control, which means that in practice just an \OPENTYPE\ -engine is not enough, but for the average font the \TEX\ model using the -traditional approach still is quite acceptable. After all, not all users use -complex scripts or need advanced features. And, in practice most readers don't -notice the difference anyway. - -\stopsection - -\startsection[title=\LUA] - -\appendixdata{\in[fontdata:lua]} - -As mentioned support for virtual fonts is built into \LUATEX\ and loading the so -called \VF\ files happens when needed. However, that concerns traditional fonts -that we already covered. In \CONTEXT\ we do use the virtual font mechanism for -creating missing glyphs out of existing ones or add fallbacks when this is not -possible. But this is not related to some kind of font format. - -In 2010 and 2011 the first public \OPENTYPE\ math fonts showed up that replace -their \TYPEONE\ originals. In \CONTEXT\ we already went forward and created -virtual \UNICODE\ fonts out of traditional fonts. Of course eventually the -defaults will change to the \OPENTYPE\ alternatives. The specification for such a -virtual font is given in \LUA\ tables and therefore you can consider \LUA\ to be -a font format as well. In \CONTEXT\ such fonts can be defined in so called -goodies files. As we use these files for much more tuning, we come back to that -in a later chapter. In a virtual font you can mix real \TYPEONE\ fonts and real -\OPENTYPE\ fonts using whatever metrics suit best. - -An extreme example is the virtual \UNICODE\ Punk font. This font is defined in -the \METAPOST\ language (derived from Don Knuths \METAFONT\ sources) where each -glyph is one graphic. Normally we get \POSTSCRIPT, but in \LUATEX\ we can also -get output in a comparable \LUA\ table. That output is converted to \PDF\ -literals that become part of the virtual font definitions and these eventually -end up in the \PDF\ page stream. So, at the \TEX\ end we have regular (virtual) -characters and all \TEX\ needs is their dimensions, but in the \PDF\ each glyph -is shown using drawing operations. Of course the now available \OPENTYPE\ variant -is more efficient, but it demonstrates the possibilities. - -\stopsection - -\startsection[title=Files] - -We summarize these formats in the following table where we explain what the file -suffixes stand for: - -\starttabulate[|Tl|p|] -\HL -\NC tfm \NC This is the traditional \TEX\ font metric file format and it reflects - the internal quantities that \TEX\ uses. The internal data structures - (in \LUATEX) are an extension of the \TFM\ format. \NC \NR -\NC vf \NC This file contains information about how to construct and where to - find virtual glyphs and is meant for the backend. With \LUATEX\ this - format gets more known. \NC \NR -\NC pk \NC This is the bitmap format used for the first generation of \TEX\ - fonts but the typesetter never deals with them. Bitmap files are more - or less obselete. \NC \NR -\HL -\NC ofm \NC This is the \OMEGA\ variant of the \type {tfm} files that caters for - larger fonts. \NC \NR -\NC ovf \NC This is the \OMEGA\ variant of the \type {vf}. \NC \NR -\HL -\NC pfb \NC In this file we find the glyph data (outlines) and some basic - information about the font, like name|-|to|-|index mappings. A - differently byte|-|encoded variant of this format is \type {pfa}.\NC - \NR -\NC afm \NC This file accompanies the \type {pfb} file and provides additional - metrics, kerns and information about ligatures. A binary variant of - this is the \PFA\ format. For \MSWINDOWS\ there is a variant that has the - \type {pfm} suffix. \NC \NR -\NC map \NC The backend will consult this file for mapping metric file names onto - real font names. \NC \NR -\NC enc \NC The backend will include (and use) this encoding vector to map - internal indices to font indices using glyph names, if needed. \NC - \NR -\HL -\NC otf \NC This binary format describes not only the font in terms of metrics, - features and properties but also contains the shapes. \NC \NR -\NC ttf \NC This is the \MICROSOFT\ variant of \OPENTYPE. \NC \NR -\NC ttc \NC This is the \MICROSOFT\ container format that combines multiple fonts - in one. \NC \NR -\HL -\NC fea \NC A (\FONTFORGE) feature definition file. Such a file can be loaded and - applied to a font. This is no longer supported in \CONTEXT\ as we have - other means to achieve the same goals. \NC \NR -\NC cid \NC A glyph index (name) to \UNICODE\ mapping file that is referenced - from an \OPENTYPE\ font and is shared between fonts. \NC \NR -\HL -\NC lfg \NC These are \CONTEXT\ specific \LUA\ font goodie files providing - additional information. \NC \NR -\HL -\stoptabulate - -If you look at how files are organized in a \TEX\ distribution, you will notice -that these files all get their own place. Therefore adding a \TYPEONE\ font to -the distribution is not that trivial if you want to avoid clashes. Also, files -are simply not found when they are not in the right spot. Just to mention a few -paths: - -\starttyping -<root>/fonts/tfm/vendor/typeface -<root>/fonts/vf/vendor/typeface -<root>/fonts/type1/vendor/typeface -<root>/fonts/truetype/vendor/typeface -<root>/fonts/opentype/vendor/typeface -<root>/fonts/fea -<root>/fonts/cid -<root>/fonts/dvips/enc -<root>/fonts/dvips/map -\stoptyping - -There can be multiple roots and the right locations are specified in a -configuration file. Currently all engines can use the \DVIPS\ encoding and map -files, so luckily we don't need to duplicate this. For some reason \TRUETYPE\ and -\OPENTYPE\ fonts have different locations and you need to be aware of the fact -that some fonts come in both formats (just to confuse users) so you might end up -with conflicts. - -In \CONTEXT\ we try to make live somewhat easier by also supporting a simple path -structure: - -\starttyping -<root>/fonts/data/vendor/typeface -\stoptyping - -This way files are kept together and installing commercial fonts is less complex -and error prone. Also, in practice we only have one set of files now: one of the -other \OPENTYPE\ formats. - -If you want to see the difference between a traditional (\PDFTEX\ or \XETEX\ plus -\CONTEXT\ \MKII) setup or a modern one (\LUATEX\ with \CONTEXT\ \MKIV) you can -install the \CONTEXT\ suite (formerly known as minimals). If you explicitly -choose for a \LUATEX\ only setup, you will notice that far less files get -installed. - -\stopsection - -\startsection[title=Text] - -This is not an in|-|depth explanation of how to define and load fonts in -\CONTEXT. First of all this is covered in other manuals, but more important is -that we assume that the reader is already familiar with the way \CONTEXT\ deals -with fonts. Therefore we limit ourselves to some remarks and expand on this a bit -in later chapters. - -The font subsystem has evolved over years and when you look at the low level code -you will probably find it complex. This is true, although in some aspects it is -not as complex as in \MKII\ where we also had to deal with encodings due to the -eight bit limitations. In fact, setting up fonts is easier due the fact that we -have less files to deal with. - -The main properties of a (modern) font subsystem for typesetting text are the -following: - -\startitemize[n] - \startitem - We need to be able to switch the look and feel efficiently and - consistently, for instance going from regular to bold or italic. So, - when we load a font family we not only load one file, but often - at least four: regular, bold, italic (oblique) and bolditalic - (boldoblique). - \stopitem - \startitem - When we change the size we also need to make sure that these related - sets are changed accordingly. You really want the bold shapes to scale - along with the regular ones. - \stopitem - \startitem - Shapes are organized in serif, sans serif, mono spaced and math and for - proper working of a typesetter that has math all over you need always - need the math. Again, when you change size, all these shapes need to - scale in sync. - \stopitem - \startitem - In one document several families can be combined so the subsystem should - make it possible to switch from one to the other without too much - overhead. - \stopitem - \startitem - Because section heads and other structural elements have their own sizes - there has to be a consistent way to deal with that. It should also be - possible to specify exceptions for them. - \stopitem -\stopitemize - -In the next chapters we will cover some details, for instance font features. You -can actually control these when setting up a body font, simply by redefining -the \type {default} feature set, but not all features are dealt with this way. -So let's continue the demands put on a font subsystem. - -\startitemize[continue] - \startitem - Sometimes inter|-|character kerning is needed. In \CONTEXT\ this is not a - property of a font because glyphs can be mixed with basically anything. - This kind of features is applied independent of a font. - \stopitem - \startitem - The same is true for casing (like uppercasing and such) which is not - related to a font but applied to a selected (or marked) piece of the - input stream. - \stopitem - \startitem - Using so called \quotation {small caps} or \quotation {old style} - numerals or \unknown\ can be dealt with by setting the default features - but often these are applied selectively. As these are applied using the - information in a font they do belong to the font subsystem but in - practice they can be seen as independent (assuming that the font supports - them at all). - \stopitem - \startitem - Protrusion (into margins) and expansion (to improve whitespace) are - applied to the font at load time because the engine needs to know about - them. But they two can selectively be turned on and off. They are more - related to line break handling than font defining. - \stopitem - \startitem - Slanting (to fake oblique) and expanding (to fake bold) are regular - features but are applied to the font because the engine needs to know - about them. They permanently influence the shape. - \stopitem -\stopitemize - -We will discuss these in this manual too. What we will not discuss in depth is -spacing, even when it depends on the (main body) font size. These use properties -of fonts (like the ex|-|height or em|-|width and maybe the width of the space, -but normally they are controlled by the spacing subsystem. We will however -mention some rather specific possibilities: - -\startitemize[continue] - \startitem - The \CONTEXT\ font subsystem provides ways to combine multiple fonts - into one. - \stopitem - \startitem - You can construct artificial fonts, using existing fonts or \METAPOST\ - graphics. - \stopitem - \startitem - Fonts can be fixed (dimensions) and completed (for instance accented - characters) when loading/ - \stopitem - \startitem - There are extensive tracing options, not only for applied features but - also for loading, checking etc. There is a set of styles that can be - used to study fonts. - \stopitem -\stopitemize - -Sometimes users ask for very special trickery and it no surprise then that some -of that is now widely know (or even discussed in detail). When we get notice of -that we can mention it in this manual. - -So how does this all relate to font formats? We mentioned that when loading we -basically load some four files per family (and more if we use specific fonts for -titling). These files just provide the data: metric information, shapes and ways -to remap characters (or sequences) into glyphs, either of not positioned relative -to each other. In traditional \TEX\ only dimensions, kerns and ligatures -mattered, but in nowadays we also deal with specific \OPENTYPE\ features. But -still, as you can deduce from the above, this is only part of the story. You need -a complete and properly integrated system. It is no big deal to set up some -environment that uses font files to achieve some typesetting goal, but to provide -users with some consistent and extensible system is a bit more work. - -There are basically three font formats: good old bitmaps, \TYPEONE\ and -\OPENTYPE. All need to be supported and expectations are that we also support -their features. But is should be noticed that whatever font you use, the quality -of the outcome depends on what information the font can provide. We can improve -processing but are often stuck with the font. There are many thousands of -fonts out there and we need to be able to use them all. - -\stopsection - -\startsection[title=Math] - -In the previous section we already mentioned math fonts. The fonts are just one -aspect of typesetting math and math fonts are special in the sense that they have -to provide the relevant information. For instance a parenthesis comes in several -sizes and at some point turns in a symbol made out of pieces (like a top curve, -middle lines and bottom curve) that overlap. The user never sees such details. In -fact, there are ot that many math fonts and these are already set up so there is -not much to mess up here. Nevertheless we mention: - -\startitemize [n] - \startitem - Math fonts are loaded in three sizes: text, script and scriptscript. The - optimal relative sizes ar defined in the font. - \stopitem - \startitem - There are direction aware math fonts and we support this in \CONTEXT. - \stopitem - \startitem - Bold math is in fact a bolder version of a regular math font (that can - have bold symbols too). Again this is supported. - \stopitem -\stopitemize - -The way math is dealt with in \CONTEXT\ is different from the way it is done -traditionally. Already when we started with \MKIV\ we moved to \UNICODE\ and -the setup at the font level is kept simple by delegating some of the work to -the \LUA\ end. We will see some of the mentioned aspects in more detail later. - -Because of it's complexity and because in a math text there can be many times -activation of math fonts (and related settings) quite some effort has been put in -making it efficient. But you need to keep in mind that when we discuss math -related topics later on, this is hardly of concern. Math fonts are loaded only -once so manipulating them a bit has no penalty. And using them later on is hardly -related to the font subsystem. - -Concerning formats we can notice that traditional \TEX\ comes with math fonts -that have properties that the engine can use. Because there were not many math -fonts, this was no problem. The \OPENTYPE\ math fonts however are also used in -other applications and therefore are a bit more generic. \footnote {Their -internals are now defined in the \OPENTYPE\ specification.} For this we not only -had to adapt the math engine in \LUATEX\ (although we kept that to the minimum) -but we also had to think different about loading them. In later chapters we will -see that in the transition to \UNICODE\ math fonts we implemented a mechanism for -combining \TYPEONE\ fonts into virtual \UNICODE\ fonts. We did that because it -made no sense to keep an old and new loader alongside. - -There will not be thousands of math fonts flying around. A few dozen is already a -lot and the developers of macro packages can set them up for the users. So, in -practice there is not much that a user needs to know about math font formats. - -\stopsection - -\startsection[title=Caching] - -Because fonts can be large and because we use \LUA\ tables to describe them -a bit of effort has been put into managing them efficiently. Once converted -to the representation that we need they get cached. You can peek into the cache -which is someplace on your system (depending on the setup): - -\starttabulate[|l|p|] -\NC \type{fonts/data} \NC font name databases \NC \NR -\NC \type{fonts/mp} \NC fonts created using \METAPOST \NC \NR -\NC \type{fonts/one} \NC type one fonts, converted from \type {afm} and \type - {pfb} files \NC \NR -\NC \type{fonts/otl} \NC open type fonts, converted from \type {ttf}, \type {otf}, - \type {ttc} and \type {ttx} files loaded using the - \CONTEXT\ \LUA\ loader \NC \NR -\NC \type{fonts/pdf} \NC font shapes for color fonts \NC \NR -\NC \type{fonts/shapes} \NC outlines of fonts (for instance for use in \METAFUN) \NC \NR -\NC \type{fonts/streams} \NC font programs for variable font instances \NC \NR -\stoptabulate - -There can be three types of files there. The \type{tma} files are just \LUA\ -tables and they can be large. These files can be compiled to bytecode where \type -{tmc} is for stock \LUATEX\ and \type {tmb} for \LUAJITTEX. The \type {tma} files -are optimized for space and memory (aka: packed) but you can expand them with -\type {mtxrun --script font}. - -Fonts in the cache are automatically updated when you install new versions of a -font or when the \CONTEXT\ font loader has been updated. - -\stopsection - -\startsection[title=Paths] - -The search for fonts happens on paths defined in \type {texmf.cnf}. The information -in there is used to generate a file database for fast access with priorities based -on file type. The \TDS\ is starting point. The environment variable driven paths -\type {OSFONTDIR} (set automatically) and \type {EXTRAFONTDIR} are taken into account. - -In addition you can set \type {RUNTIMEFONTS} which is, when set, consulted at -runtime. You can also add a path in your style: - -\starttyping -\usefontpath[c:/data/projects/myproject/fonts] -\stoptyping - -although in general we recommend to put fonts in - -\starttyping -<texroot>/tex/texmf-fonts/fonts/data] -\stoptyping - -which is more efficient. - -\stopsection - -\stopchapter - -\stopcomponent |