diff options
Diffstat (limited to 'doc/context/sources/general/manuals/luatex')
17 files changed, 4789 insertions, 2762 deletions
diff --git a/doc/context/sources/general/manuals/luatex/luatex-contents.tex b/doc/context/sources/general/manuals/luatex/luatex-contents.tex index 6d06b3ef0..2582a81c7 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-contents.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-contents.tex @@ -1,5 +1,4 @@ \environment luatex-style -\environment luatex-logos \startcomponent luatex-contents diff --git a/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex b/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex index e0119bf7e..b81e1dbda 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex @@ -1,22 +1,23 @@ % language=uk \environment luatex-style -\environment luatex-logos \startcomponent luatex-enhancements \startchapter[reference=enhancements,title={Basic \TEX\ enhancements}] -\section{Introduction} +\startsection[title={Introduction}] + +\startsubsection[title={Primitive behaviour}] From day one, \LUATEX\ has offered extra features compared to the superset of -\PDFTEX\ and \ALEPH. This has not been limited to the possibility to execute -\LUA\ code via \type {\directlua}, but \LUATEX\ also adds functionality via new -\TEX|-|side primitives or extensions to existing ones. +\PDFTEX, which includes \ETEX, and \ALEPH. This has not been limited to the +possibility to execute \LUA\ code via \prm {directlua}, but \LUATEX\ also adds +functionality via new \TEX|-|side primitives or extensions to existing ones. When \LUATEX\ starts up in \quote {iniluatex} mode (\type {luatex -ini}), it defines only the primitive commands known by \TEX82 and the one extra command -\type {\directlua}. As is fitting, a \LUA\ function has to be called to add the +\prm {directlua}. As is fitting, a \LUA\ function has to be called to add the extra primitives to the user environment. The simplest method to get access to all of the new primitive commands is by adding this line to the format generation file: @@ -25,7 +26,7 @@ file: \directlua { tex.enableprimitives('',tex.extraprimitives()) } \stoptyping -But be aware that the curly braces may not have the proper \type {\catcode} +But be aware that the curly braces may not have the proper \prm {catcode} assigned to them at this early time (giving a \quote {Missing number} error), so it may be needed to put these assignments before the above line: @@ -36,44 +37,62 @@ it may be needed to put these assignments before the above line: More fine|-|grained primitives control is possible and you can look up the details in \in {section} [luaprimitives]. For simplicity's sake, this manual -assumes that you have executed the \type {\directlua} command as given above. +assumes that you have executed the \prm {directlua} command as given above. The startup behaviour documented above is considered stable in the sense that there will not be backward|-|incompatible changes any more. We have promoted some -rather generic \PDFTEX\ primitives to core \LUATEX\ ones, and the ones inherited -frome \ALEPH\ (\OMEGA) are also promoted. Effectively this means that we now only -have the \type {tex}, \type {etex} and \type {luatex} sets left. +rather generic \PDFTEX\ primitives to core \LUATEX\ ones, and the few that we +inherited from \ALEPH\ (\OMEGA) are also promoted. Effectively this means that we +now only have the \type {tex}, \type {etex} and \type {luatex} sets left. In \in {Chapter} [modifications] we discuss several primitives that are derived from \PDFTEX\ and \ALEPH\ (\OMEGA). Here we stick to real new ones. In the chapters on fonts and math we discuss a few more new ones. -\section{Version information} +\stopsubsection + +\startsubsection[title={Version information}] -\subsection {\type {\luatexbanner}, \type {\luatexversion} and \type {\luatexrevision}} +\startsubsubsection[title={\lpr {luatexbanner}, \lpr {luatexversion} and \lpr {luatexrevision}}] + +\topicindex{version} +\topicindex{banner} There are three new primitives to test the version of \LUATEX: -\starttabulate[|l|pl|pl|] -\BC primitive \BC explanation \BC value \NC \NR -\NC \type {\luatexbanner} \NC the banner reported on the command line \NC \luatexbanner \NC \NR -\NC \type {\luatexversion} \NC a combination of major and minor number \NC \the\luatexversion \NC \NR -\NC \type {\luatexrevision} \NC the revision number, the current value is \NC \luatexrevision \NC \NR +\unexpanded\def\VersionHack#1% otherwise different luatex and luajittex runs + {\ctxlua{% + local banner = "\luatexbanner" + local banner = string.match(banner,"(.+)\letterpercent(") or banner + context(string.gsub(banner ,"jit",""))% + }} + +\starttabulate[|l|l|pl|] +\DB primitive \BC value + \BC explanation \NC \NR +\TB +\NC \lpr {luatexbanner} \NC \VersionHack{\luatexbanner} + \NC the banner reported on the command line \NC \NR +\NC \lpr {luatexversion} \NC \the\luatexversion + \NC a combination of major and minor number \NC \NR +\NC \lpr {luatexrevision} \NC \luatexrevision + \NC the revision number, the current value is \NC \NR +\LL \stoptabulate The official \LUATEX\ version is defined as follows: \startitemize \startitem - The major version is the integer result of \type {\luatexversion} divided by + The major version is the integer result of \lpr {luatexversion} divided by 100. The primitive is an \quote {internal variable}, so you may need to prefix - its use with \type {\the} depending on the context. + its use with \prm {the} depending on the context. \stopitem \startitem - The minor version is the two-digit result of \type {\luatexversion} modulo 100. + The minor version is the two|-|digit result of \lpr {luatexversion} modulo 100. \stopitem \startitem - The revision is the given by \type {\luatexrevision}. This primitive expands to + The revision is reported by \lpr {luatexrevision}. This primitive expands to a positive integer. \stopitem \startitem @@ -82,16 +101,28 @@ The official \LUATEX\ version is defined as follows: \stopitem \stopitemize -\subsection{\type {\formatname}} +\stopsubsubsection + +\startsubsubsection[title={\lpr {formatname}}] -The \type {\formatname} syntax is identical to \type {\jobname}. In \INITEX, the -expansion is empty. Otherwise, the expansion is the value that \type {\jobname} had +\topicindex{format} + +The \lpr {formatname} syntax is identical to \prm {jobname}. In \INITEX, the +expansion is empty. Otherwise, the expansion is the value that \prm {jobname} had during the \INITEX\ run that dumped the currently loaded format. You can use this token list to provide your own version info. -\section{\UNICODE\ text support} +\stopsubsubsection + +\stopsubsection + +\stopsection -\subsection {Extended ranges} +\startsection[title={\UNICODE\ text support}] + +\startsubsection[title={Extended ranges}] + +\topicindex{\UNICODE} Text input and output is now considered to be \UNICODE\ text, so input characters can use the full range of \UNICODE\ ($2^{20}+2^{16}-1 = \hbox{0x10FFFF}$). Later @@ -101,22 +132,25 @@ always converted to a suitable graphic representation of that character in a specific font. However, while processing a list of to|-|be|-|typeset nodes, its contents may still be seen as a character. Inside \LUATEX\ there is no clear separation between the two concepts. Because the subtype of a glyph node can be -changed in \LUA\ it is up to the user: subtypes larger than 255 indicate that +changed in \LUA\ it is up to the user. Subtypes larger than 255 indicate that font processing has happened. A few primitives are affected by this, all in a similar fashion: each of them has -to accommodate for a larger range of acceptable numbers. For instance, \type -{\char} now accepts values between~0 and $1{,}114{,}111$. This should not be a +to accommodate for a larger range of acceptable numbers. For instance, \prm +{char} now accepts values between~0 and $1{,}114{,}111$. This should not be a problem for well|-|behaved input files, but it could create incompatibilities for input that would have generated an error when processed by older \TEX|-|based -engines. The affected commands with an altered initial (left of the equals sign) -or secondary (right of the equals sign) value are: \type {\char}, \type -{\lccode}, \type {\uccode}, \type {\catcode}, \type {\sfcode}, \type {\efcode}, -\type {\lpcode}, \type {\rpcode}, \type {\chardef}. +engines. The affected commands with an altered initial (left of the equal sign) +or secondary (right of the equal sign) value are: \prm {char}, \prm {lccode}, +\prm {uccode}, \lpr {hjcode}, \prm {catcode}, \prm {sfcode}, \lpr {efcode}, \lpr +{lpcode}, \lpr {rpcode}, \prm {chardef}. As far as the core engine is concerned, all input and output to text files is \UTF-8 encoded. Input files can be pre|-|processed using the \type {reader} -callback. This will be explained in a later chapter. +callback. This will be explained in \in {section} [iocallback]. Normalization of +the \UNICODE\ input is on purpose not built|-|in and can be handled by a macro +package during callback processing. We have made some practical choices and the +user has to live with those. Output in byte|-|sized chunks can be achieved by using characters just outside of the valid \UNICODE\ range, starting at the value $1{,}114{,}112$ (0x110000). When @@ -129,58 +163,106 @@ are considered \quote {safe} and therefore printed as|-|is. You can disable escaping with \type {texio.setescape(false)} in which case you get the normal characters on the console. -Normalization of the \UNICODE\ input can be handled by a macro package during -callback processing (this will be explained in \in {section} [iocallback]). +\stopsubsection -\subsection{\type {\Uchar}} +\startsubsection[title={\lpr {Uchar}}] -The expandable command \type {\Uchar} reads a number between~0 and $1{,}114{,}111$ +\topicindex{\UNICODE} + +The expandable command \lpr {Uchar} reads a number between~0 and $1{,}114{,}111$ and expands to the associated \UNICODE\ character. -\section{Extended tables} +\stopsubsection + +\startsubsection[title={Extended tables}] All traditional \TEX\ and \ETEX\ registers can be 16-bit numbers. The affected commands are: \startfourcolumns -\starttyping -\count -\dimen -\skip -\muskip -\marks -\toks -\countdef -\dimendef -\skipdef -\muskipdef -\toksdef -\insert -\box -\unhbox -\unvbox -\copy -\unhcopy -\unvcopy -\wd -\ht -\dp -\setbox -\vsplit -\stoptyping +\startlines +\prm {count} +\prm {dimen} +\prm {skip} +\prm {muskip} +\prm {marks} +\prm {toks} +\prm {countdef} +\prm {dimendef} +\prm {skipdef} +\prm {muskipdef} +\prm {toksdef} +\prm {insert} +\prm {box} +\prm {unhbox} +\prm {unvbox} +\prm {copy} +\prm {unhcopy} +\prm {unvcopy} +\prm {wd} +\prm {ht} +\prm {dp} +\prm {setbox} +\prm {vsplit} +\stoplines \stopfourcolumns Because font memory management has been rewritten, character properties in fonts -are no longer shared among fonts instances that originate from the same metric -file. +are no longer shared among font instances that originate from the same metric +file. Of course we share fonts in the backend when possible so that the resulting +\PDF\ file is as efficient as possible, but for instance also expansion and +protrusion no longer use copies as in \PDFTEX. + +\stopsubsection -\section{Attributes} +\stopsection -\subsection{Attribute registers} +\startsection[title={Attributes}] + +\startsubsection[title={Nodes}] + +\topicindex {nodes} + +When \TEX\ reads input it will interpret the stream according to the properties +of the characters. Some signal a macro name and trigger expansion, others open +and close groups, trigger math mode, etc. What's left over becomes the typeset +text. Internally we get linked list of nodes. Characters become \nod {glyph} +nodes that have for instance a \type {font} and \type {char} property and \typ +{\kern 10pt} becomes a \nod {kern} node with a \type {width} property. Spaces are +alien to \TEX\ as they are turned into \nod {glue} nodes. So, a simple paragraph +is mostly a mix of sequences of \nod {glyph} nodes (words) and \nod {glue} nodes +(spaces). + +The sequences of characters at some point are extended with \nod {disc} nodes +that relate to hyphenation. After that font logic can be applied and we get a +list where some characters can be replaced, for instance multiple characters can +become one ligature, and font kerns can be injected. This is driven by the +font properties. + +Boxes (like \prm {hbox} and \prm {vbox}) become \nod {hlist} or \nod {vlist} +nodes with \type {width}, \type {height}, \type {depth} and \type {shift} +properties and a pointer \type {list} to its actual content. Boxes can be +constructed explicitly or can be the result of subprocesses. For instance, when +lines are broken into paragraphs, the lines are a linked list of \nod {hlist} +nodes. + +So, to summarize: all that you enter as content eventually becomes a node, often +as part of a (nested) list structure. They have a relative small memory footprint +and carry only the minimal amount of information needed. In traditional \TEX\ a +character node only held the font and slot number, in \LUATEX\ we also store some +language related information, the expansion factor, etc. Now that we have access +to these nodes from \LUA\ it makes sense to be able to carry more information +with an node and this is where attributes kick in. + +\stopsubsection + +\startsubsection[title={Attribute registers}] + +\topicindex {attributes} Attributes are a completely new concept in \LUATEX. Syntactically, they behave a lot like counters: attributes obey \TEX's nesting stack and can be used after -\type {\the} etc.\ just like the normal \type {\count} registers. +\prm {the} etc.\ just like the normal \prm {count} registers. \startsyntax \attribute <16-bit number> <optional equals> <32-bit number>!crlf @@ -202,13 +284,22 @@ attached to all nodes created in their scope. These can then be queried from any attributes for node list processing from \LUA\ is given in~\in {chapter}[nodes]. Attributes are stored in a sorted (sparse) linked list that are shared when -possible. This permits efficient testing and updating. +possible. This permits efficient testing and updating. You can define many +thousands of attributes but normally such a large number makes no sense and is +also not that efficient because each node carries a (possibly shared) link to a +list of currently set attributes. But they are a convenient extension and one of +the first extensions we implemented in \LUATEX. + +\stopsubsection -\subsection{Box attributes} +\startsubsection[title={Box attributes}] + +\topicindex {attributes} +\topicindex {boxes} Nodes typically receive the list of attributes that is in effect when they are created. This moment can be quite asynchronous. For example: in paragraph -building, the individual line boxes are created after the \type {\par} command has +building, the individual line boxes are created after the \prm {par} command has been processed, so they will receive the list of attributes that is in effect then, not the attributes that were in effect in, say, the first or third line of the paragraph. @@ -229,28 +320,64 @@ incompatibility is mostly due to the fact that separate specials and literals ar a more unnatural approach to colors than attributes. It is possible to fine-tune the list of attributes that are applied to a \type -{hbox}, \type {vbox} or \type {vtop} by the use of the keyword \type {attr}. An -example: +{hbox}, \type {vbox} or \type {vtop} by the use of the keyword \type {attr}. The +\type {attr} keyword(s) should come before a \type {to} or \type {spread}, if +that is also specified. An example is: -\starttyping -\attribute2=5 +\startbuffer[tex] +\attribute997=123 +\attribute998=456 \setbox0=\hbox {Hello} -\setbox2=\hbox attr1=12 attr2=-"7FFFFFFF{Hello} -\stoptyping +\setbox2=\hbox attr 999 = 789 attr 998 = -"7FFFFFFF{Hello} +\stopbuffer + +\startbuffer[lua] + for b=0,2,2 do + for a=997, 999 do + tex.sprint("box ", b, " : attr ",a," : ",tostring(tex.box[b] [a])) + tex.sprint("\\quad\\quad") + tex.sprint("list ",b, " : attr ",a," : ",tostring(tex.box[b].list[a])) + tex.sprint("\\par") + end + end +\stopbuffer + +\typebuffer[tex] + +Box 0 now has attributes 997 and 998 set while box 2 has attributes 997 and 999 +set while the nodes inside that box will all have attributes 997 and 998 set. +Assigning the maximum negative value causes an attribute to be ignored. + +To give you an idea of what this means at the \LUA\ end, take the following +code: + +\typebuffer[lua] -This will set the attribute list of box~2 to $1=12$, and the attributes of box~0 -will be $2=5$. As you can see, assigning the maximum negative value causes an -attribute to be ignored. +Later we will see that you can access properties of a node. The boxes here are so +called \nod {hlist} nodes that have a field \type {list} that points to the +content. Because the attributes are a list themselves you can access them by +indexing the node (here we do that with \type {[a]}. Running this snippet gives: -The \type {attr} keyword(s) should come before a \type {to} or \type {spread}, if -that is also specified. +\start + \getbuffer[tex] + \startpacked \tt + \ctxluabuffer[lua] + \stoppacked +\stop -\section{\LUA\ related primitives} +Because some values are not set we need to apply the \type {tostring} function +here so that we get the word \type {nil}. -\subsection{\type {\directlua}} +\stopsubsection + +\stopsection + +\startsection[title={\LUA\ related primitives}] + +\startsubsection[title={\prm {directlua}}] In order to merge \LUA\ code with \TEX\ input, a few new primitives are needed. -The primitive \type {\directlua} is used to execute \LUA\ code immediately. The +The primitive \prm {directlua} is used to execute \LUA\ code immediately. The syntax is \startsyntax @@ -261,10 +388,10 @@ syntax is The \syntax {<general text>} is expanded fully, and then fed into the \LUA\ interpreter. After reading and expansion has been applied to the \syntax {<general text>}, the resulting token list is converted to a string as if it was -displayed using \type {\the\toks}. On the \LUA\ side, each \type {\directlua} -block is treated as a separate chunk. In such a chunk you can use the \type -{local} directive to keep your variables from interfering with those used by the -macro package. +displayed using \type {\the\toks}. On the \LUA\ side, each \prm {directlua} block +is treated as a separate chunk. In such a chunk you can use the \type {local} +directive to keep your variables from interfering with those used by the macro +package. The conversion to and from a token list means that you normally can not use \LUA\ line comments (starting with \type {--}) within the argument. As there typically @@ -281,15 +408,16 @@ say: \stoptyping Then \LUA\ line comments can be used, since \TEX\ does not replace line endings -with spaces. +with spaces. Of course such an approach depends on the macro package that you +use. -Likewise, the \syntax {<16-bit number>} designates a name of a \LUA\ chunk and is +The \syntax {<16-bit number>} designates a name of a \LUA\ chunk and is taken from the \type {lua.name} array (see the documentation of the \type {lua} table further in this manual). When a chunk name starts with a \type {@} it will be displayed as a file name. This is a side effect of the way \LUA\ implements error handling. -The \type {\directlua} command is expandable. Since it passes \LUA\ code to the +The \prm {directlua} command is expandable. Since it passes \LUA\ code to the \LUA\ interpreter its expansion from the \TEX\ viewpoint is usually empty. However, there are some \LUA\ functions that produce material to be read by \TEX, the so called print functions. The most simple use of these is \type @@ -320,10 +448,10 @@ will result in \getbuffer -Note that the expansion of \type {\directlua} is a sequence of characters, not of +Note that the expansion of \prm {directlua} is a sequence of characters, not of tokens, contrary to all \TEX\ commands. So formally speaking its expansion is null, but it places material on a pseudo-file to be immediately read by \TEX, as -\ETEX's \type {\scantokens}. For a description of print functions look at \in +\ETEX's \prm {scantokens}. For a description of print functions look at \in {section} [sec:luaprint]. Because the \syntax {<general text>} is a chunk, the normal \LUA\ error handling @@ -337,15 +465,14 @@ can break up \LUATEX\ pretty bad. If you are not careful while working with the node list interface, you may even end up with assertion errors from within the \TEX\ portion of the executable. -The behaviour documented in the above subsection is considered stable in the sense -that there will not be backward-incompatible changes any more. +\stopsubsection -\subsection{\type {\latelua}} +\startsubsection[title={\lpr {latelua} and \lpr {lateluafunction}}] -Contrary to \type {\directlua}, \type {\latelua} stores \LUA\ code in a whatsit +Contrary to \prm {directlua}, \lpr {latelua} stores \LUA\ code in a whatsit that will be processed at the time of shipping out. Its intended use is a cross -between \PDF\ literals (often available as \type {\pdfliteral}) and the -traditional \TEX\ extension \type {\write}. Within the \LUA\ code you can print +between \PDF\ literals (often available as \orm {pdfliteral}) and the +traditional \TEX\ extension \prm {write}. Within the \LUA\ code you can print \PDF\ statements directly to the \PDF\ file via \type {pdf.print}, or you can write to other output streams via \type {texio.write} or simply using \LUA\ \IO\ routines. @@ -356,12 +483,19 @@ routines. \stopsyntax Expansion of macros in the final \type {<general text>} is delayed until just -before the whatsit is executed (like in \type {\write}). With regard to \PDF\ -output stream \type {\latelua} behaves as \PDF\ page literals. The \syntax +before the whatsit is executed (like in \prm {write}). With regard to \PDF\ +output stream \lpr {latelua} behaves as \PDF\ page literals. The \syntax {name <general text>} and \syntax {<16-bit number>} behave in the same way as -they do for \type {\directlua} +they do for \prm {directlua}. + +The \lpr {lateluafunction} primitive takes a number and is similar to \lpr +{luafunction} but gets delated to shipout time. It's just there for completeness. + +\stopsubsection + +\startsubsection[title={\lpr {luaescapestring}}] -\subsection{\type {\luaescapestring}} +\topicindex {escaping} This primitive converts a \TEX\ token sequence so that it can be safely used as the contents of a \LUA\ string: embedded backslashes, double and single quotes, @@ -375,23 +509,24 @@ sequence is fully expanded. \stopsyntax Most often, this command is not actually the best way to deal with the -differences between the \TEX\ and \LUA. In very short bits of \LUA\ -code it is often not needed, and for longer stretches of \LUA\ code it -is easier to keep the code in a separate file and load it using \LUA's -\type {dofile}: +differences between \TEX\ and \LUA. In very short bits of \LUA\ code it is often +not needed, and for longer stretches of \LUA\ code it is easier to keep the code +in a separate file and load it using \LUA's \type {dofile}: \starttyping \directlua { dofile('mysetups.lua') } \stoptyping -\subsection{\type {\luafunction} and \type {\luafunctioncall}} +\stopsubsection -The \type {\directlua} commands involves tokenization of its argument (after +\startsubsection[title={\lpr {luafunction}, \lpr {luafunctioncall} and \lpr {luadef}}] + +The \prm {directlua} commands involves tokenization of its argument (after picking up an optional name or number specification). The tokenlist is then converted into a string and given to \LUA\ to turn into a function that is -called. The overhead is rather small but when you use this primitive hundreds of -thousands of times, it can become noticeable. For this reason there is a variant -call available: \type {\luafunction}. This command is used as follows: +called. The overhead is rather small but when you have millions of calls it can +have some impact. For this reason there is a variant call available: \lpr +{luafunction}. This command is used as follows: \starttyping \directlua { @@ -417,73 +552,117 @@ in the following example the number \type {8} gets typeset. } \stoptyping -The \type {\luafunctioncall} primitive does the same but is unexpandable, for instance -in an \type {\edef}. +The \lpr {luafunctioncall} primitive does the same but is unexpandable, for +instance in an \prm {edef}. In addition \LUATEX\ provides a definer: + +\starttyping + \luadef\MyFunctionA 1 + \global\luadef\MyFunctionB 2 +\protected\global\luadef\MyFunctionC 3 +\stoptyping -\section {Alignments} +You should really use these commands with care. Some references get stored in +tokens and assume that the function is available when that token expands. On the +other hand, as we have tested this functionality in relative complex situations +normal usage should not give problems. -\subsection{\tex {alignmark}} +\stopsubsection -This primitive duplicates the functionality of \type {#} inside alignment -preambles. +\startsubsection[title={\lpr {luabytecode} and \lpr {luabytecodecall}}] -\subsection{\tex {aligntab}} +Analogue to the function callers discussed in the previous section we have byte +code callers. Again the call variant is unexpandable. + +\starttyping +\directlua { + lua.bytecode[9998] = function(s) + tex.sprint(s*token.scan_int()) + end + lua.bytecode[5555] = function(s) + tex.sprint(s*token.scan_dimen()) + end +} +\stoptyping + +This works with: + +\starttyping +\luabytecode 9998 5 \luabytecode 5555 5sp +\luabytecodecall9998 5 \luabytecodecall5555 5sp +\stoptyping -This primitive duplicates the functionality of \type {&} inside alignments and -preambles. +The variable \type {s} in the code is the number of the byte code register that +can be used for diagnostic purposes. The advantage of bytecode registers over +function calls is that they are stored in the format (but without upvalues). -\section{Catcode tables} +\stopsubsection + +\stopsection + +\startsection[title={Catcode tables}] + +\startsubsection[title={Catcodes}] + +\topicindex {catcodes} Catcode tables are a new feature that allows you to switch to a predefined catcode regime in a single statement. You can have a practically unlimited number of different tables. This subsystem is backward compatible: if you never use the following commands, your document will not notice any difference in behaviour compared to traditional \TEX. The contents of each catcode table is independent -from any other catcode tables, and their contents is stored and retrieved from -the format file. +from any other catcode table, and its contents is stored and retrieved from the +format file. -\subsection{\type {\catcodetable}} +\stopsubsection + +\startsubsection[title={\lpr {catcodetable}}] \startsyntax \catcodetable <15-bit number> \stopsyntax -The primitive \type {\catcodetable} switches to a different catcode table. Such a +The primitive \lpr {catcodetable} switches to a different catcode table. Such a table has to be previously created using one of the two primitives below, or it has to be zero. Table zero is initialized by \INITEX. -\subsection{\type {\initcatcodetable}} +\stopsubsection + +\startsubsection[title={\lpr {initcatcodetable}}] \startsyntax \initcatcodetable <15-bit number> \stopsyntax -The primitive \type {\initcatcodetable} creates a new table with catcodes identical -to those defined by \INITEX: - -\starttabulate[|r|l|l|l|] -\NC 0 \NC \tttf \letterbackslash \NC \NC \type {escape} \NC\NR -\NC 5 \NC \tttf \letterhat\letterhat M \NC return \NC \type {car_ret} \NC\NR -\NC 9 \NC \tttf \letterhat\letterhat @ \NC null \NC \type {ignore} \NC\NR -\NC 10 \NC \tttf <space> \NC space \NC \type {spacer} \NC\NR -\NC 11 \NC {\tttf a} \endash\ {\tttf z} \NC \NC \type {letter} \NC\NR -\NC 11 \NC {\tttf A} \endash\ {\tttf Z} \NC \NC \type {letter} \NC\NR -\NC 12 \NC everything else \NC \NC \type {other} \NC\NR -\NC 14 \NC \tttf \letterpercent \NC \NC \type {comment} \NC\NR -\NC 15 \NC \tttf \letterhat\letterhat ? \NC delete \NC \type {invalid_char} \NC\NR +The primitive \lpr {initcatcodetable} creates a new table with catcodes +identical to those defined by \INITEX. The new catcode table is allocated +globally: it will not go away after the current group has ended. If the supplied +number is identical to the currently active table, an error is raised. The +initial values are: + +\starttabulate[|c|c|l|l|] +\DB catcode \BC character \BC equivalent \BC category \NC \NR +\TB +\NC 0 \NC \tttf \letterbackslash \NC \NC \type {escape} \NC \NR +\NC 5 \NC \tttf \letterhat\letterhat M \NC return \NC \type {car_ret} \NC \NR +\NC 9 \NC \tttf \letterhat\letterhat @ \NC null \NC \type {ignore} \NC \NR +\NC 10 \NC \tttf <space> \NC space \NC \type {spacer} \NC \NR +\NC 11 \NC {\tttf a} \endash\ {\tttf z} \NC \NC \type {letter} \NC \NR +\NC 11 \NC {\tttf A} \endash\ {\tttf Z} \NC \NC \type {letter} \NC \NR +\NC 12 \NC everything else \NC \NC \type {other} \NC \NR +\NC 14 \NC \tttf \letterpercent \NC \NC \type {comment} \NC \NR +\NC 15 \NC \tttf \letterhat\letterhat ? \NC delete \NC \type {invalid_char} \NC \NR +\LL \stoptabulate -The new catcode table is allocated globally: it will not go away after the -current group has ended. If the supplied number is identical to the currently -active table, an error is raised. +\stopsubsection -\subsection{\type {\savecatcodetable}} +\startsubsection[title={\lpr {savecatcodetable}}] \startsyntax \savecatcodetable <15-bit number> \stopsyntax -\type {\savecatcodetable} copies the current set of catcodes to a new table with +\lpr {savecatcodetable} copies the current set of catcodes to a new table with the requested number. The definitions in this new table are all treated as if they were made in the outermost level. @@ -491,55 +670,78 @@ The new table is allocated globally: it will not go away after the current group has ended. If the supplied number is the currently active table, an error is raised. -\section{Suppressing errors} +\stopsubsection + +\stopsection -\subsection{\type {\suppressfontnotfounderror}} +\startsection[title={Suppressing errors}] + +\startsubsection[title={\lpr {suppressfontnotfounderror}}] + +\topicindex {errors} + +If this integer parameter is non|-|zero, then \LUATEX\ will not complain about +font metrics that are not found. Instead it will silently skip the font +assignment, making the requested csname for the font \prm {ifx} equal to \prm +{nullfont}, so that it can be tested against that without bothering the user. \startsyntax \suppressfontnotfounderror = 1 \stopsyntax -If this integer parameter is non|-|zero, then \LUATEX\ will not complain about -font metrics that are not found. Instead it will silently skip the font -assignment, making the requested csname for the font \type {\ifx} equal to \type -{\nullfont}, so that it can be tested against that without bothering the user. +\stopsubsection + +\startsubsection[title={\lpr {suppresslongerror}}] -\subsection{\type {\suppresslongerror}} +\topicindex {errors} + +If this integer parameter is non|-|zero, then \LUATEX\ will not complain about +\prm {par} commands encountered in contexts where that is normally prohibited +(most prominently in the arguments of macros not defined as \prm {long}). \startsyntax \suppresslongerror = 1 \stopsyntax -If this integer parameter is non|-|zero, then \LUATEX\ will not complain about -\type {\par} commands encountered in contexts where that is normally prohibited -(most prominently in the arguments of non-long macros). +\stopsubsection -\subsection{\type {\suppressifcsnameerror}} +\startsubsection[title={\lpr {suppressifcsnameerror}}] -\startsyntax -\suppressifcsnameerror = 1 -\stopsyntax +\topicindex {errors} If this integer parameter is non|-|zero, then \LUATEX\ will not complain about -non-expandable commands appearing in the middle of a \type {\ifcsname} expansion. +non-expandable commands appearing in the middle of a \prm {ifcsname} expansion. Instead, it will keep getting expanded tokens from the input until it encounters -an \type {\endcsname} command. If the input expansion is unbalanced with respect -to \type {\csname} \ldots \type {\endcsname} pairs, the \LUATEX\ process may hang +an \prm {endcsname} command. If the input expansion is unbalanced with respect +to \prm {csname} \ldots \prm {endcsname} pairs, the \LUATEX\ process may hang indefinitely. -\subsection{\type {\suppressoutererror}} - \startsyntax -\suppressoutererror = 1 +\suppressifcsnameerror = 1 \stopsyntax +\stopsubsection + +\startsubsection[title={\lpr {suppressoutererror}}] + +\topicindex {errors} + If this new integer parameter is non|-|zero, then \LUATEX\ will not complain -about \type {\outer} commands encountered in contexts where that is normally +about \prm {outer} commands encountered in contexts where that is normally prohibited. -\subsection{\type {\suppressmathparerror}} +\startsyntax +\suppressoutererror = 1 +\stopsyntax + +\stopsubsection + +\startsubsection[title={\lpr {suppressmathparerror}}] + +\topicindex {errors} +\topicindex {math} -The following setting will permit \type {\par} tokens in a math formula: +The following setting will permit \prm {par} tokens in a math formula: \startsyntax \suppressmathparerror = 1 @@ -553,41 +755,30 @@ $ x + 1 = a $ \stoptyping -\subsection{\type {\suppressprimitiveerror}} +\stopsubsection + +\startsubsection[title={\lpr {suppressprimitiveerror}}] + +\topicindex {errors} +\topicindex {primitives} When set to a non|-|zero value the following command will not issue an error: -\starttyping +\startsyntax \suppressprimitiveerror = 1 \primitive\notaprimitive -\stoptyping - -\section {Math} - -\subsection{Extensions} +\stopsyntax -We will cover math in its own chapter because not only the font subsystem and -spacing model have been enhanced (thereby introducing many new primitives) but -also because some more control has been added to existing functionality. +\stopsubsection -\subsection{\type {\matheqnogapstep}} +\stopsection -By default \TEX\ will add one quad between the equation and the number. This is -hard coded. A new primitive can control this: +\startsection[title={Fonts}] -\startsyntax -\matheqnogapstep = 1000 -\stopsyntax +\startsubsection[title={Font syntax}] -Because a math quad from the math text font is used instead of a dimension, we -use a step to control the size. A value of zero will suppress the gap. The step -is divided by 1000 which is the usual way to mimmick floating point factors in -\TEX. - -\section{Fonts} - -\subsection{Font syntax} +\topicindex {fonts} \LUATEX\ will accept a braced argument as a font name: @@ -598,20 +789,26 @@ is divided by 1000 which is the usual way to mimmick floating point factors in This allows for embedded spaces, without the need for double quotes. Macro expansion takes place inside the argument. -\subsection{\type {\fontid}} +\stopsubsection + +\startsubsection[title={\lpr {fontid} and \lpr {setfontid}}] \startsyntax \fontid\font \stopsyntax This primitive expands into a number. It is not a register so there is no need to -prefix with \type {\number} (and using \type {\the} gives an error). The currently +prefix with \prm {number} (and using \prm {the} gives an error). The currently used font id is \fontid\font. Here are some more: -\starttabulate[|l|c|] -\NC \type {\bf} \NC \bf \fontid\font \NC \NR -\NC \type {\it} \NC \it \fontid\font \NC \NR -\NC \type {\bi} \NC \bi \fontid\font \NC \NR +\starttabulate[|l|c|c|] +\DB style \BC command \BC font id \NC \NR +\TB +\NC normal \NC \type {\tf} \NC \bf \fontid\font \NC \NR +\NC bold \NC \type {\bf} \NC \bf \fontid\font \NC \NR +\NC italic \NC \type {\it} \NC \it \fontid\font \NC \NR +\NC bold italic \NC \type {\bi} \NC \bi \fontid\font \NC \NR +\LL \stoptabulate These numbers depend on the macro package used because each one has its own way @@ -620,12 +817,15 @@ order of loading fonts. For instance, when in \CONTEXT\ virtual math \UNICODE\ fonts are used, we can easily get over a hundred ids in use. Not all ids have to be bound to a real font, after all it's just a number. -\subsection{\type {\setfontid}} +The primitive \lpr {setfontid} can be used to enable a font with the given id, +which of course needs to be a valid one. -The primitive \type {\setfontid} can be used to enable a font with the given id -(which of course needs to be a valid one). +\stopsubsection -\subsection{\type {\noligs} and \type {\nokerns}} +\startsubsection[title={\lpr {noligs} and \lpr {nokerns}}] + +\topicindex {ligatures+suppress} +\topicindex {kerns+suppress} These primitives prohibit ligature and kerning insertion at the time when the initial node list is built by \LUATEX's main control loop. You can enable these @@ -642,15 +842,19 @@ kerning functions, i.e.\ by assigning dummy functions to their associated callbacks. Keep in mind that when you define a font (using \LUA) you can also omit the kern and ligature tables, which has the same effect as the above. -\subsection{\type{\nospaces}} +\stopsubsection + +\startsubsection[title={\type{\nospaces}}] + +\topicindex {spaces+suppress} -This new primitive can be used to overrule the usual \type {\spaceskip} -related heuristics when a space character is seen in a text flow. The -value~\type{1} triggers no injection while \type{2} results in injection of -a zero skip. Below we see the results for four characters separated by a +This new primitive can be used to overrule the usual \prm {spaceskip} related +heuristics when a space character is seen in a text flow. The value~\type{1} +triggers no injection while \type{2} results in injection of a zero skip. In \in +{figure} [fig:nospaces] we see the results for four characters separated by a space. -\startlinecorrection +\startplacefigure[reference=fig:nospaces,title={The \lpr {nospaces} options.}] \startcombination[3*2] {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 10mm}} {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 10mm}} @@ -659,24 +863,30 @@ space. {\ruledhbox to 5cm{\vtop{\hsize 1mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 1mm}} {\ruledhbox to 5cm{\vtop{\hsize 1mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 1mm}} \stopcombination -\stoplinecorrection +\stopplacefigure + +\stopsubsection + +\stopsection + +\startsection[title={Tokens, commands and strings}] -\section{Tokens, commands and strings} +\startsubsection[title={\lpr {scantextokens}}] -\subsection{\type {\scantextokens}} +\topicindex {tokens+scanning} -The syntax of \type {\scantextokens} is identical to \type {\scantokens}. This -primitive is a slightly adapted version of \ETEX's \type {\scantokens}. The +The syntax of \lpr {scantextokens} is identical to \prm {scantokens}. This +primitive is a slightly adapted version of \ETEX's \prm {scantokens}. The differences are: \startitemize \startitem - The last (and usually only) line does not have a \type {\endlinechar} + The last (and usually only) line does not have a \prm {endlinechar} appended. \stopitem \startitem - \type {\scantextokens} never raises an EOF error, and it does not execute - \type {\everyeof} tokens. + \lpr {scantextokens} never raises an EOF error, and it does not execute + \prm {everyeof} tokens. \stopitem \startitem There are no \quote {\unknown\ while end of file \unknown} error tests @@ -685,7 +895,10 @@ differences are: \stopitem \stopitemize -\subsection{\type {\toksapp}, \type {\tokspre}, \type {\etoksapp} and \type {\etokspre}} +\stopsubsection + +\startsubsection[title={\lpr {toksapp}, \lpr {tokspre}, \lpr {etoksapp}, \lpr {etokspre}, +\lpr {gtoksapp}, \lpr {gtokspre}, \lpr {xtoksapp}, \lpr {xtokspre}}] Instead of: @@ -700,15 +913,17 @@ you can use: \stoptyping The \type {pre} variants prepend instead of append, and the \type {e} variants -expand the passed general text. +expand the passed general text. The \type {g} and \type {x} variants are global. + +\stopsubsection -\subsection{\type {\csstring}, \type {\begincsname} and \type {\lastnamedcs}} +\startsubsection[title={\prm {csstring}, \lpr {begincsname} and \lpr {lastnamedcs}}] -These are somewhat special. The \type {\csstring} primitive is like -\type {\string} but it omits the leading escape character. This can be -somewhat more efficient that stripping it of afterwards. +These are somewhat special. The \prm {csstring} primitive is like +\prm {string} but it omits the leading escape character. This can be +somewhat more efficient than stripping it afterwards. -The \type {\begincsname} primitive is like \type {\csname} but doesn't create +The \lpr {begincsname} primitive is like \prm {csname} but doesn't create a relaxed equivalent when there is no such name. It is equivalent to \starttyping @@ -718,10 +933,8 @@ a relaxed equivalent when there is no such name. It is equivalent to \stoptyping The advantage is that it saves a lookup (don't expect much speedup) but more -important is that it avoids using the \type {\if}. - -The \type {\lastnamedcs} is one that should be used with care. The above -example could be written as: +important is that it avoids using the \prm {if} test. The \lpr {lastnamedcs} +is one that should be used with care. The above example could be written as: \starttyping \ifcsname foo\endcsname @@ -731,9 +944,13 @@ example could be written as: This is slightly more efficient than constructing the string twice (deep down in \LUATEX\ this also involves some \UTF8 juggling), but probably more relevant is -that it saves a few tokens and can make code a bit more more readable. +that it saves a few tokens and can make code a bit more readable. + +\stopsubsection -\subsection{\type {\clearmarks}} +\startsubsection[title={\lpr {clearmarks}}] + +\topicindex {marks} This primitive complements the \ETEX\ mark primitives and clears a mark class completely, resetting all three connected mark texts to empty. It is an @@ -743,71 +960,241 @@ immediate command. \clearmarks <16-bit number> \stopsyntax -\subsection{\type{\letcharcode}} +\stopsubsection + +\startsubsection[title={\lpr {alignmark} and \lpr {aligntab}}] + +The primitive \lpr {alignmark} duplicates the functionality of \type {#} inside +alignment preambles, while \lpr {aligntab} duplicates the functionality of \type +{&}. -This primitive is still experimental but can be used to assign a meaning to an active -character, as in: +\stopsubsection + +\startsubsection[title={\lpr {letcharcode}}] + +This primitive can be used to assign a meaning to an active character, as in: \starttyping \def\foo{bar} \letcharcode123=\foo \stoptyping -This can be a bit nicer that using the uppercase tricks (using the property of -\type {\uppercase} that it treats active characters special). +This can be a bit nicer than using the uppercase tricks (using the property of +\prm {uppercase} that it treats active characters special). -\section{Boxes, rules and leaders} +\stopsubsection -\subsection{\type {\outputbox}} +\startsubsection[title={\lpr {glet}}] -\startsyntax -\outputbox = 65535 -\stopsyntax +This primitive is similar to: + +\starttyping +\protected\def\glet{\global\let} +\stoptyping + +but faster (only measurable with millions of calls) and probably more convenient +(after all we also have \type {\gdef}). + +\stopsubsection + +\startsubsection[title={\lpr {expanded}, \lpr {immediateassignment} and \lpr {immediateassigned}}] + +\topicindex {expansion} + +The \lpr {expanded} primitive takes a token list and expands it content which can +come in handy: it avoids a tricky mix of \prm {expandafter} and \prm {noexpand}. +You can compare it with what happens inside the body of an \prm {edef}. But this +kind of expansion it still doesn't expand some primitive operations. + +\startbuffer +\newcount\NumberOfCalls + +\def\TestMe{\advance\NumberOfCalls1 } + +\edef\Tested{\TestMe foo:\the\NumberOfCalls} +\edef\Tested{\TestMe foo:\the\NumberOfCalls} +\edef\Tested{\TestMe foo:\the\NumberOfCalls} + +\meaning\Tested +\stopbuffer + +\typebuffer + +The result is a macro that has the not expanded code in its body: + +\getbuffer + +Instead we can define \tex {TestMe} in a way that expands the assignment +immediately. You need of course to be aware of preventing look ahead interference +by using a space or \tex {relax} (often an expression works better as it doesn't +leave an \tex {relax}). + +\startbuffer +\def\TestMe{\immediateassignment\advance\NumberOfCalls1 } + +\edef\Tested{\TestMe foo:\the\NumberOfCalls} +\edef\Tested{\TestMe foo:\the\NumberOfCalls} +\edef\Tested{\TestMe foo:\the\NumberOfCalls} + +\meaning\Tested +\stopbuffer + +\typebuffer + +This time the counter gets updates and we don't see interference in the +resulting \tex {Tested} macro: + +\getbuffer + +Here is a somewhat silly example of expanded comparison: + +\startbuffer +\def\expandeddoifelse#1#2#3#4% + {\immediateassignment\edef\tempa{#1}% + \immediateassignment\edef\tempb{#2}% + \ifx\tempa\tempb + \immediateassignment\def\next{#3}% + \else + \immediateassignment\def\next{#4}% + \fi + \next} + +\edef\Tested + {(\expandeddoifelse{abc}{def}{yes}{nop}/% + \expandeddoifelse{abc}{abc}{yes}{nop})} + +\meaning\Tested +\stopbuffer + +\typebuffer + +It gives: + +\getbuffer + +A variant is: + +\starttyping +\def\expandeddoifelse#1#2#3#4% + {\immediateassigned{ + \edef\tempa{#1}% + \edef\tempb{#2}% + }% + \ifx\tempa\tempb + \immediateassignment\def\next{#3}% + \else + \immediateassignment\def\next{#4}% + \fi + \next} +\stoptyping + +The possible error messages are the same as using assignments in preambles of +alignments and after the \prm {accent} command. The supported assignments are the +so called prefixed commands (except box assignments). + +\stopsubsection + +\startsubsection[title={\lpr {ifcondition}}] + +\topicindex {conditions} + +This is a somewhat special one. When you write macros conditions need to be +properly balanced in order to let \TEX's fast branch skipping work well. This new +primitive is basically a no||op flagged as a condition so that the scanner can +recognize it as an if|-|test. However, when a real test takes place the work is +done by what follows, in the next example \tex {something}. + +\starttyping +\unexpanded\def\something#1#2% + {\edef\tempa{#1}% + \edef\tempb{#2} + \ifx\tempa\tempb} + +\ifcondition\something{a}{b}% + \ifcondition\something{a}{a}% + true 1 + \else + false 1 + \fi +\else + \ifcondition\something{a}{a}% + true 2 + \else + false 2 + \fi +\fi +\stoptyping + +If you are familiar with \METAPOST, this is a bit like \type {vardef} where the macro +has a return value. Here the return value is a test. + +\stopsubsection -This new integer parameter allows you to alter the number of the box that will be +\stopsection + +\startsection[title={Boxes, rules and leaders}] + +\startsubsection[title={\lpr {outputbox}}] + +\topicindex {output} + +This integer parameter allows you to alter the number of the box that will be used to store the page sent to the output routine. Its default value is 255, and the acceptable range is from 0 to 65535. -\subsection{\type {\vpack}, \type {\hpack} and \type {\tpack}} +\startsyntax +\outputbox = 12345 +\stopsyntax + +\stopsubsection -These three primitives are like \type {\vbox}, \type {\hbox} and \type {\vtop} +\startsubsection[title={\prm {vpack}, \prm {hpack} and \prm {tpack}}] + +These three primitives are like \prm {vbox}, \prm {hbox} and \prm {vtop} but don't apply the related callbacks. -\subsection{\type {\vsplit}} +\stopsubsection + +\startsubsection[title={\prm {vsplit}}] + +\topicindex {splitting} -The \type {\vsplit} primitive has to be followed by a specification of the -required height. As alternative for the \type {to} keyword you can use \type -{upto} to get a split of the given size but result has the natural dimensions -then. +The \prm {vsplit} primitive has to be followed by a specification of the required +height. As alternative for the \type {to} keyword you can use \type {upto} to get +a split of the given size but result has the natural dimensions then. -\subsection{Images and Forms} +\stopsubsection + +\startsubsection[title={Images and reused box objects},reference=sec:imagedandforms] These two concepts are now core concepts and no longer whatsits. They are in fact now implemented as rules with special properties. Normal rules have subtype~0, saved boxes have subtype~1 and images have subtype~2. This has the positive side -effect that whenever we need to take content with dimensions into account, when we -look at rule nodes, we automatically also deal with these two types. +effect that whenever we need to take content with dimensions into account, when +we look at rule nodes, we automatically also deal with these two types. The syntax of the \type {\save...resource} is the same as in \PDFTEX\ but you should consider them to be backend specific. This means that a macro package should treat them as such and check for the current output mode if applicable. -Here are the equivalents: -\starttabulate[|l|l|] -\NC \type {\saveboxresource} \EQ \type {\pdfxform} \NC \NR -\NC \type {\saveimageresource} \EQ \type {\pdfximage} \NC \NR -\NC \type {\useboxresource} \EQ \type {\pdfrefxform} \NC \NR -\NC \type {\useimageresource} \EQ \type {\pdfrefximage} \NC \NR -\NC \type {\lastsavedboxresourceindex} \EQ \type {\pdflastxform} \NC \NR -\NC \type {\lastsavedimageresourceindex} \EQ \type {\pdflastximage} \NC \NR -\NC \type {\lastsavedimageresourcepages} \EQ \type {\pdflastximagepages} \NC \NR +\starttabulate[|l|p|] +\DB command \BC explanation \NC \NR +\TB +\NC \lpr {saveboxresource} \NC save the box as an object to be included later \NC \NR +\NC \lpr {saveimageresource} \NC save the image as an object to be included later \NC \NR +\NC \lpr {useboxresource} \NC include the saved box object here (by index) \NC \NR +\NC \lpr {useimageresource} \NC include the saved image object here (by index) \NC \NR +\NC \lpr {lastsavedboxresourceindex} \NC the index of the last saved box object \NC \NR +\NC \lpr {lastsavedimageresourceindex} \NC the index of the last saved image object \NC \NR +\NC \lpr {lastsavedimageresourcepages} \NC the number of pages in the last saved image object \NC \NR +\LL \stoptabulate \LUATEX\ accepts optional dimension parameters for \type {\use...resource} in the same format as for rules. With images, these dimensions are then used instead of -the ones given to \type {\useimageresource} but the original dimensions are not -overwritten, so that a \type {\useimageresource} without dimensions still -provides the image with dimensions defined by \type {\saveimageresource}. These -optional parameters are not implemented for \type {\saveboxresource}. +the ones given to \lpr {useimageresource} but the original dimensions are not +overwritten, so that a \lpr {useimageresource} without dimensions still +provides the image with dimensions defined by \lpr {saveimageresource}. These +optional parameters are not implemented for \lpr {saveboxresource}. \starttyping \useimageresource width 20mm height 10mm depth 5mm \lastsavedimageresourceindex @@ -820,35 +1207,51 @@ is the \type {type} key. When set to non|-|zero the \type {/Type} entry is omitted. A value of 1 or 3 still writes a \type {/BBox}, while 2 or 3 will write a \type {/Matrix}. -\subsection{\type {\nohrule} and \type {\novrule}} +\stopsubsection + +\startsubsection[title={\lpr {nohrule} and \lpr {novrule}}] + +\topicindex {rules} Because introducing a new keyword can cause incompatibilities, two new primitives -were introduced: \type {\nohrule} and \type {\novrule}. These can be used to +were introduced: \lpr {nohrule} and \lpr {novrule}. These can be used to reserve space. This is often more efficient than creating an empty box with fake -dimensions). +dimensions. + +\stopsubsection -\subsection{\type {\gleaders}} +\startsubsection[title={\lpr {gleaders}}] + +\topicindex {leaders} This type of leaders is anchored to the origin of the box to be shipped out. So -they are like normal \type {\leaders} in that they align nicely, except that the +they are like normal \prm {leaders} in that they align nicely, except that the alignment is based on the {\it largest\/} enclosing box instead of the {\it smallest\/}. The \type {g} stresses this global nature. -\section {Languages} +\stopsubsection + +\stopsection -\subsection{\type {\hyphenationmin}} +\startsection[title={Languages}] + +\startsubsection[title={\lpr {hyphenationmin}}] + +\topicindex {languages} +\topicindex {hyphenation} This primitive can be used to set the minimal word length, so setting it to a value of~$5$ means that only words of 6 characters and more will be hyphenated, of course -within the constraints of the \type {\lefthyphenmin} and \type {\righthyphenmin} +within the constraints of the \prm {lefthyphenmin} and \prm {righthyphenmin} values (as stored in the glyph node). This primitive accepts a number and stores the value with the language. -\subsection{\type {\boundary}, \type {\noboundary}, \type {\protrusionboundary} and \type -{\wordboundary}} +\stopsubsection + +\startsubsection[title={\prm {boundary}, \prm {noboundary}, \prm {protrusionboundary} and \prm {wordboundary}}] -The \type {\noboundary} commands used to inject a whatsit node but now injects a normal -node with type \type {boundary} and subtype~0. In addition you can say: +The \prm {noboundary} command is used to inject a whatsit node but now injects a normal +node with type \nod {boundary} and subtype~0. In addition you can say: \starttyping x\boundary 123\relax y @@ -858,31 +1261,57 @@ This has the same effect but the subtype is now~1 and the value~123 is stored. The traditional ligature builder still sees this as a cancel boundary directive but at the \LUA\ end you can implement different behaviour. The added benefit of passing this value is a side effect of the generalization. The subtypes~2 and~3 -are used to control protrusion and word boundaries in hyphenation. +are used to control protrusion and word boundaries in hyphenation and have +related primitives. -\section{Control and debugging} +\stopsubsection -\subsection {Tracing} +\stopsection -If \type {\tracingonline} is larger than~2, the node list display will also print +\startsection[title={Control and debugging}] + +\startsubsection[title={Tracing}] + +\topicindex {tracing} + +If \prm {tracingonline} is larger than~2, the node list display will also print the node number of the nodes. -\subsection{\type {\outputmode} and \type {\draftmode}} +\stopsubsection -The \type {\outputmode} variable tells \LUATEX\ what it has to produce: +\startsubsection[title={\lpr {outputmode}}] + +\topicindex {output} +\topicindex {backend} + +The \lpr {outputmode} variable tells \LUATEX\ what it has to produce: \starttabulate[|l|l|] +\DB value \BC output \NC \NR +\TB \NC \type {0} \NC \DVI\ code \NC \NR \NC \type {1} \NC \PDF\ code \NC \NR +\LL \stoptabulate -The value of the \type {\draftmode} counter signals the backend if it should -output less. The \PDF\ backend accepts a value of~$1$, while the \DVI\ backend -ignores the value. +\stopsubsection + +\startsubsection[title={\lpr {draftmode}}] -\section {Files} +The value of the \lpr {draftmode} counter signals the backend if it should output +less. The \PDF\ backend accepts a value of~1, while the \DVI\ backend ignores the +value. This is no critical feature so we can remove it in future versions when it +can make the backend cleaner. -\subsection{File syntax} +\stopsubsection + +\stopsection + +\startsection[title={Files}] + +\startsubsection[title={File syntax}] + +\topicindex {files+names} \LUATEX\ will accept a braced argument as a file name: @@ -894,26 +1323,49 @@ ignores the value. This allows for embedded spaces, without the need for double quotes. Macro expansion takes place inside the argument. -The \type {\tracingfonts} primitive that has been inherited from \PDFTEX\ has +The \lpr {tracingfonts} primitive that has been inherited from \PDFTEX\ has been adapted to support variants in reporting the font. The reason for this extension is that a csname not always makes sense. The zero case is the default. -\starttabulate[|T||] -\NC 0 \EQ \type{\foo xyz} \NC \NR -\NC 1 \EQ \type{\foo (bar)} \NC \NR -\NC 2 \EQ \type{<bar> xyz} \NC \NR -\NC 3 \EQ \type{<bar @ ..pt> xyz} \NC \NR -\NC 4 \EQ \type{<id>} \NC \NR -\NC 5 \EQ \type{<id: bar>} \NC \NR -\NC 6 \EQ \type{<id: bar @ ..pt> xyz} \NC \NR +\starttabulate[|l|l|] +\DB value \BC reported \NC \NR +\TB +\NC \type{0} \NC \type{\foo xyz} \NC \NR +\NC \type{1} \NC \type{\foo (bar)} \NC \NR +\NC \type{2} \NC \type{<bar> xyz} \NC \NR +\NC \type{3} \NC \type{<bar @ ..pt> xyz} \NC \NR +\NC \type{4} \NC \type{<id>} \NC \NR +\NC \type{5} \NC \type{<id: bar>} \NC \NR +\NC \type{6} \NC \type{<id: bar @ ..pt> xyz} \NC \NR +\LL \stoptabulate -\subsection{Writing to file} +\stopsubsection + +\startsubsection[title={Writing to file}] + +\topicindex {files+writing} -You can now open upto 127 files with \type {\openout}. When no file is open +You can now open upto 127 files with \prm {openout}. When no file is open writes will go to the console and log. As a consequence a system command is no longer possible but one can use \type {os.execute} to do the same. +\stopsubsection + +\stopsection + +\startsection[title={Math}] + +\topicindex {math} + +We will cover math extensions in its own chapter because not only the font +subsystem and spacing model have been enhanced (thereby introducing many new +primitives) but also because some more control has been added to existing +functionality. Much of this relates to the different approaches of traditional +\TEX\ fonts and \OPENTYPE\ math. + +\stopsection + \stopchapter \stopcomponent diff --git a/doc/context/sources/general/manuals/luatex/luatex-firstpage.tex b/doc/context/sources/general/manuals/luatex/luatex-firstpage.tex new file mode 100644 index 000000000..772fbb3fe --- /dev/null +++ b/doc/context/sources/general/manuals/luatex/luatex-firstpage.tex @@ -0,0 +1,32 @@ +\startcomponent luatex-firstpage + +\startstandardmakeup + + \start + \raggedleft + \definedfont[Bold*default at 48pt] + \setupinterlinespace + \blue \documentvariable{manual} \endgraf Reference \endgraf Manual \endgraf + \stop + + \vfill + + \definedfont[Bold*default at 12pt] + + \starttabulate[|l|l|] + \NC copyright \EQ Lua\TeX\ development team \NC \NR + \NC more info \EQ www.luatex.org \NC \NR + \NC version \EQ \currentdate \doifsomething{\documentvariable{snapshot}}{(snapshot \documentvariable{snapshot})} \NC \NR + \stoptabulate + +\stopstandardmakeup + +\setupbackgrounds + [leftpage] + [setups=pagenumber:left] + +\setupbackgrounds + [rightpage] + [setups=pagenumber:right] + +\stopcomponent diff --git a/doc/context/sources/general/manuals/luatex/luatex-fonts.tex b/doc/context/sources/general/manuals/luatex/luatex-fonts.tex index ddb64d946..d49c63afe 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-fonts.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-fonts.tex @@ -1,29 +1,31 @@ % language=uk \environment luatex-style -\environment luatex-logos \startcomponent luatex-fonts \startchapter[reference=fonts,title={Font structure}] -\section {The font tables} +\startsection[title={The font tables}] + +\topicindex {fonts} +\topicindex {fonts+tables} All \TEX\ fonts are represented to \LUA\ code as tables, and internally as \CCODE~structures. All keys in the table below are saved in the internal font -structure if they are present in the table returned by the \type {define_font} +structure if they are present in the table returned by the \cbk {define_font} callback, or if they result from the normal \TFM|/|\VF\ reading routines if there -is no \type {define_font} callback defined. +is no \cbk {define_font} callback defined. The column \quote {\VF} means that this key will be created by the \type {font.read_vf()} routine, \quote {\TFM} means that the key will be created by the -\type {font.read_tfm()} routine, and \quote{used} means whether or not the -\LUATEX\ engine itself will do something with the key. - -The top|-|level keys in the table are as follows: +\type {font.read_tfm()} routine, and \quote {used} means whether or not the +\LUATEX\ engine itself will do something with the key. The top|-|level keys in +the table are as follows: -\starttabulate[|l|c|c|c|l|p|] -\BC key \BC vf \BC tfm \BC used \BC value type \BC description \NC \NR +\starttabulate[|l|c|c|c|l|pl|] +\DB key \BC vf \BC tfm \BC used \BC value type \BC description \NC \NR +\TB \NC \type{name} \NC yes \NC yes \NC yes \NC string \NC metric (file) name \NC \NR \NC \type{area} \NC no \NC yes \NC yes \NC string \NC (directory) location, typically empty \NC \NR \NC \type{used} \NC no \NC yes \NC yes \NC boolean \NC indicates usage (initial: false) \NC \NR @@ -42,52 +44,61 @@ The top|-|level keys in the table are as follows: \NC \type{fullname} \NC no \NC no \NC yes \NC string \NC output font name, used as a fallback in the \PDF\ output if the \type {psname} is not set \NC \NR \NC \type{header} \NC yes \NC no \NC no \NC string \NC header comments, if any \NC \NR -\NC \type{hyphenchar} \NC no \NC no \NC yes \NC number \NC default: \TEX's \type {\hyphenchar} \NC \NR +\NC \type{hyphenchar} \NC no \NC no \NC yes \NC number \NC default: \TEX's \prm {hyphenchar} \NC \NR \NC \type{parameters} \NC no \NC yes \NC yes \NC hash \NC default: 7 parameters, all zero \NC \NR -\NC \type{size} \NC no \NC yes \NC yes \NC number \NC loaded (at) size. (default: same as designsize) \NC \NR -\NC \type{skewchar} \NC no \NC no \NC yes \NC number \NC default: \TEX's \type {\skewchar} \NC \NR +\NC \type{size} \NC no \NC yes \NC yes \NC number \NC the required scaling (by default the same as designsize) \NC \NR +\NC \type{skewchar} \NC no \NC no \NC yes \NC number \NC default: \TEX's \prm {skewchar} \NC \NR \NC \type{type} \NC yes \NC no \NC yes \NC string \NC basic type of this font \NC \NR \NC \type{format} \NC no \NC no \NC yes \NC string \NC disk format type \NC \NR \NC \type{embedding} \NC no \NC no \NC yes \NC string \NC \PDF\ inclusion \NC \NR \NC \type{filename} \NC no \NC no \NC yes \NC string \NC the name of the font on disk \NC \NR \NC \type{tounicode} \NC no \NC yes \NC yes \NC number \NC When this is set to~1 \LUATEX\ assumes per|-|glyph tounicode entries are present in the font. \NC \NR -\NC \type{stretch} \NC no \NC no \NC yes \NC number \NC the \quote {stretch} value from \type - {\expandglyphsinfont} \NC \NR -\NC \type{shrink} \NC no \NC no \NC yes \NC number \NC the \quote {shrink} value from \type - {\expandglyphsinfont} \NC \NR -\NC \type{step} \NC no \NC no \NC yes \NC number \NC the \quote {step} value from \type - {\expandglyphsinfont} \NC \NR +\NC \type{stretch} \NC no \NC no \NC yes \NC number \NC the \quote {stretch} value from \lpr {expandglyphsinfont} \NC \NR +\NC \type{shrink} \NC no \NC no \NC yes \NC number \NC the \quote {shrink} value from \lpr {expandglyphsinfont} \NC \NR +\NC \type{step} \NC no \NC no \NC yes \NC number \NC the \quote {step} value from \lpr {expandglyphsinfont} \NC \NR \NC \type{expansion_factor} \NC no \NC no \NC no \NC number \NC the actual expansion factor of an expanded font \NC \NR -\NC \type{attributes} \NC no \NC no \NC yes \NC string \NC the \type {\pdffontattr} \NC \NR +\NC \type{attributes} \NC no \NC no \NC yes \NC string \NC the \orm {pdffontattr} \NC \NR \NC \type{cache} \NC no \NC no \NC yes \NC string \NC This key controls caching of the \LUA\ table on the \TEX\ end where \type {yes} means: use a reference to the table that is passed to \LUATEX\ (this is the - default), and no \type {no} means: don't store the + default), and \type {no} means: don't store the table reference, don't cache any \LUA\ data for this font while \type {renew} means: don't store the table reference, but save a reference to the table that is - created at the first access to one of its fields in font. - Note: the saved reference is thread|-|local, so be - careful when you are using coroutines: an error will be - thrown if the table has been cached in one thread, but - you reference it from another thread. \NC \NR + created at the first access to one of its fields in the + font. \NC \NR \NC \type{nomath} \NC no \NC no \NC yes \NC boolean \NC This key allows a minor speedup for text fonts. If it is present and true, then \LUATEX\ will not check the character entries for math|-|specific keys. \NC \NR \NC \type{oldmath} \NC no \NC no \NC yes \NC boolean \NC This key flags a font as representing an old school \TEX\ math font and disables the \OPENTYPE\ code path. \NC \NR -\NC \type{slant} \NC no \NC no \NC yes \NC number \NC This has the same semantics as the \type {SlantFont} - operator in font map files. \NC \NR -\NC \type{extent} \NC no \NC no \NC yes \NC number \NC This has the same semantics as the \type {ExtendFont} - operator in font map files. \NC \NR +\NC \type{slant} \NC no \NC no \NC yes \NC number \NC This parameter will tilt the font and + does the same as \type {SlantFont} in the map file for + \TYPEONE\ fonts. \NC \NR +\NC \type{extend} \NC no \NC no \NC yes \NC number \NC This parameter will scale the font horizontally and + does the same as \type {ExtendFont} in the map file for + \TYPEONE\ fonts. \NC \NR +\NC \type{squeeze} \NC no \NC no \NC yes \NC number \NC This parameter will scale the font vertically and has + no equivalent in the map file. \NC \NR +\NC \type{width} \NC no \NC no \NC yes \NC number \NC The backend will inject \PDF\ operators that set the + penwidth. The value is (as usual in \TEX) divided by 1000. + It works with the \type {mode} file. \NC \NR +\NC \type{mode} \NC no \NC no \NC yes \NC number \NC The backend will inject \PDF\ operators that relate to the + drawing mode with 0~being a fill, 1~being an outline, + 2~both draw and fill and 3~no painting at all. \NC \NR +\LL \stoptabulate +The saved reference in the \type {cache} option is thread|-|local, so be careful +when you are using coroutines: an error will be thrown if the table has been +cached in one thread, but you reference it from another thread. + The key \type {name} is always required. The keys \type {stretch}, \type {shrink}, \type {step} only have meaning when used together: they can be used to -replace a post|-|loading \type {\expandglyphsinfont} command. The \type -{auto_expand} option is not supported in \LUATEX. In fact, the primitives -that create expanded or protruding copies are probably only useful when used with +replace a post|-|loading \lpr {expandglyphsinfont} command. The \type +{auto_expand} option is not supported in \LUATEX. In fact, the primitives that +create expanded or protruding copies are probably only useful when used with traditional fonts because all these extra \OPENTYPE\ properties are kept out of the picture. The \type {expansion_factor} is value that can be present inside a font in \type {font.fonts}. It is the actual expansion factor (a value between @@ -112,16 +123,14 @@ makes sure that the font's definition is written to the output file (\DVI\ or signalling the \quote {normal} direction for this font. There are sixteen possibilities: -\starttabulate[|c|c|c|c|] -\BC number \BC meaning \BC number \BC meaning \NC \NR -\NC \type{0} \NC \type{LT} \NC \type {8} \NC \type{TT} \NC \NR -\NC \type{1} \NC \type{LL} \NC \type {9} \NC \type{TL} \NC \NR -\NC \type{2} \NC \type{LB} \NC \type{10} \NC \type{TB} \NC \NR -\NC \type{3} \NC \type{LR} \NC \type{11} \NC \type{TR} \NC \NR -\NC \type{4} \NC \type{RT} \NC \type{12} \NC \type{BT} \NC \NR -\NC \type{5} \NC \type{RL} \NC \type{13} \NC \type{BL} \NC \NR -\NC \type{6} \NC \type{RB} \NC \type{14} \NC \type{BB} \NC \NR -\NC \type{7} \NC \type{RR} \NC \type{15} \NC \type{BR} \NC \NR +\starttabulate[|Tc|c|Tc|c|Tc|c|Tc|c|] +\DB \# \BC dir \BC \# \BC dir \BC \# \BC dir \BC \# \BC dir \NC \NR +\TB +\NC 0 \NC LT \NC 4 \NC RT \NC 8 \NC TT \NC 12 \NC BT \NC \NR +\NC 1 \NC LL \NC 5 \NC RL \NC 9 \NC TL \NC 13 \NC BL \NC \NR +\NC 2 \NC LB \NC 6 \NC RB \NC 10 \NC TB \NC 14 \NC BB \NC \NR +\NC 3 \NC LR \NC 7 \NC RR \NC 11 \NC TR \NC 15 \NC BR \NC \NR +\LL \stoptabulate These are \OMEGA|-|style direction abbreviations: the first character indicates @@ -138,7 +147,8 @@ gives a nicer user interface. The names and their internal remapping are: \starttabulate[|l|c|] -\BC name \BC remapping \NC \NR +\DB name \BC remapping \NC \NR +\TB \NC \type {slant} \NC 1 \NC \NR \NC \type {space} \NC 2 \NC \NR \NC \type {space_stretch} \NC 3 \NC \NR @@ -146,6 +156,7 @@ The names and their internal remapping are: \NC \type {x_height} \NC 5 \NC \NR \NC \type {quad} \NC 6 \NC \NR \NC \type {extra_space} \NC 7 \NC \NR +\LL \stoptabulate The keys \type {type}, \type {format}, \type {embedding}, \type {fullname} and @@ -159,37 +170,27 @@ virtual character whose ligatures and kerns are used to handle word boundary processing. \type {right_boundary} is similar but not actually used for anything (yet). -Other index keys are ignored. - Each character hash itself is a hash. For example, here is the character \quote -{f} (decimal 102) in the font \type {cmr10 at 10pt}: +{f} (decimal 102) in the font \type {cmr10 at 10pt}. The numbers that represent +dimensions are in scaled points. \starttyping [102] = { - ['width'] = 200250, - ['height'] = 455111, - ['depth'] = 0, - ['italic'] = 50973, - ['kerns'] = { + ["width"] = 200250, + ["height"] = 455111, + ["depth"] = 0, + ["italic"] = 50973, + ["kerns"] = { [63] = 50973, [93] = 50973, [39] = 50973, [33] = 50973, [41] = 50973 }, - ['ligatures'] = { - [102] = { - ['char'] = 11, - ['type'] = 0 - }, - [108] = { - ['char'] = 13, - ['type'] = 0 - }, - [105] = { - ['char'] = 12, - ['type'] = 0 - } + ["ligatures"] = { + [102] = { ["char"] = 11, ["type"] = 0 }, + [108] = { ["char"] = 13, ["type"] = 0 }, + [105] = { ["char"] = 12, ["type"] = 0 } } } \stoptyping @@ -197,16 +198,17 @@ Each character hash itself is a hash. For example, here is the character \quote The following top|-|level keys can be present inside a character hash: \starttabulate[|l|c|c|c|l|p|] -\BC key \BC vf \BC tfm \BC used \BC type \BC description \NC\NR +\DB key \BC vf \BC tfm \BC used \BC type \BC description \NC\NR +\TB \NC \type{width} \NC yes \NC yes \NC yes \NC number \NC character's width, in sp (default 0) \NC\NR \NC \type{height} \NC no \NC yes \NC yes \NC number \NC character's height, in sp (default 0) \NC\NR \NC \type{depth} \NC no \NC yes \NC yes \NC number \NC character's depth, in sp (default 0) \NC\NR \NC \type{italic} \NC no \NC yes \NC yes \NC number \NC character's italic correction, in sp (default zero) \NC\NR \NC \type{top_accent} \NC no \NC no \NC maybe \NC number \NC character's top accent alignment place, in sp (default zero) \NC\NR \NC \type{bot_accent} \NC no \NC no \NC maybe \NC number \NC character's bottom accent alignment place, in sp (default zero) \NC\NR -\NC \type{left_protruding} \NC no \NC no \NC maybe \NC number \NC character's \type {\lpcode} \NC\NR -\NC \type{right_protruding} \NC no \NC no \NC maybe \NC number \NC character's \type {\rpcode} \NC\NR -\NC \type{expansion_factor} \NC no \NC no \NC maybe \NC number \NC character's \type {\efcode} \NC\NR +\NC \type{left_protruding} \NC no \NC no \NC maybe \NC number \NC character's \lpr {lpcode} \NC\NR +\NC \type{right_protruding} \NC no \NC no \NC maybe \NC number \NC character's \lpr {rpcode} \NC\NR +\NC \type{expansion_factor} \NC no \NC no \NC maybe \NC number \NC character's \lpr {efcode} \NC\NR \NC \type{tounicode} \NC no \NC no \NC maybe \NC string \NC character's \UNICODE\ equivalent(s), in \UTF|-|16BE hexadecimal format \NC\NR \NC \type{next} \NC no \NC yes \NC yes \NC number \NC the \quote {next larger} character index \NC\NR \NC \type{extensible} \NC no \NC yes \NC yes \NC table \NC the constituent parts of an extensible recipe \NC\NR @@ -217,19 +219,17 @@ The following top|-|level keys can be present inside a character hash: \NC \type{commands} \NC yes \NC no \NC yes \NC array \NC virtual font commands \NC\NR \NC \type{name} \NC no \NC no \NC no \NC string \NC the character (\POSTSCRIPT) name \NC\NR \NC \type{index} \NC no \NC no \NC yes \NC number \NC the (\OPENTYPE\ or \TRUETYPE) font glyph index \NC\NR -\NC \type{used} \NC no \NC yes \NC yes \NC boolean \NC typeset already (default: false)? \NC\NR +\NC \type{used} \NC no \NC yes \NC yes \NC boolean \NC typeset already (default: false) \NC\NR \NC \type{mathkern} \NC no \NC no \NC yes \NC table \NC math cut-in specifications \NC\NR +\LL \stoptabulate The values of \type {top_accent}, \type {bot_accent} and \type {mathkern} are -used only for math accent and superscript placement, see the \at {math chapter} -[math] in this manual for details. - -The values of \type {left_protruding} and \type {right_protruding} are used only -when \type {\protrudechars} is non-zero. - -Whether or not \type {expansion_factor} is used depends on the font's global -expansion settings, as well as on the value of \type {\adjustspacing}. +used only for math accent and superscript placement, see \at {page} [math] in +this manual for details. The values of \type {left_protruding} and \type +{right_protruding} are used only when \lpr {protrudechars} is non-zero. Whether +or not \type {expansion_factor} is used depends on the font's global expansion +settings, as well as on the value of \lpr {adjustspacing}. The usage of \type {tounicode} is this: if this font specifies a \type {tounicode=1} at the top level, then \LUATEX\ will construct a \type {/ToUnicode} @@ -245,24 +245,28 @@ enclosing angle brackets. For instance the \type {tounicode} for a \type {fi} ligature would be \type {00660069}. When you pass a number the conversion will be done for you. -The presence of \type {extensible} will overrule \type {next}, if that is also -present. It in in turn can be overruled by \type {vert_variants}. - -The \type {extensible} table is very simple: +A math character can have a \type {next} field that points to a next larger +shape. However, the presence of \type {extensible} will overrule \type {next}, if +that is also present. The \type {extensible} field in turn can be overruled by +\type {vert_variants}, the \OPENTYPE\ version. The \type {extensible} table is +very simple: \starttabulate[|l|l|p|] -\BC key \BC type \BC description \NC\NR +\DB key \BC type \BC description \NC\NR +\TB \NC \type{top} \NC number \NC top character index \NC\NR \NC \type{mid} \NC number \NC middle character index \NC\NR \NC \type{bot} \NC number \NC bottom character index \NC\NR \NC \type{rep} \NC number \NC repeatable character index \NC\NR +\LL \stoptabulate The \type {horiz_variants} and \type {vert_variants} are arrays of components. Each of those components is itself a hash of up to five keys: \starttabulate[|l|l|p|] -\BC key \BC type \BC explanation \NC \NR +\DB key \BC type \BC explanation \NC \NR +\TB \NC \type{glyph} \NC number \NC The character index. Note that this is an encoding number, not a name. \NC \NR \NC \type{extender} \NC number \NC One (1) if this part is repeatable, zero (0) otherwise. \NC \NR \NC \type{start} \NC number \NC The maximum overlap at the starting side (in scaled points). \NC \NR @@ -270,11 +274,12 @@ Each of those components is itself a hash of up to five keys: \NC \type{advance} \NC number \NC The total advance width of this item. It can be zero or missing, then the natural size of the glyph for character \type {component} is used. \NC \NR +\LL \stoptabulate The \type {kerns} table is a hash indexed by character index (and \quote {character index} is defined as either a non|-|negative integer or the string -value \type {right_boundary}), with the values the kerning to be applied, in +value \type {right_boundary}), with the values of the kerning to be applied, in scaled points. The \type {ligatures} table is a hash indexed by character index (and \quote @@ -283,23 +288,25 @@ value \type {right_boundary}), with the values being yet another small hash, wit two fields: \starttabulate[|l|l|p|] -\BC key \BC type \BC description \NC \NR +\DB key \BC type \BC description \NC \NR +\TB \NC \type{type} \NC number \NC the type of this ligature command, default 0 \NC \NR \NC \type{char} \NC number \NC the character index of the resultant ligature \NC \NR +\LL \stoptabulate -The \type {char} field in a ligature is required. - -The \type {type} field inside a ligature is the numerical or string value of one -of the eight possible ligature types supported by \TEX. When \TEX\ inserts a new -ligature, it puts the new glyph in the middle of the left and right glyphs. The -original left and right glyphs can optionally be retained, and when at least one -of them is kept, it is also possible to move the new \quote {insertion point} -forward one or two places. The glyph that ends up to the right of the insertion -point will become the next \quote {left}. +The \type {char} field in a ligature is required. The \type {type} field inside a +ligature is the numerical or string value of one of the eight possible ligature +types supported by \TEX. When \TEX\ inserts a new ligature, it puts the new glyph +in the middle of the left and right glyphs. The original left and right glyphs +can optionally be retained, and when at least one of them is kept, it is also +possible to move the new \quote {insertion point} forward one or two places. The +glyph that ends up to the right of the insertion point will become the next +\quote {left}. \starttabulate[|l|c|l|l|] -\BC textual (Knuth) \BC number \BC string \BC result \NC\NR +\DB textual (Knuth) \BC number \BC string \BC result \NC\NR +\TB \NC \type{l + r =: n} \NC 0 \NC \type{=:} \NC \type{|n} \NC\NR \NC \type{l + r =:| n} \NC 1 \NC \type{=:|} \NC \type{|nr} \NC\NR \NC \type{l + r |=: n} \NC 2 \NC \type{|=:} \NC \type{|ln} \NC\NR @@ -308,6 +315,7 @@ point will become the next \quote {left}. \NC \type{l + r |=:> n} \NC 6 \NC \type{|=:>} \NC \type{l|n} \NC\NR \NC \type{l + r |=:|> n} \NC 7 \NC \type{|=:|>} \NC \type{l|nr} \NC\NR \NC \type{l + r |=:|>> n} \NC 11 \NC \type{|=:|>>} \NC \type{ln|r} \NC\NR +\LL \stoptabulate The default value is~0, and can be left out. That signifies a \quote {normal} @@ -316,7 +324,12 @@ indicates the final insertion point. The \type {commands} array is explained below. -\section {Real fonts} +\stopsection + +\startsection[title={Real fonts}] + +\topicindex {fonts+real} +\topicindex {fonts+virtual} Whether or not a \TEX\ font is a \quote {real} font that should be written to the \PDF\ document is decided by the \type {type} value in the top|-|level font @@ -325,141 +338,150 @@ inclusion mechanism will attempt to add the needed font object definitions to th \PDF. Values for \type {type} are: \starttabulate[|l|p|] -\BC value \BC description \NC\NR +\DB value \BC description \NC\NR +\TB \NC \type{real} \NC this is a base font \NC\NR \NC \type{virtual} \NC this is a virtual font \NC\NR +\LL \stoptabulate The actions to be taken depend on a number of different variables: \startitemize[packed] \startitem - Whether the used font fits in an 8-bit encoding scheme or not. + Whether the used font fits in an 8-bit encoding scheme or not. This is true for + traditional \TEX\ fonts that communicate via \TFM\ files. \stopitem \startitem - The type of the disk font file. + The type of the disk font file, for instance a bitmap file or an outline + \TYPEONE, \TRUETYPE\ or \OPENTYPE\ font. \stopitem \startitem - The level of embedding requested. + The level of embedding requested, although in most cases a subset of + characters is embedded. The times when nothing got embedded are (in our + opinion at least) basically gone. \stopitem \stopitemize A font that uses anything other than an 8-bit encoding vector has to be written -to the \PDF\ in a different way. - -The rule is: if the font table has \type {encodingbytes} set to~2, then this is a -wide font, in all other cases it isn't. The value~2 is the default for \OPENTYPE\ -and \TRUETYPE\ fonts loaded via \LUA. For \TYPEONE\ fonts, you have to set \type -{encodingbytes} to~2 explicitly. For \PK\ bitmap fonts, wide font encoding is not -supported at all. - -If no special care is needed, \LUATEX\ currently falls back to the -mapfile|-|based solution used by \PDFTEX\ and \DVIPS. This behaviour might -silently be removed in the future, in which case the related primitives and \LUA\ -functions will become no|-|ops. - -If a \quote {wide} font is used, the new subsystem kicks in, and some -extra fields have to be present in the font structure. In this case, \LUATEX\ -does not use a map file at all. - -The extra fields are: \type {format}, \type {embedding}, \type {fullname}, \type -{cidinfo} (as explained above), \type {filename}, and the \type {index} key in -the separate characters. - -Values for \type {format} are: +to the \PDF\ in a different way. When the font table has \type {encodingbytes} +set to~2, then it is a wide font, in all other cases it isn't. The value~2 is the +default for \OPENTYPE\ and \TRUETYPE\ fonts loaded via \LUA. For \TYPEONE\ fonts, +you have to set \type {encodingbytes} to~2 explicitly. For \PK\ bitmap fonts, +wide font encoding is not supported at all. + +If no special care is needed, \LUATEX\ falls back to the mapfile|-|based solution +used by \PDFTEX\ and \DVIPS, so that legacy fonts are supported transparently. If +a \quote {wide} font is used, the new subsystem kicks in, and some extra fields +have to be present in the font structure. In this case, \LUATEX\ does not use a +map file at all. These extra fields are: \type {format}, \type {embedding}, \type +{fullname}, \type {cidinfo} (as explained above), \type {filename}, and the \type +{index} key in the separate characters. + +The \type {format} variable can have the following values. \type {type3} fonts +are provided for backward compatibility only, and do not support the new wide +encoding options. \starttabulate[|l|p|] -\BC value \BC description \NC \NR +\DB value \BC description \NC \NR +\TB \NC \type{type1} \NC this is a \POSTSCRIPT\ \TYPEONE\ font \NC \NR \NC \type{type3} \NC this is a bitmapped (\PK) font \NC \NR \NC \type{truetype} \NC this is a \TRUETYPE\ or \TRUETYPE|-|based \OPENTYPE\ font \NC \NR \NC \type{opentype} \NC this is a \POSTSCRIPT|-|based \OPENTYPE\ font \NC \NR +\LL \stoptabulate -\type {type3} fonts are provided for backward compatibility only, and do not -support the new wide encoding options. - -Values for \type {embedding} are: +Valid values for the \type {embedding} variable are: \starttabulate[|l|p|] -\BC value \BC description \NC \NR +\DB value \BC description \NC \NR +\TB \NC \type{no} \NC don't embed the font at all \NC \NR \NC \type{subset} \NC include and atttempt to subset the font \NC \NR \NC \type{full} \NC include this font in its entirety \NC \NR +\LL \stoptabulate -The other fields are used as follows: The \type {fullname} will be the +The other fields are used as follows. The \type {fullname} will be the \POSTSCRIPT|/|\PDF\ font name. The \type {cidinfo} will be used as the character -set (the CID \type {/Ordering} and \type {/Registry} keys). The \type {filename} +set: the CID \type {/Ordering} and \type {/Registry} keys. The \type {filename} points to the actual font file. If you include the full path in the \type {filename} or if the file is in the local directory, \LUATEX\ will run a little -bit more efficient because it will not have to re|-|run the \type {find_xxx_file} +bit more efficient because it will not have to re|-|run the \type {find_*_file} callback in that case. Be careful: when mixing old and new fonts in one document, it is possible to create \POSTSCRIPT\ name clashes that can result in printing errors. When this -happens, you have to change the \type {fullname} of the font. +happens, you have to change the \type {fullname} of the font to a more unique +one. Typeset strings are written out in a wide format using 2~bytes per glyph, using the \type {index} key in the character information as value. The overall effect is like having an encoding based on numbers instead of traditional (\POSTSCRIPT) -name|-|based reencoding. The way to get the correct \type {index} numbers for -\TYPEONE\ fonts is by loading the font via \type {fontloader.open} and use the table -indices as \type {index} fields. +name|-|based reencoding. One way to get the correct \type {index} numbers for +\TYPEONE\ fonts is by loading the font via \type {fontloader.open} and use the +table indices as \type {index} fields. In order to make sure that cut and paste of the final document works okay you can -best make sure that there is a \type {tounicode} vector enforced. +best make sure that there is a \type {tounicode} vector enforced. Not all \PDF\ +viewers handle this right so take \ACROBAT\ as reference. + +\stopsection -\section[virtualfonts]{Virtual fonts} +\startsection[reference=virtualfonts,title={Virtual fonts}] \subsection{The structure} +\topicindex {fonts+virtual} + You have to take the following steps if you want \LUATEX\ to treat the returned -table from \type {define_font} as a virtual font: +table from \cbk {define_font} as a virtual font: \startitemize[packed] \startitem - Set the top|-|level key \type {type} to \type {virtual}. + Set the top|-|level key \type {type} to \type {virtual}. In most cases it's + optional because we look at the \type {commands} entry anyway. \stopitem \startitem - Make sure there is at least one valid entry in \type {fonts} (see below). + Make sure there is at least one valid entry in \type {fonts} (see below), + although recent versions of \LUATEX\ add a default entry when this table is + missing. \stopitem \startitem - Give a \type {commands} array to every character (see below). + Add a \type {commands} array to those characters that matter. A virtual + character can itself point to virtual characters but be careful with nesting + as you can create loops and overflow the stack (which often indicates an + error anyway). \stopitem \stopitemize The presence of the toplevel \type {type} key with the specific value \type {virtual} will trigger handling of the rest of the special virtual font fields in the table, but the mere existence of 'type' is enough to prevent \LUATEX\ from -looking for a virtual font on its own. - -Therefore, this also works \quote {in reverse}: if you are absolutely certain -that a font is not a virtual font, assigning the value \type {base} or \type -{real} to \type {type} will inhibit \LUATEX\ from looking for a virtual font -file, thereby saving you a disk search. +looking for a virtual font on its own. This also works \quote {in reverse}: if +you are absolutely certain that a font is not a virtual font, assigning the value +\type {real} to \type {type} will inhibit \LUATEX\ from looking for a virtual +font file, thereby saving you a disk search. This only matters when we load a +\TFM\ file. -The \type {fonts} is another \LUA\ array. The values are one- or two|-|key +The \type {fonts} is an (indexed) \LUA\ table. The values are one- or two|-|key hashes themselves, each entry indicating one of the base fonts in a virtual font. In case your font is referring to itself, you can use the \type {font.nextid()} function which returns the index of the next to be defined font which is probably -the currently defined one. - -An example makes this easy to understand +the currently defined one. So, a table looks like this: \starttyping fonts = { - { name = 'ptmr8a', size = 655360 }, - { name = 'psyr', size = 600000 }, - { id = 38 } + { name = "ptmr8a", size = 655360 }, + { name = "psyr", size = 600000 }, + { id = 38 } } \stoptyping -says that the first referenced font (index 1) in this virtual font is \type -{ptrmr8a} loaded at 10pt, and the second is \type {psyr} loaded at a little over -9pt. The third one is previously defined font that is known to \LUATEX\ as font id -\quote {38}. - +The first referenced font (at index~1) in this virtual font is \type {ptrmr8a} +loaded at 10pt, and the second is \type {psyr} loaded at a little over 9pt. The +third one is a previously defined font that is known to \LUATEX\ as font id~38. The array index numbers are used by the character command definitions that are part of each character. @@ -468,7 +490,8 @@ with the first entry representing a command and the extra items being the parameters to that command. The allowed commands and their arguments are: \starttabulate[|l|l|l|p|] -\BC command name \BC arguments \BC type \BC description \NC \NR +\DB command \BC arguments \BC type \BC description \NC \NR +\TB \NC \type{font} \NC 1 \NC number \NC select a new font from the local \type {fonts} table \NC \NR \NC \type{char} \NC 1 \NC number \NC typeset this character number from the current font, and move right by the character's width \NC \NR @@ -481,18 +504,20 @@ parameters to that command. The allowed commands and their arguments are: \NC \type{rule} \NC 2 \NC 2 numbers \NC output a rule $ht*wd$, and move right. \NC \NR \NC \type{down} \NC 1 \NC number \NC move down on the page \NC \NR \NC \type{right} \NC 1 \NC number \NC move right on the page \NC \NR -\NC \type{special} \NC 1 \NC string \NC output a \type {\special} command \NC \NR +\NC \type{special} \NC 1 \NC string \NC output a \prm {special} command \NC \NR \NC \type{pdf} \NC 2 \NC 2 strings \NC output a \PDF\ literal, the first string is one of \type {origin}, \type {page}, \type {text}, \type {font}, \type {direct} or \type {raw}; if you have one string only \type {origin} is assumed \NC \NR -\NC \type{lua} \NC 1 \NC string \NC execute a \LUA\ script (at \type {\latelua} time) \NC \NR -\NC \type{image} \NC 1 \NC image \NC output an image (the argument can be either an \type - {<image>} variable or an \type {image_spec} table) \NC \NR +\NC \type{lua} \NC 1 \NC string, + function \NC execute a \LUA\ script when the glyph is embedded; in case of a + function it gets the font id and character code passed \NC \NR +\NC \type{image} \NC 1 \NC image \NC output an image (the argument can be either an \type {<image>} variable or an \type {image_spec} table) \NC \NR \NC \type{comment} \NC any \NC any \NC the arguments of this command are ignored \NC \NR +\LL \stoptabulate When a font id is set to~0 then it will be replaced by the currently assigned -font id. This prevents the need for hackery with future id's (normally one could +font id. This prevents the need for hackery with future id's. Normally one could use \type {font.nextid} but when more complex fonts are built in the meantime other instances could have been loaded. @@ -500,24 +525,24 @@ The \type {pdf} option also accepts a \type {mode} keyword in which case the third argument sets the mode. That option will change the mode in an efficient way (passing an empty string would result in an extra empty lines in the \PDF\ file. This option only makes sense for virtual fonts. The \type {font} mode only -makes sense in virtual fonts. - -These modes are somewhat fuzzy and partially inherited from \PDFTEX. +makes sense in virtual fonts. Modes are somewhat fuzzy and partially inherited +from \PDFTEX. \starttabulate[|l|p|] -\BC mode \BC description \NC \NR +\DB mode \BC description \NC \NR +\TB \NC \type {origin} \NC enter page mode and set the position \NC \NR \NC \type {page} \NC enter page mode \NC \NR \NC \type {text} \NC enter text mode \NC \NR \NC \type {font} \NC enter font mode (kind of text mode, only in virtual fonts) \NC \NR \NC \type {always} \NC finish the current string and force a transform if needed \NC \NR \NC \type {raw} \NC finish the current string \NC \NR +\LL \stoptabulate -You always need to check what \PDF\ code is generated because there can be all kind of -interferences with optimizations in the backend and fonts are complicated anyway. - -Here is a rather elaborate glyph commands example: +You always need to check what \PDF\ code is generated because there can be all +kind of interferences with optimization in the backend and fonts are complicated +anyway. Here is a rather elaborate glyph commands example using such keys: \starttyping ... @@ -526,6 +551,7 @@ commands = { { "right", 5000 }, -- move right about 0.08pt { "font", 3 }, -- select the fonts[3] entry { "char", 97 }, -- place character 97 (ASCII 'a') + -- { "slot", 2, 97 }, -- an alternative for the previous two { "pop" }, -- go all the way back { "down", -200000 }, -- move upwards by about 3pt { "special", "pdf: 1 0 0 rg" } -- switch to red color @@ -539,7 +565,7 @@ commands = { The default value for \type {font} is always~1 at the start of the \type {commands} array. Therefore, if the virtual font is essentially only a -re|-|encoding, then you do usually not have create an explicit \quote {font} +re|-|encoding, then you do usually not have created an explicit \quote {font} command in the array. Rules inside of \type {commands} arrays are built up using only two dimensions: @@ -572,6 +598,8 @@ characters that are already present cannot be altered). \subsection{Example virtual font} +\topicindex {fonts+virtual} + Finally, here is a plain \TEX\ input file with a virtual font demonstration: \startbuffer @@ -579,58 +607,65 @@ Finally, here is a plain \TEX\ input file with a virtual font demonstration: callback.register('define_font', function (name,size) if name == 'cmr10-red' then - f = font.read_tfm('cmr10',size) - f.name = 'cmr10-red' - f.type = 'virtual' - f.fonts = {{ name = 'cmr10', size = size }} + local f = font.read_tfm('cmr10',size) + f.name = 'cmr10-red' + f.type = 'virtual' + f.fonts = { + { name = 'cmr10', size = size } + } for i,v in pairs(f.characters) do - if (string.char(i)):find('[tacohanshartmut]') then + if string.char(i):find('[tacohanshartmut]') then v.commands = { - {'special','pdf: 1 0 0 rg'}, - {'char',i}, - {'special','pdf: 0 g'}, + { "special", "pdf: 1 0 0 rg" }, + { "char", i }, + { "special", "pdf: 0 g" }, } - else - v.commands = {{'char',i}} end end + return f else - f = font.read_tfm(name,size) + return font.read_tfm(name,size) end - return f end ) } -\font\myfont = cmr10-red at 10pt \myfont This is a line of text \par -\font\myfontx= cmr10 at 10pt \myfontx Here is another line of text \par +\font\myfont = cmr10-red at 10pt \myfont This is a line of text \par +\font\myfontx = cmr10 at 10pt \myfontx Here is another line of text \par \stopbuffer \typebuffer -\section{The \type {vf} library} +\stopsection + +\startsection[title={The \type {vf} library}] The \type {vf} library can be used when \LUA\ code, as defined in the \type {commands} of the font, is executed. The functions provided are similar as the commands: \type {char}, \type {down}, \type {fontid}, \type {image}, \type -{node}, \type {nop}, \type {pop}, \type {push}, \type {right}, \type {rule}, -\type {special} and \type {pdf}. This library has been present for a while but -not been advertised and tested much, if only because it's easy to define an -invalid font (or mess up the \PDF\ stream). Keep in mind that the \LUA\ snippets -are executed each time when a character is output. +{node}, \type {nop}, \type {pop}, \type {push}, \type {right}, \nod {rule}, \type +{special} and \type {pdf}. This library has been present for a while but not been +advertised and tested much, if only because it's easy to define an invalid font +(or mess up the \PDF\ stream). Keep in mind that the \LUA\ snippets are executed +each time when a character is output. + +\stopsection +\startsection[title={The \type {font} library}] -\section{The \type {font} library} +\topicindex {fonts+library} The font library provides the interface into the internals of the font system, -and also it contains helper functions to load traditional \TEX\ font metrics +and it also contains helper functions to load traditional \TEX\ font metrics formats. Other font loading functionality is provided by the \type {fontloader} library that will be discussed in the next section. \subsection{Loading a \TFM\ file} -The behavior documented in this subsection is considered stable in the sense that -there will not be backward-incompatible changes any more. +\topicindex {fonts+tfm} + +The behaviour documented in this subsection is considered stable in the sense that +there will not be backward|-|incompatible changes any more. \startfunctioncall <table> fnt = @@ -649,11 +684,10 @@ The number is a bit special: \stopitem \stopitemize -The internal structure of the metrics font table that is returned is explained in -\in {chapter} [fonts]. - \subsection{Loading a \VF\ file} +\topicindex {fonts+vf} + The behavior documented in this subsection is considered stable in the sense that there will not be backward-incompatible changes any more. @@ -663,10 +697,12 @@ there will not be backward-incompatible changes any more. \stopfunctioncall The meaning of the number \type {s} and the format of the returned table are -similar to the ones in the \type {read_tfm()} function. +similar to the ones in the \type {read_tfm} function. \subsection{The fonts array} +\topicindex {fonts+virtual} + The whole table of \TEX\ fonts is accessible from \LUA\ using a virtual array. \starttyping @@ -674,9 +710,8 @@ font.fonts[n] = { ... } <table> f = font.fonts[n] \stoptyping -See \in {chapter} [fonts] for the structure of the tables. Because this is a -virtual array, you cannot call \type {pairs} on it, but see below for the \type -{font.each} iterator. +Because this is a virtual array, you cannot call \type {pairs} on it, but see +below for the \type {font.each} iterator. The two metatable functions implementing the virtual array are: @@ -686,9 +721,15 @@ font.setfont(<number> n, <table> f) \stopfunctioncall Note that at the moment, each access to the \type {font.fonts} or call to \type -{font.getfont} creates a \LUA\ table for the whole font. This process can be quite -slow. In a later version of \LUATEX, this interface will change (it will start -using userdata objects instead of actual tables). +{font.getfont} creates a \LUA\ table for the whole font unless you cached it. +This process can be quite slow. + +\startfunctioncall +<table> p = font.getparameters(<number> n) +\stopfunctioncall + +This one will return a table of the parameters as known to \TEX. These can be +different from the ones in the cached table. Also note the following: assignments can only be made to fonts that have already been defined in \TEX, but have not been accessed {\it at all\/} since that @@ -709,6 +750,8 @@ changed) or \type {nil} (not a valid font at all). \subsection{Defining a font directly} +\topicindex {fonts+define} + You can define your own font into \type {font.fonts} by calling this function: \startfunctioncall @@ -717,9 +760,8 @@ You can define your own font into \type {font.fonts} by calling this function: \stopfunctioncall The return value is the internal id number of the defined font (the index into -\type {font.fonts}). If the font creation fails, an error is raised. The table -is a font structure, as explained in \in {chapter} [fonts]. An alternative call -is: +\type {font.fonts}). If the font creation fails, an error is raised. The table is +a font structure. An alternative call is: \startfunctioncall <number> i = @@ -730,6 +772,8 @@ Where the first argument is a reserved font id (see below). \subsection{Extending a font} +\topicindex {fonts+extend} + Within reasonable bounds you can extend a font after it has been defined. Because some properties are best left unchanged this is limited to adding characters. @@ -747,6 +791,8 @@ a more drastic replacer.) \subsection{Projected next font id} +\topicindex {fonts+id} + \startfunctioncall <number> i = font.nextid() @@ -763,36 +809,37 @@ This can be handy when you create complex virtual fonts. font.nextid(true) \stopfunctioncall -\subsection{Font id} +\subsection{Font ids} + +\topicindex {fonts+id} +\topicindex {fonts+current} \startfunctioncall <number> i = font.id(<string> csname) \stopfunctioncall -This returns the font id associated with \type {csname} string, or $-1$ if \type +This returns the font id associated with \type {csname}, or $-1$ if \type {csname} is not defined. -\subsection{Currently active font} - \startfunctioncall -<number> i = font.current() -font.current(<number> i) +<number> i = + font.max() \stopfunctioncall -This gets or sets the currently used font number. - -\subsection{Maximum font id} +This is the largest used index in \type {font.fonts}. \startfunctioncall -<number> i = - font.max() +<number> i = font.current() +font.current(<number> i) \stopfunctioncall -This is the largest used index in \type {font.fonts}. +This gets or sets the currently used font number. \subsection{Iterating over all fonts} +\topicindex {fonts+iterate} + \startfunctioncall for i,v in font.each() do ... @@ -804,6 +851,8 @@ value is the index in \type {font.fonts}, the second the font itself, as a \LUA\ table. The indices are listed incrementally, but they do not always form an array of consecutive numbers: in some cases there can be holes in the sequence. +\stopsection + \stopchapter \stopcomponent diff --git a/doc/context/sources/general/manuals/luatex/luatex-introduction.tex b/doc/context/sources/general/manuals/luatex/luatex-introduction.tex index d0899147d..5ed80f4c1 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-introduction.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-introduction.tex @@ -1,7 +1,6 @@ % language=uk \environment luatex-style -\environment luatex-logos \startcomponent luatex-introduction @@ -15,9 +14,9 @@ Additional reference material is published in journals of user groups and It took about a decade to reach stable version 1.0, but for good reason. Successive versions brought new functionality, more control, some cleanup of -internals and experimental features evolved into stable ones or were dropped. +internals. Experimental features evolved into stable ones or were dropped. Already quite early \LUATEX\ could be used for production and it was used on a -daily basis by the authors. Successive versions sometimes demanded a adaption to +daily basis by the authors. Successive versions sometimes demanded an adaption to the \LUA\ interfacing, but the concepts were unchanged. The current version can be considered stable in functionality and there will be no fundamental changes. Of course we then can decide to move towards version 2.00 with different @@ -26,9 +25,10 @@ properties. Don't expect \LUATEX\ to behave the same as \PDFTEX ! Although the core functionality of that 8 bit engine was starting point, it has been combined with the directional support of \OMEGA\ (\ALEPH). But, \LUATEX\ can behave different -due to its wide (32 bit) characters, many registers and large memory support. -There is native \UTF\ input, support for large (more that 8 bit) fonts, and the -math machinery is tuned for \OPENTYPE\ math. There is support for directional +due to its wide (32 bit) characters, many registers and large memory support. The +\PDF\ code produced differs from \PDFTEX\ but users will normally not notice +that. There is native \UTF\ input, support for large (more than 8 bit) fonts, and +the math machinery is tuned for \OPENTYPE\ math. There is support for directional typesetting too. The log output can differ from other engines and will likely differ more as we move forward. When you run plain \TEX\ for sure \LUATEX\ runs slower than \PDFTEX\ but when you run for instance \CONTEXT\ \MKIV\ in many cases @@ -47,27 +47,29 @@ The organization of the source code is adapted so that it can glue all these components together. We continue cleaning up side effects of the accumulated code in \TEX\ engines (especially code that is not needed any longer). -\startitemize[packed] +\startitemize [unpacked] \startitem - Most of \PDFTEX\ version 1.40.9, converted to \CCODE. Some experimental - features have been removed and some utility macros are not inherited as - their functionality can be done in \LUA. The number of backend interface - commands has been reduced to a few. The extensions are separated from the - core (which we keep close to the original \TEX\ core). Some mechanisms - like expansion and protrusion can behave different from the original due - to some cleanup and optimization. Some whatsit based functionality (image - support and reusable content) is now core functionality. + We started out with most of \PDFTEX\ version 1.40.9. The code base was + converted to \CCODE\ and split in modules. Experimental features were + removed and utility macros are not inherited because their functionality + can be programmed in \LUA. The number of backend interface commands has + been reduced to a few. The so called extensions are separated from the + core (which we try to keep close to the original \TEX\ core). Some + mechanisms like expansion and protrusion can behave different from the + original due to some cleanup and optimization. Some whatsit based + functionality (image support and reusable content) is now core + functionality. We don't stay in sync with \PDFTEX\ development. \stopitem \startitem - The direction model and some other bits from \ALEPH\ RC4 (derived from - \OMEGA) is included. The related primitives are part of core \LUATEX\ but - at the node level directional support is no longer based on so called - whatsits but on real nodes. In fact, whatsits are now only used for - backend specific extensions. + The direction model from \ALEPH\ RC4 (which is derived from \OMEGA) is + included. The related primitives are part of core \LUATEX\ but at the + node level directional support is no longer based on so called whatsits + but on real nodes with relevant properties. The number of directions is + limited to the useful set and the backend has been made direction aware. \stopitem \startitem Neither \ALEPH's I/O translation processes, nor tcx files, nor \ENCTEX\ - can be used, these encoding|-|related functions are superseded by a + are available. These encoding|-|related functions are superseded by a \LUA|-|based solution (reader callbacks). In a similar fashion all file \IO\ can be intercepted. \stopitem @@ -80,14 +82,18 @@ code in \TEX\ engines (especially code that is not needed any longer). \startitem There are various \TEX\ extensions but only those that cannot be done using the \LUA\ interfaces. The math machinery often has two code paths: - one traditional and the other more suitable for wide \OPENTYPE\ fonts. + one traditional and the other more suitable for wide \OPENTYPE\ fonts. Here + we follow the \MICROSOFT\ specifications as much as possible. Some math + functionality has been opened up a bit so that users have more control. \stopitem \startitem The fontloader uses parts of \FONTFORGE\ 2008.11.17 combined with additional code specific for usage in a \TEX\ engine. We try to minimize specific font support to what \TEX\ needs: character references and dimensions and delegate everything else to \LUA. That way we keep \TEX\ - open for extensions without touching the core. + open for extensions without touching the core. In order to minimize + dependencies at some point we may decide to make this an optional + library. \stopitem \startitem The \METAPOST\ library is integral part of \LUATEX. This gives \TEX\ some @@ -95,33 +101,71 @@ code in \TEX\ engines (especially code that is not needed any longer). Again \LUA\ is used as glue between the frontend and backend. Further development of \METAPOST\ is closely related to \LUATEX. \stopitem + \startitem + The virtual font technology that comes with \TEX\ has been integrated + into the font machinery in a way that permits creating virtual fonts + at runtime. Because \LUATEX\ can also act as a \LUA\ interpreter this + means that a complete \TEX\ workflow can be built without the need for + additional programs. + \stopitem + \startitem + The versions starting from 1.09 no longer use the poppler library for + inclusion but a lightweight dedicated one. This removes a dependency but + also makes the inclusion code of \LUATEX\ different from \PDFTEX. In fact + it was already much different due to the \LUA\ image interfacing. + \stopitem \stopitemize We try to keep upcoming versions compatible but intermediate releases can contain -experimental features. A general rule is that versions that end up on \TEX live -and|/|or are released around \CONTEXT\ meetings are stable. Future versions will -probably become a bit leaner and meaner. Some libraries might become external as -we don't want to bloat the binary and also don't want to add more hard coded -solutions. After all, with \LUA\ you can extend the core functionality. The less -dependencies, the better. - -The \TEXLIVE\ version is to be considered the current stable version. Any version -between the yearly \TEXLIVE\ releases are to be considered beta. The beta -releases are normally available via the \CONTEXT\ distribution channels (the -garden and so called minimals). - -\blank[1*big] - -Hans Hagen, Harmut Henkel, \crlf -Taco Hoekwater \& Luigi Scarso +experimental features. A general rule is that versions that end up on \TEXLIVE\ +and|/|or are released around \CONTEXT\ meetings are stable. Any version between +the yearly \TEXLIVE\ releases are to be considered beta and in the repository end +up as trunk releases. We have an experimental branch that we use for development +but there is no support for any of its experimental features. Intermediate +releases (from trunk) are normally available via the \CONTEXT\ distribution +channels (the garden and so called minimals). + +Version 1.10 is more or less an endpoint in development: this is what you get. +Because not only \CONTEXT, that we can adapt rather easily, uses \LUATEX, we +cannot change fundamentals without unforeseen consequences. By now it has been +proven that \LUA\ can be used to extend the core functionality so there is no +need to add more, and definitely no hard coded solutions for (not so) common +problems. Of course there will be bug fixes, maybe some optimization, and there +might even be some additions or non|-|intrusive improvements, but only after +testing outside the stable release. After all, the binary is already more than +large enough and there is not that much to gain. + +You might find \LUA\ helpers that are not yet documented. These are considered +experimental, although when you encounter them in a \CONTEXT\ version that has +been around for a while you can assume that they will stay. Of course it can just +be that we forgot to document them yet. + +A manual like this is not really meant as tutorial, for that we refer to +documents that ship with \CONTEXT, articles, etc. It is also never complete +enough for all readers. We try to keep up but the reader needs to realize that +it's all volunteer work done in spare time. And for sure, complaining about a bad +manual or crappy documentation will not really motivate us to spend more time on +it. That being said, we hope that this document is useful. + +\blank[big] + +\testpage[5] + +\startlines +Hans Hagen +Harmut Henkel +Taco Hoekwater +Luigi Scarso +\stoplines \blank[3*big] -\starttabulate +\starttabulate[|||] \NC Version \EQ \currentdate \NC \NR -\NC \LUATEX \EQ version \cldcontext{status.luatex_version/100}, - revision \cldcontext{status.luatex_revision}, - number \cldcontext{environment.luatexversion} \NC \NR +\NC \LUATEX \EQ \cldcontext{LUATEXENGINE} % + \cldcontext{LUATEXVERSION} / % + \cldcontext{LUATEXFUNCTIONALITY} + \NC \NR \NC \CONTEXT \EQ MkIV \contextversion \NC \NR \stoptabulate diff --git a/doc/context/sources/general/manuals/luatex/luatex-languages.tex b/doc/context/sources/general/manuals/luatex/luatex-languages.tex index 365e87f26..d4a7bda60 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-languages.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-languages.tex @@ -1,19 +1,22 @@ % language=uk \environment luatex-style -\environment luatex-logos \startcomponent luatex-languages \startchapter[reference=languages,title={Languages, characters, fonts and glyphs}] +\startsection[title={Introduction}] + +\topicindex {languages} + \LUATEX's internal handling of the characters and glyphs that eventually become typeset is quite different from the way \TEX82 handles those same objects. The easiest way to explain the difference is to focus on unrestricted horizontal mode (i.e.\ paragraphs) and hyphenation first. Later on, it will be easy to deal with the differences that occur in horizontal and math modes. -In \TEX82, the characters you type are converted into \type {char_node} records +In \TEX82, the characters you type are converted into \type {char} node records when they are encountered by the main control loop. \TEX\ attaches and processes the font information while creating those records, so that the resulting \quote {horizontal list} contains the final forms of ligatures and implicit kerning. @@ -21,7 +24,7 @@ This packaging is needed because we may want to get the effective width of for instance a horizontal box. When it becomes necessary to hyphenate words in a paragraph, \TEX\ converts (one -word at time) the \type {char_node} records into a string by replacing ligatures +word at time) the \type {char} node records into a string by replacing ligatures with their components and ignoring the kerning. Then it runs the hyphenation algorithm on this string, and converts the hyphenated result back into a \quote {horizontal list} that is consecutively spliced back into the paragraph stream. @@ -29,14 +32,14 @@ Keep in mind that the paragraph may contain unboxed horizontal material, which then already contains ligatures and kerns and the words therein are part of the hyphenation process. -Those \type {char_node} records are somewhat misnamed, as they are glyph +Those \type {char} node records are somewhat misnamed, as they are glyph positions in specific fonts, and therefore not really \quote {characters} in the -linguistic sense. There is no language information inside the \type {char_node} +linguistic sense. There is no language information inside the \type {char} node records at all. Instead, language information is passed along using \type -{language whatsit} records inside the horizontal list. +{language whatsit} nodes inside the horizontal list. In \LUATEX, the situation is quite different. The characters you type are always -converted into \type {glyph_node} records with a special subtype to identify them +converted into \nod {glyph} node records with a special subtype to identify them as being intended as linguistic characters. \LUATEX\ stores the needed language information in those records, but does not do any font|-|related processing at the time of node creation. It only stores the index of the current font and a @@ -48,93 +51,83 @@ font information in the whole list (creating ligatures and adjusting kerning), and finally it adjusts all the subtype identifiers so that the records are \quote {glyph nodes} from now on. -\section[charsandglyphs]{Characters and glyphs} +\stopsection + +\startsection[title={Characters, glyphs and discretionaries},reference=charsandglyphs] + +\topicindex {characters} +\topicindex {glyphs} +\topicindex {hyphenation} -\TEX82 (including \PDFTEX) differentiates between \type {char_node}s and \type -{lig_node}s. The former are simple items that contained nothing but a \quote +\TEX82 (including \PDFTEX) differentiates between \type {char} nodes and \type +{lig} nodes. The former are simple items that contained nothing but a \quote {character} and a \quote {font} field, and they lived in the same memory as tokens did. The latter also contained a list of components, and a subtype indicating whether this ligature was the result of a word boundary, and it was stored in the same place as other nodes like boxes and kerns and glues. In \LUATEX, these two types are merged into one, somewhat larger structure called -a \type {glyph_node}. Besides having the old character, font, and component -fields, and the new special fields like \quote {attr} (see~\in {section} -[glyphnodes]), these nodes also contain: +a \nod {glyph} node. Besides having the old character, font, and component +fields there are a few more, like \quote {attr} that we will see in \in {section} +[glyphnodes], these nodes also contain a subtype, that codes four main types and +two additional ghost types. For ligatures, multiple bits can be set at the same +time (in case of a single|-|glyph word). \startitemize - -\startitem A subtype, split into four main types: - - \startitemize - \startitem - \type {character}, for characters to be hyphenated: the lowest bit - (bit 0) is set to 1. - \stopitem - \startitem - \type {glyph}, for specific font glyphs: the lowest bit (bit 0) is - not set. - \stopitem - \startitem - \type {ligature}, for ligatures (bit 1 is set) - \stopitem - \startitem - \type {ghost}, for \quote {ghost objects} (bit 2 is set) - \stopitem - \stopitemize - - The latter two make further use of two extra fields (bits 3 and 4): - - \startitemize - \startitem - \type {left}, for ligatures created from a left word boundary and for - ghosts created from \type {\leftghost} - \stopitem - \startitem - \type {right}, for ligatures created from a right word boundary and - for ghosts created from \type {\rightghost} - \stopitem - \stopitemize - - For ligatures, both bits can be set at the same time (in case of a - single|-|glyph word). - -\stopitem - -\startitem - \type {glyph_node}s of type \quote {character} also contain language data, - split into four items that were current when the node was created: the - \type {\setlanguage} (15 bits), \type {\lefthyphenmin} (8 bits), \type - {\righthyphenmin} (8 bits), and \type {\uchyph} (1 bit). -\stopitem - + \startitem + \type {character}, for characters to be hyphenated: the lowest bit + (bit 0) is set to 1. + \stopitem + \startitem + \nod {glyph}, for specific font glyphs: the lowest bit (bit 0) is + not set. + \stopitem + \startitem + \type {ligature}, for constructed ligatures bit 1 is set. + \stopitem + \startitem + \type {ghost}, for so called \quote {ghost objects} bit 2 is set. + \stopitem + \startitem + \type {left}, for ligatures created from a left word boundary and for + ghosts created from \lpr {leftghost} bit 3 gets set. + \stopitem + \startitem + \type {right}, for ligatures created from a right word boundary and + for ghosts created from \lpr {rightghost} bit 4 is set. + \stopitem \stopitemize +The \nod {glyph} nodes also contain language data, split into four items that +were current when the node was created: the \prm {setlanguage} (15~bits), \prm +{lefthyphenmin} (8~bits), \prm {righthyphenmin} (8~bits), and \prm {uchyph} +(1~bit). + Incidentally, \LUATEX\ allows 16383 separate languages, and words can be 256 characters long. The language is stored with each character. You can set -\type {\firstvalidlanguage} to for instance~1 and make thereby language~0 +\prm {firstvalidlanguage} to for instance~1 and make thereby language~0 an ignored hyphenation language. -The new primitive \type {\hyphenationmin} can be used to signal the minimal length -of a word. This value stored with the (current) language. +The new primitive \lpr {hyphenationmin} can be used to signal the minimal length +of a word. This value is stored with the (current) language. -Because the \type {\uchyph} value is saved in the actual nodes, its handling is -subtly different from \TEX82: changes to \type {\uchyph} become effective +Because the \prm {uchyph} value is saved in the actual nodes, its handling is +subtly different from \TEX82: changes to \prm {uchyph} become effective immediately, not at the end of the current partial paragraph. Typeset boxes now always have their language information embedded in the nodes themselves, so there is no longer a possible dependency on the surrounding -language settings. In \TEX82, a mid-paragraph statement like \type {\unhbox0} would -process the box using the current paragraph language unless there was a -\type {\setlanguage} issued inside the box. In \LUATEX, all language variables are -already frozen. +language settings. In \TEX82, a mid|-|paragraph statement like \type {\unhbox0} +would process the box using the current paragraph language unless there was a +\prm {setlanguage} issued inside the box. In \LUATEX, all language variables +are already frozen. In traditional \TEX\ the process of hyphenation is driven by \type {lccode}s. In \LUATEX\ we made this dependency less strong. There are several strategies possible. When you do nothing, the currently used \type {lccode}s are used, when loading patterns, setting exceptions or hyphenating a list. -When you set \type {\savinghyphcodes} to a value larger than zero the current set +When you set \prm {savinghyphcodes} to a value greater than zero the current set of \type {lccode}s will be saved with the language. In that case changing a \type {lccode} afterwards has no effect. However, you can adapt the set with: @@ -144,96 +137,88 @@ of \type {lccode}s will be saved with the language. In that case changing a \typ This change is global which makes sense if you keep in mind that the moment that hyphenation happens is (normally) when the paragraph or a horizontal box is -constructed. When \type {\savinghyphcodes} was zero when the language got +constructed. When \prm {savinghyphcodes} was zero when the language got initialized you start out with nothing, otherwise you already have a set. -When a \type {\hjcode} is larger than $0$ but smaller than $32$ is indicates the +When a \lpr {hjcode} is greater than 0 but less than 32 is indicates the to be used length. In the following example we map a character (\type {x}) onto another one in the patterns and tell the engine that \type {œ} counts as one character. Because traditionally zero itself is reserved for inhibiting -hyphenation, a value of $32$ counts as zero. - -\starttyping -% assuming french patterns: -foobar % foo-bar - -\hjcode`x=`o - -fxxbar % fxx-bar - -\lefthyphenmin3 - -œdipus % œdi-pus - -\lefthyphenmin4 - -œdipus % œdipus - -\hjcode`œ=2 - -œdipus % œdi-pus - -\hjcode`i=32 -\hjcode`d=32 - -œdipus % œdipus -\stoptyping +hyphenation, a value of 32 counts as zero. + +Here are some examples (we assume that French patterns are used): + +\starttabulate[||||] +\NC \NC \type{foobar} \NC \type{foo-bar} \NC \NR +\NC \type{\hjcode`x=`o} \NC \type{fxxbar} \NC \type{fxx-bar} \NC \NR +\NC \type{\lefthyphenmin3} \NC \type{œdipus} \NC \type{œdi-pus} \NC \NR +\NC \type{\lefthyphenmin4} \NC \type{œdipus} \NC \type{œdipus} \NC \NR +\NC \type{\hjcode`œ=2} \NC \type{œdipus} \NC \type{œdi-pus} \NC \NR +\NC \type{\hjcode`i=32 \hjcode`d=32} \NC \type{œdipus} \NC \type{œdipus} \NC \NR +\NC +\stoptabulate Carrying all this information with each glyph would give too much overhead and -also make the process of setting up thee codes more complex. A solution with +also make the process of setting up these codes more complex. A solution with \type {hjcode} sets was considered but rejected because in practice the current approach is sufficient and it would not be compatible anyway. Beware: the values are always saved in the format, independent of the setting -of \type {\savinghyphcodes} at the moment the format is dumped. +of \prm {savinghyphcodes} at the moment the format is dumped. A boundary node normally would mark the end of a word which interferes with for -instance discretionary injection. For this you can use the \type {\wordboundary} -as trigger. Here are a few examples of usage: +instance discretionary injection. For this you can use the \prm {wordboundary} +as a trigger. Here are a few examples of usage: \startbuffer discrete---discrete \stopbuffer -\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop +\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower \startbuffer discrete\discretionary{}{}{---}discrete \stopbuffer -\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop +\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower \startbuffer discrete\wordboundary\discretionary{}{}{---}discrete \stopbuffer -\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop +\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower \startbuffer discrete\wordboundary\discretionary{}{}{---}\wordboundary discrete \stopbuffer -\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop +\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower \startbuffer discrete\wordboundary\discretionary{---}{}{}\wordboundary discrete \stopbuffer -\typebuffer \start \dontcomplain \hsize 1pt \getbuffer \par \stop +\typebuffer \startnarrower \dontcomplain \hsize 1pt \getbuffer \par \stopnarrower We only accept an explicit hyphen when there is a preceding glyph and we skip a -sequence of explicit hyphens as that normally indicates a \type {--} or \type +sequence of explicit hyphens since that normally indicates a \type {--} or \type {---} ligature in which case we can in a worse case usage get bad node lists later on due to messed up ligature building as these dashes are ligatures in base -fonts. This is a side effect of the separating the hyphenation, ligaturing and +fonts. This is a side effect of separating the hyphenation, ligaturing and kerning steps. -The start and end of a characters is signalled by a glue, penalty, kern or boundary -node. But by default also a hlist, vlist, rule, dir, whatsit, ins, and adjust node -indicate a start or end. You can omit the last set from the test by setting -\type {\hyphenationbounds} to a non|-|zero value: +The start and end of a sequence of characters is signalled by a \nod {glue}, \nod +{penalty}, \nod {kern} or \nod {boundary} node. But by default also a \nod +{hlist}, \nod {vlist}, \nod {rule}, \nod {dir}, \nod {whatsit}, \nod {ins}, and +\nod {adjust} node indicate a start or end. You can omit the last set from the +test by setting \lpr {hyphenationbounds} to a non|-|zero value: -\starttabulate[|l|l|] +\starttabulate[|c|l|] +\DB value \BC behaviour \NC \NR +\TB \NC \type{0} \NC not strict \NC \NR \NC \type{1} \NC strict start \NC \NR \NC \type{2} \NC strict end \NC \NR \NC \type{3} \NC strict start and strict end \NC \NR +\LL \stoptabulate The word start is determined as follows: \starttabulate[|l|l|] +\DB node \BC behaviour \NC \NR +\TB \BC boundary \NC yes when wordboundary \NC \NR \BC hlist \NC when hyphenationbounds 1 or 3 \NC \NR \BC vlist \NC when hyphenationbounds 1 or 3 \NC \NR @@ -244,11 +229,14 @@ The word start is determined as follows: \BC math \NC skipped \NC \NR \BC glyph \NC exhyphenchar (one only) : yes (so no -- ---) \NC \NR \BC otherwise \NC yes \NC \NR +\LL \stoptabulate The word end is determined as follows: \starttabulate[|l|l|] +\DB node \BC behaviour \NC \NR +\TB \BC boundary \NC yes \NC \NR \BC glyph \NC yes when different language \NC \NR \BC glue \NC yes \NC \NR @@ -261,10 +249,11 @@ The word end is determined as follows: \BC whatsit \NC when hyphenationbounds 2 or 3 \NC \NR \BC ins \NC when hyphenationbounds 2 or 3 \NC \NR \BC adjust \NC when hyphenationbounds 2 or 3 \NC \NR +\LL \stoptabulate -\in{Figures}[hb:1] upto \in[hb:5] show some examples. In all cases we set the min -values to 1 and make sure that the words hyphenate at each character. +\in {Figures} [hb:1] upto \in [hb:5] show some examples. In all cases we set the +min values to 1 and make sure that the words hyphenate at each character. \hyphenation{o-n-e t-w-o} @@ -283,12 +272,13 @@ values to 1 and make sure that the words hyphenate at each character. \startplacefigure[reference=hb:1,title={\type{one}}] \startcombination[4*1] - {\SomeTest{0}{one}} {\type{0}} - {\SomeTest{1}{one}} {\type{1}} - {\SomeTest{2}{one}} {\type{2}} - {\SomeTest{3}{one}} {\type{3}} + {\SomeTest{0}{one}} {\type{0}} + {\SomeTest{1}{one}} {\type{1}} + {\SomeTest{2}{one}} {\type{2}} + {\SomeTest{3}{one}} {\type{3}} \stopcombination \stopplacefigure + \startplacefigure[reference=hb:2,title={\type{one\null two}}] \startcombination[4*1] {\SomeTest{0}{one\null two}} {\type{0}} @@ -297,6 +287,7 @@ values to 1 and make sure that the words hyphenate at each character. {\SomeTest{3}{one\null two}} {\type{3}} \stopcombination \stopplacefigure + \startplacefigure[reference=hb:3,title={\type{\null one\null two}}] \startcombination[4*1] {\SomeTest{0}{\null one\null two}} {\type{0}} @@ -305,6 +296,7 @@ values to 1 and make sure that the words hyphenate at each character. {\SomeTest{3}{\null one\null two}} {\type{3}} \stopcombination \stopplacefigure + \startplacefigure[reference=hb:4,title={\type{one\null two\null}}] \startcombination[4*1] {\SomeTest{0}{one\null two\null}} {\type{0}} @@ -313,6 +305,7 @@ values to 1 and make sure that the words hyphenate at each character. {\SomeTest{3}{one\null two\null}} {\type{3}} \stopcombination \stopplacefigure + \startplacefigure[reference=hb:5,title={\type{\null one\null two\null}}] \startcombination[4*1] {\SomeTest{0}{\null one\null two\null}} {\type{0}} @@ -330,7 +323,7 @@ deal differently with (a sequence of) explicit hyphens. We already have added some control over aspects of the hyphenation and yet another one concerns automatic hyphens (e.g.\ \type {-} characters in the input). -When \type {\automatichyphenmode} has a value of 0, a hyphen will be turned into +When \lpr {automatichyphenmode} has a value of 0, a hyphen will be turned into an automatic discretionary. The snippets before and after it will not be hyphenated. A side effect is that a leading hyphen can lead to a split but one will seldom run into that situation. Setting a pre and post character makes this @@ -359,12 +352,6 @@ before--after \par before---after \par \stopbuffer -We show three samples: - -Input A: \typebuffer[a] -Input B: \typebuffer[b] -Input C: \typebuffer[c] - \startbuffer[demo] \startcombination[nx=4,ny=3,location=top] {\framed[align=normal,strut=no,top=\vskip.5ex,bottom=\vskip.5ex]{\automatichyphenmode\zerocount \hsize6em \getbuffer[a]}} {A~0~6em} @@ -382,104 +369,139 @@ Input C: \typebuffer[c] \stopcombination \stopbuffer -\startplacefigure[reference=automatic:1,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with a \type {\hsize} +\startplacefigure[locationreference=automatichyphenmode:1,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with a \prm {hsize} of 6em and 2pt (which triggers a linebreak).}] \dontcomplain \tt \getbuffer[demo] \stopplacefigure -\startplacefigure[reference=automatic:2,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with \type -{\preexhyphenchar} and \type {\postexhyphenchar} set to characters \type {A} and \type {B}.}] +\startplacefigure[reference=automatichyphenmode:2,title={The automatic modes \type {0} (default), \type {1} and \type {2}, with \lpr {preexhyphenchar} and \lpr {postexhyphenchar} set to characters \type {A} and \type {B}.}] \postexhyphenchar`A\relax \preexhyphenchar `B\relax \dontcomplain \tt \getbuffer[demo] \stopplacefigure -As with primitive companions of other single character commands, the \type {\-} -command has a more verbose primitive version in \type {\explicitdiscretionary} +In \in {figure} [automatichyphenmode:1] \in {and} [automatichyphenmode:2] we show +what happens with three samples: + +Input A: \typebuffer[a] +Input B: \typebuffer[b] +Input C: \typebuffer[c] + +As with primitive companions of other single character commands, the \prm {-} +command has a more verbose primitive version in \lpr {explicitdiscretionary} and the normally intercepted in the hyphenator character \type {-} (or whatever -is configured) is available as \type {\automaticdiscretionary}. +is configured) is available as \lpr {automaticdiscretionary}. -\section{The main control loop} +\stopsection + +\startsection[title={The main control loop}] + +\topicindex {main loop} +\topicindex {hyphenation} In \LUATEX's main loop, almost all input characters that are to be typeset are -converted into \type {glyph} node records with subtype \quote {character}, but +converted into \nod {glyph} node records with subtype \quote {character}, but there are a few exceptions. -First, the \type {\accent} primitives creates nodes with subtype \quote {glyph} -instead of \quote {character}: one for the actual accent and one for the -accentee. The primary reason for this is that \type {\accent} in \TEX82 is -explicitly dependent on the current font encoding, so it would not make much -sense to attach a new meaning to the primitive's name, as that would invalidate -many old documents and macro packages. \footnote {Of course, modern packages will -not use the \type {\accent} primitive at all but try to map directly on composed -characters.} A secondary reason is that in \TEX82, \type {\accent} prohibits -hyphenation of the current word. Since in \LUATEX\ hyphenation only takes place -on \quote {character} nodes, it is possible to achieve the same effect. - -This change of meaning did happen with \type {\char}, that now generates \quote -{glyph} nodes with a character subtype. In traditional \TEX\ there was a strong -relationship between the 8|-|bit input encoding, hyphenation and glyphs taken -from a font. In \LUATEX\ we have \UTF\ input, and in most cases this maps -directly to a character in a font, apart from glyph replacement in the font -engine. If you want to access arbitrary glyphs in a font directly you can always -use \LUA\ to do so, because fonts are available as \LUA\ table. - -Second, all the results of processing in math mode eventually become nodes with -\quote {glyph} subtypes. - -Third, the \ALEPH|-|derived commands \type {\leftghost} and \type {\rightghost} -create nodes of a third subtype: \quote {ghost}. These nodes are ignored -completely by all further processing until the stage where inter|-|glyph kerning -is added. - -Fourth, automatic discretionaries are handled differently. \TEX82 inserts an -empty discretionary after sensing an input character that matches the \type -{\hyphenchar} in the current font. This test is wrong in our opinion: whether or -not hyphenation takes place should not depend on the current font, it is a -language property. \footnote {When \TEX\ showed up we didn't have \UNICODE\ yet -and being limited to eight bits meant that one sometimes had to compromise -between supporting character input, glyph rendering, hyphenation.} - -In \LUATEX, it works like this: if \LUATEX\ senses a string of input characters -that matches the value of the new integer parameter \type {\exhyphenchar}, it will -insert an explicit discretionary after that series of nodes. Initex sets the \type -{\exhyphenchar=`\-}. Incidentally, this is a global parameter instead of a -language-specific one because it may be useful to change the value depending on -the document structure instead of the text language. - -The insertion of discretionaries after a sequence of explicit hyphens happens at -the same time as the other hyphenation processing, {\it not\/} inside the main -control loop. - -The only use \LUATEX\ has for \type {\hyphenchar} is at the check whether a word -should be considered for hyphenation at all. If the \type {\hyphenchar} of the -font attached to the first character node in a word is negative, then hyphenation -of that word is abandoned immediately. This behaviour is added for backward -compatibility only, and the use of \type {\hyphenchar=-1} as a means of -preventing hyphenation should not be used in new \LUATEX\ documents. - -Fifth, \type {\setlanguage} no longer creates whatsits. The meaning of \type -{\setlanguage} is changed so that it is now an integer parameter like all others. -That integer parameter is used in \type {\glyph_node} creation to add language -information to the glyph nodes. In conjunction, the \type {\language} primitive is -extended so that it always also updates the value of \type {\setlanguage}. - -Sixth, the \type {\noboundary} command (that prohibits word boundary processing -where that would normally take place) now does create nodes. These nodes are -needed because the exact place of the \type {\noboundary} command in the input -stream has to be retained until after the ligature and font processing stages. - -Finally, there is no longer a \type {main_loop} label in the code. Remember that -\TEX82 did quite a lot of processing while adding \type {char_nodes} to the -horizontal list? For speed reasons, it handled that processing code outside of -the \quote {main control} loop, and only the first character of any \quote {word} -was handled by that \quote {main control} loop. In \LUATEX, there is no longer a -need for that (all hard work is done later), and the (now very small) bits of -character|-|handling code have been moved back inline. When \type -{\tracingcommands} is on, this is visible because the full word is reported, -instead of just the initial character. - -Because we tend to make hard codes behaviour configurable a few new primitives +\startitemize[n] + +\startitem + The \prm {accent} primitive creates nodes with subtype \quote {glyph} + instead of \quote {character}: one for the actual accent and one for the + accentee. The primary reason for this is that \prm {accent} in \TEX82 is + explicitly dependent on the current font encoding, so it would not make much + sense to attach a new meaning to the primitive's name, as that would + invalidate many old documents and macro packages. A secondary reason is that + in \TEX82, \prm {accent} prohibits hyphenation of the current word. Since + in \LUATEX\ hyphenation only takes place on \quote {character} nodes, it is + possible to achieve the same effect. Of course, modern \UNICODE\ aware macro + packages will not use the \prm {accent} primitive at all but try to map + directly on composed characters. + + This change of meaning did happen with \prm {char}, that now generates + \quote {glyph} nodes with a character subtype. In traditional \TEX\ there was + a strong relationship between the 8|-|bit input encoding, hyphenation and + glyphs taken from a font. In \LUATEX\ we have \UTF\ input, and in most cases + this maps directly to a character in a font, apart from glyph replacement in + the font engine. If you want to access arbitrary glyphs in a font directly + you can always use \LUA\ to do so, because fonts are available as \LUA\ + table. +\stopitem + +\startitem + All the results of processing in math mode eventually become nodes with + \quote {glyph} subtypes. In fact, the result of processing math is just + a regular list of glyphs, kerns, glue, penalties, boxes etc. +\stopitem + +\startitem + The \ALEPH|-|derived commands \lpr {leftghost} and \lpr {rightghost} + create nodes of a third subtype: \quote {ghost}. These nodes are ignored + completely by all further processing until the stage where inter|-|glyph + kerning is added. +\stopitem + +\startitem + Automatic discretionaries are handled differently. \TEX82 inserts an empty + discretionary after sensing an input character that matches the \prm + {hyphenchar} in the current font. This test is wrong in our opinion: whether + or not hyphenation takes place should not depend on the current font, it is a + language property. \footnote {When \TEX\ showed up we didn't have \UNICODE\ + yet and being limited to eight bits meant that one sometimes had to + compromise between supporting character input, glyph rendering, hyphenation.} + + In \LUATEX, it works like this: if \LUATEX\ senses a string of input + characters that matches the value of the new integer parameter \prm + {exhyphenchar}, it will insert an explicit discretionary after that series of + nodes. Initially \TEX\ sets the \type {\exhyphenchar=`\-}. Incidentally, this + is a global parameter instead of a language-specific one because it may be + useful to change the value depending on the document structure instead of the + text language. + + The insertion of discretionaries after a sequence of explicit hyphens happens + at the same time as the other hyphenation processing, {\it not\/} inside the + main control loop. + + The only use \LUATEX\ has for \prm {hyphenchar} is at the check whether a + word should be considered for hyphenation at all. If the \prm {hyphenchar} + of the font attached to the first character node in a word is negative, then + hyphenation of that word is abandoned immediately. This behaviour is added + for backward compatibility only, and the use of \type {\hyphenchar=-1} as a + means of preventing hyphenation should not be used in new \LUATEX\ documents. +\stopitem + +\startitem + The \prm {setlanguage} command no longer creates whatsits. The meaning of + \prm {setlanguage} is changed so that it is now an integer parameter like all + others. That integer parameter is used in \type {\glyph_node} creation to add + language information to the glyph nodes. In conjunction, the \prm {language} + primitive is extended so that it always also updates the value of \prm + {setlanguage}. +\stopitem + +\startitem + The \prm {noboundary} command (that prohibits word boundary processing + where that would normally take place) now does create nodes. These nodes are + needed because the exact place of the \prm {noboundary} command in the + input stream has to be retained until after the ligature and font processing + stages. +\stopitem + +\startitem + There is no longer a \type {main_loop} label in the code. Remember that + \TEX82 did quite a lot of processing while adding \type {char_nodes} to the + horizontal list? For speed reasons, it handled that processing code outside + of the \quote {main control} loop, and only the first character of any \quote + {word} was handled by that \quote {main control} loop. In \LUATEX, there is + no longer a need for that (all hard work is done later), and the (now very + small) bits of character|-|handling code have been moved back inline. When + \prm {tracingcommands} is on, this is visible because the full word is + reported, instead of just the initial character. +\stopitem + +\stopitemize + +Because we tend to make hard coded behaviour configurable a few new primitives have been added: \starttyping @@ -489,50 +511,59 @@ have been added: \stoptyping The first parameter has the following consequences for automatic discs (the ones -resulting from an \type {\exhyphenchar}: +resulting from an \prm {exhyphenchar}: \starttabulate[|c|l|l|] -\BC mode \BC automatic disc \type{-} \BC explicit disc \type{\-} \NC \NR -\HL -\NC \type{0} \NC \type {\exhyphenpenalty} \NC \type {\exhyphenpenalty} \NC \NR -\NC \type{1} \NC \type {\hyphenpenalty} \NC \type {\hyphenpenalty} \NC \NR -\NC \type{2} \NC \type {\exhyphenpenalty} \NC \type {\hyphenpenalty} \NC \NR -\NC \type{3} \NC \type {\hyphenpenalty} \NC \type {\exhyphenpenalty} \NC \NR -\NC \type{4} \NC \type {\automatichyphenpenalty} \NC \type {\explicithyphenpenalty} \NC \NR -\NC \type{5} \NC \type {\exhyphenpenalty} \NC \type {\explicithyphenpenalty} \NC \NR -\NC \type{6} \NC \type {\hyphenpenalty} \NC \type {\explicithyphenpenalty} \NC \NR -\NC \type{7} \NC \type {\automatichyphenpenalty} \NC \type {\exhyphenpenalty} \NC \NR -\NC \type{8} \NC \type {\automatichyphenpenalty} \NC \type {\hyphenpenalty} \NC \NR +\DB mode \BC automatic disc \type {-} \BC explicit disc \prm{-} \NC \NR +\TB +\NC \type{0} \NC \prm {exhyphenpenalty} \NC \prm {exhyphenpenalty} \NC \NR +\NC \type{1} \NC \prm {hyphenpenalty} \NC \prm {hyphenpenalty} \NC \NR +\NC \type{2} \NC \prm {exhyphenpenalty} \NC \prm {hyphenpenalty} \NC \NR +\NC \type{3} \NC \prm {hyphenpenalty} \NC \prm {exhyphenpenalty} \NC \NR +\NC \type{4} \NC \lpr {automatichyphenpenalty} \NC \lpr {explicithyphenpenalty} \NC \NR +\NC \type{5} \NC \prm {exhyphenpenalty} \NC \lpr {explicithyphenpenalty} \NC \NR +\NC \type{6} \NC \prm {hyphenpenalty} \NC \lpr {explicithyphenpenalty} \NC \NR +\NC \type{7} \NC \lpr {automatichyphenpenalty} \NC \prm {exhyphenpenalty} \NC \NR +\NC \type{8} \NC \lpr {automatichyphenpenalty} \NC \prm {hyphenpenalty} \NC \NR +\LL \stoptabulate -other values do what we always did in \LUATEX: insert \type {\exhyphenpenalty}. +other values do what we always did in \LUATEX: insert \prm {exhyphenpenalty}. -\section[patternsexceptions]{Loading patterns and exceptions} +\stopsection -The hyphenation algorithm in \LUATEX\ is quite different from the one in \TEX82, -although it uses essentially the same user input. +\startsection[title={Loading patterns and exceptions},reference=patternsexceptions] -After expansion, the argument for \type {\patterns} has to be proper \UTF8 with -individual patterns separated by spaces, no \type {\char} or \type {\chardef}d -commands are allowed. The current implementation quite strict and will reject all -non|-|\UNICODE\ characters. +\topicindex {hyphenation} +\topicindex {hyphenation+patterns} +\topicindex {hyphenation+exceptions} +\topicindex {patterns} +\topicindex {exceptions} -Likewise, the expanded argument for \type {\hyphenation} also has to be proper -\UTF8, but here a bit of extra syntax is provided: +Although we keep the traditional approach towards hyphenation (which is still +superior) the implementation of the hyphenation algorithm in \LUATEX\ is quite +different from the one in \TEX82. + +After expansion, the argument for \prm {patterns} has to be proper \UTF8 with +individual patterns separated by spaces, no \prm {char} or \prm {chardef}d +commands are allowed. The current implementation is quite strict and will reject +all non|-|\UNICODE\ characters. Likewise, the expanded argument for \prm +{hyphenation} also has to be proper \UTF8, but here a bit of extra syntax is +provided: \startitemize[n] \startitem - Three sets of arguments in curly braces (\type {{}{}{}}) indicates a desired - complex discretionary, with arguments as in \type {\discretionary}'s command in + Three sets of arguments in curly braces (\type {{}{}{}}) indicate a desired + complex discretionary, with arguments as in \prm {discretionary}'s command in normal document input. \stopitem \startitem - A \type {-} indicates a desired simple discretionary, cf.\ \type {\-} and \type - {\discretionary{-}{}{}} in normal document input. + A \type {-} indicates a desired simple discretionary, cf.\ \type {\-} and + \type {\discretionary{-}{}{}} in normal document input. \stopitem \startitem - Internal command names are ignored. This rule is provided especially for \type - {\discretionary}, but it also helps to deal with \type {\relax} commands that + Internal command names are ignored. This rule is provided especially for \prm + {discretionary}, but it also helps to deal with \prm {relax} commands that may sneak in. \stopitem \startitem @@ -540,22 +571,24 @@ Likewise, the expanded argument for \type {\hyphenation} also has to be proper \stopitem \stopitemize -The expanded argument is first converted back to a space-separated string while +The expanded argument is first converted back to a space|-|separated string while dropping the internal command names. This string is then converted into a dictionary by a routine that creates key|-|value pairs by converting the other listed items. It is important to note that the keys in an exception dictionary can always be generated from the values. Here are a few examples: \starttabulate[|l|l|l|] -\BC value \BC implied key (input) \NC effect \NC\NR +\DB value \BC implied key (input) \BC effect \NC\NR +\TB \NC \type {ta-ble} \NC table \NC \type {ta\-ble} ($=$ \type {ta\discretionary{-}{}{}ble}) \NC\NR \NC \type {ba{k-}{}{c}ken} \NC backen \NC \type {ba\discretionary{k-}{}{c}ken} \NC\NR +\LL \stoptabulate The resultant patterns and exception dictionary will be stored under the language -code that is the present value of \type {\language}. +code that is the present value of \prm {language}. -In the last line of the table, you see there is no \type {\discretionary} command +In the last line of the table, you see there is no \prm {discretionary} command in the value: the command is optional in the \TEX-based input syntax. The underlying reason for that is that it is conceivable that a whole dictionary of words is stored as a plain text file and loaded into \LUATEX\ using one of the @@ -573,20 +606,64 @@ actual explicit hyphen character if needed). For example, this matches the word \hyphenation{multi{-}{}{-}word{-}{}{-}boun-daries} \stoptyping -The motivation behind the \ETEX\ extension \type {\savinghyphcodes} was that +The motivation behind the \ETEX\ extension \prm {savinghyphcodes} was that hyphenation heavily depended on font encodings. This is no longer true in \LUATEX, and the corresponding primitive is basically ignored. Because we now -have \type {hjcode}, the case relate codes can be used exclusively for \type -{\uppercase} and \type {\lowercase}. - -\section{Applying hyphenation} +have \lpr {hjcode}, the case relate codes can be used exclusively for \prm +{uppercase} and \prm {lowercase}. + +The three curly brace pair pattern in an exception can be somewhat unexpected so +we will try to explain it by example. The pattern \type {foo{}{}{x}bar} pattern +creates a lookup \type {fooxbar} and the pattern \type {foo{}{}{}bar} creates +\type {foobar}. Then, when a hit happens there is a replacement text (\type {x}) +or none. Because we introduced penalties in discretionary nodes, the exception +syntax now also can take a penalty specification. The value between square brackets +is a multiplier for \lpr {exceptionpenalty}. Here we have set it to 10000 so +effectively we get 30000 in the example. + +\def\ShowSample#1#2% + {\startlinecorrection[blank] + \hyphenation{#1}% + \exceptionpenalty=10000 + \bTABLE[foregroundstyle=type] + \bTR + \bTD[align=middle,nx=4] \type{#1} \eTD + \eTR + \bTR + \bTD[align=middle] \type{10em} \eTD + \bTD[align=middle] \type {3em} \eTD + \bTD[align=middle] \type {0em} \eTD + \bTD[align=middle] \type {6em} \eTD + \eTR + \bTR + \bTD[width=10em]\vtop{\hsize 10em 123 #2 123\par}\eTD + \bTD[width=10em]\vtop{\hsize 3em 123 #2 123\par}\eTD + \bTD[width=10em]\vtop{\hsize 0em 123 #2 123\par}\eTD + \bTD[width=10em]\vtop{\setupalign[verytolerant,stretch]\rmtf\hsize 6em 123 #2 #2 #2 #2 123\par}\eTD + \eTR + \eTABLE + \stoplinecorrection} + +\ShowSample{x{a-}{-b}{}x{a-}{-b}{}x{a-}{-b}{}x{a-}{-b}{}xx}{xxxxxx} +\ShowSample{x{a-}{-b}{}x{a-}{-b}{}[3]x{a-}{-b}{}[1]x{a-}{-b}{}xx}{xxxxxx} + +\ShowSample{z{a-}{-b}{z}{a-}{-b}{z}{a-}{-b}{z}{a-}{-b}{z}z}{zzzzzz} +\ShowSample{z{a-}{-b}{z}{a-}{-b}{z}[3]{a-}{-b}{z}[1]{a-}{-b}{z}z}{zzzzzz} + +\stopsection + +\startsection[title={Applying hyphenation}] + +\topicindex {hyphenation+how it works} +\topicindex {hyphenation+discretionaries} +\topicindex {discretionaries} The internal structures \LUATEX\ uses for the insertion of discretionaries in words is very different from the ones in \TEX82, and that means there are some noticeable differences in handling as well. First and foremost, there is no \quote {compressed trie} involved in hyphenation. -The algorithm still reads \PATGEN-generated pattern files, but \LUATEX\ uses a +The algorithm still reads pattern files generated by \PATGEN, but \LUATEX\ uses a finite state hash to match the patterns against the word to be hyphenated. This algorithm is based on the \quote {libhnj} library used by \OPENOFFICE, which in turn is inspired by \TEX. @@ -605,12 +682,12 @@ of the implementation: \stopitem \startitem Because there is no \quote {trie preparation} stage, language patterns never - become frozen. This means that the primitive \type {\patterns} (and its \LUA\ + become frozen. This means that the primitive \prm {patterns} (and its \LUA\ counterpart \type {lang.patterns}) can be used at any time, not only in ini\TEX. \stopitem \startitem - Only the string representation of \type {\patterns} and \type {\hyphenation} is + Only the string representation of \prm {patterns} and \prm {hyphenation} is stored in the format file. At format load time, they are simply re|-|evaluated. It follows that there is no real reason to preload languages in the format file. In fact, it is usually not a good idea to do so. It is @@ -618,23 +695,27 @@ of the implementation: needed. \stopitem \startitem - \LUATEX\ uses the language-specific variables \type {\prehyphenchar} and \type - {\posthyphenchar} in the creation of implicit discretionaries, instead of - \TEX82's \type {\hyphenchar}, and the values of the language|-|specific variables - \type {\preexhyphenchar} and \type {\postexhyphenchar} for explicit + \LUATEX\ uses the language-specific variables \lpr {prehyphenchar} and \lpr + {posthyphenchar} in the creation of implicit discretionaries, instead of + \TEX82's \prm {hyphenchar}, and the values of the language|-|specific + variables \lpr {preexhyphenchar} and \lpr {postexhyphenchar} for explicit discretionaries (instead of \TEX82's empty discretionary). \stopitem \startitem - The value of the two counters related to hyphenation, \type {\hyphenpenalty} - and \type {\exhyphenpenalty}, are now stored in the discretionary nodes. This - permits a local overload for explicit \type {\discretionary} commands. The + The value of the two counters related to hyphenation, \prm {hyphenpenalty} + and \prm {exhyphenpenalty}, are now stored in the discretionary nodes. This + permits a local overload for explicit \prm {discretionary} commands. The value current when the hyphenation pass is applied is used. When no callbacks are used this is compatible with traditional \TEX. When you apply the \LUA\ \type {lang.hyphenate} function the current values are used. \stopitem +\startitem + The hyphenation exception dictionary is maintained as key|-|value hash, and + that is also dynamic, so the \type {hyph_size} setting is not used either. +\stopitem \stopitemize -Because we store penalties in the disc node the \type {\discretionary} command has +Because we store penalties in the disc node the \prm {discretionary} command has been extended to accept an optional penalty specification, so you can do the following: @@ -657,18 +738,18 @@ inserted at the left-hand side of a word). Word boundaries are no longer implied by font switches, but by language switches. One word can have two separate fonts and still be hyphenated correctly (but it -can not have two different languages, the \type {\setlanguage} command forces a +can not have two different languages, the \prm {setlanguage} command forces a word boundary). All languages start out with \type {\prehyphenchar=`\-}, \type {\posthyphenchar=0}, \type {\preexhyphenchar=0} and \type {\postexhyphenchar=0}. When you assign the values of one of these four parameters, you are actually changing the settings -for the current \type {\language}, this behaviour is compatible with \type {\patterns} -and \type {\hyphenation}. +for the current \prm {language}, this behaviour is compatible with \prm {patterns} +and \prm {hyphenation}. \LUATEX\ also hyphenates the first word in a paragraph. Words can be up to 256 -characters long (up from 64 in \TEX82). Longer words generate an error right now, -but eventually either the limitation will be removed or perhaps it will become +characters long (up from 64 in \TEX82). Longer words are ignored right now, but +eventually either the limitation will be removed or perhaps it will become possible to silently ignore the excess characters (this is what happens in \TEX82, but there the behaviour cannot be controlled). @@ -677,10 +758,12 @@ that this function expects to receive a list of \quote {character} nodes. It wil not operate properly in the presence of \quote {glyph}, \quote {ligature}, or \quote {ghost} nodes, nor does it know how to deal with kerning. -The hyphenation exception dictionary is maintained as key|-|value hash, and that -is also dynamic, so the \type {hyph_size} setting is not used either. +\stopsection -\section{Applying ligatures and kerning} +\startsection[title={Applying ligatures and kerning}] + +\topicindex {ligatures} +\topicindex {kerning} After all possible hyphenation points have been inserted in the list, \LUATEX\ will process the list to convert the \quote {character} nodes into \quote {glyph} @@ -689,24 +772,28 @@ ligatures are processed, then all kerning information is applied to the result list. But those two stages are somewhat dependent on each other: If the used font makes it possible to do so, the ligaturing stage adds virtual \quote {character} nodes to the word boundaries in the list. While doing so, it removes and -interprets \type {\noboundary} nodes. The kerning stage deletes those word +interprets \prm {noboundary} nodes. The kerning stage deletes those word boundary items after it is done with them, and it does the same for \quote {ghost} nodes. Finally, at the end of the kerning stage, all remaining \quote {character} nodes are converted to \quote {glyph} nodes. -This work separation is worth mentioning because, if you overrule from \LUA\ only +This word separation is worth mentioning because, if you overrule from \LUA\ only one of the two callbacks related to font handling, then you have to make sure you perform the tasks normally done by \LUATEX\ itself in order to make sure that the other, non|-|overruled, routine continues to function properly. -Work in this area is not yet complete, but most of the possible cases are handled -by our rewritten ligaturing engine. At some point all of the possible inputs will -become supported. \footnote {Not all of this makes sense because we nowadays have -\OPENTYPE\ fonts and ligature building can happen in ,any different ways there.} +Although we could improve the situation the reality is that in modern \OPENTYPE\ +fonts ligatures can be constructed in many ways: by replacing a sequence of +characters by one glyph, or by selectively replacing individual glyphs, or by +kerning, or any combination of this. Add to that contextual analysis and it will +be clear that we have to let \LUA\ do that job instead. The generic font handler +that we provide (which is part of \CONTEXT) distinguishes between base mode +(which essentially is what we describe here and which delegates the task to \TEX) +and node mode (which deals with more complex fonts. -For example, take the word \type {office}, hyphenated \type {of-fice}, using a -\quote {normal} font with all the \type {f}-\type {f} and \type {f}-\type {i} -type ligatures: +Let's look at an example. Take the word \type {office}, hyphenated \type +{of-fice}, using a \quote {normal} font with all the \type {f}-\type {f} and +\type {f}-\type {i} type ligatures: \starttabulate[|l|l|] \NC initial \NC \type {{o}{f}{f}{i}{c}{e}} \NC\NR @@ -734,11 +821,15 @@ the top-level discretionary that resulted from the first hyphenation point. Here is that nested solution again, in a different representation: +\testpage[4] + \starttabulate[|l|c|c|c|c|c|c|] -\NC \BC pre \BC \BC post \BC \BC replace \BC \NC \NR +\DB \BC pre \BC \BC post \BC \BC replace \BC \NC \NR +\TB \NC topdisc \NC \type {f-} \NC (1) \NC \NC sub 1 \NC \NC sub 2 \NC \NR \NC sub 1 \NC \type {f-} \NC (2) \NC \type {i} \NC (3) \NC \type {<fi>} \NC (4) \NC \NR \NC sub 2 \NC \type {<ff>-} \NC (5) \NC \type {i} \NC (6) \NC \type {<ffi>} \NC (7) \NC \NR +\LL \stoptabulate When line breaking is choosing its breakpoints, the following fields will @@ -763,21 +854,23 @@ the first node). One can observe that the \type {of-f-ice} and \type {off-ice} cases both end with the same actual post replacement list (\type {i}), and that this would be the -case even if that \type {i} was the first item of a potential following ligature -like \type {ic}. This allows \LUATEX\ to do away with one of the fields, and thus -make the whole stuff fit into just two discretionary nodes. +case even if \type {i} was the first item of a potential following ligature like +\type {ic}. This allows \LUATEX\ to do away with one of the fields, and thus make +the whole stuff fit into just two discretionary nodes. The mapping of the seven list fields to the six fields in this discretionary node pair is as follows: \starttabulate[|l|c|c|] -\BC field \BC description \NC \NC \NR +\DB field \BC description \NC \NC \NR +\TB \NC \type {disc1.pre} \NC \type {f-} \NC (1) \NC \NR \NC \type {disc1.post} \NC \type {<fi>} \NC (4) \NC \NR \NC \type {disc1.replace} \NC \type {<ffi>} \NC (7) \NC \NR \NC \type {disc2.pre} \NC \type {f-} \NC (2) \NC \NR \NC \type {disc2.post} \NC \type {i} \NC (3,6) \NC \NR \NC \type {disc2.replace} \NC \type {<ff>-} \NC (5) \NC \NR +\LL \stoptabulate What is actually generated after ligaturing has been applied is therefore: @@ -806,13 +899,21 @@ mapping a sequence of glyphs onto one glyph, but also by selective replacement a kerning. This means that the above examples are just representing the traditional approach. -\section{Breaking paragraphs into lines} +\stopsection + +\startsection[title={Breaking paragraphs into lines}] -This code is still almost unchanged, but because of the above|-|mentioned changes +\topicindex {linebreaks} +\topicindex {paragraphs} +\topicindex {discretionaries} + +This code is almost unchanged, but because of the above|-|mentioned changes with respect to discretionaries and ligatures, line breaking will potentially be different from traditional \TEX. The actual line breaking code is still based on the \TEX82 algorithms, and it does not expect there to be discretionaries inside -of discretionaries. +of discretionaries. But, as patterns evolve and font handling can influence +discretionaries, you need to be aware of the fact that long term consistency is not +an engine matter only. But that situation is now fairly common in \LUATEX, due to the changes to the ligaturing mechanism. And also, the \LUATEX\ discretionary nodes are implemented @@ -826,10 +927,19 @@ The combined effect of these two differences is that \LUATEX\ does not always us all of the potential breakpoints in a paragraph, especially when fonts with many ligatures are used. Of course kerning also complicates matters here. -\section{The \type {lang} library} +\stopsection + +\startsection[title={The \type {lang} library}][library=lang] + +\subsection {\type {new} and \type {id}} -This library provides the interface to \LUATEX's structure -representing a language, and the associated functions. +\topicindex {languages+library} + +\libindex {new} +\libindex {id} + +This library provides the interface to \LUATEX's structure representing a +language, and the associated functions. \startfunctioncall <language> l = lang.new() @@ -837,106 +947,141 @@ representing a language, and the associated functions. \stopfunctioncall This function creates a new userdata object. An object of type \type {<language>} -is the first argument to most of the other functions in the \type {lang} -library. These functions can also be used as if they were object methods, using -the colon syntax. - -Without an argument, the next available internal id number will be assigned to -this object. With argument, an object will be created that links to the internal -language with that id number. +is the first argument to most of the other functions in the \type {lang} library. +These functions can also be used as if they were object methods, using the colon +syntax. Without an argument, the next available internal id number will be +assigned to this object. With argument, an object will be created that links to +the internal language with that id number. \startfunctioncall <number> n = lang.id(<language> l) \stopfunctioncall -returns the internal \type {\language} id number this object refers to. +The number returned is the internal \prm {language} id number this object refers to. + +\subsection {\type {hyphenation}} + +\libindex {hyphenation} + +You can hyphenate a string directly with: \startfunctioncall <string> n = lang.hyphenation(<language> l) lang.hyphenation(<language> l, <string> n) \stopfunctioncall -Either returns the current hyphenation exceptions for this language, or adds new -ones. The syntax of the string is explained in~\in {section} +\subsection {\type {clear_hyphenation} and \type {clean}} + +\libindex {clear_hyphenation} +\libindex {clean} + +This either returns the current hyphenation exceptions for this language, or adds +new ones. The syntax of the string is explained in~\in {section} [patternsexceptions]. \startfunctioncall lang.clear_hyphenation(<language> l) \stopfunctioncall -Clears the exception dictionary (string) for this language. +This call clears the exception dictionary (string) for this language. \startfunctioncall <string> n = lang.clean(<language> l, <string> o) <string> n = lang.clean(<string> o) \stopfunctioncall -Creates a hyphenation key from the supplied hyphenation value. The syntax of the -argument string is explained in~\in {section} [patternsexceptions]. This function -is useful if you want to do something else based on the words in a dictionary -file, like spell|-|checking. +This function creates a hyphenation key from the supplied hyphenation value. The +syntax of the argument string is explained in \in {section} [patternsexceptions]. +This function is useful if you want to do something else based on the words in a +dictionary file, like spell|-|checking. + +\subsection {\type {patterns} and \type {clear_patterns}} + +\libindex {patterns} +\libindex {clear_patterns} \startfunctioncall <string> n = lang.patterns(<language> l) lang.patterns(<language> l, <string> n) \stopfunctioncall -Adds additional patterns for this language object, or returns the current set. -The syntax of this string is explained in~\in {section} [patternsexceptions]. +This adds additional patterns for this language object, or returns the current +set. The syntax of this string is explained in \in {section} +[patternsexceptions]. \startfunctioncall lang.clear_patterns(<language> l) \stopfunctioncall -Clears the pattern dictionary for this language. +This can be used to clear the pattern dictionary for a language. + +\subsection {\type {hyphenationmin}} + +\libindex {hyphenationmin} + +This function sets (or gets) the value of the \TEX\ parameter +\type {\hyphenationmin}. \startfunctioncall -<number> n = lang.prehyphenchar(<language> l) -lang.prehyphenchar(<language> l, <number> n) +n = lang.hyphenationmin(<language> l) +lang.hyphenationmin(<language> l, <number> n) \stopfunctioncall -Gets or sets the \quote {pre|-|break} hyphen character for implicit hyphenation -in this language (initially the hyphen, decimal 45). +\subsection {\type {[pre|post][ex|]hyphenchar}} + +\libindex {prehyphenchar} +\libindex {posthyphenchar} +\libindex {preexhyphenchar} +\libindex {postexhyphenchar} \startfunctioncall +<number> n = lang.prehyphenchar(<language> l) +lang.prehyphenchar(<language> l, <number> n) + <number> n = lang.posthyphenchar(<language> l) lang.posthyphenchar(<language> l, <number> n) \stopfunctioncall -Gets or sets the \quote {post|-|break} hyphen character for implicit hyphenation -in this language (initially null, decimal~0, indicating emptiness). +These two are used to get or set the \quote {pre|-|break} and \quote +{post|-|break} hyphen characters for implicit hyphenation in this language. The +intial values are decimal 45 (hyphen) and decimal~0 (indicating emptiness). \startfunctioncall <number> n = lang.preexhyphenchar(<language> l) lang.preexhyphenchar(<language> l, <number> n) -\stopfunctioncall -Gets or sets the \quote {pre|-|break} hyphen character for explicit hyphenation -in this language (initially null, decimal~0, indicating emptiness). - -\startfunctioncall <number> n = lang.postexhyphenchar(<language> l) lang.postexhyphenchar(<language> l, <number> n) \stopfunctioncall -Gets or sets the \quote {post|-|break} hyphen character for explicit hyphenation -in this language (initially null, decimal~0, indicating emptiness). +These gets or set the \quote {pre|-|break} and \quote {post|-|break} hyphen +characters for explicit hyphenation in this language. Both are initially +decimal~0 (indicating emptiness). + +\subsection {\type {hyphenate}} + +\libindex {hyphenate} + +The next call inserts hyphenation points (discretionary nodes) in a node list. If +\type {tail} is given as argument, processing stops on that node. Currently, +\type {success} is always true if \type {head} (and \type {tail}, if specified) +are proper nodes, regardless of possible other errors. \startfunctioncall <boolean> success = lang.hyphenate(<node> head) <boolean> success = lang.hyphenate(<node> head, <node> tail) \stopfunctioncall -Inserts hyphenation points (discretionary nodes) in a node list. If \type {tail} -is given as argument, processing stops on that node. Currently, \type {success} -is always true if \type {head} (and \type {tail}, if specified) are proper nodes, -regardless of possible other errors. - Hyphenation works only on \quote {characters}, a special subtype of all the glyph nodes with the node subtype having the value \type {1}. Glyph modes with -different subtypes are not processed. See \in {section~} [charsandglyphs] for +different subtypes are not processed. See \in {section} [charsandglyphs] for more details. +\subsection {\type {[set|get]hjcode}} + +\libindex {sethjcode} +\libindex {gethjcode} + The following two commands can be used to set or query hj codes: \startfunctioncall @@ -945,7 +1090,9 @@ lang.sethjcode(<language> l, <number> char, <number> usedchar) \stopfunctioncall When you set a hjcode the current sets get initialized unless the set was already -initialized due to \type {\savinghyphcodes} being larger than zero. +initialized due to \prm {savinghyphcodes} being larger than zero. + +\stopsection \stopchapter diff --git a/doc/context/sources/general/manuals/luatex/luatex-logos.tex b/doc/context/sources/general/manuals/luatex/luatex-logos.tex index 7406dd602..3172336ec 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-logos.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-logos.tex @@ -1,5 +1,7 @@ \startenvironment luatex-logos +\usemodule[abr-02] + \logo[DFONT] {dfont} \logo[CFF] {cff} \logo[CMAP] {CMap} diff --git a/doc/context/sources/general/manuals/luatex/luatex-lua.tex b/doc/context/sources/general/manuals/luatex/luatex-lua.tex index 82b060440..f9107fa1f 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-lua.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-lua.tex @@ -1,15 +1,17 @@ % language=uk \environment luatex-style -\environment luatex-logos \startcomponent luatex-lua -\startchapter[reference=lua,title={\LUA\ general}] +\startchapter[reference=lua,title={Using \LUATEX}] -\section[init]{Initialization} +\startsection[title={Initialization},reference=init] -\subsection{\LUATEX\ as a \LUA\ interpreter} +\startsubsection[title={\LUATEX\ as a \LUA\ interpreter}] + +\topicindex {initialization} +\topicindex {\LUA+interpreter} There are some situations that make \LUATEX\ behave like a standalone \LUA\ interpreter: @@ -35,7 +37,11 @@ positive values, just like the \LUA\ interpreter. in effect, a somewhat bulky stand alone \LUA\ interpreter with a bunch of extra preloaded libraries. -\subsection{\LUATEX\ as a \LUA\ byte compiler} +\stopsubsection + +\startsubsection[title={\LUATEX\ as a \LUA\ byte compiler}] + +\topicindex {\LUA+byte code} There are two situations that make \LUATEX\ behave like the \LUA\ byte compiler: @@ -46,9 +52,15 @@ There are two situations that make \LUATEX\ behave like the \LUA\ byte compiler: In this mode, \LUATEX\ is exactly like \type {luac} from the stand alone \LUA\ distribution, except that it does not have the \type {-l} switch, and that it -accepts (but ignores) the \type {--luaconly} switch. +accepts (but ignores) the \type {--luaconly} switch. The current version of \LUA\ +can dump bytecode using \type {string.dump} so we might decide to drop this +version of \LUATEX. -\subsection{Other commandline processing} +\stopsubsection + +\startsubsection[title={Other commandline processing}] + +\topicindex {command line} When the \LUATEX\ executable starts, it looks for the \type {--lua} command line option. If there is no \type {--lua} option, the command line is interpreted in a @@ -56,6 +68,8 @@ similar fashion as the other \TEX\ engines. Some options are accepted but have n consequence. The following command|-|line options are understood: \starttabulate[|l|p|] +\DB commandline argument \BC explanation \NC \NR +\TB \NC \type{--credits} \NC display credits and exit \NC \NR \NC \type{--debug-format} \NC enable format debugging \NC \NR \NC \type{--draftmode} \NC switch on draft mode i.e.\ generate no output in \PDF\ mode \NC \NR @@ -65,112 +79,137 @@ consequence. The following command|-|line options are understood: \NC \type{--halt-on-error} \NC stop processing at the first error\NC \NR \NC \type{--help} \NC display help and exit \NC\NR \NC \type{--ini} \NC be \type {iniluatex}, for dumping formats \NC\NR -\NC \type{--interaction=STRING} \NC set interaction mode: \type {batchmode}, \type {nonstopmode}, \type {scrollmode} or \type {errorstopmode} \NC \NR +\NC \type{--interaction=STRING} \NC set interaction mode: \type {batchmode}, \type {nonstopmode}, + \type {scrollmode} or \type {errorstopmode} \NC \NR \NC \type{--jobname=STRING} \NC set the job name to \type {STRING} \NC \NR -\NC \type{--kpathsea-debug=NUMBER} \NC set path searching debugging flags according to the bits of \type {NUMBER} \NC \NR +\NC \type{--kpathsea-debug=NUMBER} \NC set path searching debugging flags according to the bits of + \type {NUMBER} \NC \NR \NC \type{--lua=FILE} \NC load and execute a \LUA\ initialization script \NC\NR -\NC \type{--[no-]mktex=FMT} \NC disable/enable \type {mktexFMT} generation with \type {FMT} is \type {tex} or \type {tfm} \NC \NR +\NC \type{--[no-]mktex=FMT} \NC disable/enable \type {mktexFMT} generation with \type {FMT} is + \type {tex} or \type {tfm} \NC \NR \NC \type{--nosocket} \NC disable the \LUA\ socket library \NC\NR -\NC \type{--output-comment=STRING} \NC use \type {STRING} for \DVI\ file comment instead of date (no effect for \PDF) \NC \NR +\NC \type{--output-comment=STRING} \NC use \type {STRING} for \DVI\ file comment instead of date (no + effect for \PDF) \NC \NR \NC \type{--output-directory=DIR} \NC use \type {DIR} as the directory to write files to \NC \NR -\NC \type{--output-format=FORMAT} \NC use \type {FORMAT} for job output; \type {FORMAT} is \type {dvi} or \type {pdf} \NC \NR +\NC \type{--output-format=FORMAT} \NC use \type {FORMAT} for job output; \type {FORMAT} is \type {dvi} + or \type {pdf} \NC \NR \NC \type{--progname=STRING} \NC set the program name to \type {STRING} \NC \NR \NC \type{--recorder} \NC enable filename recorder \NC \NR \NC \type{--safer} \NC disable easily exploitable \LUA\ commands \NC\NR \NC \type{--[no-]shell-escape} \NC disable/enable system calls \NC \NR -\NC \type{--shell-restricted} \NC restrict system calls to a list of commands given in \type {texmf.cnf} \NC \NR +\NC \type{--shell-restricted} \NC restrict system calls to a list of commands given in \type + {texmf.cnf} \NC \NR \NC \type{--synctex=NUMBER} \NC enable \type {synctex} \NC \NR \NC \type{--utc} \NC use utc times when applicable \NC \NR \NC \type{--version} \NC display version and exit \NC \NR +\LL \stoptabulate -Some of the traditional flags are just ignored: \type {--etex}, \type -{--translate-file}, \type {--8bit}. \type {--[no-]parse-first-line}, \type -{--default-translate-file}. Also, we no longer support write18 because \type -{os.execute} can do the same. +We don't support \prm {write} 18 because \type {os.execute} can do the same. It +simplifies the code and makes more write targets possible. -The value to use for \type {\jobname} is decided as follows: +The value to use for \prm {jobname} is decided as follows: \startitemize \startitem If \type {--jobname} is given on the command line, its argument will be the - value for \type {\jobname}, without any changes. The argument will not be + value for \prm {jobname}, without any changes. The argument will not be used for actual input so it need not exist. The \type {--jobname} switch only - controls the \type {\jobname} setting. + controls the \prm {jobname} setting. \stopitem \startitem - Otherwise, \type {\jobname} will be the name of the first file that is read + Otherwise, \prm {jobname} will be the name of the first file that is read from the file system, with any path components and the last extension (the part following the last \type {.}) stripped off. \stopitem \startitem - An exception to the previous point: if the command line goes into interactive - mode (by starting with a command) and there are no files input via \type - {\everyjob} either, then the \type {\jobname} is set to \type {texput} as a - last resort. + There is an exception to the previous point: if the command line goes into + interactive mode (by starting with a command) and there are no files input + via \prm {everyjob} either, then the \prm {jobname} is set to \type + {texput} as a last resort. \stopitem \stopitemize The file names for output files that are generated automatically are created by attaching the proper extension (\type {log}, \type {pdf}, etc.) to the found -\type {\jobname}. These files are created in the directory pointed to by \type +\prm {jobname}. These files are created in the directory pointed to by \type {--output-directory}, or in the current directory, if that switch is not present. -\blank - Without the \type {--lua} option, command line processing works like it does in -any other web2c-based typesetting engine, except that \LUATEX\ has a few extra -switches. - -If the \type {--lua} option is present, \LUATEX\ will enter an alternative mode -of command line processing in comparison to the standard web2c programs. - -In this mode, a small series of actions is taken in order. First, it will parse -the command line as usual, but it will only interpret a small subset of the -options immediately: \type {--safer}, \type {--nosocket}, \type -{--[no-]shell-escape}, \type {--enable-write18}, \type {--disable-write18}, \type -{--shell-restricted}, \type {--help}, \type {--version}, and \type {--credits}. - -Next \LUATEX\ searches for the requested \LUA\ initialization script. If it -cannot be found using the actual name given on the command line, a second attempt -is made by prepending the value of the environment variable \type {LUATEXDIR}, if -that variable is defined in the environment. - -Then it checks the various safety switches. You can use those to disable some -\LUA\ commands that can easily be abused by a malicious document. At the moment, -\type {--safer} \type {nil}s the following functions: - -\starttabulate[|l|l|] -\BC library \BC functions \NC \NR -\NC \type {os} \NC \type {execute} \type {exec} \type {spawn} \type {setenv} \type {rename} \type {remove} \type {tmpdir} \NC \NR -\NC \type {io} \NC \type {popen} \type {output} \type {tmpfile} \NC \NR -\NC \type {lfs} \NC \type {rmdir} \type {mkdir} \type {chdir} \type {lock} \type {touch} \NC \NR -\stoptabulate +any other \WEBC|-|based typesetting engine, except that \LUATEX\ has a few extra +switches and lacks some others. Also, if the \type {--lua} option is present, +\LUATEX\ will enter an alternative mode of command line processing in comparison +to the standard \WEBC\ programs. In this mode, a small series of actions is taken +in the following order: -Furthermore, it disables loading of compiled \LUA\ libraries and it makes \type -{io.open()} fail on files that are opened for anything besides reading. +\startitemize[n] -When \LUATEX\ starts it set the locale to a neutral value. If for some reason you -use \type {os.locale}, you need to make sure you \type {nil} it afterwards -because otherwise it can interfere with code that for instance generates dates. -You can nil the locale with +\startitem + First, it will parse the command line as usual, but it will only interpret a + small subset of the options immediately: \type {--safer}, \type {--nosocket}, + \type {--[no-]shell-escape}, \type {--enable-write18}, \type + {--disable-write18}, \type {--shell-restricted}, \type {--help}, \type + {--version}, and \type {--credits}. +\stopitem -\starttyping -os.setlocale(nil.nil) -\stoptyping +\startitem + Next \LUATEX\ searches for the requested \LUA\ initialization script. If it + cannot be found using the actual name given on the command line, a second + attempt is made by prepending the value of the environment variable \type + {LUATEXDIR}, if that variable is defined in the environment. +\stopitem + +\startitem + Then it checks the various safety switches. You can use those to disable some + \LUA\ commands that can easily be abused by a malicious document. At the + moment, \type {--safer} \type {nil}s the following functions: + + \blank + + \starttabulate[|c|l|] + \DB library \BC functions \NC \NR + \TB + \NC \type {os} \NC \type {execute} \type {exec} \type {spawn} \type {setenv} + \type {rename} \type {remove} \type {tmpdir} \NC \NR + \NC \type {io} \NC \type {popen} \type {output} \type {tmpfile} \NC \NR + \NC \type {lfs} \NC \type {rmdir} \type {mkdir} \type {chdir} \type {lock} + \type {touch} \NC \NR + \LL + \stoptabulate -The \type {--nosocket} option makes the socket library unavailable, so that \LUA\ -cannot use networking. + \blank -The switches \type {--[no-]shell-escape}, \type {--[enable|disable]-write18}, and -\type {--shell-restricted} have the same effects as in \PDFTEX, and additionally -make \type {io.popen()}, \type {os.execute}, \type {os.exec} and \type {os.spawn} -adhere to the requested option. + Furthermore, it disables loading of compiled \LUA\ libraries and it makes + \type {io.open()} fail on files that are opened for anything besides reading. +\stopitem + +\startitem + When \LUATEX\ starts it sets the \type {locale} to a neutral value. If for + some reason you use \type {os.locale}, you need to make sure you \type {nil} + it afterwards because otherwise it can interfere with code that for instance + generates dates. You can ignore the \type {locale} with: -Next the initialization script is loaded and executed. From within the script, -the entire command line is available in the \LUA\ table \type {arg}, beginning with -\type {arg[0]}, containing the name of the executable. As consequence warnings -about unrecognized options are suppressed. + \starttyping + os.setlocale(nil,nil) + \stoptyping + + The \type {--nosocket} option makes the socket library unavailable, so that \LUA\ + cannot use networking. + + The switches \type {--[no-]shell-escape}, \type {--[enable|disable]-write18}, and + \type {--shell-restricted} have the same effects as in \PDFTEX, and additionally + make \type {io.popen()}, \type {os.execute}, \type {os.exec} and \type {os.spawn} + adhere to the requested option. +\stopitem + +\startitem + Next the initialization script is loaded and executed. From within the + script, the entire command line is available in the \LUA\ table \type {arg}, + beginning with \type {arg[0]}, containing the name of the executable. As + consequence warnings about unrecognized options are suppressed. +\stopitem + +\stopitemize Command line processing happens very early on. So early, in fact, that none of \TEX's initializations have taken place yet. For that reason, the tables that @@ -179,7 +218,7 @@ deal with typesetting, like \type {tex}, \type {token}, \type {node} and are \type {nil}'d). Special care is taken that \type {texio.write} and \type {texio.write_nl} function properly, so that you can at least report your actions to the log file when (and if) it eventually becomes opened (note that \TEX\ does -not even know its \type {\jobname} yet at this point). +not even know its \prm {jobname} yet at this point). Everything you do in the \LUA\ initialization script will remain visible during the rest of the run, with the exception of the \TEX\ specific libraries like @@ -205,17 +244,34 @@ finished: in order to initialize the built|-|in \KPATHSEA\ library properly, check \type {--progname}, or \type {--ini} and \type {--fmt}, if \type {--progname} is missing. -\section{\LUA\ behaviour} +\stopsubsection + +\stopsection + +\startsection[title={\LUA\ behaviour}] + +\startsubsection[title={The \LUA\ version}] + +\topicindex {\LUA+libraries} +\topicindex {\LUA+extensions} + +We currently use \LUA\ 5.3 and will follow developments of the language but +normally with some delay. Therefore the user needs to keep an eye on (subtle) +differences in successive versions of the language. Also, \LUAJITTEX\ lags behind +in the sense that \LUAJIT\ is not in sync with regular \LUA\ development. Here is +an example of one aspect. \LUA s \type {tostring} function (and \type {string.format} may return values in scientific notation, thereby confusing the \TEX\ end of things when it is used as -the right|-|hand side of an assignment to a \type {\dimen} or \type {\count}. +the right|-|hand side of an assignment to a \prm {dimen} or \prm {count}. The +output of these serializers also depend on the \LUA\ version, so in \LUA\ 5.3 you +can get different output than from 5.2. -Loading dynamic \LUA\ libraries will fail if there are two \LUA\ libraries loaded -at the same time (which will typically happen on \type {win32}, because there is -one \LUA\ 5.3 inside \LUATEX, and another will likely be linked to the \DLL\ file -of the module itself). +\stopsubsection + +\startsubsection[title={Integration in the \TDS\ ecosystem}] +The main \TEX\ distributions follow the \TEX\ directory structure (\TDS). \LUATEX\ is able to use the kpathsea library to find \type {require()}d modules. For this purpose, \type {package.searchers[2]} is replaced by a different loader function, that decides at runtime whether to use kpathsea or the built|-|in core @@ -226,6 +282,10 @@ Initialization of \KPATHSEA\ can happen either implicitly (when \LUATEX\ starts up and the startup script has not set \type {texconfig.kpse_init} to false), or explicitly by calling the \LUA\ function \type {kpse.set_program_name()}. +\stopsubsection + +\startsubsection[title={Loading libraries}] + \LUATEX\ is able to use dynamically loadable \LUA\ libraries, unless \type {--safer} was given as an option on the command line. For this purpose, \type {package.searchers[3]} is replaced by a different loader function, that @@ -233,12 +293,10 @@ decides at runtime whether to use \KPATHSEA\ or the built|-|in core \LUA\ function. It uses \KPATHSEA\ when that is already initialized at that point in time, otherwise it reverts to using the normal \type {package.cpath} loader. -This functionality required an extension to kpathsea: - -\startnarrower -There is a new kpathsea file format: \type {kpse_clua_format} that searches for -files with extension \type {.dll} and \type {.so}. The \type {texmf.cnf} setting -for this variable is \type {CLUAINPUTS}, and by default it has this value: +This functionality required an extension to kpathsea. There is a new kpathsea +file format: \type {kpse_clua_format} that searches for files with extension +\type {.dll} and \type {.so}. The \type {texmf.cnf} setting for this variable is +\type {CLUAINPUTS}, and by default it has this value: \starttyping CLUAINPUTS=.:$SELFAUTOLOC/lib/{$progname,$engine,}/lua// @@ -248,12 +306,18 @@ This path is imperfect (it requires a \TDS\ subtree below the binaries directory), but the architecture has to be in the path somewhere, and the currently simplest way to do that is to search below the binaries directory only. Of course it no big deal to write an alternative loader and use that in a macro -package. +package. One level up (a \type {lib} directory parallel to \type {bin}) would +have been nicer, but that is not doable because \TEXLIVE\ uses a \type +{bin/<arch>} structure. + +Loading dynamic \LUA\ libraries will fail if there are two \LUA\ libraries loaded +at the same time (which will typically happen on \type {win32}, because there is +one \LUA\ 5.3 inside \LUATEX, and another will likely be linked to the \DLL\ file +of the module itself). -One level up (a \type {lib} directory parallel to \type {bin}) would have been -nicer, but that is not doable because \TEXLIVE\ uses a \type {bin/<arch>} -structure. -\stopnarrower +\stopsubsection + +\startsubsection[title={Executing programs}] In keeping with the other \TEX|-|like programs in \TEXLIVE, the two \LUA\ functions \type {os.execute} and \type {io.popen}, as well as the two new functions \type @@ -265,26 +329,29 @@ the command line option \type {--luaonly} was not given), it will only run the four functions above if the matching \type {texmf.cnf} variable(s) or their \type {texconfig} (see \in {section} [texconfig]) counterparts allow execution of the requested system command. In \quote {script interpreter} runs of \LUATEX, these -settings have no effect, and all four functions function as normal. - -The \type {f:read("*line")} and \type {f:lines()} functions from the io library -have been adjusted so that they are line|-|ending neutral: any of \type {LF}, -\type {CR} or \type {CR+LF} are acceptable line endings. - -\type {luafilesystem} has been extended: there are two extra boolean functions -(\type {lfs.isdir(filename)} and \type {lfs.isfile(filename)}) and one extra -string field in its attributes table (\type {permissions}). There is an -additional function \type {lfs.shortname()} which takes a file name and returns -its short name on \type {win32} platforms. On other platforms, it just returns -the given argument. The file name is not tested for existence. Finally, for -non|-|\type {win32} platforms only, there is the new function \type -{lfs.readlink()} hat takes an existing symbolic link as argument and returns its -content. It returns an error on \type {win32}. - -The \type {string} library has an extra function: \type {string.explode(s[,m])}. -This function returns an array containing the string argument \type {s} split -into sub-strings based on the value of the string argument \type {m}. The second -argument is a string that is either empty (this splits the string into +settings have no effect, and all four functions have their original meaning. + +Some libraries have a few more functions, either coded in \CCODE\ or in \LUA. For +instance, when we started with \LUATEX\ we added some helpers to the \type +{luafilesystem} namespace \type {lfs}. The two boolean functions \type +{lfs.isdir} and \type {lfs.isfile} were speedy and better variants of what could +be done with \type {lfs.attributes}. The additional function \type +{lfs.shortname} takes a file name and returns its short name on \type {win32} +platforms. Finally, for non|-|\type {win32} platforms only, we provided \type +{lfs.readlink} that takes an existing symbolic link as argument and returns its +name. However, the \type library evolved so we have dropped these in favour of +pure \LUA\ variants. The \type {shortname} helper is obsolete and now just +returns the name. + +\stopsubsection + +\startsubsection[title={Multibyte \type {string} functions}] + +The \type {string} library has a few extra functions, for example \libidx +{string} {explode}. This function takes upto two arguments: \type +{string.explode(s[,m])} and returns an array containing the string argument \type +{s} split into sub-strings based on the value of the string argument \type {m}. +The second argument is a string that is either empty (this splits the string into characters), a single character (this splits on each occurrence of that character, possibly introducing empty strings), or a single character followed by the plus sign \type {+} (this special version does not create empty sub-strings). @@ -293,7 +360,9 @@ The default value for \type {m} is \quote {\type { +}} (multiple spaces). Note: written in \TEX\ macros. The \type {string} library also has six extra iterators that return strings -piecemeal: +piecemeal: \libidx {string} {utfvalues}, \libidx {string} {utfcharacters}, +\libidx {string} {characters}, \libidx {string} {characterpairs}, \libidx +{string} {bytes} and \libidx {string} {bytepairs}. \startitemize \startitem @@ -303,7 +372,7 @@ piecemeal: \type {string.utfcharacters(s)}: a string with a single \UTF-8 token in it \stopitem \startitem - \type {string.characters(s)}: a string containing one byte + \type {string.cWharacters(s)}: a string containing one byte \stopitem \startitem \type {string.characterpairs(s)}: two strings each containing one byte or an @@ -323,7 +392,8 @@ are useful especially in the conversion of \UTF16 encoded data into \UTF8. There is also a two|-|argument form of \type {string.dump()}. The second argument is a boolean which, if true, strips the symbols from the dumped data. This -matches an extension made in \type {luajit}. +matches an extension made in \type {luajit}. This is typically a function that +gets adapted as \LUA\ itself progresses. The \type {string} library functions \type {len}, \type {lower}, \type {sub} etc.\ are not \UNICODE|-|aware. For strings in the \UTF8 encoding, i.e., strings @@ -338,7 +408,8 @@ interpretation of character classes in \type {unicode.utf8} functions refer to the library sources at \hyphenatedurl {http://luaforge.net/projects/sln}. Version 5.3 of \LUA\ provides some native \UTF8 support but we have added a few -similar helpers too: +similar helpers too: \libidx {string} {utfvalue}, \libidx {string} {utfcharacter} +and \libidx {string} {utflength}. \startitemize \startitem @@ -354,13 +425,18 @@ similar helpers too: \stopitem \stopitemize -These three functions are relative fast and don't do much checking. They can be used -as building blocks for other helpers. +These three functions are relative fast and don't do much checking. They can be +used as building blocks for other helpers. +\stopsubsection -\blank +\startsubsection[title={Extra \type {os} library functions}] -The \type {os} library has a few extra functions and variables: +The \type {os} library has a few extra functions and variables: \libidx {os} +{selfdir}, \libidx {os} {exec}, \libidx {os} {spawn}, \libidx {os} {setenv}, +\libidx {os} {env}, \libidx {os} {gettimeofday}, \libidx {os} {times}, \libidx +{os} {tmpdir}, \libidx {os} {type}, \libidx {os} {name} and \libidx {os} {uname}, +that we will discuss here. \startitemize @@ -373,25 +449,37 @@ The \type {os} library has a few extra functions and variables: \type {os.exec(commandline)} is a variation on \type {os.execute}. Here \type {commandline} can be either a single string or a single table. - If the argument is a table \LUATEX\ first checks if there is a value at - integer index zero. If there is, this is the command to be executed. - Otherwise, it will use the value at integer index one. If neither are - present, nothing at all happens. - - The set of consecutive values starting at integer~1 in the table are the - arguments that are passed on to the command (the value at index~1 becomes - \type {arg[0]}). The command is searched for in the execution path, so there - is normally no need to pass on a fully qualified path name. - - If the argument is a string, then it is automatically converted into a table - by splitting on whitespace. In this case, it is impossible for the command - and first argument to differ from each other. - - In the string argument format, whitespace can be protected by putting (part - of) an argument inside single or double quotes. One layer of quotes is - interpreted by \LUATEX, and all occurrences of \type {\"}, \type {\'} or \type - {\\} within the quoted text are unescaped. In the table format, there is no - string handling taking place. + \startitemize + + \startitem + If the argument is a table \LUATEX\ first checks if there is a value at + integer index zero. If there is, this is the command to be executed. + Otherwise, it will use the value at integer index one. If neither are + present, nothing at all happens. + \stopitem + + \startitem + The set of consecutive values starting at integer~1 in the table are the + arguments that are passed on to the command (the value at index~1 becomes + \type {arg[0]}). The command is searched for in the execution path, so + there is normally no need to pass on a fully qualified path name. + \stopitem + + \startitem + If the argument is a string, then it is automatically converted into a + table by splitting on whitespace. In this case, it is impossible for the + command and first argument to differ from each other. + \stopitem + + \startitem + In the string argument format, whitespace can be protected by putting + (part of) an argument inside single or double quotes. One layer of quotes + is interpreted by \LUATEX, and all occurrences of \type {\"}, \type {\'} + or \type {\\} within the quoted text are unescaped. In the table format, + there is no string handling taking place. + \stopitem + + \stopitemize This function normally does not return control back to the \LUA\ script: the command will replace the current process. However, it will return the two @@ -470,14 +558,86 @@ The \type {os} library has a few extra functions and variables: \stopitem \startitem - \type {os.uname()} returns a table with specific operating system + \type {os.uname} returns a table with specific operating system information acquired at runtime. The keys in the returned table are all - string valued, and their names are: \type {sysname}, \type {machine}, \type + string values, and their names are: \type {sysname}, \type {machine}, \type {release}, \type {version}, and \type {nodename}. \stopitem \stopitemize +\stopsubsection + +\startsubsection[title={Binary input from files with \type {fio}}] + +There is a whole set of helpers for reading numbers and strings from a file: +\libidx {fio} {readcardinal1}, \libidx {fio} {readcardinal2}, \libidx {fio} +{readcardinal3}, \libidx {fio} {readcardinal4}, \libidx {fio} +{readcardinaltable}, \libidx {fio} {readinteger1}, \libidx {fio} {readinteger2}, +\libidx {fio} {readinteger3}, \libidx {fio} {readinteger4}, \libidx {fio} +{readintegertable}, \libidx {fio} {readfixed2}, \libidx {fio} {readfixed4}, +\libidx {fio} {read2dot14}, \libidx {fio} {setposition}, \libidx {fio} +{getposition}, \libidx {fio} {skipposition}, \libidx {fio} {readbytes}, \libidx +{fio} {readbytetable}. They work on normal \LUA\ file handles. + +%libidx{fio}{readline} +%libidx{fio}{recordfilename} +%libidx{fio}{checkpermission} + +This library provides a set of functions for reading numbers from a file and +in addition to the regular \type {io} library functions. + +\starttabulate +\NC \type{readcardinal1(f)} \NC a 1 byte unsigned integer \NC \NR +\NC \type{readcardinal2(f)} \NC a 2 byte unsigned integer \NC \NR +\NC \type{readcardinal3(f)} \NC a 3 byte unsigned integer \NC \NR +\NC \type{readcardinal4(f)} \NC a 4 byte unsigned integer \NC \NR +\NC \type{readcardinaltable(f,n,b)} \NC \type {n} cardinals of \type {b} bytes \NC \NR +\NC \type{readinteger1(f)} \NC a 1 byte signed integer \NC \NR +\NC \type{readinteger2(f)} \NC a 2 byte signed integer \NC \NR +\NC \type{readinteger3(f)} \NC a 3 byte signed integer \NC \NR +\NC \type{readinteger4(f)} \NC a 4 byte signed integer \NC \NR +\NC \type{readintegertable(f,n,b)} \NC \type {n} integers of \type {b} bytes \NC \NR +\NC \type{readfixed2(f)} \NC a 2 byte float (used in font files) \NC \NR +\NC \type{readfixed4(f)} \NC a 4 byte float (used in font files) \NC \NR +\NC \type{read2dot14(f)} \NC a 2 byte float (used in font files) \NC \NR +\NC \type{setposition(f,p)} \NC goto position \type {p} \NC \NR +\NC \type{getposition(f)} \NC get the current position \NC \NR +\NC \type{skipposition(f,n)} \NC skip \type {n} positions \NC \NR +\NC \type{readbytes(f,n)} \NC \type {n} bytes \NC \NR +\NC \type{readbytetable(f,n)} \NC \type {n} bytes\NC \NR +\stoptabulate + +\stopsubsection + +\startsubsection[title={Binary input from strings with \type {sio}}] + +A similar set of function as in the \type {fio} library is available in the \type +{sio} library: \libidx {sio} {readcardinal1}, \libidx {sio} {readcardinal2}, +\libidx {sio} {readcardinal3}, \libidx {sio} {readcardinal4}, \libidx {sio} +{readcardinaltable}, \libidx {sio} {readinteger1}, \libidx {sio} {readinteger2}, +\libidx {sio} {readinteger3}, \libidx {sio} {readinteger4}, \libidx {sio} +{readintegertable}, \libidx {sio} {readfixed2}, \libidx {sio} {readfixed4}, +\libidx {sio} {read2dot14}, \libidx {sio} {setposition}, \libidx {sio} +{getposition}, \libidx {sio} {skipposition}, \libidx {sio} {readbytes} and +\libidx {sio} {readbytetable}. Here the first argument is a string instead of a +file handle. More details can be found in the previous section. + +\stopsubsection + +\startsubsection[title={Hashes conform \type {sha2}}] + +This library is a side effect of the \type {pdfe} library that needs such +helpers. The \libidx{sha2}{digest256}, \libidx{sha2}{digest384} and +\libidx{sha2}{digest512} functions accept a string and return a string with the +hash. + +\stopsubsection + +\startsubsection[title={Locales}] + +\index {locales} + In stock \LUA, many things depend on the current locale. In \LUATEX, we can't do that, because it makes documents unportable. While \LUATEX\ is running if forces the following locale settings: @@ -488,7 +648,14 @@ LC_COLLATE=C LC_NUMERIC=C \stoptyping -\section {\LUA\ modules} +\stopsubsection + +\stopsection + +\startsection[title={\LUA\ modules}] + +\topicindex {\LUA+libraries} +\topicindex {\LUA+modules} Some modules that are normally external to \LUA\ are statically linked in with \LUATEX, because they offer useful functionality: @@ -496,11 +663,27 @@ Some modules that are normally external to \LUA\ are statically linked in with \startitemize \startitem + \type {lpeg}, by Roberto Ierusalimschy, \hyphenatedurl + {http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html}. This library is not + \UNICODE|-|aware, but interprets strings on a byte|-|per|-|byte basis. This + mainly means that \type {lpeg.S} cannot be used with \UTF8 characters encoded + in more than two bytes, and thus \type {lpeg.S} will look for one of those + two bytes when matching, not the combination of the two. The same is true for + \type {lpeg.R}, although the latter will display an error message if used + with multibyte characters. Therefore \type {lpeg.R('aä')} results in the + message \type {bad argument #1 to 'R' (range must have two characters)}, + since to \type {lpeg}, \type {ä} is two 'characters' (bytes), so \type {aä} + totals three. In practice this is no real issue and with some care you can + deal with \UNICODE\ just fine. +\stopitem + +\startitem \type {slnunicode}, from the \type {selene} libraries, \hyphenatedurl {http://luaforge.net/projects/sln}. This library has been slightly extended so that the \type {unicode.utf8.*} functions also accept the first 256 values of plane~18. This is the range \LUATEX\ uses for raw binary output, as - explained above. + explained above. We have no plans to provide more like this because you can + basically do all that you want in \LUA. \stopitem \startitem @@ -514,20 +697,6 @@ Some modules that are normally external to \LUA\ are statically linked in with \stopitem \startitem - \type {lpeg}, by Roberto Ierusalimschy, \hyphenatedurl - {http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html}. This library is not - \UNICODE|-|aware, but interprets strings on a byte|-|per|-|byte basis. This - mainly means that \type {lpeg.S} cannot be used with \UTF8 characters encoded - in more than two bytes, and thus \type {lpeg.S} will look for one of those - two bytes when matching, not the combination of the two. The same is true for - \type {lpeg.R}, although the latter will display an error message if used - with multibyte characters. Therefore \type {lpeg.R('aä')} results in the - message \type {bad argument #1 to 'R' (range must have two characters)}, - since to \type {lpeg}, \type {ä} is two 'characters' (bytes), so \type {aä} - totals three. In practice this is no real issue. -\stopitem - -\startitem \type {lzlib}, by Tiago Dionizio, \hyphenatedurl {http://luaforge.net/projects/lzlib/}. \stopitem @@ -546,11 +715,12 @@ Some modules that are normally external to \LUA\ are statically linked in with \stopitemize -At some point (this also depends on distributions) \LUATEX\ might have these -libraries loaded on demand. For this reason you can best use \type {require} to -make sure they are loaded. +\stopsection -\section{Testing} +\startsection[title={Testing}] + +\topicindex {testing} +\topicindex {\PDF+date} For development reasons you can influence the used startup date and time. This can be done in two ways. @@ -579,17 +749,11 @@ When Universal Time is needed, you can pass the flag \type {utc} to the engine. property also works when the date and time are set by \LUATEX\ itself. It has a complementary entry \type {use_utc_time} in the \type {texconfig} table. -\startnotabene - To some extend a cleaner solution would be to have a flag that disables all - variable data in one go (like filenames and so) but we just follow the method - implemented in \PDFTEX\ where primitives are used to influence other - properties. -\stopnotabene - -\startnotabene - In \CONTEXT\ we provide the command line argument \type {--nodates} that does - bit more disabling of dates. -\stopnotabene +There is some control possible, for instance prevent filename to be written to +the \PDF\ file. This is discussed elsewhere. In \CONTEXT\ we provide the command +line argument \type {--nodates} that does a bit more disabling of dates. + +\stopsection \stopchapter diff --git a/doc/context/sources/general/manuals/luatex/luatex-math.tex b/doc/context/sources/general/manuals/luatex/luatex-math.tex index 8ccae83f3..4623ce706 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-math.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-math.tex @@ -1,12 +1,15 @@ % language=uk \environment luatex-style -\environment luatex-logos \startcomponent luatex-math \startchapter[reference=math,title={Math}] +\startsection[title={Traditional alongside \OPENTYPE}] + +\topicindex {math} + The handling of mathematics in \LUATEX\ differs quite a bit from how \TEX82 (and therefore \PDFTEX) handles math. First, \LUATEX\ adds primitives and extends some others so that \UNICODE\ input can be used easily. Second, all of \TEX82's @@ -16,18 +19,133 @@ make it easier to use \OPENTYPE\ math fonts. And finally, there are some extensions that have been proposed or considered in the past that are now added to the engine. -\section{The current math style} +\stopsection + +\startsection[title={Unicode math characters}] + +\topicindex {math+\UNICODE} +\topicindex {\UNICODE+math} + +Character handling is now extended up to the full \UNICODE\ range (the \type {\U} +prefix), which is compatible with \XETEX. + +The math primitives from \TEX\ are kept as they are, except for the ones that +convert from input to math commands: \type {mathcode}, and \type {delcode}. These +two now allow for a 21-bit character argument on the left hand side of the equals +sign. + +Some of the new \LUATEX\ primitives read more than one separate value. This is +shown in the tables below by a plus sign. + +The input for such primitives would look like this: + +\starttyping +\def\overbrace{\Umathaccent 0 1 "23DE } +\stoptyping + +The altered \TEX82 primitives are: + +\starttabulate[|l|l|r|c|l|r|] +\DB primitive \BC min \BC max \BC \kern 2em \BC min \BC max \NC \NR +\TB +\NC \prm {mathcode} \NC 0 \NC 10FFFF \NC = \NC 0 \NC 8000 \NC \NR +\NC \prm {delcode} \NC 0 \NC 10FFFF \NC = \NC 0 \NC FFFFFF \NC \NR +\LL +\stoptabulate + +The unaltered ones are: + +\starttabulate[|l|l|r|] +\DB primitive \BC min \BC max \NC \NR +\TB +\NC \prm {mathchardef} \NC 0 \NC 8000 \NC \NR +\NC \prm {mathchar} \NC 0 \NC 7FFF \NC \NR +\NC \prm {mathaccent} \NC 0 \NC 7FFF \NC \NR +\NC \prm {delimiter} \NC 0 \NC 7FFFFFF \NC \NR +\NC \prm {radical} \NC 0 \NC 7FFFFFF \NC \NR +\LL +\stoptabulate + +For practical reasons \prm {mathchardef} will silently accept values larger +that \type {0x8000} and interpret it as \lpr {Umathcharnumdef}. This is needed +to satisfy older macro packages. + +The following new primitives are compatible with \XETEX: + +% somewhat fuzzy: + +\starttabulate[|l|l|r|c|l|r|] +\DB primitive \BC min \BC max \BC \kern 2em \BC min \BC max \NC \NR +\TB +\NC \lpr {Umathchardef} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR +\NC \lpr {Umathcharnumdef}\rlap{\high{5}} \NC -80000000 \NC 7FFFFFFF \NC \NC \NC \NC \NR +\NC \lpr {Umathcode} \NC 0 \NC 10FFFF \NC = \NC 0+0+0 \NC 7+FF+10FFFF \NC \NR +\NC \lpr {Udelcode} \NC 0 \NC 10FFFF \NC = \NC 0+0 \NC FF+10FFFF \NC \NR +\NC \lpr {Umathchar} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR +\NC \lpr {Umathaccent} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR +\NC \lpr {Udelimiter} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR +\NC \lpr {Uradical} \NC 0+0 \NC FF+10FFFF \NC \NC \NC \NC \NR +\NC \lpr {Umathcharnum} \NC -80000000 \NC 7FFFFFFF \NC \NC \NC \NC \NR +\NC \lpr {Umathcodenum} \NC 0 \NC 10FFFF \NC = \NC -80000000 \NC 7FFFFFFF \NC \NR +\NC \lpr {Udelcodenum} \NC 0 \NC 10FFFF \NC = \NC -80000000 \NC 7FFFFFFF \NC \NR +\LL +\stoptabulate + +Specifications typically look like: + +\starttyping +\Umathchardef\xx="1"0"456 +\Umathcode 123="1"0"789 +\stoptyping + +The new primitives that deal with delimiter|-|style objects do not set up a +\quote {large family}. Selecting a suitable size for display purposes is expected +to be dealt with by the font via the \lpr {Umathoperatorsize} parameter. + +For some of these primitives, all information is packed into a single signed +integer. For the first two (\lpr {Umathcharnum} and \lpr {Umathcodenum}), the +lowest 21 bits are the character code, the 3 bits above that represent the math +class, and the family data is kept in the topmost bits. This means that the values +for math families 128--255 are actually negative. For \lpr {Udelcodenum} there +is no math class. The math family information is stored in the bits directly on +top of the character code. Using these three commands is not as natural as using +the two- and three|-|value commands, so unless you know exactly what you are +doing and absolutely require the speedup resulting from the faster input +scanning, it is better to use the verbose commands instead. + +The \lpr {Umathaccent} command accepts optional keywords to control various +details regarding math accents. See \in {section} [mathacc] below for details. + +There are more new primitives and all of these will be explained in following +sections: + +\starttabulate[|l|l|] +\DB primitive \BC value range (in hex) \NC \NR +\TB +\NC \lpr {Uroot} \NC 0 + 0--FF + 10FFFF \NC \NR +\NC \lpr {Uoverdelimiter} \NC 0 + 0--FF + 10FFFF \NC \NR +\NC \lpr {Uunderdelimiter} \NC 0 + 0--FF + 10FFFF \NC \NR +\NC \lpr {Udelimiterover} \NC 0 + 0--FF + 10FFFF \NC \NR +\NC \lpr {Udelimiterunder} \NC 0 + 0--FF + 10FFFF \NC \NR +\LL +\stoptabulate + +\stopsection + +\startsection[title={Math styles}] + +\subsection{\lpr {mathstyle}} + +\topicindex {math+styles} It is possible to discover the math style that will be used for a formula in an expandable fashion (while the math list is still being read). To make this -possible, \LUATEX\ adds the new primitive: \type {\mathstyle}. This is a \quote -{convert command} like e.g. \type {\romannumeral}: its value can only be read, +possible, \LUATEX\ adds the new primitive: \lpr {mathstyle}. This is a \quote +{convert command} like e.g. \prm {romannumeral}: its value can only be read, not set. -\subsection{\type {\mathstyle}} - The returned value is between 0 and 7 (in math mode), or $-1$ (all other modes). -For easy testing, the eight math style commands have been altered so that the can +For easy testing, the eight math style commands have been altered so that they can be used as numeric values, so you can write code like this: \starttyping @@ -48,16 +166,15 @@ thereby reusing numbers) because the number that got used is stored and used in the second pass (so changing \type {\fam 12} mid|-|formula spoils over to preceding use of that family). -The style switching primitives like \type {\textstyle} are turned into nodes so -the styles set there are frozen. The \type {\mathchoice} primitive results in -four lists being constructed of which one is used in the second pass. The fact -that some automatic styles are not yet known also means that the \type -{\mathstyle} primitive expands to the current style which can of course be -different from the one really used. It's a snapshot of the first pass state. As a -consequence in the following example you get a style number (first pass) typeset -that can actually differ from the used style (second pass). In the case of a math -choice used ungrouped, the chosen style is used after the choice too, unless you -group. +The style switching primitives like \prm {textstyle} are turned into nodes so the +styles set there are frozen. The \prm {mathchoice} primitive results in four +lists being constructed of which one is used in the second pass. The fact that +some automatic styles are not yet known also means that the \lpr {mathstyle} +primitive expands to the current style which can of course be different from the +one really used. It's a snapshot of the first pass state. As a consequence in the +following example you get a style number (first pass) typeset that can actually +differ from the used style (second pass). In the case of a math choice used +ungrouped, the chosen style is used after the choice too, unless you group. \startbuffer[1] [a:\mathstyle]\quad @@ -120,134 +237,39 @@ This gives: \blank $\displaystyle \getbuffer[1]$ \blank \blank $\textstyle \getbuffer[1]$ \blank -Using \type {\begingroup} \unknown\ \type {\endgroup} instead gives: +Using \prm {begingroup} \unknown\ \prm {endgroup} instead gives: \blank $\displaystyle \getbuffer[2]$ \blank \blank $\textstyle \getbuffer[2]$ \blank -This might look wrong but it's just a side effect of \type {\mathstyle} expanding +This might look wrong but it's just a side effect of \lpr {mathstyle} expanding to the current (first pass) style and the number being injected in the list that gets converted in the second pass. It all makes sense and it illustrates the importance of grouping. In fact, the math choice style being effective afterwards has advantages. It would be hard to get it otherwise. -\subsection{\type {\Ustack}} +\subsection{\lpr {Ustack}} + +\topicindex {math+stacks} There are a few math commands in \TEX\ where the style that will be used is not -known straight from the start. These commands (\type {\over}, \type {\atop}, -\type {\overwithdelims}, \type {\atopwithdelims}) would therefore normally return -wrong values for \type {\mathstyle}. To fix this, \LUATEX\ introduces a special -prefix command: \type {\Ustack}: +known straight from the start. These commands (\prm {over}, \prm {atop}, +\prm {overwithdelims}, \prm {atopwithdelims}) would therefore normally return +wrong values for \lpr {mathstyle}. To fix this, \LUATEX\ introduces a special +prefix command: \lpr {Ustack}: \starttyping $\Ustack {a \over b}$ \stoptyping -The \type {\Ustack} command will scan the next brace and start a new math group +The \lpr {Ustack} command will scan the next brace and start a new math group with the correct (numerator) math style. -\section{Unicode math characters} - -Character handling is now extended up to the full \UNICODE\ range (the \type {\U} -prefix), which is compatible with \XETEX. - -The math primitives from \TEX\ are kept as they are, except for the ones that -convert from input to math commands: \type {mathcode}, and \type {delcode}. These -two now allow for a 21-bit character argument on the left hand side of the equals -sign. - -Some of the new \LUATEX\ primitives read more than one separate value. This is -shown in the tables below by a plus sign in the second column. - -The input for such primitives would look like this: - -\starttyping -\def\overbrace{\Umathaccent 0 1 "23DE } -\stoptyping - -The altered \TEX82 primitives are: - -\starttabulate[|l|l|r|c|l|r|] -\BC primitive \BC min \BC max \BC \kern 2em \BC min \BC max \NC \NR -\NC \type {\mathcode} \NC 0 \NC 10FFFF \NC = \NC 0 \NC 8000 \NC \NR -\NC \type {\delcode} \NC 0 \NC 10FFFF \NC = \NC 0 \NC FFFFFF \NC \NR -\stoptabulate - -The unaltered ones are: - -\starttabulate[|l|l|r|] -\BC primitive \BC min \BC max \NC \NR -\NC \type {\mathchardef} \NC 0 \NC 8000 \NC \NR -\NC \type {\mathchar} \NC 0 \NC 7FFF \NC \NR -\NC \type {\mathaccent} \NC 0 \NC 7FFF \NC \NR -\NC \type {\delimiter} \NC 0 \NC 7FFFFFF \NC \NR -\NC \type {\radical} \NC 0 \NC 7FFFFFF \NC \NR -\stoptabulate +\subsection{Cramped math styles} -For practical reasons \type {\mathchardef} will silently accept values larger -that \type {0x8000} and interpret it as \type {\Umathcharnumdef}. This is needed -to satisfy older macro packages. - -The following new primitives are compatible with \XETEX: - -% somewhat fuzzy: - -\starttabulate[|l|l|r|c|l|r|] -\BC primitive \BC min \BC max \BC \kern 2em \BC min \BC max \NC \NR -\NC \type {\Umathchardef} \NC 0+0+0 \NC 7+FF+10FFFF\rlap{\high{1}} \NC \NC \NC \NC \NR -\NC \type {\Umathcharnumdef}\rlap{\high{5}} \NC -80000000 \NC 7FFFFFFF\rlap{\high{3}} \NC \NC \NC \NC \NR -\NC \type {\Umathcode} \NC 0 \NC 10FFFF \NC = \NC 0+0+0 \NC 7+FF+10FFFF\rlap{\high{1}} \NC \NR -\NC \type {\Udelcode} \NC 0 \NC 10FFFF \NC = \NC 0+0 \NC FF+10FFFF\rlap{\high{2}} \NC \NR -\NC \type {\Umathchar} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR -\NC \type {\Umathaccent} \NC 0+0+0 \NC 7+FF+10FFFF\rlap{\high{2,4}} \NC \NC \NC \NC \NR -\NC \type {\Udelimiter} \NC 0+0+0 \NC 7+FF+10FFFF\rlap{\high{2}} \NC \NC \NC \NC \NR -\NC \type {\Uradical} \NC 0+0 \NC FF+10FFFF\rlap{\high{2}} \NC \NC \NC \NC \NR -\NC \type {\Umathcharnum} \NC -80000000 \NC 7FFFFFFF\rlap{\high{3}} \NC \NC \NC \NC \NR -\NC \type {\Umathcodenum} \NC 0 \NC 10FFFF \NC = \NC -80000000 \NC 7FFFFFFF\rlap{\high{3}} \NC \NR -\NC \type {\Udelcodenum} \NC 0 \NC 10FFFF \NC = \NC -80000000 \NC 7FFFFFFF\rlap{\high{3}} \NC \NR -\stoptabulate - -Specifications typically look like: - -\starttyping -\Umathchardef\xx="1"0"456 -\Umathcode 123="1"0"789 -\stoptyping - -Note 1: The new primitives that deal with delimiter|-|style objects do not set up a -\quote {large family}. Selecting a suitable size for display purposes is expected -to be dealt with by the font via the \type {\Umathoperatorsize} parameter (more -information can be found in a following section). - -Note 2: For these three primitives, all information is packed into a single -signed integer. For the first two (\type {\Umathcharnum} and \type -{\Umathcodenum}), the lowest 21 bits are the character code, the 3 bits above -that represent the math class, and the family data is kept in the topmost bits -(This means that the values for math families 128--255 are actually negative). -For \type {\Udelcodenum} there is no math class. The math family information is -stored in the bits directly on top of the character code. Using these three -commands is not as natural as using the two- and three|-|value commands, so -unless you know exactly what you are doing and absolutely require the speedup -resulting from the faster input scanning, it is better to use the verbose -commands instead. - -Note 3: The \type {\Umathaccent} command accepts optional keywords to control -various details regarding math accents. See \in {section} [mathacc] below for -details. - -New primitives that exist in \LUATEX\ only (all of these will be explained -in following sections): - -\starttabulate[|l|l|] -\BC primitive \BC value range (in hex) \NC \NR -\NC \type {\Uroot} \NC 0+0--FF+10FFFF$^2$ \NC \NR -\NC \type {\Uoverdelimiter} \NC 0+0--FF+10FFFF$^2$ \NC \NR -\NC \type {\Uunderdelimiter} \NC 0+0--FF+10FFFF$^2$ \NC \NR -\NC \type {\Udelimiterover} \NC 0+0--FF+10FFFF$^2$ \NC \NR -\NC \type {\Udelimiterunder} \NC 0+0--FF+10FFFF$^2$ \NC \NR -\stoptabulate - -\section{Cramped math styles} +\topicindex {math+styles} +\topicindex {math+spacing} +\topicindex {math+cramped} \LUATEX\ has four new primitives to set the cramped math styles directly: @@ -282,20 +304,23 @@ are described as follows: style if the original style was cramped. \stopitem \startitem - Formulas under a \type {\sqrt} or \type {\overline} are in cramped style. + Formulas under a \type {\sqrt} or \prm {overline} are in cramped style. \stopitem \stopitemize In \LUATEX\ one can set the styles in more detail which means that you sometimes -have to set both normal and cramped styles to get the effect you want. If we -force styles in the script using \type {\scriptstyle} and \type {\crampedscriptstyle} -we get this: +have to set both normal and cramped styles to get the effect you want. (Even) if +we force styles in the script using \prm {scriptstyle} and \lpr +{crampedscriptstyle} we get this: \startbuffer[demo] \starttabulate +\DB style \BC example \NC \NR +\TB \NC default \NC $b_{x=xx}^{x=xx}$ \NC \NR \NC script \NC $b_{\scriptstyle x=xx}^{\scriptstyle x=xx}$ \NC \NR \NC crampedscript \NC $b_{\crampedscriptstyle x=xx}^{\crampedscriptstyle x=xx}$ \NC \NR +\LL \stoptabulate \stopbuffer @@ -310,7 +335,7 @@ Now we set the following parameters \typebuffer[setup] -This gives: +This gives a different result: \start\getbuffer[setup,demo]\stop @@ -329,69 +354,77 @@ Now we get: \start\getbuffer[setup,demo]\stop -\section{Math parameter settings} +\stopsection + +\startsection[title={Math parameter settings}] + +\subsection {Many new \lpr {Umath*} primitives} + +\topicindex {math+parameters} In \LUATEX, the font dimension parameters that \TEX\ used in math typesetting are now accessible via primitive commands. In fact, refactoring of the math engine -has resulted in many more parameters than were accessible before. +has resulted in many more parameters than were not accessible before. \starttabulate -\BC primitive name \BC description \NC \NR -\NC \type {\Umathquad} \NC the width of 18 mu's \NC \NR -\NC \type {\Umathaxis} \NC height of the vertical center axis of +\DB primitive name \BC description \NC \NR +\TB +\NC \lpr {Umathquad} \NC the width of 18 mu's \NC \NR +\NC \lpr {Umathaxis} \NC height of the vertical center axis of the math formula above the baseline \NC \NR -\NC \type {\Umathoperatorsize} \NC minimum size of large operators in display mode \NC \NR -\NC \type {\Umathoverbarkern} \NC vertical clearance above the rule \NC \NR -\NC \type {\Umathoverbarrule} \NC the width of the rule \NC \NR -\NC \type {\Umathoverbarvgap} \NC vertical clearance below the rule \NC \NR -\NC \type {\Umathunderbarkern} \NC vertical clearance below the rule \NC \NR -\NC \type {\Umathunderbarrule} \NC the width of the rule \NC \NR -\NC \type {\Umathunderbarvgap} \NC vertical clearance above the rule \NC \NR -\NC \type {\Umathradicalkern} \NC vertical clearance above the rule \NC \NR -\NC \type {\Umathradicalrule} \NC the width of the rule \NC \NR -\NC \type {\Umathradicalvgap} \NC vertical clearance below the rule \NC \NR -\NC \type {\Umathradicaldegreebefore}\NC the forward kern that takes place before placement of - the radical degree \NC \NR -\NC \type {\Umathradicaldegreeafter} \NC the backward kern that takes place after placement of - the radical degree \NC \NR -\NC \type {\Umathradicaldegreeraise} \NC this is the percentage of the total height and depth of - the radical sign that the degree is raised by; it is - expressed in \type {percents}, so 60\% is expressed as the - integer $60$ \NC \NR -\NC \type {\Umathstackvgap} \NC vertical clearance between the two - elements in a \type {\atop} stack \NC \NR -\NC \type {\Umathstacknumup} \NC numerator shift upward in \type {\atop} stack \NC \NR -\NC \type {\Umathstackdenomdown} \NC denominator shift downward in \type {\atop} stack \NC \NR -\NC \type {\Umathfractionrule} \NC the width of the rule in a \type {\over} \NC \NR -\NC \type {\Umathfractionnumvgap} \NC vertical clearance between the numerator and the rule \NC \NR -\NC \type {\Umathfractionnumup} \NC numerator shift upward in \type {\over} \NC \NR -\NC \type {\Umathfractiondenomvgap} \NC vertical clearance between the denominator and the rule \NC \NR -\NC \type {\Umathfractiondenomdown} \NC denominator shift downward in \type {\over} \NC \NR -\NC \type {\Umathfractiondelsize} \NC minimum delimiter size for \type {\...withdelims} \NC \NR -\NC \type {\Umathlimitabovevgap} \NC vertical clearance for limits above operators \NC \NR -\NC \type {\Umathlimitabovebgap} \NC vertical baseline clearance for limits above operators \NC \NR -\NC \type {\Umathlimitabovekern} \NC space reserved at the top of the limit \NC \NR -\NC \type {\Umathlimitbelowvgap} \NC vertical clearance for limits below operators \NC \NR -\NC \type {\Umathlimitbelowbgap} \NC vertical baseline clearance for limits below operators \NC \NR -\NC \type {\Umathlimitbelowkern} \NC space reserved at the bottom of the limit \NC \NR -\NC \type {\Umathoverdelimitervgap} \NC vertical clearance for limits above delimiters \NC \NR -\NC \type {\Umathoverdelimiterbgap} \NC vertical baseline clearance for limits above delimiters \NC \NR -\NC \type {\Umathunderdelimitervgap} \NC vertical clearance for limits below delimiters \NC \NR -\NC \type {\Umathunderdelimiterbgap} \NC vertical baseline clearance for limits below delimiters \NC \NR -\NC \type {\Umathsubshiftdrop} \NC subscript drop for boxes and subformulas \NC \NR -\NC \type {\Umathsubshiftdown} \NC subscript drop for characters \NC \NR -\NC \type {\Umathsupshiftdrop} \NC superscript drop (raise, actually) for boxes and subformulas \NC \NR -\NC \type {\Umathsupshiftup} \NC superscript raise for characters \NC \NR -\NC \type {\Umathsubsupshiftdown} \NC subscript drop in the presence of a superscript \NC \NR -\NC \type {\Umathsubtopmax} \NC the top of standalone subscripts cannot be higher than this - above the baseline \NC \NR -\NC \type {\Umathsupbottommin} \NC the bottom of standalone superscripts cannot be less than - this above the baseline \NC \NR -\NC \type {\Umathsupsubbottommax} \NC the bottom of the superscript of a combined super- and subscript - be at least as high as this above the baseline \NC \NR -\NC \type {\Umathsubsupvgap} \NC vertical clearance between super- and subscript \NC \NR -\NC \type {\Umathspaceafterscript} \NC additional space added after a super- or subscript \NC \NR -\NC \type {\Umathconnectoroverlapmin}\NC minimum overlap between parts in an extensible recipe \NC \NR +\NC \lpr {Umathoperatorsize} \NC minimum size of large operators in display mode \NC \NR +\NC \lpr {Umathoverbarkern} \NC vertical clearance above the rule \NC \NR +\NC \lpr {Umathoverbarrule} \NC the width of the rule \NC \NR +\NC \lpr {Umathoverbarvgap} \NC vertical clearance below the rule \NC \NR +\NC \lpr {Umathunderbarkern} \NC vertical clearance below the rule \NC \NR +\NC \lpr {Umathunderbarrule} \NC the width of the rule \NC \NR +\NC \lpr {Umathunderbarvgap} \NC vertical clearance above the rule \NC \NR +\NC \lpr {Umathradicalkern} \NC vertical clearance above the rule \NC \NR +\NC \lpr {Umathradicalrule} \NC the width of the rule \NC \NR +\NC \lpr {Umathradicalvgap} \NC vertical clearance below the rule \NC \NR +\NC \lpr {Umathradicaldegreebefore}\NC the forward kern that takes place before placement of + the radical degree \NC \NR +\NC \lpr {Umathradicaldegreeafter} \NC the backward kern that takes place after placement of + the radical degree \NC \NR +\NC \lpr {Umathradicaldegreeraise} \NC this is the percentage of the total height and depth of + the radical sign that the degree is raised by; it is + expressed in \type {percents}, so 60\% is expressed as the + integer $60$ \NC \NR +\NC \lpr {Umathstackvgap} \NC vertical clearance between the two + elements in a \prm {atop} stack \NC \NR +\NC \lpr {Umathstacknumup} \NC numerator shift upward in \prm {atop} stack \NC \NR +\NC \lpr {Umathstackdenomdown} \NC denominator shift downward in \prm {atop} stack \NC \NR +\NC \lpr {Umathfractionrule} \NC the width of the rule in a \prm {over} \NC \NR +\NC \lpr {Umathfractionnumvgap} \NC vertical clearance between the numerator and the rule \NC \NR +\NC \lpr {Umathfractionnumup} \NC numerator shift upward in \prm {over} \NC \NR +\NC \lpr {Umathfractiondenomvgap} \NC vertical clearance between the denominator and the rule \NC \NR +\NC \lpr {Umathfractiondenomdown} \NC denominator shift downward in \prm {over} \NC \NR +\NC \lpr {Umathfractiondelsize} \NC minimum delimiter size for \type {\...withdelims} \NC \NR +\NC \lpr {Umathlimitabovevgap} \NC vertical clearance for limits above operators \NC \NR +\NC \lpr {Umathlimitabovebgap} \NC vertical baseline clearance for limits above operators \NC \NR +\NC \lpr {Umathlimitabovekern} \NC space reserved at the top of the limit \NC \NR +\NC \lpr {Umathlimitbelowvgap} \NC vertical clearance for limits below operators \NC \NR +\NC \lpr {Umathlimitbelowbgap} \NC vertical baseline clearance for limits below operators \NC \NR +\NC \lpr {Umathlimitbelowkern} \NC space reserved at the bottom of the limit \NC \NR +\NC \lpr {Umathoverdelimitervgap} \NC vertical clearance for limits above delimiters \NC \NR +\NC \lpr {Umathoverdelimiterbgap} \NC vertical baseline clearance for limits above delimiters \NC \NR +\NC \lpr {Umathunderdelimitervgap} \NC vertical clearance for limits below delimiters \NC \NR +\NC \lpr {Umathunderdelimiterbgap} \NC vertical baseline clearance for limits below delimiters \NC \NR +\NC \lpr {Umathsubshiftdrop} \NC subscript drop for boxes and subformulas \NC \NR +\NC \lpr {Umathsubshiftdown} \NC subscript drop for characters \NC \NR +\NC \lpr {Umathsupshiftdrop} \NC superscript drop (raise, actually) for boxes and subformulas \NC \NR +\NC \lpr {Umathsupshiftup} \NC superscript raise for characters \NC \NR +\NC \lpr {Umathsubsupshiftdown} \NC subscript drop in the presence of a superscript \NC \NR +\NC \lpr {Umathsubtopmax} \NC the top of standalone subscripts cannot be higher than this + above the baseline \NC \NR +\NC \lpr {Umathsupbottommin} \NC the bottom of standalone superscripts cannot be less than + this above the baseline \NC \NR +\NC \lpr {Umathsupsubbottommax} \NC the bottom of the superscript of a combined super- and subscript + be at least as high as this above the baseline \NC \NR +\NC \lpr {Umathsubsupvgap} \NC vertical clearance between super- and subscript \NC \NR +\NC \lpr {Umathspaceafterscript} \NC additional space added after a super- or subscript \NC \NR +\NC \lpr {Umathconnectoroverlapmin}\NC minimum overlap between parts in an extensible recipe \NC \NR +\LL \stoptabulate Each of the parameters in this section can be set by a command like this: @@ -403,22 +436,9 @@ Each of the parameters in this section can be set by a command like this: they obey grouping, and you can use \type {\the\Umathquad\displaystyle} if needed. -\section{Skips around display math} +\subsection{Font|-|based math parameters} -The injection of \type {\abovedisplayskip} and \type {\belowdisplayskip} is not -symmetrical. An above one is always inserted, also when zero, but the below is -only inserted when larger than zero. Especially the later makes it sometimes hard -to fully control spacing. Therefore \LUATEX\ comes with a new directive: \type -{\mathdisplayskipmode}. The following values apply: - -\starttabulate -\NC 0 \NC normal \TEX\ behaviour \NC \NR -\NC 1 \NC always (same as 0) \NC \NR -\NC 2 \NC only when not zero \NC \NR -\NC 3 \NC never, not even when not zero \NC \NR -\stoptabulate - -\section{Font-based Math Parameters} +\topicindex {math+parameters} While it is nice to have these math parameters available for tweaking, it would be tedious to have to set each of them by hand. For this reason, \LUATEX\ @@ -432,121 +452,118 @@ case no attention is paid to which family is being assigned to: the \type {MathConstants} tables in the last assigned family sets all parameters. In the table below, the one|-|letter style abbreviations and symbolic tfm font -dimension names match those using in the \TeX book. Assignments to \type -{\textfont} set the values for the cramped and uncramped display and text styles, -\type {\scriptfont} sets the script styles, and \type {\scriptscriptfont} sets -the scriptscript styles, so we have eight parameters for three font sizes. In the +dimension names match those used in the \TeX book. Assignments to \prm +{textfont} set the values for the cramped and uncramped display and text styles, +\prm {scriptfont} sets the script styles, and \prm {scriptscriptfont} sets the +scriptscript styles, so we have eight parameters for three font sizes. In the \TFM\ case, assignments only happen in family~2 and family~3 (and of course only for the parameters for which there are font dimensions). Besides the parameters below, \LUATEX\ also looks at the \quote {space} font dimension parameter. For math fonts, this should be set to zero. -\start +\def\MathLine#1#2#3#4#5% + {\TB + \NC \llap{\high{\tx #2\enspace}}\ttbf \string #1 \NC \tt #5 \NC \NR + \NC \tx #3 \NC \tt #4 \NC \NR} -\switchtobodyfont[8pt] - -\starttabulate[|l|l|l|p|] -\BC variable \BC style \BC default value opentype \BC default value tfm \NC \NR -\NC \type {\Umathaxis} \NC -- \NC AxisHeight \NC axis_height \NC \NR -\NC \type {\Umathoperatorsize} \NC D, D' \NC DisplayOperatorMinHeight \NC $^6$ \NC \NR -\NC \type {\Umathfractiondelsize} \NC D, D' \NC FractionDelimiterDisplayStyleSize$^9$ \NC delim1 \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC FractionDelimiterSize$^9$ \NC delim2 \NC \NR -\NC \type {\Umathfractiondenomdown} \NC D, D' \NC FractionDenominatorDisplayStyleShiftDown \NC denom1 \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC FractionDenominatorShiftDown \NC denom2 \NC \NR -\NC \type {\Umathfractiondenomvgap} \NC D, D' \NC FractionDenominatorDisplayStyleGapMin \NC 3*default_rule_thickness \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC FractionDenominatorGapMin \NC default_rule_thickness \NC \NR -\NC \type {\Umathfractionnumup} \NC D, D' \NC FractionNumeratorDisplayStyleShiftUp \NC num1 \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC FractionNumeratorShiftUp \NC num2 \NC \NR -\NC \type {\Umathfractionnumvgap} \NC D, D' \NC FractionNumeratorDisplayStyleGapMin \NC 3*default_rule_thickness \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC FractionNumeratorGapMin \NC default_rule_thickness \NC \NR -\NC \type {\Umathfractionrule} \NC -- \NC FractionRuleThickness \NC default_rule_thickness \NC \NR -\NC \type {\Umathskewedfractionhgap} \NC -- \NC SkewedFractionHorizontalGap \NC math_quad/2 \NC \NR -\NC \type {\Umathskewedfractionvgap} \NC -- \NC SkewedFractionVerticalGap \NC math_x_height \NC \NR -\NC \type {\Umathlimitabovebgap} \NC -- \NC UpperLimitBaselineRiseMin \NC big_op_spacing3 \NC \NR -\NC \type {\Umathlimitabovekern} \NC -- \NC 0$^1$ \NC big_op_spacing5 \NC \NR -\NC \type {\Umathlimitabovevgap} \NC -- \NC UpperLimitGapMin \NC big_op_spacing1 \NC \NR -\NC \type {\Umathlimitbelowbgap} \NC -- \NC LowerLimitBaselineDropMin \NC big_op_spacing4 \NC \NR -\NC \type {\Umathlimitbelowkern} \NC -- \NC 0$^1$ \NC big_op_spacing5 \NC \NR -\NC \type {\Umathlimitbelowvgap} \NC -- \NC LowerLimitGapMin \NC big_op_spacing2 \NC \NR -\NC \type {\Umathoverdelimitervgap} \NC -- \NC StretchStackGapBelowMin \NC big_op_spacing1 \NC \NR -\NC \type {\Umathoverdelimiterbgap} \NC -- \NC StretchStackTopShiftUp \NC big_op_spacing3 \NC \NR -\NC \type {\Umathunderdelimitervgap} \NC-- \NC StretchStackGapAboveMin \NC big_op_spacing2 \NC \NR -\NC \type {\Umathunderdelimiterbgap} \NC-- \NC StretchStackBottomShiftDown \NC big_op_spacing4 \NC \NR -\NC \type {\Umathoverbarkern} \NC -- \NC OverbarExtraAscender \NC default_rule_thickness \NC \NR -\NC \type {\Umathoverbarrule} \NC -- \NC OverbarRuleThickness \NC default_rule_thickness \NC \NR -\NC \type {\Umathoverbarvgap} \NC -- \NC OverbarVerticalGap \NC 3*default_rule_thickness \NC \NR -\NC \type {\Umathquad} \NC -- \NC <font_size(f)>$^1$ \NC math_quad \NC \NR -\NC \type {\Umathradicalkern} \NC -- \NC RadicalExtraAscender \NC default_rule_thickness \NC \NR -\NC \type {\Umathradicalrule} \NC -- \NC RadicalRuleThickness \NC <not set>$^2$ \NC \NR -\NC \type {\Umathradicalvgap} \NC D, D' \NC RadicalDisplayStyleVerticalGap \NC (default_rule_thickness+\crlf - (abs(math_x_height)/4))$^3$ \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC RadicalVerticalGap \NC (default_rule_thickness+\crlf - (abs(default_rule_thickness)/4))$^3$ \NC \NR -\NC \type {\Umathradicaldegreebefore} \NC -- \NC RadicalKernBeforeDegree \NC <not set>$^2$ \NC \NR -\NC \type {\Umathradicaldegreeafter} \NC -- \NC RadicalKernAfterDegree \NC <not set>$^2$ \NC \NR -\NC \type {\Umathradicaldegreeraise} \NC -- \NC RadicalDegreeBottomRaisePercent \NC <not set>$^{2,7}$ \NC \NR -\NC \type {\Umathspaceafterscript} \NC -- \NC SpaceAfterScript \NC script_space$^4$ \NC \NR -\NC \type {\Umathstackdenomdown} \NC D, D' \NC StackBottomDisplayStyleShiftDown \NC denom1 \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC StackBottomShiftDown \NC denom2 \NC \NR -\NC \type {\Umathstacknumup} \NC D, D' \NC StackTopDisplayStyleShiftUp \NC num1 \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC StackTopShiftUp \NC num3 \NC \NR -\NC \type {\Umathstackvgap} \NC D, D' \NC StackDisplayStyleGapMin \NC 7*default_rule_thickness \NC \NR -\NC \NC T, T', S, S', SS, SS' \NC StackGapMin \NC 3*default_rule_thickness \NC \NR -\NC \type {\Umathsubshiftdown} \NC -- \NC SubscriptShiftDown \NC sub1 \NC \NR -\NC \type {\Umathsubshiftdrop} \NC -- \NC SubscriptBaselineDropMin \NC sub_drop \NC \NR -\NC \type {\Umathsubsupshiftdown} \NC -- \NC SubscriptShiftDownWithSuperscript$^8$ \NC \NC \NR -\NC \NC \NC \quad\ or SubscriptShiftDown \NC sub2 \NC \NR -\NC \type {\Umathsubtopmax} \NC -- \NC SubscriptTopMax \NC (abs(math_x_height * 4) / 5) \NC \NR -\NC \type {\Umathsubsupvgap} \NC -- \NC SubSuperscriptGapMin \NC 4*default_rule_thickness \NC \NR -\NC \type {\Umathsupbottommin} \NC -- \NC SuperscriptBottomMin \NC (abs(math_x_height) / 4) \NC \NR -\NC \type {\Umathsupshiftdrop} \NC -- \NC SuperscriptBaselineDropMax \NC sup_drop \NC \NR -\NC \type {\Umathsupshiftup} \NC D \NC SuperscriptShiftUp \NC sup1 \NC \NR -\NC \NC T, S, SS, \NC SuperscriptShiftUp \NC sup2 \NC \NR -\NC \NC D', T', S', SS' \NC SuperscriptShiftUpCramped \NC sup3 \NC \NR -\NC \type {\Umathsupsubbottommax} \NC -- \NC SuperscriptBottomMaxWithSubscript \NC (abs(math_x_height * 4) / 5) \NC \NR -\NC \type {\Umathunderbarkern} \NC -- \NC UnderbarExtraDescender \NC default_rule_thickness \NC \NR -\NC \type {\Umathunderbarrule} \NC -- \NC UnderbarRuleThickness \NC default_rule_thickness \NC \NR -\NC \type {\Umathunderbarvgap} \NC -- \NC UnderbarVerticalGap \NC 3*default_rule_thickness \NC \NR -\NC \type {\Umathconnectoroverlapmin} \NC -- \NC MinConnectorOverlap \NC 0$^5$ \NC \NR +\starttabulate[|l|l|] +\DB variable / style \BC tfm / opentype \NC \NR +\MathLine{\Umathaxis} {} {} {AxisHeight} {axis_height} +\MathLine{\Umathoperatorsize} {6} {D, D'} {DisplayOperatorMinHeight} {\emdash} +\MathLine{\Umathfractiondelsize} {9} {D, D'} {FractionDelimiterDisplayStyleSize} {delim1} +\MathLine{\Umathfractiondelsize} {9} {T, T', S, S', SS, SS'}{FractionDelimiterSize} {delim2} +\MathLine{\Umathfractiondenomdown} {} {D, D'} {FractionDenominatorDisplayStyleShiftDown}{denom1} +\MathLine{\Umathfractiondenomdown} {} {T, T', S, S', SS, SS'}{FractionDenominatorShiftDown} {denom2} +\MathLine{\Umathfractiondenomvgap} {} {D, D'} {FractionDenominatorDisplayStyleGapMin} {3*default_rule_thickness} +\MathLine{\Umathfractiondenomvgap} {} {T, T', S, S', SS, SS'}{FractionDenominatorGapMin} {default_rule_thickness} +\MathLine{\Umathfractionnumup} {} {D, D'} {FractionNumeratorDisplayStyleShiftUp} {num1} +\MathLine{\Umathfractionnumup} {} {T, T', S, S', SS, SS'}{FractionNumeratorShiftUp} {num2} +\MathLine{\Umathfractionnumvgap} {} {D, D'} {FractionNumeratorDisplayStyleGapMin} {3*default_rule_thickness} +\MathLine{\Umathfractionnumvgap} {} {T, T', S, S', SS, SS'}{FractionNumeratorGapMin} {default_rule_thickness} +\MathLine{\Umathfractionrule} {} {} {FractionRuleThickness} {default_rule_thickness} +\MathLine{\Umathskewedfractionhgap} {} {} {SkewedFractionHorizontalGap} {math_quad/2} +\MathLine{\Umathskewedfractionvgap} {} {} {SkewedFractionVerticalGap} {math_x_height} +\MathLine{\Umathlimitabovebgap} {} {} {UpperLimitBaselineRiseMin} {big_op_spacing3} +\MathLine{\Umathlimitabovekern} {1} {} {0} {big_op_spacing5} +\MathLine{\Umathlimitabovevgap} {} {} {UpperLimitGapMin} {big_op_spacing1} +\MathLine{\Umathlimitbelowbgap} {} {} {LowerLimitBaselineDropMin} {big_op_spacing4} +\MathLine{\Umathlimitbelowkern} {1} {} {0} {big_op_spacing5} +\MathLine{\Umathlimitbelowvgap} {} {} {LowerLimitGapMin} {big_op_spacing2} +\MathLine{\Umathoverdelimitervgap} {} {} {StretchStackGapBelowMin} {big_op_spacing1} +\MathLine{\Umathoverdelimiterbgap} {} {} {StretchStackTopShiftUp} {big_op_spacing3} +\MathLine{\Umathunderdelimitervgap} {} {} {StretchStackGapAboveMin} {big_op_spacing2} +\MathLine{\Umathunderdelimiterbgap} {} {} {StretchStackBottomShiftDown} {big_op_spacing4} +\MathLine{\Umathoverbarkern} {} {} {OverbarExtraAscender} {default_rule_thickness} +\MathLine{\Umathoverbarrule} {} {} {OverbarRuleThickness} {default_rule_thickness} +\MathLine{\Umathoverbarvgap} {} {} {OverbarVerticalGap} {3*default_rule_thickness} +\MathLine{\Umathquad} {1} {} {<font_size(f)>} {math_quad} +\MathLine{\Umathradicalkern} {} {} {RadicalExtraAscender} {default_rule_thickness} +\MathLine{\Umathradicalrule} {2} {} {RadicalRuleThickness} {<not set>} +\MathLine{\Umathradicalvgap} {3} {D, D'} {RadicalDisplayStyleVerticalGap} {default_rule_thickness+abs(math_x_height)/4} +\MathLine{\Umathradicalvgap} {3} {T, T', S, S', SS, SS'}{RadicalVerticalGap} {default_rule_thickness+abs(default_rule_thickness)/4} +\MathLine{\Umathradicaldegreebefore}{2} {} {RadicalKernBeforeDegree} {<not set>} +\MathLine{\Umathradicaldegreeafter} {2} {} {RadicalKernAfterDegree} {<not set>} +\MathLine{\Umathradicaldegreeraise} {2,7}{} {RadicalDegreeBottomRaisePercent} {<not set>} +\MathLine{\Umathspaceafterscript} {4} {} {SpaceAfterScript} {script_space} +\MathLine{\Umathstackdenomdown} {} {D, D'} {StackBottomDisplayStyleShiftDown} {denom1} +\MathLine{\Umathstackdenomdown} {} {T, T', S, S', SS, SS'}{StackBottomShiftDown} {denom2} +\MathLine{\Umathstacknumup} {} {D, D'} {StackTopDisplayStyleShiftUp} {num1} +\MathLine{\Umathstacknumup} {} {T, T', S, S', SS, SS'}{StackTopShiftUp} {num3} +\MathLine{\Umathstackvgap} {} {D, D'} {StackDisplayStyleGapMin} {7*default_rule_thickness} +\MathLine{\Umathstackvgap} {} {T, T', S, S', SS, SS'}{StackGapMin} {3*default_rule_thickness} +\MathLine{\Umathsubshiftdown} {} {} {SubscriptShiftDown} {sub1} +\MathLine{\Umathsubshiftdrop} {} {} {SubscriptBaselineDropMin} {sub_drop} +\MathLine{\Umathsubsupshiftdown} {8} {} {SubscriptShiftDownWithSuperscript} {\emdash} +\MathLine{\Umathsubtopmax} {} {} {SubscriptTopMax} {abs(math_x_height*4)/5} +\MathLine{\Umathsubsupvgap} {} {} {SubSuperscriptGapMin} {4*default_rule_thickness} +\MathLine{\Umathsupbottommin} {} {} {SuperscriptBottomMin} {abs(math_x_height/4)} +\MathLine{\Umathsupshiftdrop} {} {} {SuperscriptBaselineDropMax} {sup_drop} +\MathLine{\Umathsupshiftup} {} {D} {SuperscriptShiftUp} {sup1} +\MathLine{\Umathsupshiftup} {} {T, S, SS,} {SuperscriptShiftUp} {sup2} +\MathLine{\Umathsupshiftup} {} {D', T', S', SS'} {SuperscriptShiftUpCramped} {sup3} +\MathLine{\Umathsupsubbottommax} {} {} {SuperscriptBottomMaxWithSubscript} {abs(math_x_height*4)/5} +\MathLine{\Umathunderbarkern} {} {} {UnderbarExtraDescender} {default_rule_thickness} +\MathLine{\Umathunderbarrule} {} {} {UnderbarRuleThickness} {default_rule_thickness} +\MathLine{\Umathunderbarvgap} {} {} {UnderbarVerticalGap} {3*default_rule_thickness} +\MathLine{\Umathconnectoroverlapmin}{5} {} {MinConnectorOverlap} {0} +\LL \stoptabulate -\stop - -Note 1: \OPENTYPE\ fonts set \type {\Umathlimitabovekern} and \type -{\Umathlimitbelowkern} to zero and set \type {\Umathquad} to the font size of the +Note 1: \OPENTYPE\ fonts set \lpr {Umathlimitabovekern} and \lpr +{Umathlimitbelowkern} to zero and set \lpr {Umathquad} to the font size of the used font, because these are not supported in the \type {MATH} table, -Note 2: Traditional \TFM\ fonts do not set \type {\Umathradicalrule} because +Note 2: Traditional \TFM\ fonts do not set \lpr {Umathradicalrule} because \TEX82\ uses the height of the radical instead. When this parameter is indeed not set when \LUATEX\ has to typeset a radical, a backward compatibility mode will kick in that assumes that an oldstyle \TEX\ font is used. Also, they do not set -\type {\Umathradicaldegreebefore}, \type {\Umathradicaldegreeafter}, and \type -{\Umathradicaldegreeraise}. These are then automatically initialized to +\lpr {Umathradicaldegreebefore}, \lpr {Umathradicaldegreeafter}, and \lpr +{Umathradicaldegreeraise}. These are then automatically initialized to $5/18$quad, $-10/18$quad, and 60. -Note 3: If \TFM\ fonts are used, then the \type {\Umathradicalvgap} is not set +Note 3: If \TFM\ fonts are used, then the \lpr {Umathradicalvgap} is not set until the first time \LUATEX\ has to typeset a formula because this needs parameters from both family~2 and family~3. This provides a partial backward -compatibility with \TEX82, but that compatibility is only partial: once the \type -{\Umathradicalvgap} is set, it will not be recalculated any more. +compatibility with \TEX82, but that compatibility is only partial: once the \lpr +{Umathradicalvgap} is set, it will not be recalculated any more. -Note 4: When \TFM\ fonts are used a similar situation arises with respect to -\type {\Umathspaceafterscript}: it is not set until the first time \LUATEX\ has -to typeset a formula. This provides some backward compatibility with \TEX82. But -once the \type {\Umathspaceafterscript} is set, \type {\scriptspace} will never -be looked at again. +Note 4: When \TFM\ fonts are used a similar situation arises with respect to \lpr +{Umathspaceafterscript}: it is not set until the first time \LUATEX\ has to +typeset a formula. This provides some backward compatibility with \TEX82. But +once the \lpr {Umathspaceafterscript} is set, \prm {scriptspace} will never be +looked at again. -Note 5: Traditional \TFM\ fonts set \type {\Umathconnectoroverlapmin} to zero +Note 5: Traditional \TFM\ fonts set \lpr {Umathconnectoroverlapmin} to zero because \TEX82\ always stacks extensibles without any overlap. -Note 6: The \type {\Umathoperatorsize} is only used in \type {\displaystyle}, and -is only set in \OPENTYPE\ fonts. In \TFM\ font mode, it is artificially set to -one scaled point more than the initial attempt's size, so that always the \quote +Note 6: The \lpr {Umathoperatorsize} is only used in \prm {displaystyle}, and is +only set in \OPENTYPE\ fonts. In \TFM\ font mode, it is artificially set to one +scaled point more than the initial attempt's size, so that always the \quote {first next} will be tried, just like in \TEX82. -Note 7: The \type {\Umathradicaldegreeraise} is a special case because it is the -only parameter that is expressed in a percentage instead of as a number of scaled +Note 7: The \lpr {Umathradicaldegreeraise} is a special case because it is the +only parameter that is expressed in a percentage instead of a number of scaled points. Note 8: \type {SubscriptShiftDownWithSuperscript} does not actually exist in the @@ -557,16 +574,198 @@ Note 9: \type {FractionDelimiterDisplayStyleSize} and \type {FractionDelimiterSize} do not actually exist in the \quote {standard} \OPENTYPE\ math font Cambria, but were useful enough to be added. -\section{Nolimit correction} +\stopsection + +\startsection[title={Math spacing}] + +\subsection{Inline surrounding space} + +\topicindex {math+spacing} + +Inline math is surrounded by (optional) \prm {mathsurround} spacing but that is a fixed +dimension. There is now an additional parameter \lpr {mathsurroundskip}. When set to a +non|-|zero value (or zero with some stretch or shrink) this parameter will replace +\prm {mathsurround}. By using an additional parameter instead of changing the nature +of \prm {mathsurround}, we can remain compatible. In the meantime a bit more +control has been added via \lpr {mathsurroundmode}. This directive can take 6 values +with zero being the default behaviour. + +\start + +\def\OneLiner#1#2% + {\NC \type{#1} + \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x$x$x} + \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x $x$ x} + \NC #2 + \NC \NR} + +\startbuffer +\mathsurround 10pt +\mathsurroundskip20pt +\stopbuffer + +\typebuffer \getbuffer + +\starttabulate[|c|c|c|pl|] +\DB mode \BC x\$x\$x \BC x \$x\$ x \BC effect \NC \NR +\TB +\OneLiner{0}{obey \prm {mathsurround} when \lpr {mathsurroundskip} is 0pt} +\OneLiner{1}{only add skip to the left} +\OneLiner{2}{only add skip to the right} +\OneLiner{3}{add skip to the left and right} +\OneLiner{4}{ignore the skip setting, obey \prm {mathsurround}} +\OneLiner{5}{disable all spacing around math} +\OneLiner{6}{only apply \lpr {mathsurroundskip} when also spacing} +\OneLiner{7}{only apply \lpr {mathsurroundskip} when no spacing} +\LL +\stoptabulate + +\stop + +Method six omits the surround glue when there is (x)spacing glue present while +method seven does the opposite, the glue is only applied when there is (x)space +glue present too. Anything more fancy, like checking the begining or end of a +paragraph (or edges of a box) would not be robust anyway. If you want that you +can write a callback that runs over a list and analyzes a paragraph. Actually, in +that case you could also inject glue (or set the properties of a math node) +explicitly. So, these modes are in practice mostly useful for special purposes +and experiments (they originate in a tracker item). Keep in mind that this glue +is part of the math node and not always treated as normal glue: it travels with +the begin and end math nodes. Also, method 6 and 7 will zero the skip related +fields in a node when applicable in the first occasion that checks them +(linebreaking or packaging). + +\subsection{Pairwise spacing} + +\topicindex {math+spacing} + +Besides the parameters mentioned in the previous sections, there are also 64 new +primitives to control the math spacing table (as explained in Chapter~18 of the +\TEX book). The primitive names are a simple matter of combining two math atom +types, but for completeness' sake, here is the whole list: + +\starttwocolumns +\startlines +\lpr {Umathordordspacing} +\lpr {Umathordopspacing} +\lpr {Umathordbinspacing} +\lpr {Umathordrelspacing} +\lpr {Umathordopenspacing} +\lpr {Umathordclosespacing} +\lpr {Umathordpunctspacing} +\lpr {Umathordinnerspacing} +\lpr {Umathopordspacing} +\lpr {Umathopopspacing} +\lpr {Umathopbinspacing} +\lpr {Umathoprelspacing} +\lpr {Umathopopenspacing} +\lpr {Umathopclosespacing} +\lpr {Umathoppunctspacing} +\lpr {Umathopinnerspacing} +\lpr {Umathbinordspacing} +\lpr {Umathbinopspacing} +\lpr {Umathbinbinspacing} +\lpr {Umathbinrelspacing} +\lpr {Umathbinopenspacing} +\lpr {Umathbinclosespacing} +\lpr {Umathbinpunctspacing} +\lpr {Umathbininnerspacing} +\lpr {Umathrelordspacing} +\lpr {Umathrelopspacing} +\lpr {Umathrelbinspacing} +\lpr {Umathrelrelspacing} +\lpr {Umathrelopenspacing} +\lpr {Umathrelclosespacing} +\lpr {Umathrelpunctspacing} +\lpr {Umathrelinnerspacing} +\lpr {Umathopenordspacing} +\lpr {Umathopenopspacing} +\lpr {Umathopenbinspacing} +\lpr {Umathopenrelspacing} +\lpr {Umathopenopenspacing} +\lpr {Umathopenclosespacing} +\lpr {Umathopenpunctspacing} +\lpr {Umathopeninnerspacing} +\lpr {Umathcloseordspacing} +\lpr {Umathcloseopspacing} +\lpr {Umathclosebinspacing} +\lpr {Umathcloserelspacing} +\lpr {Umathcloseopenspacing} +\lpr {Umathcloseclosespacing} +\lpr {Umathclosepunctspacing} +\lpr {Umathcloseinnerspacing} +\lpr {Umathpunctordspacing} +\lpr {Umathpunctopspacing} +\lpr {Umathpunctbinspacing} +\lpr {Umathpunctrelspacing} +\lpr {Umathpunctopenspacing} +\lpr {Umathpunctclosespacing} +\lpr {Umathpunctpunctspacing} +\lpr {Umathpunctinnerspacing} +\lpr {Umathinnerordspacing} +\lpr {Umathinneropspacing} +\lpr {Umathinnerbinspacing} +\lpr {Umathinnerrelspacing} +\lpr {Umathinneropenspacing} +\lpr {Umathinnerclosespacing} +\lpr {Umathinnerpunctspacing} +\lpr {Umathinnerinnerspacing} +\stoplines +\stoptwocolumns + +These parameters are of type \prm {muskip}, so setting a parameter can be done +like this: + +\starttyping +\Umathopordspacing\displaystyle=4mu plus 2mu +\stoptyping + +They are all initialized by \type {initex} to the values mentioned in the table +in Chapter~18 of the \TEX book. + +Note 1: for ease of use as well as for backward compatibility, \prm {thinmuskip}, +\prm {medmuskip} and \prm {thickmuskip} are treated specially. In their case a +pointer to the corresponding internal parameter is saved, not the actual \prm +{muskip} value. This means that any later changes to one of these three +parameters will be taken into account. + +Note 2: Careful readers will realise that there are also primitives for the items +marked \type {*} in the \TEX book. These will not actually be used as those +combinations of atoms cannot actually happen, but it seemed better not to break +orthogonality. They are initialized to zero. + +\subsection{Skips around display math} + +\topicindex {math+spacing} + +The injection of \prm {abovedisplayskip} and \prm {belowdisplayskip} is not +symmetrical. An above one is always inserted, also when zero, but the below is +only inserted when larger than zero. Especially the latter makes it sometimes hard +to fully control spacing. Therefore \LUATEX\ comes with a new directive: \lpr +{mathdisplayskipmode}. The following values apply: + +\starttabulate[|c|l|] +\DB value \BC meaning \NC \NR +\TB +\NC 0 \NC normal \TEX\ behaviour \NC \NR +\NC 1 \NC always (same as 0) \NC \NR +\NC 2 \NC only when not zero \NC \NR +\NC 3 \NC never, not even when not zero \NC \NR +\LL +\stoptabulate + +\subsection {Nolimit correction} -There are two extra math parameters \type {\Umathnolimitsupfactor} and \type -{\Umathnolimitsubfactor} that were added to provide some control over how limits +\topicindex {math+limits} + +There are two extra math parameters \lpr {Umathnolimitsupfactor} and \lpr +{Umathnolimitsubfactor} that were added to provide some control over how limits are spaced (for example the position of super and subscripts after integral -operators). They relate to an extra parameter \type {\mathnolimitsmode}. The half -corrections are what happens when scripts are placed on above and below. The +operators). They relate to an extra parameter \lpr {mathnolimitsmode}. The half +corrections are what happens when scripts are placed above and below. The problem with italic corrections is that officially that correction italic is used for above|/|below placement while advanced kerns are used for placement at the -right end. The question is: how often is this implemented, and if so, does the +right end. The question is: how often is this implemented, and if so, do the kerns assume correction too. Anyway, with this parameter one can control it. \starttabulate[|l|ck1|ck1|ck1|ck1|ck1|ck1|] @@ -609,13 +808,15 @@ When the mode is set to one, the math parameters are used. This way a macro package writer can decide what looks best. Given the current state of fonts in \CONTEXT\ we currently use mode 1 with factor 0 for the superscript and 750 for the subscripts. Positive values are used for both parameters but the subscript -shifts to the left. A \type {\mathnolimitsmode} larger that 15 is considered to +shifts to the left. A \lpr {mathnolimitsmode} larger that 15 is considered to be a factor for the subscript correction. This feature can be handy when experimenting. -\section{Math italic mess} +\subsection {Math italic mess} + +\topicindex {math+italics} -The \type {\mathitalicsmode} parameter can be set to~1 to force italic correction +The \lpr {mathitalicsmode} parameter can be set to~1 to force italic correction before noads that represent some more complex structure (read: everything that is not an ord, bin, rel, open, close, punct or inner). We show a Cambria example. @@ -641,20 +842,26 @@ example. This kind of parameters relate to the fact that italic correction in \OPENTYPE\ math is bound to fuzzy rules. So, control is the solution. -\section{Script boxes} +\subsection {Script and kerning} -If you want typeset text in math macro packages often provide something \type -{\text} which obeys the script sizes. As the definition can be anything there is -a good change that the kerning doesn't come out well when used in a script. Given -that the first glyph ends up in an \type {\hbox} we have some control over this. -And, as a bonus we also added control over the normal sublist kerning. The \type -{\mathscriptboxmode} parameter defaults to~1. +\topicindex {math+kerning} +\topicindex {math+scripts} -\starttabulate[|l|l|] +If you want to typeset text in math macro packages often provide something \type +{\text} which obeys the script sizes. As the definition can be anything there is +a good chance that the kerning doesn't come out well when used in a script. Given +that the first glyph ends up in a \prm {hbox} we have some control over this. +And, as a bonus we also added control over the normal sublist kerning. The \lpr +{mathscriptboxmode} parameter defaults to~1. + +\starttabulate[|c|l|] +\DB value \BC meaning \NC \NR +\TB \NC \type {0} \NC forget about kerning \NC \NR \NC \type {1} \NC kern math sub lists with a valid glyph \NC \NR \NC \type {2} \NC also kern math sub boxes that have a valid glyph \NC \NR \NC \type {2} \NC only kern math sub boxes with a boundary node present\NC \NR +\LL \stoptabulate Here we show some examples. Of course this doesn't solve all our problems, if @@ -675,12 +882,12 @@ italics, while other fonts can lack kerns. \unexpanded\def\Show#1#2#3% {\doifelsenothing{#3} - {\small\typeinlinebuffer[#1]} + {\small\tx\typeinlinebuffer[#1]} {\doifelse{#3}{-} - {\small\type{mode #2}} + {\small\bf\tt mode #2} {\switchtobodyfont[#3]\showfontkerns\showglyphs\mathscriptboxmode#2\relax\inlinebuffer[#1]}}} -\starttabulate[|lT|c|c|c|c|c|] +\starttabulate[|lBT|c|c|c|c|c|] \NC \NC \Show{1}{0}{} \NC\Show{1}{1}{} \NC \Show{2}{1}{} \NC \Show{2}{2}{} \NC \Show{3}{3}{} \NC \NR \NC \NC \Show{1}{0}{-} \NC\Show{1}{1}{-} \NC \Show{2}{1}{-} \NC \Show{2}{2}{-} \NC \Show{3}{3}{-} \NC \NR \NC modern \NC \Show{1}{0}{modern} \NC\Show{1}{1}{modern} \NC \Show{2}{1}{modern} \NC \Show{2}{2}{modern} \NC \Show{3}{3}{modern} \NC \NR @@ -690,15 +897,129 @@ italics, while other fonts can lack kerns. \NC dejavu \NC \Show{1}{0}{dejavu} \NC\Show{1}{1}{dejavu} \NC \Show{2}{1}{dejavu} \NC \Show{2}{2}{dejavu} \NC \Show{3}{3}{dejavu} \NC \NR \stoptabulate -\section{Unscaled fences} +Kerning between a character subscript is controlled by \lpr {mathscriptcharmode} +which also defaults to~1. + +Here is another example. Internally we tag kerns as italic kerns or font kerns +where font kerns result from the staircase kern tables. In 2018 fonts like Latin +Modern and Pagella rely on cheats with the boundingbox, Cambria uses staircase +kerns and Lucida a mixture. Depending on how fonts evolve we might add some more +control over what one can turn on and off. + +\def\MathSample#1#2#3% + {\NC + #1 \NC + #2 \NC + \showglyphdata \switchtobodyfont[#2,17.3pt]$#3T_{f}$ \NC + \showglyphdata \switchtobodyfont[#2,17.3pt]$#3\gamma_{e}$ \NC + \showglyphdata \switchtobodyfont[#2,17.3pt]$#3\gamma_{ee}$ \NC + \showglyphdata \switchtobodyfont[#2,17.3pt]$#3T_{\tf fluff}$ \NC + \NR} + +\starttabulate[|Tl|Tl|l|l|l|l|] + \FL + \MathSample{normal}{modern} {\mr} + \MathSample{} {pagella} {\mr} + \MathSample{} {cambria} {\mr} + \MathSample{} {lucidaot}{\mr} + \ML + \MathSample{bold} {modern} {\mb} + \MathSample{} {pagella} {\mb} + \MathSample{} {cambria} {\mb} + \MathSample{} {lucidaot}{\mb} + \LL +\stoptabulate + +\subsection{Fixed scripts} + +We have three parameters that are used for this fixed anchoring: + +\starttabulate[|c|l|] +\DB parameter \BC register \NC \NR +\NC $d$ \NC \lpr {Umathsubshiftdown} \NC \NR +\NC $u$ \NC \lpr {Umathsupshiftup} \NC \NR +\NC $s$ \NC \lpr {Umathsubsupshiftdown} \NC \NR +\LL +\stoptabulate + +When we set \lpr {mathscriptsmode} to a value other than zero these are used +for calculating fixed positions. This is something that is needed for instance +for chemistry. You can manipulate the mentioned variables to achieve different +effects. + +\def\SampleMath#1% + {$\mathscriptsmode#1\mathupright CH_2 + CH^+_2 + CH^2_2$} + +\starttabulate[|c|c|c|p|] +\DB mode \BC down \BC up \BC example \NC \NR +\TB +\NC 0 \NC dynamic \NC dynamic \NC \SampleMath{0} \NC \NR +\NC 1 \NC $d$ \NC $u$ \NC \SampleMath{1} \NC \NR +\NC 2 \NC $s$ \NC $u$ \NC \SampleMath{2} \NC \NR +\NC 3 \NC $s$ \NC $u + s - d$ \NC \SampleMath{3} \NC \NR +\NC 4 \NC $d + (s-d)/2$ \NC $u + (s-d)/2$ \NC \SampleMath{4} \NC \NR +\NC 5 \NC $d$ \NC $u + s - d$ \NC \SampleMath{5} \NC \NR +\LL +\stoptabulate + +The value of this parameter obeys grouping but applies to the whole current +formula. + +% if needed we can put the value in stylenodes but maybe more should go there -The \type {\mathdelimitersmode} primitive is experimental and deals with the -following (potential) problems. Three bits can be set. The first bit prevents -an unwanted shift when the fence symbol is not scaled (a cambria side effect). The -second bit forces italic correction between a preceding character ordinal and -the fenced subformula, while the third bit turns that subformula into a ordinary -so that the same spacing applies as with unfenced variants. Here we show Cambria -(with \type {\mathitalicsmode} enabled). +\subsection{Penalties: \lpr {mathpenaltiesmode}} + +\topicindex {math+penalties} + +Only in inline math penalties will be added in a math list. You can force +penalties (also in display math) by setting: + +\starttyping +\mathpenaltiesmode = 1 +\stoptyping + +This primnitive is not really needed in \LUATEX\ because you can use the callback +\cbk {mlist_to_hlist} to force penalties by just calling the regular routine +with forced penalties. However, as part of opening up and control this primitive +makes sense. As a bonus we also provide two extra penalties: + +\starttyping +\prebinoppenalty = -100 % example value +\prerelpenalty = 900 % example value +\stoptyping + +They default to inifinite which signals that they don't need to be inserted. When +set they are injected before a binop or rel noad. This is an experimental feature. + +\subsection{Equation spacing: \lpr {matheqnogapstep}} + +By default \TEX\ will add one quad between the equation and the number. This is +hard coded. A new primitive can control this: + +\startsyntax +\matheqnogapstep = 1000 +\stopsyntax + +Because a math quad from the math text font is used instead of a dimension, we +use a step to control the size. A value of zero will suppress the gap. The step +is divided by 1000 which is the usual way to mimmick floating point factors in +\TEX. + +\stopsection + +\startsection[title={Math constructs}] + +\subsection {Unscaled fences} + +\topicindex {math+fences} + +The \lpr {mathdelimitersmode} primitive is experimental and deals with the +following (potential) problems. Three bits can be set. The first bit prevents an +unwanted shift when the fence symbol is not scaled (a cambria side effect). The +second bit forces italic correction between a preceding character ordinal and the +fenced subformula, while the third bit turns that subformula into an ordinary so +that the same spacing applies as with unfenced variants. Here we show Cambria +(with \lpr {mathitalicsmode} enabled). \starttexdefinition Whatever #1 \NC \type{\mathdelimitersmode = #1} @@ -716,127 +1037,34 @@ so that the same spacing applies as with unfenced variants. Here we show Cambria \stop So, when set to 7 fenced subformulas with unscaled delimiters come out the same -as unfenced ones. This can be handy for cases where one is forced to use \type -{\left} and \type {\right} always because of unpredictable content. As said, it's -an experimental features (which somehow fits in the exceptional way fences are -dealt with in the engine). The full list of flags is given in the next table: - -\starttabulate[|T|l|] -\NC "01 \NC don't apply the usual shift \NC \NR -\NC "02 \NC apply italic correction when possible \NC \NR -\NC "04 \NC force a ordinary subformula \NC \NR -\NC "08 \NC no shift when a base character \NC \NR -\NC "10 \NC only shift when an extensible \NC \NR +as unfenced ones. This can be handy for cases where one is forced to use \prm +{left} and \prm {right} always because of unpredictable content. As said, it's an +experimental feature (which somehow fits in the exceptional way fences are dealt +with in the engine). The full list of flags is given in the next table: + +\starttabulate[|c|l|] +\DB value \BC meaning \NC \NR +\TB +\NC \type{"01} \NC don't apply the usual shift \NC \NR +\NC \type{"02} \NC apply italic correction when possible \NC \NR +\NC \type{"04} \NC force an ordinary subformula \NC \NR +\NC \type{"08} \NC no shift when a base character \NC \NR +\NC \type{"10} \NC only shift when an extensible \NC \NR +\LL \stoptabulate -The effect can depend on the font (and for Cambria one can use for instance \type -{"16}). +The effect can depend on the font (and for Cambria one can use for instance \type {"16}). -\section{Math spacing setting} +\subsection[mathacc]{Accent handling} -Besides the parameters mentioned in the previous sections, there are also 64 new -primitives to control the math spacing table (as explained in Chapter~18 of the -\TEX book). The primitive names are a simple matter of combining two math atom -types, but for completeness' sake, here is the whole list: - -\starttwocolumns -\starttyping -\Umathordordspacing -\Umathordopspacing -\Umathordbinspacing -\Umathordrelspacing -\Umathordopenspacing -\Umathordclosespacing -\Umathordpunctspacing -\Umathordinnerspacing -\Umathopordspacing -\Umathopopspacing -\Umathopbinspacing -\Umathoprelspacing -\Umathopopenspacing -\Umathopclosespacing -\Umathoppunctspacing -\Umathopinnerspacing -\Umathbinordspacing -\Umathbinopspacing -\Umathbinbinspacing -\Umathbinrelspacing -\Umathbinopenspacing -\Umathbinclosespacing -\Umathbinpunctspacing -\Umathbininnerspacing -\Umathrelordspacing -\Umathrelopspacing -\Umathrelbinspacing -\Umathrelrelspacing -\Umathrelopenspacing -\Umathrelclosespacing -\Umathrelpunctspacing -\Umathrelinnerspacing -\Umathopenordspacing -\Umathopenopspacing -\Umathopenbinspacing -\Umathopenrelspacing -\Umathopenopenspacing -\Umathopenclosespacing -\Umathopenpunctspacing -\Umathopeninnerspacing -\Umathcloseordspacing -\Umathcloseopspacing -\Umathclosebinspacing -\Umathcloserelspacing -\Umathcloseopenspacing -\Umathcloseclosespacing -\Umathclosepunctspacing -\Umathcloseinnerspacing -\Umathpunctordspacing -\Umathpunctopspacing -\Umathpunctbinspacing -\Umathpunctrelspacing -\Umathpunctopenspacing -\Umathpunctclosespacing -\Umathpunctpunctspacing -\Umathpunctinnerspacing -\Umathinnerordspacing -\Umathinneropspacing -\Umathinnerbinspacing -\Umathinnerrelspacing -\Umathinneropenspacing -\Umathinnerclosespacing -\Umathinnerpunctspacing -\Umathinnerinnerspacing -\stoptyping -\stoptwocolumns - -These parameters are of type \type {\muskip}, so setting a parameter can be done -like this: - -\starttyping -\Umathopordspacing\displaystyle=4mu plus 2mu -\stoptyping - -They are all initialized by \type {initex} to the values mentioned in the table -in Chapter~18 of the \TEX book. - -Note 1: for ease of use as well as for backward compatibility, \type -{\thinmuskip}, \type {\medmuskip} and \type {\thickmuskip} are treated -especially. In their case a pointer to the corresponding internal parameter is -saved, not the actual \type {\muskip} value. This means that any later changes to -one of these three parameters will be taken into account. - -Note 2: Careful readers will realise that there are also primitives for the items -marked \type {*} in the \TEX book. These will not actually be used as those -combinations of atoms cannot actually happen, but it seemed better not to break -orthogonality. They are initialized to zero. - -\section[mathacc]{Math accent handling} +\topicindex {math+accents} \LUATEX\ supports both top accents and bottom accents in math mode, and math accents stretch automatically (if this is supported by the font the accent comes from, of course). Bottom and combined accents as well as fixed-width math accents -are controlled by optional keywords following \type {\Umathaccent}. +are controlled by optional keywords following \lpr {Umathaccent}. -The keyword \type {bottom} after \type {\Umathaccent} signals that a bottom accent +The keyword \type {bottom} after \lpr {Umathaccent} signals that a bottom accent is needed, and the keyword \type {both} signals that both a top and a bottom accent are needed (in this case two accents need to be specified, of course). @@ -852,7 +1080,7 @@ A simple example: If a math top accent has to be placed and the accentee is a character and has a non-zero \type {top_accent} value, then this value will be used to place the -accent instead of the \type {\skewchar} kern used by \TEX82. +accent instead of the \prm {skewchar} kern used by \TEX82. The \type {top_accent} value represents a vertical line somewhere in the accentee. The accent will be shifted horizontally such that its own \type @@ -861,7 +1089,7 @@ accentee. The accent will be shifted horizontally such that its own \type followed by its italic correction is used instead. The vertical placement of a top accent depends on the \type {x_height} of the -font of the accentee (as explained in the \TEX book), but if value that turns out +font of the accentee (as explained in the \TEX book), but if a value turns out to be zero and the font had a \type {MathConstants} table, then \type {AccentBaseHeight} is used instead. @@ -870,28 +1098,30 @@ correction takes place. Possible locations are \type {top}, \type {bottom}, \type {both} and \type {center}. When no location is given \type {top} is assumed. An additional -parameter \type {fraction} can be specified followed by a number; a value of for +parameter \nod {fraction} can be specified followed by a number; a value of for instance 1200 means that the criterium is 1.2 times the width of the nucleus. The fraction only applies to the stepwise selected shapes and is mostly meant for the \type {overlay} location. It also works for the other locations but then it concerns the width. -\section{Math root extension} +\subsection{Radical extensions} + +\topicindex {math+radicals} -The new primitive \type {\Uroot} allows the construction of a radical noad -including a degree field. Its syntax is an extension of \type {\Uradical}: +The new primitive \lpr {Uroot} allows the construction of a radical noad +including a degree field. Its syntax is an extension of \lpr {Uradical}: \starttyping \Uradical <fam integer> <char integer> <radicand> \Uroot <fam integer> <char integer> <degree> <radicand> \stoptyping -The placement of the degree is controlled by the math parameters \type -{\Umathradicaldegreebefore}, \type {\Umathradicaldegreeafter}, and \type -{\Umathradicaldegreeraise}. The degree will be typeset in \type -{\scriptscriptstyle}. +The placement of the degree is controlled by the math parameters \lpr +{Umathradicaldegreebefore}, \lpr {Umathradicaldegreeafter}, and \lpr +{Umathradicaldegreeraise}. The degree will be typeset in \prm +{scriptscriptstyle}. -\section{Math kerning in super- and subscripts} +\subsection{Super- and subscripts} The character fields in a \LUA|-|loaded \OPENTYPE\ math font can have a \quote {mathkern} table. The format of this table is the same as the \quote {mathkern} @@ -950,11 +1180,15 @@ next higher height and kern pair, or the highest one in the character (if there value high enough in the character), or simply zero (if the character has no math kern pairs at all). -\section{Scripts on horizontally extensible items like arrows} +\subsection{Scripts on extensibles} -The primitives \type {\Uunderdelimiter} and \type {\Uoverdelimiter} allow the +\topicindex {math+scripts} +\topicindex {math+delimiters} +\topicindex {math+extensibles} + +The primitives \lpr {Uunderdelimiter} and \lpr {Uoverdelimiter} allow the placement of a subscript or superscript on an automatically extensible item and -\type {\Udelimiterunder} and \type {\Udelimiterover} allow the placement of an +\lpr {Udelimiterunder} and \lpr {Udelimiterover} allow the placement of an automatically extensible item as a subscript or superscript on a nucleus. The input: @@ -972,18 +1206,18 @@ $\Udelimiterunder 0 "2194 {\hbox{\strut delimiterunder}}$ \blank \startnarrower \getbuffer \stopnarrower \blank -The vertical placements are controlled by \type {\Umathunderdelimiterbgap}, \type -{\Umathunderdelimitervgap}, \type {\Umathoverdelimiterbgap}, and \type -{\Umathoverdelimitervgap} in a similar way as limit placements on large operators. -The superscript in \type {\Uoverdelimiter} is typeset in a suitable scripted style, -the subscript in \type {\Uunderdelimiter} is cramped as well. +The vertical placements are controlled by \lpr {Umathunderdelimiterbgap}, \lpr +{Umathunderdelimitervgap}, \lpr {Umathoverdelimiterbgap}, and \lpr +{Umathoverdelimitervgap} in a similar way as limit placements on large operators. +The superscript in \lpr {Uoverdelimiter} is typeset in a suitable scripted style, +the subscript in \lpr {Uunderdelimiter} is cramped as well. These primitives accepts an option \type {width} specification. When used the also optional keywords \type {left}, \type {middle} and \type {right} will determine what happens when a requested size can't be met (which can happen when we step to successive larger variants). -An extra primitive \type {\Uhextensible} is available that can be used like this: +An extra primitive \lpr {Uhextensible} is available that can be used like this: \startbuffer $\Uhextensible width 10cm 0 "2194$ @@ -1008,38 +1242,11 @@ $\Uhextensible width 1pt middle 0 "2194$ font metrics are involved we have a different code path for traditional fonts end \OPENTYPE\ fonts. -\section {Extracting values} - -You can extract the components of a math character. Say that we have defined: +\subsection{Fractions} -\starttyping -\Umathcode 1 2 3 4 -\stoptyping - -then - -\starttyping -[\Umathcharclass1] [\Umathcharfam1] [\Umathcharslot1] -\stoptyping +\topicindex {math+fractions} -will return: - -\starttyping -[2] [3] [4] -\stoptyping - -These commands are provides as convenience. Before they came available you could -do the following: - -\starttyping -\def\Umathcharclass{\directlua{tex.print(tex.getmathcode(token.scan_int())[1])}} -\def\Umathcharfam {\directlua{tex.print(tex.getmathcode(token.scan_int())[2])}} -\def\Umathcharslot {\directlua{tex.print(tex.getmathcode(token.scan_int())[3])}} -\stoptyping - -\section{fractions} - -The \type {\abovewithdelims} command accepts a keyword \type {exact}. When issued +The \prm {abovewithdelims} command accepts a keyword \type {exact}. When issued the extra space relative to the rule thickness is not added. One can of course use the \type {\Umathfraction..gap} commands to influence the spacing. Also the rule is still positioned around the math axis. @@ -1053,7 +1260,7 @@ vertical gap for skewed fractions. Of course some guessing is needed in order to implement something that uses them. And so we now provide a primitive similar to the other fraction related ones but with a few options so that one can influence the rendering. Of course a user can also mess around a bit with the parameters -\type {\Umathskewedfractionhgap} and \type {\Umathskewedfractionvgap}. +\lpr {Umathskewedfractionhgap} and \lpr {Umathskewedfractionvgap}. The syntax used here is: @@ -1064,7 +1271,7 @@ The syntax used here is: where the options can be \type {noaxis} and \type {exact}. By default we add half the axis to the shifts and by default we zero the width of the middle character. -For Latin Modern The result looks as follows: +For Latin Modern the result looks as follows: \def\ShowA#1#2#3{$x + { {#1} \Uskewed / #3 {#2} } + x$} \def\ShowB#1#2#3{$x + { {#1} \Uskewedwithdelims / () #3 {#2} } + x$} @@ -1099,7 +1306,85 @@ For Latin Modern The result looks as follows: \stoptabulate \stop -\section {Last lines} +\subsection {Delimiters: \type{\Uleft}, \prm {Umiddle} and \prm {Uright}} + +\topicindex {math+delimiters} + +Normally you will force delimiters to certain sizes by putting an empty box or +rule next to it. The resulting delimiter will either be a character from the +stepwise size range or an extensible. The latter can be quite differently +positioned than the characters as it depends on the fit as well as the fact if +the used characters in the font have depth or height. Commands like (plain \TEX +s) \type {\big} need use this feature. In \LUATEX\ we provide a bit more control +by three variants that support optional parameters \type {height}, \type {depth} +and \type {axis}. The following example uses this: + +\startbuffer +\Uleft height 30pt depth 10pt \Udelimiter "0 "0 "000028 +\quad x\quad +\Umiddle height 40pt depth 15pt \Udelimiter "0 "0 "002016 +\quad x\quad +\Uright height 30pt depth 10pt \Udelimiter "0 "0 "000029 +\quad \quad \quad +\Uleft height 30pt depth 10pt axis \Udelimiter "0 "0 "000028 +\quad x\quad +\Umiddle height 40pt depth 15pt axis \Udelimiter "0 "0 "002016 +\quad x\quad +\Uright height 30pt depth 10pt axis \Udelimiter "0 "0 "000029 +\stopbuffer + +\typebuffer + +\startlinecorrection +\ruledhbox{\mathematics{\getbuffer}} +\stoplinecorrection + +The keyword \type {exact} can be used as directive that the real dimensions +should be applied when the criteria can't be met which can happen when we're +still stepping through the successively larger variants. When no dimensions are +given the \type {noaxis} command can be used to prevent shifting over the axis. + +You can influence the final class with the keyword \type {class} which will +influence the spacing. The numbers are the same as for character classes. + +\stopsection + +\startsection[title={Extracting values}] + +\subsection{Codes} + +\topicindex {math+codes} + +You can extract the components of a math character. Say that we have defined: + +\starttyping +\Umathcode 1 2 3 4 +\stoptyping + +then + +\starttyping +[\Umathcharclass1] [\Umathcharfam1] [\Umathcharslot1] +\stoptyping + +will return: + +\starttyping +[2] [3] [4] +\stoptyping + +These commands are provides as convenience. Before they come available you could +do the following: + +\starttyping +\def\Umathcharclass{\directlua{tex.print(tex.getmathcode(token.scan_int())[1])}} +\def\Umathcharfam {\directlua{tex.print(tex.getmathcode(token.scan_int())[2])}} +\def\Umathcharslot {\directlua{tex.print(tex.getmathcode(token.scan_int())[3])}} +\stoptyping + +\subsection {Last lines} + +\topicindex {math+last line} There is a new primitive to control the overshoot in the calculation of the previous line in mid|-|paragraph display math. The default value is 2 times @@ -1121,32 +1406,41 @@ get the length of the last line, the following will often work too: \relax} \stoptyping -\section {Other Math changes} +\stopsection + +\startsection[title={Math mode}] + +\subsection {Verbose versions of single|-|character math commands} -\subsection {Verbose versions of single-character math commands} +\topicindex {math+styles} \LUATEX\ defines six new primitives that have the same function as \type {^}, \type {_}, \type {$}, and \type {$$}: \starttabulate[|l|l|] -\BC primitive \BC explanation \NC \NR -\NC \type {\Usuperscript} \NC Duplicates the functionality of \type {^} \NC \NR -\NC \type {\Usubscript} \NC Duplicates the functionality of \type {_} \NC \NR -\NC \type {\Ustartmath} \NC Duplicates the functionality of \type {$}, % $ +\DB primitive \BC explanation \NC \NR +\TB +\NC \lpr {Usuperscript} \NC duplicates the functionality of \type {^} \NC \NR +\NC \lpr {Usubscript} \NC duplicates the functionality of \type {_} \NC \NR +\NC \lpr {Ustartmath} \NC duplicates the functionality of \type {$}, % $ when used in non-math mode. \NC \NR -\NC \type {\Ustopmath} \NC Duplicates the functionality of \type {$}, % $ +\NC \lpr {Ustopmath} \NC duplicates the functionality of \type {$}, % $ when used in inline math mode. \NC \NR -\NC \type {\Ustartdisplaymath} \NC Duplicates the functionality of \type {$$}, % $$ +\NC \lpr {Ustartdisplaymath} \NC duplicates the functionality of \type {$$}, % $$ when used in non-math mode. \NC \NR -\NC \type {\Ustopdisplaymath} \NC Duplicates the functionality of \type {$$}, % $$ +\NC \lpr {Ustopdisplaymath} \NC duplicates the functionality of \type {$$}, % $$ when used in display math mode. \NC \NR +\LL \stoptabulate -The \type {\Ustopmath} and \type {\Ustopdisplaymath} primitives check if the current +The \lpr {Ustopmath} and \lpr {Ustopdisplaymath} primitives check if the current math mode is the correct one (inline vs.\ displayed), but you can freely intermix the four mathon|/|mathoff commands with explicit dollar sign(s). -\subsection{Script commands \type {\Unosuperscript} and \type {\Unosubscript}} +\subsection{Script commands \lpr {Unosuperscript} and \lpr {Unosubscript}} + +\topicindex {math+styles} +\topicindex {math+scripts} These two commands result in super- and subscripts but with the current style (at the time of rendering). So, @@ -1164,71 +1458,17 @@ $ results in \inlinebuffer[script]. +\subsection{Allowed math commands in non|-|math modes} -\subsection{Allowed math commands in non-math modes} - -The commands \type {\mathchar}, and \type {\Umathchar} and control sequences that -are the result of \type {\mathchardef} or \type {\Umathchardef} are also -acceptable in the horizontal and vertical modes. In those cases, the \type -{\textfont} from the requested math family is used. - -\section{Math surrounding skips} - -Inline math is surrounded by (optional) \type {\mathsurround} spacing but that is fixed -dimension. There is now an additional parameter \type {\mathsurroundskip}. When set to a -non|-|zero value (or zero with some stretch or shrink) this parameter will replace -\type {\mathsurround}. By using an additional parameter instead of changing the nature -of \type {\mathsurround}, we can remain compatible. In the meantime a bit more -control has been added via \type {\mathsurroundmode}. This directive can take 6 values -with zero being the default behaviour. - -\start +\topicindex {math+text} +\topicindex {text+math} -\def\OneLiner#1#2% - {\NC \type{#1} - \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x$x$x} - \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x $x$ x} - \NC #2 - \NC \NR} +The commands \prm {mathchar}, and \lpr {Umathchar} and control sequences that are +the result of \prm {mathchardef} or \lpr {Umathchardef} are also acceptable in +the horizontal and vertical modes. In those cases, the \prm {textfont} from the +requested math family is used. -\startbuffer -\mathsurround 10pt -\mathsurroundskip20pt -\stopbuffer - -\typebuffer \getbuffer - -\starttabulate[|c|c|c|pl|] -\HL -\BC mode \BC \type {x$x$x} \BC \type {x $x$ x} \BC effect \NC \NR -\HL -\OneLiner{0}{obey \type {\mathsurround} when \type {\mathsurroundskip} is 0pt} -\OneLiner{1}{only add skip to the left} -\OneLiner{2}{only add skip to the right} -\OneLiner{3}{add skip to the left and right} -\OneLiner{4}{ignore the skip setting, obey \type {\mathsurround}} -\OneLiner{5}{disable all spacing around math} -\OneLiner{6}{only apply \type {\mathsurroundskip} when also spacing} -\OneLiner{7}{only apply \type {\mathsurroundskip} when no spacing} -\HL -\stoptabulate - -\stop - -Method six omits the surround glue when there is (x)spacing glue present while -method seven does the opposite, the glue is only applied when there is (x)space -glue present too. Anything more fancy, like checking the begining or end of a -paragraph (or edges of a box) would not be robust anyway. If you want that you -can write a callback that runs over a list and analyzes a paragraph. Actually, in -that case you could also inject glue (or set the properties of a math node) -explicitly. So, these modes are in practice mostly useful for special purposes -and experiments (they originate in a tracker item). Keep in mind that this glue -is part of the math node and not always treated as normal glue: it travels with -the begin and end math nodes. Also, method 6 and 7 will zero the skip related -fields in a node when applicable in the first occasion that checks them -(linebreaking or packaging). - -% \section{Math todo} +% \startsection[title={Math todo}] % % The following items are still todo. % @@ -1249,194 +1489,158 @@ fields in a node when applicable in the first occasion that checks them % Support for multi|-|line displays using \MATHML\ style alignment points. % \stopitem % \stopitemize +% +% \stopsection -\subsection {Delimiters: \type{\Uleft}, \type {\Umiddle} and \type {\Uright}} - -Normally you will force delimiters to certain sizes by putting an empty box or -rule next to it. The resulting delimiter will either be a character from the -stepwise size range or an extensible. The latter can be quite differently -positioned that the characters as it depends on the fit as well as the fact if -the used characters in the font have depth or height. Commands like (plain \TEX -s) \type {\big} need use this feature. In \LUATEX\ we provide a bit more control -by three variants that supporting optional parameters \type {height}, \type -{depth} and \type {axis}. The following example uses this: - -\startbuffer -\Uleft height 30pt depth 10pt \Udelimiter "0 "0 "000028 -\quad x\quad -\Umiddle height 40pt depth 15pt \Udelimiter "0 "0 "002016 -\quad x\quad -\Uright height 30pt depth 10pt \Udelimiter "0 "0 "000029 -\quad \quad \quad -\Uleft height 30pt depth 10pt axis \Udelimiter "0 "0 "000028 -\quad x\quad -\Umiddle height 40pt depth 15pt axis \Udelimiter "0 "0 "002016 -\quad x\quad -\Uright height 30pt depth 10pt axis \Udelimiter "0 "0 "000029 -\stopbuffer - -\typebuffer - -\startlinecorrection -\ruledhbox{\mathematics{\getbuffer}} -\stoplinecorrection - -The keyword \type {exact} can be used as directive that the real dimensions -should be applied when the criteria can't be met which can happen when we're -still stepping through the successively larger variants. When no dimensions are -given the \type {noaxis} command can be used to prevent shifting over the axis. +\stopsection -You can influence the final class with the keyword \type {class} which will -influence the spacing. The numbers are the same as for character classes. +\startsection[title={Goodies}] -\subsection{Fixed scripts} +\subsection {Flattening: \lpr {mathflattenmode}} -We have three parameters that are used for this fixed anchoring: +\topicindex {math+flattening} -\starttabulate[|l|l|] -\NC $d$ \NC \type {\Umathsubshiftdown} \NC \NR -\NC $u$ \NC \type {\Umathsupshiftup} \NC \NR -\NC $s$ \NC \type {\Umathsubsupshiftdown} \NC \NR -\stoptabulate +The \TEX\ math engine collapses \type {ord} noads without sub- and superscripts +and a character as nucleus. and which has the side effect that in \OPENTYPE\ mode +italic corrections are applied (given that they are enabled). -When we set \type {\mathscriptsmode} to a value other than zero these are used -for calculating fixed positions. This is something that is needed for instance -for chemistry. You can manipulate the mentioned variables to achive different -effects. +\startbuffer[sample] +\switchtobodyfont[modern] +$V \mathbin{\mathbin{v}} V$\par +$V \mathord{\mathord{v}} V$\par +\stopbuffer -\def\SampleMath#1% - {$\mathscriptsmode#1\mathupright CH_2 + CH^+_2 + CH^2_2$} +\typebuffer[sample] -\starttabulate[|c|c|c|l|] -\BC mode \BC down \BC up \BC \NC \NR -\NC 0 \NC dynamic \NC dynamic \NC \SampleMath{0} \NC \NR -\NC 1 \NC $d$ \NC $u$ \NC \SampleMath{1} \NC \NR -\NC 2 \NC $s$ \NC $u$ \NC \SampleMath{2} \NC \NR -\NC 3 \NC $s$ \NC $u + s - d$ \NC \SampleMath{3} \NC \NR -\NC 4 \NC $d + (s-d)/2$ \NC $u + (s-d)/2$ \NC \SampleMath{4} \NC \NR -\NC 5 \NC $d$ \NC $u + s - d$ \NC \SampleMath{5} \NC \NR -\stoptabulate +This renders as: -The value of this parameter obeys grouping but applies to the whole current -formula. +\blank \start \mathflattenmode\plusone \getbuffer[sample] \stop \blank -% if needed we can put the value in stylenodes but maybe more should go there +When we set \lpr {mathflattenmode} to 31 we get: -\subsection{Penalties: \type {\mathpenaltiesmode}} +\blank \start \mathflattenmode\numexpr1+2+4+8+16\relax \getbuffer[sample] \stop \blank -Only in inline math penalties will be added in a math list. You can force -penalties (also in display math) by setting: +When you see no difference, then the font probably has the proper character +dimensions and no italic correction is needed. For Latin Modern (at least till +2018) there was a visual difference. In that respect this parameter is not always +needed unless of course you want efficient math lists anyway. -\starttyping -\mathpenaltiesmode = 1 -\stoptyping - -This primnitive is not really needed in \LUATEX\ because you can use the callback -\type {mlist_to_hlist} to force penalties by just calling the regular routine -with forced penalties. However, as part of opening up and control this primitive -makes sense. As a bonus we also provide two extra penalties: +You can influence flattening by adding the appropriate number to the value of the +mode parameter. The default value is~1. -\starttyping -\prebinoppenalty = -100 % example value -\prerelpenalty = 900 % example value -\stoptyping +\starttabulate[|Tc|c|] +\DB mode \BC class \NC \NR +\TB +\NC 1 \NC ord \NC \NR +\NC 2 \NC bin \NC \NR +\NC 4 \NC rel \NC \NR +\NC 8 \NC punct \NC \NR +\NC 16 \NC inner \NC \NR +\LL +\stoptabulate -They default to inifinite which signals that they don't need to be inserted. When -set they are injected before a binop or rel noad. This is an experimental feature. +\subsection {Less Tracing} -\subsection {Tracing} +\topicindex {math+tracing} Because there are quite some math related parameters and values, it is possible to limit tracing. Only when \type {tracingassigns} and|/|or \type {tracingrestores} are set to~2 or more they will be traced. -\subsection {Math options} +\subsection {Math options with \lpr {mathoption}} The logic in the math engine is rather complex and there are often no universal solutions (read: what works out well for one font, fails for another). Therefore -some variations in the implementation will be driven by options for which a new -primitive \type {\mathoption} has been introduced (so that we don't end up with -many new commands). The approach of options also permits us to see what effect a -specific solution has. - -\subsubsection {\type {\mathoption old}} +some variations in the implementation are driven by parameters (modes). In +addition there is a new primitive \lpr {mathoption} which will be used for +testing. Don't rely on any option to be there in a production version as they are +meant for development. This option was introduced for testing purposes when the math engine got split code paths and it forces the engine to treat new fonts as old ones with respect to italic correction etc. There are no guarantees given with respect to the final -result and unexpected side effects are not seens as bugs as they relate to font -properties. +result and unexpected side effects are not seen as bugs as they relate to font +properties. Ther eis currently only one option: \startbuffer \mathoption old 1 \stopbuffer The \type {oldmath} boolean flag in the \LUA\ font table is the official way to -force old treatment as it's bound to fonts. - -\subsubsection {\type {\mathoption noitaliccompensation}} - -This option compensates placement for characters with a built|-|in italic -correction. - -\startbuffer -{\showboxes\int}\quad -{\showboxes\int_{|}^{|}}\quad -{\showboxes\int\limits_{|}^{|}} -\stopbuffer - -\typebuffer - -Gives (with computer modern that has such italics): - -\startlinecorrection[blank] - \switchtobodyfont[modern] - \startcombination[nx=2,ny=2,distance=5em] - {\mathoption noitaliccompensation 0\relax \mathematics{\getbuffer}} - {\nohyphens\type{0:inline}} - {\mathoption noitaliccompensation 0\relax \mathematics{\displaymath\getbuffer}} - {\nohyphens\type{0:display}} - {\mathoption noitaliccompensation 1\relax \mathematics{\getbuffer}} - {\nohyphens\type{1:inline}} - {\mathoption noitaliccompensation 1\relax \mathematics{\displaymath\getbuffer}} - {\nohyphens\type{1:display}} - \stopcombination -\stoplinecorrection - -\subsubsection {\type {\mathoption nocharitalic}} +force old treatment as it's bound to fonts. Like with all options we may +temporarily introduce with this command this feature is not meant for production. -When two characters follow each other italic correction can interfere. The -following example shows what this option does: - -\startbuffer -\catcode"1D443=11 -\catcode"1D444=11 -\catcode"1D445=11 -P( PP PQR -\stopbuffer - -\typebuffer - -Gives (with computer modern that has such italics): - -\startlinecorrection[blank] - \switchtobodyfont[modern] - \startcombination[nx=2,ny=2,distance=5em] - {\mathoption nocharitalic 0\relax \mathematics{\getbuffer}} - {\nohyphens\type{0:inline}} - {\mathoption nocharitalic 0\relax \mathematics{\displaymath\getbuffer}} - {\nohyphens\type{0:display}} - {\mathoption nocharitalic 1\relax \mathematics{\getbuffer}} - {\nohyphens\type{1:inline}} - {\mathoption nocharitalic 1\relax \mathematics{\displaymath\getbuffer}} - {\nohyphens\type{1:display}} - \stopcombination -\stoplinecorrection - -\subsubsection {\type {\mathoption useoldfractionscaling}} +% % obsolete: +% +% \subsubsection {\type {\mathoption noitaliccompensation}} +% +% This option compensates placement for characters with a built|-|in italic +% correction. +% +% \startbuffer +% {\showboxes\int}\quad +% {\showboxes\int_{|}^{|}}\quad +% {\showboxes\int\limits_{|}^{|}} +% \stopbuffer +% +% \typebuffer +% +% Gives (with computer modern that has such italics): +% +% \startlinecorrection[blank] +% \switchtobodyfont[modern] +% \startcombination[nx=2,ny=2,distance=5em] +% {\mathoption noitaliccompensation 0\relax \mathematics{\getbuffer}} +% {\nohyphens\type{0:inline}} +% {\mathoption noitaliccompensation 0\relax \mathematics{\displaymath\getbuffer}} +% {\nohyphens\type{0:display}} +% {\mathoption noitaliccompensation 1\relax \mathematics{\getbuffer}} +% {\nohyphens\type{1:inline}} +% {\mathoption noitaliccompensation 1\relax \mathematics{\displaymath\getbuffer}} +% {\nohyphens\type{1:display}} +% \stopcombination +% \stoplinecorrection + +% % obsolete: +% +% \subsubsection {\type {\mathoption nocharitalic}} +% +% When two characters follow each other italic correction can interfere. The +% following example shows what this option does: +% +% \startbuffer +% \catcode"1D443=11 +% \catcode"1D444=11 +% \catcode"1D445=11 +% P( PP PQR +% \stopbuffer +% +% \typebuffer +% +% Gives (with computer modern that has such italics): +% +% \startlinecorrection[blank] +% \switchtobodyfont[modern] +% \startcombination[nx=2,ny=2,distance=5em] +% {\mathoption nocharitalic 0\relax \mathematics{\getbuffer}} +% {\nohyphens\type{0:inline}} +% {\mathoption nocharitalic 0\relax \mathematics{\displaymath\getbuffer}} +% {\nohyphens\type{0:display}} +% {\mathoption nocharitalic 1\relax \mathematics{\getbuffer}} +% {\nohyphens\type{1:inline}} +% {\mathoption nocharitalic 1\relax \mathematics{\displaymath\getbuffer}} +% {\nohyphens\type{1:display}} +% \stopcombination +% \stoplinecorrection + +% % obsolete: +% +% \subsubsection {\type {\mathoption useoldfractionscaling}} +% +% This option has been introduced as solution for tracker item 604 for fuzzy cases +% around either or not present fraction related settings for new fonts. -This option has been introduced as solution for tracker item 604 for fuzzy cases -around either or not present fraction related settings for new fonts. +\stopsection \stopchapter diff --git a/doc/context/sources/general/manuals/luatex/luatex-modifications.tex b/doc/context/sources/general/manuals/luatex/luatex-modifications.tex index b5d8f2750..50d23fb1b 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-modifications.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-modifications.tex @@ -1,7 +1,6 @@ % language=uk \environment luatex-style -\environment luatex-logos \startcomponent luatex-modifications @@ -11,6 +10,9 @@ \startsubsection[title=The need for change] +\topicindex {engines} +\topicindex {history} + The first version of \LUATEX\ only had a few extra primitives and it was largely the same as \PDFTEX. Then we merged substantial parts of \ALEPH\ into the code and got more primitives. When we got more stable the decision was made to clean @@ -22,13 +24,15 @@ most of the adapted ones. Besides the expected changes caused by new functionality, there are a number of not|-|so|-|expected changes. These are sometimes a side|-|effect of a new -(conflicting) feature, or, more often than not, a change neccessary to clean up +(conflicting) feature, or, more often than not, a change necessary to clean up the internal interfaces. These will also be mentioned. \stopsubsection \startsubsection[title=Changes from \TEX\ 3.1415926] +\topicindex {\TEX} + Of course it all starts with traditional \TEX. Even if we started with \PDFTEX, most still comes from the original. But we divert a bit. @@ -38,7 +42,7 @@ most still comes from the original. But we divert a bit. The current code base is written in \CCODE, not \PASCAL. We use \CWEB\ when possible. As a consequence instead of one large file plus change files, we now have multiple files organized in categories like \type {tex}, \type - {pdf}, \type {lang}, \type {font}, \type {lua}, etc. There are some artefacts + {pdf}, \type {lang}, \type {font}, \type {lua}, etc. There are some artifacts of the conversion to \CCODE, but in due time we will clean up the source code and make sure that the documentation is done right. Many files are in the \CWEB\ format, but others, like those interfacing to \LUA, are \CCODE\ files. @@ -63,21 +67,21 @@ most still comes from the original. But we divert a bit. \stopitem \startitem - The upper limit to \type {\endlinechar} and \type {\newlinechar} is 127. + The upper limit to \prm {endlinechar} and \prm {newlinechar} is 127. \stopitem \startitem - Magnification (\type {\mag}) is only supported in \DVI\ output mode. You can + Magnification (\prm {mag}) is only supported in \DVI\ output mode. You can set this parameter and it even works with \type {true} units till you switch to \PDF\ output mode. When you use \PDF\ output you can best not touch the - \type {\mag} variable. This fuzzy behaviour is not much different from using + \prm {mag} variable. This fuzzy behaviour is not much different from using \PDF\ backend related functionality while eventually \DVI\ output is required. After the output mode has been frozen (normally that happens when the first page is shipped out) or when \PDF\ output is enabled, the \type {true} specification is ignored. When you preload a plain format adapted to - \LUATEX\ it can be that the \type {\mag} parameter already has been set. + \LUATEX\ it can be that the \prm {mag} parameter already has been set. \stopitem \stopitemize @@ -86,6 +90,8 @@ most still comes from the original. But we divert a bit. \startsubsection[title=Changes from \ETEX\ 2.2] +\topicindex {\ETEX} + Being the de factor standard extension of course we provide the \ETEX\ functionality, but with a few small adaptations. @@ -99,18 +105,19 @@ functionality, but with a few small adaptations. \startitem The \TEXXET\ extension is not present, so the primitives \type {\TeXXeTstate}, \type {\beginR}, \type {\beginL}, \type {\endR} and \type - {\endL} are missing. Instead we use the \OMEGA\ approach to directionality. + {\endL} are missing. Instead we used the \OMEGA/\ALEPH\ approach to + directionality as starting point. \stopitem \startitem - Some of the tracing information that is output by \ETEX's \type - {\tracingassigns} and \type {\tracingrestores} is not there. + Some of the tracing information that is output by \ETEX's \prm + {tracingassigns} and \prm {tracingrestores} is not there. \stopitem \startitem - Register management in \LUATEX\ uses the \ALEPH\ model, so the maximum value - is 65535 and the implementation uses a flat array instead of the mixed - flat|\&|sparse model from \ETEX. + Register management in \LUATEX\ uses the \OMEGA/\ALEPH\ model, so the maximum + value is 65535 and the implementation uses a flat array instead of the mixed + flat & sparse model from \ETEX. \stopitem \startitem @@ -127,6 +134,8 @@ functionality, but with a few small adaptations. \startsubsection[title=Changes from \PDFTEX\ 1.40] +\topicindex {\PDFTEX} + Because we want to produce \PDF\ the most natural starting point was the popular \PDFTEX\ program. We inherit the stable features, dropped most of the experimental code and promoted some functionality to core \LUATEX\ functionality @@ -134,76 +143,73 @@ which in turn triggered renaming primitives. For compatibility reasons we still refer to \type {\pdf...} commands but \LUATEX\ has a different backend interface. Instead of these primitives there are three -interfacing primitives: \type {\pdfextension}, \type {\pdfvariable} and -\type {\pdffeedback} that take keywords and optional further arguments. This way -we can extend the features when needed but don't need to adapt the core engine. -The front- and backend are decoupled as much as possible. +interfacing primitives: \lpr {pdfextension}, \lpr {pdfvariable} and \lpr +{pdffeedback} that take keywords and optional further arguments (below we will +still use the \tex {pdf} prefix names as reference). This way we can extend the +features when needed but don't need to adapt the core engine. The front- and +backend are decoupled as much as possible. \startitemize \startitem The (experimental) support for snap nodes has been removed, because it is much more natural to build this functionality on top of node processing and - attributes. The associated primitives that are now gone are: \type - {\pdfsnaprefpoint}, \type {\pdfsnapy}, and \type {\pdfsnapycomp}. + attributes. The associated primitives that are gone are: \orm + {pdfsnaprefpoint}, \orm {pdfsnapy}, and \orm {pdfsnapycomp}. \stopitem \startitem The (experimental) support for specialized spacing around nodes has also been - removed. The associated primitives that are now gone are: \type - {\pdfadjustinterwordglue}, \type {\pdfprependkern}, and \type - {\pdfappendkern}, as well as the five supporting primitives \type - {\knbscode}, \type {\stbscode}, \type {\shbscode}, \type {\knbccode}, and - \type {\knaccode}. + removed. The associated primitives that are gone are: \orm + {pdfadjustinterwordglue}, \orm {pdfprependkern}, and \orm {pdfappendkern}, as + well as the five supporting primitives \orm {knbscode}, \orm {stbscode}, \orm + {shbscode}, \orm {knbccode}, and \orm {knaccode}. \stopitem \startitem A number of \quote {\PDFTEX\ primitives} have been removed as they can be - implemented using \LUA: - - \start \raggedright - \type {\pdfelapsedtime}, \type {\pdfescapehex}, \type {\pdfescapename}, \type - {\pdfescapestring}, \type {\pdffiledump}, \type {\pdffilemoddate}, \type - {\pdffilesize}, \type {\pdfforcepagebox}, \type {\pdflastmatch}, \type - {\pdfmatch}, \type {\pdfmdfivesum}, \type {\pdfmovechars}, \type - {\pdfoptionalwaysusepdfpagebox}, \type {\pdfoptionpdfinclusionerrorlevel}, - \type {\pdfresettimer}, \type {\pdfshellescape}, \type {\pdfstrcmp} and \type - {\pdfunescapehex} - \par \stop + implemented using \LUA: \orm {pdfelapsedtime}, \orm {pdfescapehex}, \orm + {pdfescapename}, \orm {pdfescapestring}, \orm {pdffiledump}, \orm + {pdffilemoddate}, \orm {pdffilesize}, \orm {pdfforcepagebox}, \orm + {pdflastmatch}, \orm {pdfmatch}, \orm {pdfmdfivesum}, \orm {pdfmovechars}, + \orm {pdfoptionalwaysusepdfpagebox}, \orm {pdfoptionpdfinclusionerrorlevel}, + \orm {pdfresettimer}, \orm {pdfshellescape}, \orm {pdfstrcmp} and \orm + {pdfunescapehex}. \stopitem \startitem - The version related primitives \type {\pdftexbanner}, \type {\pdftexversion} - and \type {\pdftexrevision} are no longer present as there is no longer a + The version related primitives \orm {pdftexbanner}, \orm {pdftexversion} + and \orm {pdftexrevision} are no longer present as there is no longer a relationship with \PDFTEX\ development. \stopitem \startitem The experimental snapper mechanism has been removed and therefore also the - primitives: - - \start \raggedright - \type {\pdfignoreddimen}, \type {\pdffirstlineheight}, \type - {\pdfeachlineheight}, \type {\pdfeachlinedepth} and \type - {\pdflastlinedepth} - \par \stop + primitives \orm {pdfignoreddimen}, \orm {pdffirstlineheight}, \orm + {pdfeachlineheight}, \orm {pdfeachlinedepth} and \orm {pdflastlinedepth}. \stopitem \startitem - The experimental primitives \type {\primitive}, \type {\ifprimitive}, \type - {\ifabsnum} and \type {\ifabsdim} are promoted to core primitives. The \type + The experimental primitives \lpr {primitive}, \lpr {ifprimitive}, \lpr + {ifabsnum} and \lpr {ifabsdim} are promoted to core primitives. The \type {\pdf*} prefixed originals are not available. \stopitem \startitem - The \PNG\ transparency fix from 1.40.6 is not applied as high|-|level support - is pending. Because \LUATEX\ has a different subsystem for managing images, - more diversion from its ancestor happened in the meantime. + Because \LUATEX\ has a different subsystem for managing images, more + diversion from its ancestor happened in the meantime. We don't adapt to + changes in \PDFTEX. +\stopitem + +\startitem + Two extra token lists are provided, \orm {pdfxformresources} and \orm + {pdfxformattr}, as an alternative to \orm {pdfxform} keywords. \stopitem \startitem - Two extra token lists are provides, \type {\pdfxformresources} and \type - {\pdfxformattr}, as an alternative to \type {\pdfxform} keywords. + Image specifications also support \type {visiblefilename}, \type + {userpassword} and \type {ownerpassword}. The password options are only + relevant for encrypted \PDF\ files. \stopitem \startitem @@ -214,27 +220,27 @@ The front- and backend are decoupled as much as possible. \stopitem \startitem - The primitives \type {\pdfpagewidth} and \type {\pdfpageheight} have been removed - because \type {\pagewidth} and \type {\pageheight} have that purpose. + The primitives \orm {pdfpagewidth} and \orm {pdfpageheight} have been removed + because \lpr {pagewidth} and \lpr {pageheight} have that purpose. \stopitem \startitem - The primitives \type {\pdfnormaldeviate}, \type {\pdfuniformdeviate}, \type - {\pdfsetrandomseed} and \type {\pdfrandomseed} have been promoted to core + The primitives \orm {pdfnormaldeviate}, \orm {pdfuniformdeviate}, \orm + {pdfsetrandomseed} and \orm {pdfrandomseed} have been promoted to core primitives without \type {pdf} prefix so the original commands are no longer recognized. \stopitem \startitem - The primitives \type {\ifincsname}, \type {\expanded} and \type {\quitvmode} + The primitives \lpr {ifincsname}, \lpr {expanded} and \lpr {quitvmode} are now core primitives. \stopitem \startitem As the hz and protrusion mechanism are part of the core the related - primitives \type {\lpcode}, \type {\rpcode}, \type {\efcode}, \type - {\leftmarginkern}, \type {\rightmarginkern} are promoted to core primitives. The - two commands \type {\protrudechars} and \type {\adjustspacing} replace their + primitives \lpr {lpcode}, \lpr {rpcode}, \lpr {efcode}, \lpr + {leftmarginkern}, \lpr {rightmarginkern} are promoted to core primitives. The + two commands \lpr {protrudechars} and \lpr {adjustspacing} replace their prefixed with \type {\pdf} originals. \stopitem @@ -245,36 +251,36 @@ The front- and backend are decoupled as much as possible. \stopitem \startitem - When \type {\adjustspacing} has value~2, hz optimization will be applied to + When \lpr {adjustspacing} has value~2, hz optimization will be applied to glyphs and kerns. When the value is~3, only glyphs will be treated. A value smaller than~2 disables this feature. \stopitem \startitem - The \type {\tagcode} primitive is promoted to core primitive. + The \lpr {tagcode} primitive is promoted to core primitive. \stopitem \startitem - The \type {\letterspacefont} feature is now part of the core but will not be + The \lpr {letterspacefont} feature is now part of the core but will not be changed (improved). We just provide it for legacy use. \stopitem \startitem - The \type {\pdfnoligatures} primitive is now \type {\ignoreligaturesinfont}. + The \orm {pdfnoligatures} primitive is now \lpr {ignoreligaturesinfont}. \stopitem \startitem - The \type {\pdfcopyfont} primitive is now \type {\copyfont}. + The \orm {pdfcopyfont} primitive is now \lpr {copyfont}. \stopitem \startitem - The \type {\pdffontexpand} primitive is now \type {\expandglyphsinfont}. + The \orm {pdffontexpand} primitive is now \lpr {expandglyphsinfont}. \stopitem \startitem - Because position tracking is also available in \DVI\ mode the \type - {\savepos}, \type {\lastxpos} and \type {\lastypos} commands now replace - their \type {pdf} prefixed originals. + Because position tracking is also available in \DVI\ mode the \lpr {savepos}, + \lpr {lastxpos} and \lpr {lastypos} commands now replace their \type {pdf} + prefixed originals. \stopitem \startitem @@ -284,27 +290,31 @@ The front- and backend are decoupled as much as possible. \stopitem \startitem - The initializers \type {\pdfoutput} has been replaced by \type {\outputmode} and - \type {\pdfdraftmode} is now \type {\draftmode}. + The initializers \orm {pdfoutput} has been replaced by \lpr {outputmode} and + \orm {pdfdraftmode} is now \lpr {draftmode}. \stopitem \startitem - The pixel multiplier dimension \type {\pdfpxdimen} lots its prefix and is now calles - \type {\pxdimen}. + The pixel multiplier dimension \orm {pdfpxdimen} lost its prefix and is now + called \lpr {pxdimen}. \stopitem \startitem - An extra \type {\pdfimageaddfilename} option has been added that can be used to block - writing the filename to the \PDF\ file. + An extra \orm {pdfimageaddfilename} option has been added that can be used to + block writing the filename to the \PDF\ file. \stopitem \startitem - The primitive \type {\pdftracingfonts} is now \type {\tracingfonts} as it + The primitive \orm {pdftracingfonts} is now \lpr {tracingfonts} as it doesn't relate to the backend. \stopitem \startitem - The experimental primitive \type {\pdfinsertht} is kept as \type {\insertht}. + The experimental primitive \orm {pdfinsertht} is kept as \lpr {insertht}. +\stopitem + +\startitem + There is some more control over what metadata goes into the \PDF\ file. \stopitem \startitem @@ -318,25 +328,28 @@ The front- and backend are decoupled as much as possible. One change involves the so called xforms and ximages. In \PDFTEX\ these are implemented as so called whatsits. But contrary to other whatsits they have dimensions that need to be taken into account when for instance calculating -optimal line breaks. In \LUATEX\ these are now promoted to normal nodes, which -simplifies code that needs those dimensions. +optimal line breaks. In \LUATEX\ these are now promoted to a special type of rule +nodes, which simplifies code that needs those dimensions. Another reason for promotion is that these are useful concepts. Backends can -provide the ability to use content that has been rendered in several places, -and images are also common. For that reason we also changed the names: +provide the ability to use content that has been rendered in several places, and +images are also common. As already mentioned in \in {section} +[sec:imagedandforms], we now have: \starttabulate[|l|l|] -\BC new name \BC old name \NC \NR -\NC \type {\saveboxresource} \NC \type {\pdfxform} \NC \NR -\NC \type {\saveimageresource} \NC \type {\pdfximage} \NC \NR -\NC \type {\useboxresource} \NC \type {\pdfrefxform} \NC \NR -\NC \type {\useimageresource} \NC \type {\pdfrefximage} \NC \NR -\NC \type {\lastsavedboxresourceindex} \NC \type {\pdflastxform} \NC \NR -\NC \type {\lastsavedimageresourceindex} \NC \type {\pdflastximage} \NC \NR -\NC \type {\lastsavedimageresourcepages} \NC \type {\pdflastximagepages} \NC \NR +\DB \LUATEX \BC \PDFTEX \NC \NR +\TB +\NC \lpr {saveboxresource} \NC \orm {pdfxform} \NC \NR +\NC \lpr {saveimageresource} \NC \orm {pdfximage} \NC \NR +\NC \lpr {useboxresource} \NC \orm {pdfrefxform} \NC \NR +\NC \lpr {useimageresource} \NC \orm {pdfrefximage} \NC \NR +\NC \lpr {lastsavedboxresourceindex} \NC \orm {pdflastxform} \NC \NR +\NC \lpr {lastsavedimageresourceindex} \NC \orm {pdflastximage} \NC \NR +\NC \lpr {lastsavedimageresourcepages} \NC \orm {pdflastximagepages} \NC \NR +\LL \stoptabulate -There are a few \type {\pdffeedback} features that relate to this but these are +There are a few \lpr {pdffeedback} features that relate to this but these are typical backend specific ones. The index that gets returned is to be considered as \quote {just a number} and although it still has the same meaning (object related) as before, you should not depend on that. @@ -344,7 +357,7 @@ related) as before, you should not depend on that. The protrusion detection mechanism is enhanced a bit to enable a bit more complex situations. When protrusion characters are identified some nodes are skipped: -\startitemize[packed] +\startitemize[packed,columns,two] \startitem zero glue \stopitem \startitem penalties \stopitem \startitem empty discretionaries \stopitem @@ -374,6 +387,8 @@ instance content moved into the margin: \startsubsection[title=Changes from \ALEPH\ RC4] +\topicindex {\ALEPH} + Because we wanted proper directional typesetting the \ALEPH\ mechanisms looked most attractive. These are rather close to the ones provided by \OMEGA, so what we say next applies to both these programs. @@ -381,46 +396,40 @@ we say next applies to both these programs. \startitemize \startitem - The extended 16-bit math primitives (\type {\omathcode} etc.) have been + The extended 16-bit math primitives (\orm {omathcode} etc.) have been removed. \stopitem \startitem The \OCP\ processing has been removed completely and as a consequence, the - following primitives have been removed: - - \start \raggedright - \type {\ocp}, \type {\externalocp}, \type {\ocplist}, \type {\pushocplist}, - \type {\popocplist}, \type {\clearocplists}, \type {\addbeforeocplist}, \type - {\addafterocplist}, \type {\removebeforeocplist}, \type {\removeafterocplist} - and \type {\ocptracelevel} - \par \stop + following primitives have been removed: \orm {ocp}, \orm {externalocp}, \orm + {ocplist}, \orm {pushocplist}, \orm {popocplist}, \orm {clearocplists}, \orm + {addbeforeocplist}, \orm {addafterocplist}, \orm {removebeforeocplist}, \orm + {removeafterocplist} and \orm {ocptracelevel}. \stopitem \startitem \LUATEX\ only understands 4~of the 16~direction specifiers of \ALEPH: \type - {TLT} (latin), \type {TRT} (arabic), \type {RTT} (cjk), \type {LTL} - (mongolian). All other direction specifiers generate an error. + {TLT} (latin), \type {TRT} (arabic), \type {RTT} (cjk), \type {LTL} (mongolian). + All other direction specifiers generate an error. In addition to a keyword + driven model we also provide an integer driven one. \stopitem \startitem The input translations from \ALEPH\ are not implemented, the related - primitives are not available: - - \start \raggedright - \type {\DefaultInputMode}, \type {\noDefaultInputMode}, \type {\noInputMode}, - \type {\InputMode}, \type {\DefaultOutputMode}, \type {\noDefaultOutputMode}, - \type {\noOutputMode}, \type {\OutputMode}, \type {\DefaultInputTranslation}, - \type {\noDefaultInputTranslation}, \type {\noInputTranslation}, \type - {\InputTranslation}, \type {\DefaultOutputTranslation}, \type - {\noDefaultOutputTranslation}, \type {\noOutputTranslation} and \type - {\OutputTranslation} - \par \stop + primitives are not available: \orm {DefaultInputMode}, \orm + {noDefaultInputMode}, \orm {noInputMode}, \orm {InputMode}, \orm + {DefaultOutputMode}, \orm {noDefaultOutputMode}, \orm {noOutputMode}, \orm + {OutputMode}, \orm {DefaultInputTranslation}, \orm + {noDefaultInputTranslation}, \orm {noInputTranslation}, \orm + {InputTranslation}, \orm {DefaultOutputTranslation}, \orm + {noDefaultOutputTranslation}, \orm {noOutputTranslation} and \orm + {OutputTranslation}. \stopitem \startitem - Several bugs have been fixed an confusing implementation details have been sorted - out. + Several bugs have been fixed and confusing implementation details have been + sorted out. \stopitem \startitem @@ -429,13 +438,13 @@ we say next applies to both these programs. \stopitem \startitem - The \type {^^} notation has been extended: after \type {^^^^} four hexadecimal - characters are expected and after \type {^^^^^^} six hexadecimal characters - have to be given. The original \TEX\ interpretation is still valid for the - \type {^^} case but the four and six variants do no backtracking, i.e.\ when - they are not followed by the right number of hexadecimal digits they issue an - error message. Because \type{^^^} is a normal \TEX\ case, we don't support the - odd number of \type {^^^^^} either. + The \type {^^} notation has been extended: after \type {^^^^} four + hexadecimal characters are expected and after \type {^^^^^^} six hexadecimal + characters have to be given. The original \TEX\ interpretation is still valid + for the \type {^^} case but the four and six variants do no backtracking, + i.e.\ when they are not followed by the right number of hexadecimal digits + they issue an error message. Because \type{^^^} is a normal \TEX\ case, we + don't support the odd number of \type {^^^^^} either. \stopitem \startitem @@ -449,9 +458,9 @@ we say next applies to both these programs. \stopitem \startitem - The page dimension related primitives \type {\pagewidth} and \type - {\pageheight} have been promoted to core primitives. The \type {\hoffset} and - \type {\voffset} primitives have been fixed. + The page dimension related primitives \lpr {pagewidth} and \lpr {pageheight} + have been promoted to core primitives. The \prm {hoffset} and \prm {voffset} + primitives have been fixed. \stopitem \startitem @@ -461,30 +470,35 @@ we say next applies to both these programs. \stopitem \startitem - The two dimension registers \type {\pagerightoffset} and \type - {\pagebottomoffset} are now core primitives. + The two dimension registers \lpr {pagerightoffset} and \lpr + {pagebottomoffset} are now core primitives. \stopitem \startitem - The direction related primitives \type {\pagedir}, \type {\bodydir}, \type - {\pardir}, \type {\textdir}, \type {\mathdir} and \type {\boxdir} are now - core primitives. + The direction related primitives \lpr {pagedir}, \lpr {bodydir}, \lpr + {pardir}, \lpr {textdir}, \lpr {mathdir} and \lpr {boxdir} are now core + primitives. \stopitem \startitem - The promotion of primitives to core primitives as well as the removed of all - others means that the initialization namespace \type {aleph} is gone. + The promotion of primitives to core primitives as well as removing of all + others means that the initialization namespace \type {aleph} that early + versions of \LUATEX\ provided is gone. \stopitem \stopitemize The above let's itself summarize as: we took the 32 bit aspects and much of the -directional mechanisms. +directional mechanisms and merged it into the \PDFTEX\ code base as starting +point for further development. Then we simplified directionality, fixed it and +opened it up. \stopsubsection \startsubsection[title=Changes from standard \WEBC] +\topicindex {\WEBC} + The compilation framework is \WEBC\ and we keep using that but without the \PASCAL\ to \CCODE\ step. This framework also provides some common features that deal with reading bytes from files and locating files in \TDS. This is what we do @@ -507,7 +521,7 @@ different: \stopitem \startitem - The \type {\openout} whatsits are not written to the log file. + The \prm {openout} whatsits are not written to the log file. \stopitem \startitem @@ -515,7 +529,8 @@ different: mode because \type {texmf.cnf} is not read: \type {shell-escape} is off (but that is not a problem because of \LUA's \type {os.execute}), and the paranoia checks on \type {openin} and \type {openout} do not happen. However, it is - easy for a \LUA\ script to do this itself by overloading \type {io.open}. + easy for a \LUA\ script to do this itself by overloading \type {io.open} and + alike. \stopitem \startitem @@ -528,17 +543,22 @@ different: \stopsection -\startsection[reference=backendprimitives,title=The backend primitives \type {\pdf*}] +\startsection[reference=backendprimitives,title=The backend primitives] + +\startsubsection[title={Less primitives}] + +\topicindex {backend} +\topicindex {\PDF+backend} In a previous section we mentioned that some \PDFTEX\ primitives were removed and others promoted to core \LUATEX\ primitives. That is only part of the story. In order to separate the backend specific primitives in de code these commands are now replaced by only a few. In traditional \TEX\ we only had the \DVI\ backend but now we have two: \DVI\ and \PDF. Additional functionality is implemented as -\quote {extensions} in \TEX speak. By separating more strickly we are able to -keep the core (fontend) clean and stable. If for some reason an extra backend -option is needed, it can be implemented without touching the core. The three -\PDF\ backend related primitives are +\quote {extensions} in \TEX\ speak. By separating more strickly we are able to +keep the core (frontend) clean and stable and isolate these extensions. If for +some reason an extra backend option is needed, it can be implemented without +touching the core. The three \PDF\ backend related primitives are: \starttyping \pdfextension command [specification] @@ -550,8 +570,12 @@ An extension triggers further parsing, depending on the command given. A variabl a (kind of) register and can be read and written, while a feedback is reporting something (as it comes from the backend it's normally a sequence of tokens). +\stopsubsection + +\startsubsection[title={\lpr{pdfextension}, \lpr {pdfvariable} and \lpr {pdffeedback}},reference=sec:pdfextensions] + In order for \LUATEX\ to be more than just \TEX\ you need to enable primitives. That -has already be the case right from the start. If you want the traditional \PDFTEX\ +has already been the case right from the start. If you want the traditional \PDFTEX\ primitives (for as far their functionality is still around) you now can do this: \starttyping @@ -604,6 +628,7 @@ The configuration related registers have become: \starttyping \edef\pdfcompresslevel {\pdfvariable compresslevel} \edef\pdfobjcompresslevel {\pdfvariable objcompresslevel} +\edef\pdfrecompress {\pdfvariable recompress} \edef\pdfdecimaldigits {\pdfvariable decimaldigits} \edef\pdfgamma {\pdfvariable gamma} \edef\pdfimageresolution {\pdfvariable imageresolution} @@ -618,6 +643,7 @@ The configuration related registers have become: \edef\pdfignoreunknownimages {\pdfvariable ignoreunknownimages} \edef\pdfgentounicode {\pdfvariable gentounicode} \edef\pdfomitcidset {\pdfvariable omitcidset} +\edef\pdfomitcharset {\pdfvariable omitcharset} \edef\pdfpagebox {\pdfvariable pagebox} \edef\pdfminorversion {\pdfvariable minorversion} \edef\pdfuniqueresname {\pdfvariable uniqueresname} @@ -657,100 +683,16 @@ macro:->[internal backend integer] macro:->[internal backend tokenlist] \stoptyping -The \type {\edef} can also be an \type {\def} but it's a bit more efficient -to expand the lookup related register beforehand. After that you can adapt -the defaults; these are: - -\starttyping -\pdfcompresslevel 9 -\pdfobjcompresslevel 1 % used: (0,9) -\pdfdecimaldigits 4 % used: (3,6) -\pdfgamma 1000 -\pdfimageresolution 71 -\pdfimageapplygamma 0 -\pdfimagegamma 2200 -\pdfimagehicolor 1 -\pdfimageaddfilename 1 -\pdfpkresolution 72 -\pdfpkfixeddpi 0 -\pdfinclusioncopyfonts 0 -\pdfinclusionerrorlevel 0 -\pdfignoreunknownimages 0 -\pdfgentounicode 0 -\pdfomitcidset 0 -\pdfpagebox 0 -\pdfminorversion 4 -\pdfuniqueresname 0 - -\pdfhorigin 1in -\pdfvorigin 1in -\pdflinkmargin 0pt -\pdfdestmargin 0pt -\pdfthreadmargin 0pt -\pdfxformmargin 0pt -\stoptyping - -If you also want some backward compatibility, you can add: - -\starttyping -\let\pdfpagewidth \pagewidth -\let\pdfpageheight \pageheight - -\let\pdfadjustspacing \adjustspacing -\let\pdfprotrudechars \protrudechars -\let\pdfnoligatures \ignoreligaturesinfont -\let\pdffontexpand \expandglyphsinfont -\let\pdfcopyfont \copyfont - -\let\pdfxform \saveboxresource -\let\pdflastxform \lastsavedboxresourceindex -\let\pdfrefxform \useboxresource - -\let\pdfximage \saveimageresource -\let\pdflastximage \lastsavedimageresourceindex -\let\pdflastximagepages\lastsavedimageresourcepages -\let\pdfrefximage \useimageresource - -\let\pdfsavepos \savepos -\let\pdflastxpos \lastxpos -\let\pdflastypos \lastypos - -\let\pdfoutput \outputmode -\let\pdfdraftmode \draftmode - -\let\pdfpxdimen \pxdimen - -\let\pdfinsertht \insertht - -\let\pdfnormaldeviate \normaldeviate -\let\pdfuniformdeviate \uniformdeviate -\let\pdfsetrandomseed \setrandomseed -\let\pdfrandomseed \randomseed - -\let\pdfprimitive \primitive -\let\ifpdfprimitive \ifprimitive - -\let\ifpdfabsnum \ifabsnum -\let\ifpdfabsdim \ifabsdim -\stoptyping - -And even: - -\starttyping -\newdimen\pdfeachlineheight -\newdimen\pdfeachlinedepth -\newdimen\pdflastlinedepth -\newdimen\pdffirstlineheight -\newdimen\pdfignoreddimen -\stoptyping +The \prm {edef} can also be a \prm {def} but it's a bit more efficient to expand +the lookup related register beforehand. The backend is derived from \PDFTEX\ so the same syntax applies. However, the \type {outline} command accepts a \type {objnum} followed by a number. No checking takes place so when this is used it had better be a valid (flushed) object. -In order to be (more or less) compatible with \PDFTEX\ we also support the -option to suppress some info: +In order to be (more or less) compatible with \PDFTEX\ we also support the option +to suppress some info but we do so via a bitset: \starttyping \pdfvariable suppressoptionalinfo \numexpr @@ -770,7 +712,7 @@ option to suppress some info: In addition you can overload the trailer id, but we don't do any checking on validity, so you have to pass a valid array. The following is like the ones -normally generated by the engine: +normally generated by the engine. You even need to include the brackets here! \starttyping \pdfvariable trailerid {[ @@ -779,8 +721,6 @@ normally generated by the engine: ]} \stoptyping -So, you even need to include the brackets! - Although we started from a merge of \PDFTEX\ and \ALEPH, by now the code base as well as functionality has diverted from those parents. Here we show the options that can be passed to the extensions. @@ -928,10 +868,113 @@ that can be passed to the extensions. {tokens} \stoptexsyntax +\stopsubsection + +\startsubsection[title={Defaults}] + +The engine sets the following defaults. + +\starttyping +\pdfcompresslevel 9 +\pdfobjcompresslevel 1 % used: (0,9) +\pdfrecompress 0 % mostly for debugging +\pdfdecimaldigits 4 % used: (3,6) +\pdfgamma 1000 +\pdfimageresolution 71 +\pdfimageapplygamma 0 +\pdfimagegamma 2200 +\pdfimagehicolor 1 +\pdfimageaddfilename 1 +\pdfpkresolution 72 +\pdfpkfixeddpi 0 +\pdfinclusioncopyfonts 0 +\pdfinclusionerrorlevel 0 +\pdfignoreunknownimages 0 +\pdfgentounicode 0 +\pdfomitcidset 0 +\pdfomitcharset 0 +\pdfpagebox 0 +\pdfminorversion 4 +\pdfuniqueresname 0 + +\pdfhorigin 1in +\pdfvorigin 1in +\pdflinkmargin 0pt +\pdfdestmargin 0pt +\pdfthreadmargin 0pt +\pdfxformmargin 0pt +\stoptyping + +\stopsubsection + +\startsubsection[title={Backward compatibility}] + +If you also want some backward compatibility, you can add: + +\starttyping +\let\pdfpagewidth \pagewidth +\let\pdfpageheight \pageheight + +\let\pdfadjustspacing \adjustspacing +\let\pdfprotrudechars \protrudechars +\let\pdfnoligatures \ignoreligaturesinfont +\let\pdffontexpand \expandglyphsinfont +\let\pdfcopyfont \copyfont + +\let\pdfxform \saveboxresource +\let\pdflastxform \lastsavedboxresourceindex +\let\pdfrefxform \useboxresource + +\let\pdfximage \saveimageresource +\let\pdflastximage \lastsavedimageresourceindex +\let\pdflastximagepages\lastsavedimageresourcepages +\let\pdfrefximage \useimageresource + +\let\pdfsavepos \savepos +\let\pdflastxpos \lastxpos +\let\pdflastypos \lastypos + +\let\pdfoutput \outputmode +\let\pdfdraftmode \draftmode + +\let\pdfpxdimen \pxdimen + +\let\pdfinsertht \insertht + +\let\pdfnormaldeviate \normaldeviate +\let\pdfuniformdeviate \uniformdeviate +\let\pdfsetrandomseed \setrandomseed +\let\pdfrandomseed \randomseed + +\let\pdfprimitive \primitive +\let\ifpdfprimitive \ifprimitive + +\let\ifpdfabsnum \ifabsnum +\let\ifpdfabsdim \ifabsdim +\stoptyping + +And even: + +\starttyping +\newdimen\pdfeachlineheight +\newdimen\pdfeachlinedepth +\newdimen\pdflastlinedepth +\newdimen\pdffirstlineheight +\newdimen\pdfignoreddimen +\stoptyping + +\stopsubsection + \stopsection \startsection[title=Directions] +\topicindex {\OMEGA} +\topicindex {\ALEPH} +\topicindex {directions} + +\startsubsection[title={Four directions}] + The directional model in \LUATEX\ is inherited from \OMEGA|/|\ALEPH\ but we tried to improve it a bit. At some point we played with recovery of modes but that was disabled later on when we found that it interfered with nested directions. That @@ -939,14 +982,23 @@ itself had as side effect that the node list was no longer balanced with respect to directional nodes which in turn can give side effects when a series of dir changes happens without grouping. -The current (0.97 onward) approach is that we again make the list balanced but -try to avoid some side effects. What happens is quite intuitive if we forget -about spaces (turned into glue) but even there what happens makes sense if you -look at it in detail. However that logic makes in|-|group switching kind of -useless when no proper nested grouping is used: switching from right to left -several times nested, results in spacing ending up after each other due to nested -mirroring. Of course a sane macro package will manage this for the user but here -we are discussing the low level dir injection. +When extending the \PDF\ backend to support directions some inconsistencies were +found and as a result we decided to support only the four models that make sense +\type {TLT} (latin), \type {TRT} (arabic), \type {RTT} (cjk) and \type {LTL} +(mongolian). + +\stopsubsection + +\startsubsection[title={How it works}] + +The approach is that we again make the list balanced but try to avoid some side +effects. What happens is quite intuitive if we forget about spaces (turned into +glue) but even there what happens makes sense if you look at it in detail. +However that logic makes in|-|group switching kind of useless when no proper +nested grouping is used: switching from right to left several times nested, +results in spacing ending up after each other due to nested mirroring. Of course +a sane macro package will manage this for the user but here we are discussing the +low level dir injection. This is what happens: @@ -1031,7 +1083,7 @@ It gets typeset as: We could define the two helpers to look back, pick up a skip, remove it and inject it after the dir node. But that way we loose the subtype information that for some applications can be handy to be kept as|-|is. This is why we now have a -variant of \type {\textdir} which injects the balanced node before the skip. +variant of \lpr {textdir} which injects the balanced node before the skip. Instead of the previous definition we can use: \startbuffer[def] @@ -1060,64 +1112,64 @@ comes out as a properly spaced: Anything more complex that this, like combination of skips and penalties, or kerns, should be handled in the input or macro package because there is no way we -can predict the expected behaviour. In fact, the \type {\linedir} is just a +can predict the expected behaviour. In fact, the \lpr {linedir} is just a convenience extra which could also have been implemented using node list parsing. +\stopsubsection + +\startsubsection[title={Controlling glue with \lpr {breakafterdirmode}}] + Glue after a dir node is ignored in the linebreak decision but you can bypass that -by setting \type {\breakafterdirmode} to~\type {1}. The following table shows the +by setting \lpr {breakafterdirmode} to~\type {1}. The following table shows the difference. Watch your spaces. \def\ShowSome#1{% - \BC - \type{#1} - \NC - \breakafterdirmode = 0 - \hsize 0pt - #1 + \BC \type{#1} + \NC \breakafterdirmode\zerocount\hsize\zeropoint#1 \NC + \NC \breakafterdirmode\plusone\hsize\zeropoint#1 \NC - \breakafterdirmode = 1 - \hsize 0pt - #1 - \NC - \NC \NR \HL + \NC \NR } -\starttabulate[|l|Tp(0pt)|w(5em)|Tp(0pt)|p|] - \HL - \BC \type{\breakafterdirmode} +\starttabulate[|l|Tp(1pt)|w(5em)|Tp(1pt)|w(5em)|] + \DB \BC \type{0} \NC \BC \type{1} \NC \NC \NR - \HL + \TB \ShowSome{pre {\textdir TLT xxx} post} \ShowSome{pre {\textdir TLT xxx }post} \ShowSome{pre{ \textdir TLT xxx} post} \ShowSome{pre{ \textdir TLT xxx }post} \ShowSome{pre { \textdir TLT xxx } post} \ShowSome{pre {\textdir TLT\relax\space xxx} post} + \LL \stoptabulate +\stopsubsection + +\startsubsection[title={Controling parshapes with \lpr {shapemode}}] Another adaptation to the \ALEPH\ directional model is control over shapes driven -by \type {\hangindent} and \type {\parshape}. This is controlled by a new parameter -\type {\shapemode}: +by \prm {hangindent} and \prm {parshape}. This is controlled by a new parameter +\lpr {shapemode}: -\starttabulate[|c|c|c|] -\BC \BC \type {\hangindent} \BC \type {\parshape} \NC \NR +\starttabulate[|c|l|l|] +\DB value \BC \prm {hangindent} \BC \prm {parshape} \NC \NR +\TB \BC \type{0} \NC normal \NC normal \NC \NR \BC \type{1} \NC mirrored \NC normal \NC \NR \BC \type{2} \NC normal \NC mirrored \NC \NR \BC \type{3} \NC mirrored \NC mirrored \NC \NR +\LL \stoptabulate -The value is reset to zero (like \type {\hangindent} and \type {\parshape}) +The value is reset to zero (like \prm {hangindent} and \prm {parshape}) after the paragraph is done with. You can use negative values to prevent -this. - -In \in {figure} [fig:shapemode] a few examples are given. +this. In \in {figure} [fig:shapemode] a few examples are given. \startplacefigure[reference=fig:shapemode,title={The effect of \type {shapemode}.}] \startcombination[2*3] @@ -1162,12 +1214,46 @@ In \in {figure} [fig:shapemode] a few examples are given. \stopcombination \stopplacefigure +\stopsubsection + +\startsubsection[title={Symbols or numbers}] + +Internally the implementation is different from \ALEPH. First of all we use no +whatsits but dedicated nodes, but also we have only 4 directions that are mapped +onto 4 numbers. A text direction node can mark the start or end of a sequence of +nodes, and therefore has two states. At the \TEX\ end we don't see these states +because \TEX\ itself will add proper end state nodes if needed. + +The symbolic names \type {TLT}, \type {TRT}, etc.\ originate in \OMEGA. In +\LUATEX\ we also have a number based model which sometimes makes more sense. + +\starttabulate[|c|l|l|] +\DB value \BC equivalent \NC \NR +\TB +\BC \type {0} \NC TLT \NC \NR +\BC \type {1} \NC TRT \NC \NR +\BC \type {2} \NC LTL \NC \NR +\BC \type {3} \NC RTT \NC \NR +\LL +\stoptabulate + +We support the \OMEGA\ primitives \orm {textdir}, \orm {pardir}, \orm {pagedir}, +\orm {pardir} and \orm {mathdir}. These accept three character keywords. The +primitives that set the direction by number are: \lpr {textdirection}, \lpr +{pardirection}, \lpr {pagedirection} and \lpr {bodydirection} and \lpr +{mathdirection}. When specifying a direction for a box you can use \type {bdir} +instead of \type {dir}. + +\stopsubsection + \stopsection \startsection[title=Implementation notes] \startsubsection[title=Memory allocation] +\topicindex {memory} + The single internal memory heap that traditional \TEX\ used for tokens and nodes is split into two separate arrays. Each of these will grow dynamically when needed. @@ -1189,14 +1275,9 @@ structures, some of the macros have been duplicated. For instance, there are now {token_info}. All access to the variable memory array is now hidden behind a macro called \type {vmem}. We mention this because using the \TEX book as reference is still quite valid but not for memory related details. Another -significate detail is that we have double linked node lists and that some nodes +significant detail is that we have double linked node lists and that most nodes carry more data. -The implementation of the growth of two arrays (via reallocation) introduces a -potential pitfall: the memory arrays should never be used as the left hand side -of a statement that can modify the array in question. Details like this are -of no concern to users. - The input line buffer and pool size are now also reallocated when needed, and the \type {texmf.cnf} settings \type {buf_size} and \type {pool_size} are silently ignored. @@ -1205,25 +1286,31 @@ ignored. \startsubsection[title=Sparse arrays] -The \type {\mathcode}, \type {\delcode}, \type {\catcode}, \type {\sfcode}, \type -{\lccode} and \type {\uccode} (and the new \type {\hjcode}) tables are now sparse -arrays that are implemented in~\CCODE. They are no longer part of the \TEX\ -\quote {equivalence table} and because each had 1.1 million entries with a few -memory words each, this makes a major difference in memory usage. +The \prm {mathcode}, \prm {delcode}, \prm {catcode}, \prm {sfcode}, \prm {lccode} +and \prm {uccode} (and the new \lpr {hjcode}) tables are now sparse arrays that +are implemented in~\CCODE. They are no longer part of the \TEX\ \quote +{equivalence table} and because each had 1.1 million entries with a few memory +words each, this makes a major difference in memory usage. Performance is not +really hurt by this. -The \type {\catcode}, \type {\sfcode}, \type {\lccode}, \type {\uccode} and \type -{\hjcode} assignments do not yet show up when using the \ETEX\ tracing routines -\type {\tracingassigns} and \type {\tracingrestores}. +The \prm {catcode}, \prm {sfcode}, \prm {lccode}, \prm {uccode} and \lpr {hjcode} +assignments don't show up when using the \ETEX\ tracing routines \prm +{tracingassigns} and \prm {tracingrestores} but we don't see that as a real +limitation. -A side|-|effect of the current implementation is that \type {\global} is now more -expensive in terms of processing than non|-|global assignments. +A side|-|effect of the current implementation is that \prm {global} is now more +expensive in terms of processing than non|-|global assignments but not many users +will notice that. The glyph ids within a font are also managed by means of a sparse array as glyph -ids can go up to index $2^{21}-1$. +ids can go up to index $2^{21}-1$ but these are never accessed directly so again +users will not notice this. \stopsubsection -\startsubsection[title=Simple single-character csnames] +\startsubsection[title=Simple single|-|character csnames] + +\topicindex {csnames} Single|-|character commands are no longer treated specially in the internals, they are stored in the hash just like the multiletter csnames. @@ -1236,16 +1323,22 @@ control sequences that uses a prefix that is otherwise impossible to obtain. \stopsubsection -\startsubsection[title=Compressed format] +\startsubsection[title=The compressed format file] + +\topicindex {format} The format is passed through \type {zlib}, allowing it to shrink to roughly half of the size it would have had in uncompressed form. This takes a bit more \CPU\ -cycles but much less disk \IO, so it should still be faster. +cycles but much less disk \IO, so it should still be faster. We use a level~3 +compression which we found to be the optimal trade|-|off between filesize and +decompression speed. \stopsubsection \startsubsection[title=Binary file reading] +\topicindex {files+binary} + All of the internal code is changed in such a way that if one of the \type {read_xxx_file} callbacks is not set, then the file is read by a \CCODE\ function using basically the same convention as the callback: a single read into a buffer @@ -1257,6 +1350,9 @@ previous code (that mostly used \type {getc} calls), it can be quite a bit faste \startsubsection[title=Tabs and spaces] +\topicindex {space} +\topicindex {newline} + We conform to the way other \TEX\ engines handle trailing tabs and spaces. For decades trailing tabs and spaces (before a newline) were removed from the input but this behaviour was changed in September 2017 to only handle spaces. We are @@ -1273,7 +1369,7 @@ to be kept. We are aware of the fact that this contradicts some of our other choices but consistency with other engines and the fact that in \KPSE\ mode a common file \IO\ layer is used can have a side effect of breaking compatibility. We still stick to our view that at the log level we can (and might be) more -incompatible. +incompatible. We already expose some more details. \stopsubsection diff --git a/doc/context/sources/general/manuals/luatex/luatex-nodes.tex b/doc/context/sources/general/manuals/luatex/luatex-nodes.tex index 517d9b6c8..34a2aebe8 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-nodes.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-nodes.tex @@ -1,16 +1,21 @@ % language=uk \environment luatex-style -\environment luatex-logos \startcomponent luatex-nodes \startchapter[reference=nodes,title={Nodes}] -\section{\LUA\ node representation} +\startsection[title={\LUA\ node representation}][library=node] -\TEX's nodes are represented in \LUA\ as userdata object with a variable set of -fields. In the following syntax tables, such the type of such a userdata object +\topicindex {nodes} + +\libindex {fields} +\libindex {subtypes} +\libindex {values} + +\TEX's nodes are represented in \LUA\ as userdata objects with a variable set of +fields. In the following syntax tables, such as the type of such a userdata object is represented as \syntax {<node>}. The current return value of \type {node.types()} is: @@ -24,76 +29,43 @@ The current return value of \type {node.types()} is: \stopluacode . % period -The \type {\lastnodetype} primitive is \ETEX\ compliant. The valid range is still +The \prm {lastnodetype} primitive is \ETEX\ compliant. The valid range is still $[-1,15]$ and glyph nodes (formerly known as char nodes) have number~0 while ligature nodes are mapped to~7. That way macro packages can use the same symbolic names as in traditional \ETEX. Keep in mind that these \ETEX\ node numbers are different from the real internal ones and that there are more \ETEX\ node types than~15. -You can ask for a list of fields with the \type {node.fields} (which takes an id) -and for valid subtypes with \type {node.subtypes} (which takes a string because -eventually we might support more used enumerations). - -The \type {node.values} function reports some used values. Valid arguments are -\type {dir}, \type {direction}, \type {glue}, \type {pdf_literal}, \type -{pdf_action}, \type {pdf_window} and \type {color_stack}. Keep in mind that the -setters normally expect a number, but this helper gives you a list of what -numbers matter. For practical reason the \type {pagestate} values are also -reported with this helper. - -\subsection{Attributes} - -The newly introduced attribute registers are non|-|trivial, because the value -that is attached to a node is essentially a sparse array of key|-|value pairs. It -is generally easiest to deal with attribute lists and attributes by using the -dedicated functions in the \type {node} library, but for completeness, here is -the low|-|level interface. - -\subsubsection{attribute_list nodes} - -An \type {attribute_list} item is used as a head pointer for a list of attribute -items. It has only one user-visible field: - -\starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR -\NC \type{next} \NC node \NC pointer to the first attribute \NC \NR -\stoptabulate - -\subsubsection{attribute nodes} - -A normal node's attribute field will point to an item of type \type -{attribute_list}, and the \type {next} field in that item will point to the first -defined \quote {attribute} item, whose \type {next} will point to the second -\quote {attribute} item, etc. +You can ask for a list of fields with \type {node.fields} and for valid subtypes +with \type {node.subtypes}. The \type {node.values} function reports some used +values. Valid arguments are \nod {dir}, \type {direction}, \nod {glue}, \whs +{pdf_literal}, \whs {pdf_action}, \whs {pdf_window} and \whs {color_stack}. Keep +in mind that the setters normally expect a number, but this helper gives you a +list of what numbers matter. For practical reason the \type {pagestate} values +are also reported with this helper. -\starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR -\NC \type{next} \NC node \NC pointer to the next attribute \NC \NR -\NC \type{number} \NC number \NC the attribute type id \NC \NR -\NC \type{value} \NC number \NC the attribute value \NC \NR -\stoptabulate +\stopsection -As mentioned it's better to use the official helpers rather than edit these -fields directly. For instance the \type {prev} field is used for other purposes -and there is no double linked list. +\startsection[title={Main text nodes}] -\subsection{Main text nodes} +\topicindex {nodes+text} These are the nodes that comprise actual typesetting commands. A few fields are present in all nodes regardless of their type, these are: \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{next} \NC node \NC the next node in a list, or nil \NC \NR \NC \type{id} \NC number \NC the node's type (\type {id}) number \NC \NR \NC \type{subtype} \NC number \NC the node \type {subtype} identifier \NC \NR +\LL \stoptabulate -The \type {subtype} is sometimes just a stub entry. Not all nodes actually use -the \type {subtype}, but this way you can be sure that all nodes accept it as a -valid field name, and that is often handy in node list traversal. In the -following tables \type {next} and \type {id} are not explicitly mentioned. +The \type {subtype} is sometimes just a dummy entry because not all nodes +actually use the \type {subtype}, but this way you can be sure that all nodes +accept it as a valid field name, and that is often handy in node list traversal. +In the following tables \type {next} and \type {id} are not explicitly mentioned. Besides these three fields, almost all nodes also have an \type {attr} field, and there is a also a field called \type {prev}. That last field is always present, @@ -101,25 +73,33 @@ but only initialized on explicit request: when the function \type {node.slide()} is called, it will set up the \type {prev} fields to be a backwards pointer in the argument node list. By now most of \TEX's node processing makes sure that the \type {prev} nodes are valid but there can be exceptions, especially when the -internal magic uses a leading \type {temp} nodes to temporarily store a state. +internal magic uses a leading \nod {temp} nodes to temporarily store a state. -\subsubsection{hlist nodes} +\subsection{\nod {hlist} nodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{list} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{width} \NC number \NC the width of the box \NC \NR \NC \type{height} \NC number \NC the height of the box \NC \NR \NC \type{depth} \NC number \NC the depth of the box \NC \NR -\NC \type{shift} \NC number \NC a displacement perpendicular to the character progression direction \NC \NR -\NC \type{glue_order} \NC number \NC a number in the range $[0,4]$, indicating the glue order \NC \NR +\NC \type{shift} \NC number \NC a displacement perpendicular to the character + progression direction \NC \NR +\NC \type{glue_order} \NC number \NC a number in the range $[0,4]$, indicating the + glue order \NC \NR \NC \type{glue_set} \NC number \NC the calculated glue ratio \NC \NR -\NC \type{glue_sign} \NC number \NC 0 = \type {normal}, 1 = \type {stretching}, 2 = \type {shrinking} \NC \NR +\NC \type{glue_sign} \NC number \NC 0 = \type {normal}, 1 = \type {stretching}, 2 = + \type {shrinking} \NC \NR \NC \type{head/list} \NC node \NC the first node of the body of this list \NC \NR -\NC \type{dir} \NC string \NC the direction of this box, see~\in[dirnodes] \NC \NR +\NC \type{dir} \NC string \NC the direction of this box, see~\in [dirnodes] \NC \NR +\LL \stoptabulate +\topicindex {nodes+lists} +\topicindex {lists} + A warning: never assign a node list to the \type {head} field unless you are sure its internal link structure is correct, otherwise an error may result. @@ -127,39 +107,72 @@ Note: the field name \type {head} and \type {list} are both valid. Sometimes it makes more sense to refer to a list by \type {head}, sometimes \type {list} makes more sense. -\subsubsection{vlist nodes} +\subsection{\nod {vlist} nodes} + +\topicindex {nodes+lists} +\topicindex {lists} -This node is similar to \type {hlist}, except that \quote {shift} is a displacement +This node is similar to \nod {hlist}, except that \quote {shift} is a displacement perpendicular to the line progression direction, and \quote {subtype} only has the values 0, 4, and~5. -\subsubsection{rule nodes} +\subsection{\nod {rule} nodes} -Contrary to traditional \TEX, \LUATEX\ has more subtypes because we also use -rules to store reuseable objects and images. User nodes are invisible and can be -intercepted by a callback. +\topicindex {nodes+rules} +\topicindex {rules} + +Contrary to traditional \TEX, \LUATEX\ has more \prm {rule} subtypes because we +also use rules to store reuseable objects and images. User nodes are invisible +and can be intercepted by a callback. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR -\NC \type{subtype} \NC number \NC \showsubtypes{rule} \NC \NR -\NC \type{attr} \NC node \NC list of attributes \NC \NR -\NC \type{width} \NC number \NC the width of the rule where the special value $-1073741824$ is used for \quote {running} glue dimensions \NC \NR -\NC \type{height} \NC number \NC the height of the rule (can be negative) \NC \NR -\NC \type{depth} \NC number \NC the depth of the rule (can be negative) \NC \NR -\NC \type{dir} \NC string \NC the direction of this rule, see~\in[dirnodes] \NC \NR -\NC \type{index} \NC number \NC an optional index that can be referred to \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes {rule} \NC \NR +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{width} \NC number \NC the width of the rule where the special value + $-1073741824$ is used for \quote {running} glue dimensions \NC \NR +\NC \type{height} \NC number \NC the height of the rule (can be negative) \NC \NR +\NC \type{depth} \NC number \NC the depth of the rule (can be negative) \NC \NR +\NC \type{left} \NC number \NC shift at the left end (also subtracted from width) \NC \NR +\NC \type{right} \NC number \NC (subtracted from width) \NC \NR +\NC \type{dir} \NC string \NC the direction of this rule, see~\in[dirnodes] \NC \NR +\NC \type{index} \NC number \NC an optional index that can be referred to \NC \NR +\NC \type{transform} \NC number \NC an private variable (also used to specify outline width) \NC \NR +\LL \stoptabulate -\subsubsection{ins nodes} +The \type {left} and type {right} keys are somewhat special (and experimental). +When rules are auto adapting to the surrounding box width you can enforce a shift +to the right by setting \type {left}. The value is also subtracted from the width +which can be a value set by the engine itself and is not entirely under user +control. The \type {right} is also subtracted from the width. It all happens in +the backend so these are not affecting the calculations in the frontend (actually +the auto settings also happen in the backend). For a vertical rule \type {left} +affects the height and \type {right} affects the depth. There is no matching +interface at the \TEX\ end (although we can have more keywords for rules it would +complicate matters and introduce a speed penalty.) However, you can just +construct a rule node with \LUA\ and write it to the \TEX\ input. The \type +{outline} subtype is just a convenient variant and the \type {transform} field +specifies the width of the outline. + +\subsection{\nod {ins} nodes} + +\topicindex {nodes+insertions} +\topicindex {insertions} + +This node relates to the \prm {insert} primitive. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC the insertion class \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{cost} \NC number \NC the penalty associated with this insert \NC \NR \NC \type{height} \NC number \NC height of the insert \NC \NR \NC \type{depth} \NC number \NC depth of the insert \NC \NR \NC \type{head/list} \NC node \NC the first node of the body of this insert \NC \NR +\LL \stoptabulate There is a set of extra fields that concern the associated glue: \type {width}, @@ -167,53 +180,76 @@ There is a set of extra fields that concern the associated glue: \type {width}, These are all numbers. A warning: never assign a node list to the \type {head} field unless you are sure -its internal link structure is correct, otherwise an error may be result. You can use -\type {list} instead (often in functions you want to use local variable swith similar +its internal link structure is correct, otherwise an error may result. You can use +\type {list} instead (often in functions you want to use local variable with similar names and both names are equally sensible). -\subsubsection{mark nodes} +\subsection{\nod {mark} nodes} + +\topicindex {nodes+marks} +\topicindex {marks} + +This one relates to the \prm {mark} primitive. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC unused \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{class} \NC number \NC the mark class \NC \NR \NC \type{mark} \NC table \NC a table representing a token list \NC \NR +\LL \stoptabulate -\subsubsection{adjust nodes} +\subsection{\nod {adjust} nodes} + +\topicindex {nodes+adjust} +\topicindex {adjust} + +This node comes from \prm {vadjust} primitive. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{adjust} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{head/list} \NC node \NC adjusted material \NC \NR +\LL \stoptabulate A warning: never assign a node list to the \type {head} field unless you are sure -its internal link structure is correct, otherwise an error may be result. +its internal link structure is correct, otherwise an error may be the result. + +\subsection{\nod {disc} nodes} + +\topicindex {nodes+discretionaries} +\topicindex {discretionaries} -\subsubsection{disc nodes} +The \prm {discretionary} and \prm {-}, the \type {-} character but also the +hyphenation mechanism produces these nodes. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{disc} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{pre} \NC node \NC pointer to the pre|-|break text \NC \NR \NC \type{post} \NC node \NC pointer to the post|-|break text \NC \NR \NC \type{replace} \NC node \NC pointer to the no|-|break text \NC \NR -\NC \type{penalty} \NC number \NC the penalty associated with the break, normally \type {\hyphenpenalty} or \type {\exhyphenpenalty} \NC \NR +\NC \type{penalty} \NC number \NC the penalty associated with the break, normally + \prm {hyphenpenalty} or \prm {exhyphenpenalty} \NC \NR +\LL \stoptabulate The subtype numbers~4 and~5 belong to the \quote {of-f-ice} explanation given -elsewhere. +elsewhere. These disc nodes are kind of special as at some point they also keep +information about breakpoints and nested ligatures. -These disc nodes are kind of special as at some point they also keep information -about breakpoints and nested ligatures. The \type {pre}, \type {post} and \type -{replace} fields at the \LUA\ end are in fact indirectly accessed and have a -\type {prev} pointer that is not \type {nil}. This means that when you mess -around with the head of these (three) lists, you also need to reassign them -because that will restore the proper \type {prev} pointer, so: +The \type {pre}, \type {post} and \type {replace} fields at the \LUA\ end are in +fact indirectly accessed and have a \type {prev} pointer that is not \type {nil}. +This means that when you mess around with the head of these (three) lists, you +also need to reassign them because that will restore the proper \type {prev} +pointer, so: \starttyping pre = d.pre @@ -225,48 +261,64 @@ Otherwise you can end up with an invalid internal perception of reality and \LUATEX\ might even decide to crash on you. It also means that running forward over for instance \type {pre} is ok but backward you need to stop at \type {pre}. And you definitely must not mess with the node that \type {prev} points to, if -only because it is not really an node but part of the disc data structure (so +only because it is not really a node but part of the disc data structure (so freeing it again might crash \LUATEX). -\subsubsection{math nodes} +\subsection{\nod {math} nodes} + +\topicindex {nodes+math} +\topicindex {math+nodes} + +Math nodes represent the boundaries of a math formula, normally wrapped into +\type {$} signs. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{math} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR -\NC \type{surround} \NC number \NC width of the \type {\mathsurround} kern \NC \NR +\NC \type{surround} \NC number \NC width of the \prm {mathsurround} kern \NC \NR +\LL \stoptabulate There is a set of extra fields that concern the associated glue: \type {width}, \type {stretch}, \type {stretch_order}, \type {shrink} and \type {shrink_order}. These are all numbers. -\subsubsection{glue nodes} +\subsection{\nod {glue} nodes} + +\topicindex {nodes+glue} +\topicindex {glue} Skips are about the only type of data objects in traditional \TEX\ that are not a -simple value. The structure that represents the glue components of a skip is -called a \type {glue_spec}, and it has the following accessible fields: +simple value. They are inserted when \TEX\ sees a space in the text flow but also +by \prm {hskip} and \prm {vskip}. The structure that represents the glue +components of a skip is called a \nod {glue_spec}, and it has the following +accessible fields: \starttabulate[|l|l|p|] -\BC key \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{width} \NC number \NC the horizontal or vertical displacement \NC \NR \NC \type{stretch} \NC number \NC extra (positive) displacement or stretch amount \NC \NR \NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR \NC \type{shrink} \NC number \NC extra (negative) displacement or shrink amount\NC \NR \NC \type{shrink_order} \NC number \NC factor applied to shrink amount \NC \NR +\LL \stoptabulate The effective width of some glue subtypes depends on the stretch or shrink needed to make the encapsulating box fit its dimensions. For instance, in a paragraph -lines normally have glue representing spaces and these stretch of shrink to make +lines normally have glue representing spaces and these stretch or shrink to make the content fit in the available space. The \type {effective_glue} function that takes a glue node and a parent (hlist or vlist) returns the effective width of -that glue item. +that glue item. When you pass \type {true} as third argument the value will be +rounded. -A gluespec node is a special kind of node that is used for storing a set of glue -values in registers. Originally they were also used to store properties of glue -nodes (using a system of reference counts) but we now keep these properties in -the glue nodes themselves, which gives a cleaner interface to \LUA. +A \nod {glue_spec} node is a special kind of node that is used for storing a set +of glue values in registers. Originally they were also used to store properties +of glue nodes (using a system of reference counts) but we now keep these +properties in the glue nodes themselves, which gives a cleaner interface to \LUA. The indirect spec approach was in fact an optimization in the original \TEX\ code. First of all it can save quite some memory because all these spaces that @@ -281,10 +333,12 @@ not that high (and nowadays memory is less an issue, also given that a glue node is only a few memory words larger than a spec). \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{glue} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{leader} \NC node \NC pointer to a box or rule for leaders \NC \NR +\LL \stoptabulate In addition there are the \type {width}, \type {stretch} \type {stretch_order}, @@ -295,64 +349,90 @@ so we decided to stick to that naming. A regular word space also results in a \type {spaceskip} subtype (this used to be a \type {userskip} with subtype zero). -\subsubsection{kern nodes} +\subsection{\nod {kern} nodes} + +\topicindex {nodes+kerns} +\topicindex {kerns} + +The \prm {kern} command creates such nodes but for instance the font and math +machinery can also add them. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{kern} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{kern} \NC number \NC fixed horizontal or vertical advance \NC \NR +\LL \stoptabulate -\subsubsection{penalty nodes} +\subsection{\nod {penalty} nodes} + +\topicindex {nodes+penalty} +\topicindex {penalty} + +The \prm {penalty} command is one that generates these nodes. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{penalty} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{penalty} \NC number \NC the penalty value \NC \NR +\LL \stoptabulate The subtypes are just informative and \TEX\ itself doesn't use them. When you run into an \type {linebreakpenalty} you need to keep in mind that it's a accumulation of \type {club}, \type{widow} and other relevant penalties. -\subsubsection[glyphnodes]{glyph nodes} +\subsection[glyphnodes]{\nod {glyph} nodes} + +\topicindex {nodes+glyph} +\topicindex {glyphs} + +These are probably the mostly used nodes and although you can push them in the +current list with for instance \prm {char} \TEX\ will normally do it for you when +it considers some input to be text. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR -\NC \type{subtype} \NC number \NC bitfield \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC bit field \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR -\NC \type{char} \NC number \NC the chatacter index in the font \NC \NR +\NC \type{char} \NC number \NC the character index in the font \NC \NR \NC \type{font} \NC number \NC the font identifier \NC \NR \NC \type{lang} \NC number \NC the language identifier \NC \NR \NC \type{left} \NC number \NC the frozen \type {\lefthyphenmnin} value \NC \NR \NC \type{right} \NC number \NC the frozen \type {\righthyphenmnin} value \NC \NR -\NC \type{uchyph} \NC boolean \NC the frozen \type {\uchyph} value \NC \NR +\NC \type{uchyph} \NC boolean \NC the frozen \prm {uchyph} value \NC \NR \NC \type{components} \NC node \NC pointer to ligature components \NC \NR \NC \type{xoffset} \NC number \NC a virtual displacement in horizontal direction \NC \NR \NC \type{yoffset} \NC number \NC a virtual displacement in vertical direction \NC \NR -%NC \type{xadvance} \NC number \NC an additional advance after the glyph (experimental) \NC \NR \NC \type{width} \NC number \NC the (original) width of the character \NC \NR \NC \type{height} \NC number \NC the (original) height of the character\NC \NR \NC \type{depth} \NC number \NC the (original) depth of the character\NC \NR \NC \type{expansion_factor} \NC number \NC the to be applied expansion_factor \NC \NR +\NC \type{data} \NC number \NC a general purpose field for users (we had room for it) \NC \NR +\LL \stoptabulate The \type {width}, \type {height} and \type {depth} values are read|-|only. The -\type {expansion_factor} is assigned in the parbuilder and used in the backend. +\type {expansion_factor} is assigned in the par builder and used in the backend. A warning: never assign a node list to the components field unless you are sure its internal link structure is correct, otherwise an error may be result. Valid bits for the \type {subtype} field are: \starttabulate[|c|l|] -\BC bit \BC meaning \NC \NR +\DB bit \BC meaning \NC \NR +\TB \NC 0 \NC character \NC \NR \NC 1 \NC ligature \NC \NR \NC 2 \NC ghost \NC \NR \NC 3 \NC left \NC \NR \NC 4 \NC right \NC \NR +\LL \stoptabulate See \in {section} [charsandglyphs] for a detailed description of the \type @@ -378,87 +458,117 @@ helpers are not always faster than separate calls but they sometimes permit making more readable tests. The \type {uses_font} helpers takes a node and font id and returns true when a glyph or disc node references that font. -\subsubsection{boundary nodes} +\subsection{\nod {boundary} nodes} + +\topicindex {nodes+boundary} +\topicindex {boundary} + +This node relates to the \prm {noboundary}, \prm {boundary}, \prm +{protrusionboundary} and \prm {wordboundary} primitives. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{boundary} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{value} \NC number \NC values 0--255 are reserved \NC \NR +\LL \stoptabulate -This node relates to the \type {\noboundary}, \type {\boundary}, \type -{\protrusionboundary} and \type {\wordboundary} primitives. +\subsection{\nod {local_par} nodes} + +\topicindex {nodes+paragraphs} +\topicindex {paragraphs} -\subsubsection{local_par nodes} +This node is inserted at the start of a paragraph. You should not mess +too much with this one. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR -\NC \type{pen_inter} \NC number \NC local interline penalty (from \type {\localinterlinepenalty}) \NC \NR -\NC \type{pen_broken} \NC number \NC local broken penalty (from \type {\localbrokenpenalty}) \NC \NR +\NC \type{pen_inter} \NC number \NC local interline penalty (from \lpr {localinterlinepenalty}) \NC \NR +\NC \type{pen_broken} \NC number \NC local broken penalty (from \lpr {localbrokenpenalty}) \NC \NR \NC \type{dir} \NC string \NC the direction of this par. see~\in [dirnodes] \NC \NR -\NC \type{box_left} \NC node \NC the \type {\localleftbox} \NC \NR -\NC \type{box_left_width} \NC number \NC width of the \type {\localleftbox} \NC \NR -\NC \type{box_right} \NC node \NC the \type {\localrightbox} \NC \NR -\NC \type{box_right_width} \NC number \NC width of the \type {\localrightbox} \NC \NR +\NC \type{box_left} \NC node \NC the \lpr {localleftbox} \NC \NR +\NC \type{box_left_width} \NC number \NC width of the \lpr {localleftbox} \NC \NR +\NC \type{box_right} \NC node \NC the \lpr {localrightbox} \NC \NR +\NC \type{box_right_width} \NC number \NC width of the \lpr {localrightbox} \NC \NR +\LL \stoptabulate A warning: never assign a node list to the \type {box_left} or \type {box_right} field unless you are sure its internal link structure is correct, otherwise an -error may be result. +error may result. + +\subsection[dirnodes]{\nod {dir} nodes} -\subsubsection[dirnodes]{dir nodes} +\topicindex {nodes+direction} +\topicindex {directions} + +Direction nodes mark parts of the running text that need a change of direction and \ +the \prm {textdir} command generates them. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{dir} \NC string \NC the direction (but see below) \NC \NR \NC \type{level} \NC number \NC nesting level of this direction whatsit \NC \NR +\LL \stoptabulate -A note on \type {dir} strings. Direction specifiers are three|-|letter -combinations of \type {T}, \type {B}, \type {R}, and \type {L}. - -These are built up out of three separate items: +Direction specifiers are three|-|letter combinations of \type {T}, \type {B}, +\type {R}, and \type {L}. These are built up out of three separate items: \startitemize[packed] \startitem - the first is the direction of the \quote{top} of paragraphs. + the first is the direction of the \quote{top} of paragraphs \stopitem \startitem - the second is the direction of the \quote{start} of lines. + the second is the direction of the \quote{start} of lines \stopitem \startitem - the third is the direction of the \quote{top} of glyphs. + the third is the direction of the \quote{top} of glyphs \stopitem \stopitemize However, only four combinations are accepted: \type {TLT}, \type {TRT}, \type -{RTT}, and \type {LTL}. +{RTT}, and \type {LTL}. Inside actual \nod {dir} nodes, the representation of +\nod {dir} is not a three|-|letter but a combination of numbers. When printed the +direction is indicated by a \type {+} or \type {-}, indicating whether the value +is pushed or popped from the direction stack. -Inside actual \type {dir} whatsit nodes, the representation of \type {dir} is not -a three-letter but a four|-|letter combination. The first character in this case -is always either \type {+} or \type {-}, indicating whether the value is pushed -or popped from the direction stack. +\subsection{\nod {marginkern} nodes} -\subsubsection{margin_kern nodes} +\topicindex {nodes+paragraphs} +\topicindex {paragraphs} +\topicindex {protrusion} + +Margin kerns result from protrusion. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR -\NC \type{subtype} \NC number \NC \showsubtypes{margin_kern} \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{subtype} \NC number \NC \showsubtypes{marginkern} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{width} \NC number \NC the advance of the kern \NC \NR \NC \type{glyph} \NC node \NC the glyph to be used \NC \NR +\LL \stoptabulate -\subsection{Math nodes} +\stopsection + +\startsection[title={Math noads}] + +\topicindex {nodes+math} +\topicindex {math+nodes} These are the so||called \quote {noad}s and the nodes that are specifically associated with math processing. Most of these nodes contain subnodes so that the list of possible fields is actually quite small. First, the subnodes: -\subsubsection{Math kernel subnodes} +\subsection{Math kernel subnodes} Many object fields in math mode are either simple characters in a specific family or math lists or node lists. There are four associated subnodes that represent @@ -467,58 +577,62 @@ these cases (in the following node descriptions these are indicated by the word The \type {next} and \type {prev} fields for these subnodes are unused. -\subsubsubsection{math_char and math_text_char subnodes} +\subsection{\nod {math_char} and \nod {math_text_char} subnodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{char} \NC number \NC the character index \NC \NR \NC \type{fam} \NC number \NC the family number \NC \NR +\LL \stoptabulate -The \type {math_char} is the simplest subnode field, it contains the character -and family for a single glyph object. The \type {math_text_char} is a special +The \nod {math_char} is the simplest subnode field, it contains the character +and family for a single glyph object. The \nod {math_text_char} is a special case that you will not normally encounter, it arises temporarily during math list conversion (its sole function is to suppress a following italic correction). -\subsubsubsection{sub_box and sub_mlist subnodes} +\subsection{\nod {sub_box} and \nod {sub_mlist} subnodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{head/list} \NC node \NC list of nodes \NC \NR +\LL \stoptabulate -These two subnode types are used for subsidiary list items. For \type {sub_box}, -the \type {head} points to a \quote {normal} vbox or hbox. For \type {sub_mlist}, +These two subnode types are used for subsidiary list items. For \nod {sub_box}, +the \type {head} points to a \quote {normal} vbox or hbox. For \nod {sub_mlist}, the \type {head} points to a math list that is yet to be converted. A warning: never assign a node list to the \type {head} field unless you are sure -its internal link structure is correct, otherwise an error may be result. +its internal link structure is correct, otherwise an error is triggered. -\subsubsection{Math delimiter subnode} +\subsection{\nod {delim} subnodes} There is a fifth subnode type that is used exclusively for delimiter fields. As before, the \type {next} and \type {prev} fields are unused. -\subsubsubsection{delim subnodes} - \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{small_char} \NC number \NC character index of base character \NC \NR \NC \type{small_fam} \NC number \NC family number of base character \NC \NR \NC \type{large_char} \NC number \NC character index of next larger character \NC \NR \NC \type{large_fam} \NC number \NC family number of next larger character \NC \NR +\LL \stoptabulate The fields \type {large_char} and \type {large_fam} can be zero, in that case the -font that is sed for the \type {small_fam} is expected to provide the large +font that is set for the \type {small_fam} is expected to provide the large version as an extension to the \type {small_char}. -\subsubsection{Math core nodes} +\subsection{Math core nodes} -First, there are the objects (the \TEX book calls then \quote {atoms}) that are +First, there are the objects (the \TEX book calls them \quote {atoms}) that are associated with the simple math objects: ord, op, bin, rel, open, close, punct, inner, over, under, vcent. These all have the same fields, and they are combined into a single node type with separate subtypes for differentiation. @@ -526,6 +640,8 @@ into a single node type with separate subtypes for differentiation. Some noads have an option field. The values in this bitset are common: \starttabulate[|l|r|] +\DB meaning \BC bits \NC \NR +\TB \NC set \NC \type{0x08} \NC \NR \NC internal \NC \type{0x00} + \type{0x08} \NC \NR \NC internal \NC \type{0x01} + \type{0x08} \NC \NR @@ -538,24 +654,28 @@ Some noads have an option field. The values in this bitset are common: \NC no sub script \NC \type{0x21} + \type{0x08} \NC \NR \NC no super script \NC \type{0x22} + \type{0x08} \NC \NR \NC no script \NC \type{0x23} + \type{0x08} \NC \NR +\LL \stoptabulate -\subsubsubsection{simple nodes} +\subsection{simple \nod {noad} nodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{noad} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{nucleus} \NC kernel node \NC base \NC \NR \NC \type{sub} \NC kernel node \NC subscript \NC \NR \NC \type{sup} \NC kernel node \NC superscript \NC \NR \NC \type{options} \NC number \NC bitset of rendering options \NC \NR +\LL \stoptabulate -\subsubsubsection{accent nodes} +\subsection{\nod {accent} nodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{accent} \NC \NR \NC \type{nucleus} \NC kernel node \NC base \NC \NR \NC \type{sub} \NC kernel node \NC subscript \NC \NR @@ -563,57 +683,65 @@ Some noads have an option field. The values in this bitset are common: \NC \type{accent} \NC kernel node \NC top accent \NC \NR \NC \type{bot_accent} \NC kernel node \NC bottom accent \NC \NR \NC \type{fraction} \NC number \NC larger step criterium (divided by 1000) \NC \NR +\LL \stoptabulate -\subsubsubsection{style nodes} +\subsection{\nod {style} nodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{style} \NC string \NC contains the style \NC \NR +\LL \stoptabulate There are eight possibilities for the string value: one of \type {display}, \type {text}, \type {script}, or \type {scriptscript}. Each of these can have be prefixed by \type {cramped}. -\subsubsubsection{choice nodes} +\subsection{\nod {choice} nodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{display} \NC node \NC list of display size alternatives \NC \NR \NC \type{text} \NC node \NC list of text size alternatives \NC \NR \NC \type{script} \NC node \NC list of scriptsize alternatives \NC \NR \NC \type{scriptscript} \NC node \NC list of scriptscriptsize alternatives \NC \NR +\LL \stoptabulate Warning: never assign a node list to the \type {display}, \type {text}, \type {script}, or \type {scriptscript} field unless you are sure its internal link -structure is correct, otherwise an error may be result. +structure is correct, otherwise an error can occur. -\subsubsubsection{radical nodes} +\subsection{\nod {radical} nodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{radical} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{nucleus} \NC kernel node \NC base \NC \NR \NC \type{sub} \NC kernel node \NC subscript \NC \NR \NC \type{sup} \NC kernel node \NC superscript \NC \NR \NC \type{left} \NC delimiter node \NC \NC \NR -\NC \type{degree} \NC kernel node \NC only set by \type {\Uroot} \NC \NR +\NC \type{degree} \NC kernel node \NC only set by \lpr {Uroot} \NC \NR \NC \type{width} \NC number \NC required width \NC \NR \NC \type{options} \NC number \NC bitset of rendering options \NC \NR +\LL \stoptabulate Warning: never assign a node list to the \type {nucleus}, \type {sub}, \type {sup}, \type {left}, or \type {degree} field unless you are sure its internal -link structure is correct, otherwise an error may be result. +link structure is correct, otherwise an error can be triggered. -\subsubsubsection{fraction nodes} +\subsection{\nod {fraction} nodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{width} \NC number \NC (optional) width of the fraction \NC \NR \NC \type{num} \NC kernel node \NC numerator \NC \NR @@ -622,16 +750,18 @@ link structure is correct, otherwise an error may be result. \NC \type{right} \NC delimiter node \NC right side symbol \NC \NR \NC \type{middle} \NC delimiter node \NC middle symbol \NC \NR \NC \type{options} \NC number \NC bitset of rendering options \NC \NR +\LL \stoptabulate Warning: never assign a node list to the \type {num}, or \type {denom} field unless you are sure its internal link structure is correct, otherwise an error -may be result. +can result. -\subsubsubsection{fence nodes} +\subsection{\nod {fence} nodes} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{subtype} \NC number \NC \showsubtypes{fence} \NC \NR \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{delim} \NC delimiter node \NC delimiter specification \NC \NR @@ -640,15 +770,18 @@ may be result. \NC \type{depth} \NC number \NC required depth \NC \NR \NC \type{options} \NC number \NC bitset of rendering options \NC \NR \NC \type{class} \NC number \NC spacing related class \NC \NR +\LL \stoptabulate Warning: some of these fields are used by the renderer and might get adapted in the process. -\subsection{whatsit nodes} +\stopsection + +\startsection[title={Front|-|end whatsits}] -Whatsit nodes come in many subtypes that you can ask for by running -\type {node.whatsits()}: +Whatsit nodes come in many subtypes that you can ask for them by running +\type {node.whatsits}: \startluacode for id, name in table.sortedpairs(node.whatsits()) do context.type(name) @@ -659,44 +792,53 @@ Whatsit nodes come in many subtypes that you can ask for by running \stopluacode . % period -\subsubsection{front|-|end whatsits} +Some of them are generic and independent of the output mode and others are +specific to the chosen backend: \DVI\ or \PDF. Here we discuss the generic +font|-|end nodes nodes. -\subsubsubsection{open whatsits} +\subsection{\whs {open}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR \NC \type{name} \NC string \NC file name \NC \NR \NC \type{ext} \NC string \NC file extension \NC \NR \NC \type{area} \NC string \NC file area (this may become obsolete) \NC \NR +\LL \stoptabulate -\subsubsubsection{write whatsits} +\subsection{\whs {write}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR \NC \type{data} \NC table \NC a table representing the token list to be written \NC \NR +\LL \stoptabulate -\subsubsubsection{close whatsits} +\subsection{\whs {close}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{stream} \NC number \NC \TEX's stream id number \NC \NR +\LL \stoptabulate -\subsubsubsection{user_defined whatsits} +\subsection{\whs {user_defined}} User|-|defined whatsit nodes can only be created and handled from \LUA\ code. In effect, they are an extension to the extension mechanism. The \LUATEX\ engine will simply step over such whatsits without ever looking at the contents. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{user_id} \NC number \NC id number \NC \NR \NC \type{type} \NC number \NC type of the value \NC \NR @@ -704,103 +846,129 @@ will simply step over such whatsits without ever looking at the contents. \NC \NC node \NC a node list \NC \NR \NC \NC string \NC a \LUA\ string \NC \NR \NC \NC table \NC a \LUA\ table \NC \NR +\LL \stoptabulate The \type {type} can have one of six distinct values. The number is the \ASCII\ -value if the first character if the type name (so you can use string.byte("l") +value if the first character of the type name (so you can use string.byte("l") instead of \type {108}). \starttabulate[|r|c|p|] -\BC value \BC meaning \BC explanation \NC \NR +\DB value \BC meaning \BC explanation \NC \NR +\TB \NC 97 \NC a \NC list of attributes (a node list) \NC \NR \NC 100 \NC d \NC a \LUA\ number \NC \NR \NC 108 \NC l \NC a \LUA\ value (table, number, boolean, etc) \NC \NR \NC 110 \NC n \NC a node list \NC \NR \NC 115 \NC s \NC a \LUA\ string \NC \NR \NC 116 \NC t \NC a \LUA\ token list in \LUA\ table form (a list of triplets) \NC \NR +\LL \stoptabulate -\subsubsubsection{save_pos whatsits} +\subsection{\whs {save_pos}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR +\LL \stoptabulate -\subsubsubsection{late_lua whatsits} +\subsection{\whs {late_lua}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR -\NC \type{attr} \NC node \NC list of attributes \NC \NR -\NC \type{data} \NC string \NC data to execute \NC \NR -\NC \type{string} \NC string \NC data to execute \NC \NR -\NC \type{name} \NC string \NC the name to use for \LUA\ error reporting \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{data} \NC string or function \NC the to be written information stored as \LUA\ value \NC \NR +\NC \type{token} \NC string \NC the to be written information stored as token list \NC \NR +\NC \type{name} \NC string \NC the name to use for \LUA\ error reporting \NC \NR +\LL \stoptabulate The difference between \type {data} and \type {string} is that on assignment, the -\type {data} field is converted to a token list, cf. use as \type {\latelua}. The +\type {data} field is converted to a token list, cf.\ use as \lpr {latelua}. The \type {string} version is treated as a literal string. -\subsubsection{\DVI\ backend whatsits} +\stopsection -\subsubsection{special whatsits} +\startsection[title={\DVI\ backend whatsits}] + +\subsection{\whs {special}} + +There is only one \DVI\ backend whatsit, and it just flushes its content to the +output file. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR -\NC \type{data} \NC string \NC the \type {\special} information \NC \NR +\NC \type{data} \NC string \NC the \prm {special} information \NC \NR +\LL \stoptabulate -\subsubsection{\PDF\ backend whatsits} +\stopsection + +\startsection[title={\PDF\ backend whatsits}] -\subsubsubsection{pdf_literal whatsits} +\subsection{\whs {pdf_literal}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR -\NC \type{attr} \NC node \NC list of attributes \NC \NR -\NC \type{mode} \NC number \NC the \quote {mode} setting of this literal \NC \NR -\NC \type{data} \NC string \NC the \type {\pdfliteral} information \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{attr} \NC node \NC list of attributes \NC \NR +\NC \type{mode} \NC number \NC the \quote {mode} setting of this literal \NC \NR +\NC \type{data} \NC string \NC the to be written information stored as \LUA\ string \NC \NR +\NC \type{token} \NC string \NC the to be written information stored as token list \NC \NR +\LL \stoptabulate Possible mode values are: -\starttabulate[|l|p|] -\BC value \BC keyword \NC \NR +\starttabulate[|c|p|] +\DB value \BC keyword \NC \NR +\TB \NC 0 \NC \type{origin} \NC \NR \NC 1 \NC \type{page} \NC \NR \NC 2 \NC \type{direct} \NC \NR \NC 3 \NC \type{raw} \NC \NR \NC 4 \NC \type{text} \NC \NR +\LL \stoptabulate -The higher the number, the less checking and the more you can run into troubles. +The higher the number, the less checking and the more you can run into trouble. Especially the \type {raw} variant can produce bad \PDF\ so you can best check what you generate. -\subsubsubsection{pdf_refobj whatsits} +\subsection{\whs {pdf_refobj}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{objnum} \NC number \NC the referenced \PDF\ object number \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_annot whatsits} +\subsection{\whs {pdf_annot}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{width} \NC number \NC the width (not used in calculations) \NC \NR \NC \type{height} \NC number \NC the height (not used in calculations) \NC \NR \NC \type{depth} \NC number \NC the depth (not used in calculations) \NC \NR \NC \type{objnum} \NC number \NC the referenced \PDF\ object number \NC \NR \NC \type{data} \NC string \NC the annotation data \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_start_link whatsits} +\subsection{\whs {pdf_start_link}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{width} \NC number \NC the width (not used in calculations) \NC \NR \NC \type{height} \NC number \NC the height (not used in calculations) \NC \NR @@ -808,19 +976,23 @@ what you generate. \NC \type{objnum} \NC number \NC the referenced \PDF\ object number \NC \NR \NC \type{link_attr} \NC table \NC the link attribute token list \NC \NR \NC \type{action} \NC node \NC the action to perform \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_end_link whatsits} +\subsection{\whs {pdf_end_link}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_dest whatsits} +\subsection{\whs {pdf_dest}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{width} \NC number \NC the width (not used in calculations) \NC \NR \NC \type{height} \NC number \NC the height (not used in calculations) \NC \NR @@ -831,45 +1003,54 @@ what you generate. \NC \type{dest_type} \NC number \NC type of destination \NC \NR \NC \type{xyz_zoom} \NC number \NC the zoom factor (times 1000) \NC \NR \NC \type{objnum} \NC number \NC the \PDF\ object number \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_action whatsits} +\subsection{\whs {pdf_action}} -These are a special kind of item that only appears inside \PDF\ start link +These are a special kind of items that only appear inside \PDF\ start link objects. \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{action_type} \NC number \NC the kind of action involved \NC \NR \NC \type{action_id} \NC number or string \NC token list reference or string \NC \NR \NC \type{named_id} \NC number \NC the index of the destination \NC \NR \NC \type{file} \NC string \NC the target filename \NC \NR \NC \type{new_window} \NC number \NC the window state of the target \NC \NR \NC \type{data} \NC string \NC the name of the destination \NC \NR +\LL \stoptabulate Valid action types are: \starttabulate[|l|l|] -\NC 0 \NC \type{page} \NC \NR -\NC 1 \NC \type{goto} \NC \NR -\NC 2 \NC \type{thread} \NC \NR -\NC 3 \NC \type{user} \NC \NR +\DB value \BC meaning \NC \NR +\TB +\NC 0 \NC \type{page} \NC \NR +\NC 1 \NC \type{goto} \NC \NR +\NC 2 \NC \type{thread} \NC \NR +\NC 3 \NC \type{user} \NC \NR +\LL \stoptabulate Valid window types are: \starttabulate[|l|l|] -\NC 0 \NC \type{notset} \NC \NR -\NC 1 \NC \type{new} \NC \NR -\NC 2 \NC \type{nonew} \NC \NR +\DB value \BC meaning \NC \NR +\TB +\NC 0 \NC \type{notset} \NC \NR +\NC 1 \NC \type{new} \NC \NR +\NC 2 \NC \type{nonew} \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_thread whatsits} +\subsection{\whs {pdf_thread}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{width} \NC number \NC the width (not used in calculations) \NC \NR \NC \type{height} \NC number \NC the height (not used in calculations) \NC \NR @@ -878,12 +1059,14 @@ Valid window types are: \NC \type{tread_id} \NC number \NC the thread id \NC \NR \NC \NC string \NC the thread name \NC \NR \NC \type{thread_attr} \NC number \NC extra thread information \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_start_thread whatsits} +\subsection{\whs {pdf_start_thread}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{width} \NC number \NC the width (not used in calculations) \NC \NR \NC \type{height} \NC number \NC the height (not used in calculations) \NC \NR @@ -892,48 +1075,63 @@ Valid window types are: \NC \type{tread_id} \NC number \NC the thread id \NC \NR \NC \NC string \NC the thread name \NC \NR \NC \type{thread_attr} \NC number \NC extra thread information \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_end_thread whatsits} +\subsection{\whs {pdf_end_thread}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_colorstack whatsits} +\subsection{\whs {pdf_colorstack}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{stack} \NC number \NC colorstack id number \NC \NR \NC \type{command} \NC number \NC command to execute \NC \NR \NC \type{data} \NC string \NC data \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_setmatrix whatsits} +\subsection{\whs {pdf_setmatrix}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR \NC \type{data} \NC string \NC data \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_save whatsits} +\subsection{\whs {pdf_save}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR +\LL \stoptabulate -\subsubsubsection{pdf_restore whatsits} +\subsection{\whs {pdf_restore}} \starttabulate[|l|l|p|] -\BC field \BC type \BC explanation \NC \NR +\DB field \BC type \BC explanation \NC \NR +\TB \NC \type{attr} \NC node \NC list of attributes \NC \NR +\LL \stoptabulate -\section{The \type {node} library} +\stopsection + +\startsection[title={The \type {node} library}][library=node] + +\subsection {Introduction} The \type {node} library contains functions that facilitate dealing with (lists of) nodes and their values. They allow you to create, alter, copy, delete, and @@ -943,8 +1141,7 @@ insert \LUATEX\ node objects, the core objects within the typesetter. \type {luatex.node}. The various parts within a node can be accessed using named fields. -Each node has at least the three fields \type {next}, \type {id}, and \type -{subtype}: +Each node has at least the three fields \type {next}, \type {id}, and \type {subtype}: \startitemize[intro] @@ -969,10 +1166,9 @@ Each node has at least the three fields \type {next}, \type {id}, and \type \stopitemize The other available fields depend on the \type {id} (and for \quote {whatsits}, -the \type {subtype}) of the node. Further details on the various fields and their -meanings are given in~\in{chapter}[nodes]. +the \type {subtype}) of the node. -Support for \type {unset} (alignment) nodes is partial: they can be queried and +Support for \nod {unset} (alignment) nodes is partial: they can be queried and modified from \LUA\ code, but not created. Nodes can be compared to each other, but: you are actually comparing indices into @@ -987,76 +1183,90 @@ call the node freeing functions yourself when you are no longer in need of a nod (list). Nodes form linked lists without reference counting, so you have to be careful that when control returns back to \LUATEX\ itself, you have not deleted nodes that are still referenced from a \type {next} pointer elsewhere, and that -you did not create nodes that are referenced more than once. +you did not create nodes that are referenced more than once. Normally the setters +and getters handle this for you. There are statistics available with regards to the allocated node memory, which can be handy for tracing. -\subsection{Node handling functions} +\subsection{\type {is_node}} -\subsubsection{\type {node.is_node}} +\topicindex {nodes+functions} + +\libindex {is_node} \startfunctioncall -<boolean> t = +<boolean|integer> t = node.is_node(<any> item) \stopfunctioncall -This function returns true if the argument is a userdata object of -type \type {<node>}. +This function returns a number (the internal index of the node) if the argument +is a userdata object of type \type {<node>} and false when no node is passed. -\subsubsection{\type {node.types}} +\subsection{\type {types} and \type {whatsits}} + +\libindex {types} +\libindex {whatsits} + +This function returns an array that maps node id numbers to node type strings, +providing an overview of the possible top|-|level \type {id} types. \startfunctioncall <table> t = node.types() \stopfunctioncall -This function returns an array that maps node id numbers to node type strings, -providing an overview of the possible top|-|level \type {id} types. - -\subsubsection{\type {node.whatsits}} +\TEX's \quote {whatsits} all have the same \type {id}. The various subtypes are +defined by their \type {subtype} fields. The function is much like \type {types}, +except that it provides an array of \type {subtype} mappings. \startfunctioncall <table> t = node.whatsits() \stopfunctioncall -\TEX's \quote{whatsits} all have the same \type {id}. The various subtypes are -defined by their \type {subtype} fields. The function is much like \type -{node.types}, except that it provides an array of \type {subtype} mappings. +\subsection{\type {id}} -\subsubsection{\type {node.id}} +\libindex{id} + +This converts a single type name to its internal numeric representation. \startfunctioncall <number> id = node.id(<string> type) \stopfunctioncall -This converts a single type name to its internal numeric representation. +\subsection{\type {type} and \type {subtype}} -\subsubsection{\type {node.subtype}} +\libindex {type} +\libindex {subtype} + +In the argument is a number, then the next function converts an internal numeric +representation to an external string representation. Otherwise, it will return +the string \type {node} if the object represents a node, and \type {nil} +otherwise. \startfunctioncall -<number> subtype = - node.subtype(<string> type) +<string> type = + node.type(<any> n) \stopfunctioncall -This converts a single whatsit name to its internal numeric representation (\type -{subtype}). - -\subsubsection{\type {node.type}} +This next one converts a single whatsit name to its internal numeric +representation (\type {subtype}). \startfunctioncall -<string> type = - node.type(<any> n) +<number> subtype = + node.subtype(<string> type) \stopfunctioncall -In the argument is a number, then this function converts an internal numeric -representation to an external string representation. Otherwise, it will return -the string \type {node} if the object represents a node, and \type {nil} -otherwise. +\subsection{\type {fields}} + +\libindex {fields} -\subsubsection{\type {node.fields}} +This function returns an array of valid field names for a particular type of +node. If you want to get the valid fields for a \quote {whatsit}, you have to +supply the second argument also. In other cases, any given second argument will +be silently ignored. \startfunctioncall <table> t = @@ -1065,24 +1275,29 @@ otherwise. node.fields(<number> id, <number> subtype) \stopfunctioncall -This function returns an array of valid field names for a particular type of -node. If you want to get the valid fields for a \quote {whatsit}, you have to -supply the second argument also. In other cases, any given second argument will -be silently ignored. +The function accepts string \type {id} and \type {subtype} values as well. -This function accepts string \type {id} and \type {subtype} values as well. +\subsection{\type {has_field}} -\subsubsection{\type {node.has_field}} +\libindex {has_field} + +This function returns a boolean that is only true if \type {n} is +actually a node, and it has the field. \startfunctioncall <boolean> t = node.has_field(<node> n, <string> field) \stopfunctioncall -This function returns a boolean that is only true if \type {n} is -actually a node, and it has the field. +\subsection{\type {new}} -\subsubsection{\type {node.new}} +\libindex{new} + +The \type {new} function creates a new node. All its fields are initialized to +either zero or \type {nil} except for \type {id} and \type {subtype}. Instead of +numbers you can also use strings (names). If you create a new \nod {whatsit} node +the second argument is required. As with all node functions, this function +creates a node at the \TEX\ level. \startfunctioncall <node> n = @@ -1091,15 +1306,16 @@ actually a node, and it has the field. node.new(<number> id, <number> subtype) \stopfunctioncall -Creates a new node. All of the new node's fields are initialized to either zero -or \type {nil} except for \type {id} and \type {subtype} (if supplied). If you -want to create a new whatsit, then the second argument is required, otherwise it -need not be present. As with all node functions, this function creates a node on -the \TEX\ level. +\subsection{\type {free}, \type {flush_node} and \type {flush_list}} -This function accepts string \type {id} and \type {subtype} values as well. +\libindex{free} +\libindex{flush_node} +\libindex{flush_list} -\subsubsection{\type {node.free} and \type {node.flush_node}} +The next one the node \type {n} from \TEX's memory. Be careful: no checks are +done on whether this node is still pointed to from a register or some \type +{next} field: it is up to you to make sure that the internal data structures +remain correct. \startfunctioncall <node> next = @@ -1107,35 +1323,33 @@ This function accepts string \type {id} and \type {subtype} values as well. flush_node(<node> n) \stopfunctioncall -Removes the node \type {n} from \TEX's memory. Be careful: no checks are done on -whether this node is still pointed to from a register or some \type {next} field: -it is up to you to make sure that the internal data structures remain correct. - The \type {free} function returns the next field of the freed node, while the \type {flush_node} alternative returns nothing. -\subsubsection{\type {node.flush_list}} +A list starting with node \type {n} can be flushed from \TEX's memory too. Be +careful: no checks are done on whether any of these nodes is still pointed to +from a register or some \type {next} field: it is up to you to make sure that the +internal data structures remain correct. \startfunctioncall node.flush_list(<node> n) \stopfunctioncall -Removes the node list \type {n} and the complete node list following \type {n} -from \TEX's memory. Be careful: no checks are done on whether any of these nodes -is still pointed to from a register or some \type {next} field: it is up to you -to make sure that the internal data structures remain correct. +\subsection{\type {copy} and \type {copy_list}} -\subsubsection{\type {node.copy}} +\libindex{copy} +\libindex{copy_list} + +This creates a deep copy of node \type {n}, including all nested lists as in the case +of a hlist or vlist node. Only the \type {next} field is not copied. \startfunctioncall <node> m = node.copy(<node> n) \stopfunctioncall -Creates a deep copy of node \type {n}, including all nested lists as in the case -of a hlist or vlist node. Only the \type {next} field is not copied. - -\subsubsection{\type {node.copy_list}} +A deep copy of the node list that starts at \type {n} can be created too. If +\type {m} is also given, the copy stops just before node \type {m}. \startfunctioncall <node> m = @@ -1144,42 +1358,37 @@ of a hlist or vlist node. Only the \type {next} field is not copied. node.copy_list(<node> n, <node> m) \stopfunctioncall -Creates a deep copy of the node list that starts at \type {n}. If \type {m} is -also given, the copy stops just before node \type {m}. +Note that you cannot copy attribute lists this way. However, there is normally no +need to copy attribute lists as when you do assignments to the \type {attr} field +or make changes to specific attributes, the needed copying and freeing takes +place automatically. + +\subsection{\type {prev} and \type{next}} -Note that you cannot copy attribute lists this way, specialized functions for -dealing with attribute lists will be provided later but are not there yet. -However, there is normally no need to copy attribute lists as when you do -assignments to the \type {attr} field or make changes to specific attributes, the -needed copying and freeing takes place automatically. +\libindex{prev} +\libindex{next} -\subsubsection{\type {node.next}} +These returns the node preceding or following the given node, or \type {nil} if +there is no such node. \startfunctioncall <node> m = node.next(<node> n) -\stopfunctioncall - -Returns the node following this node, or \type {nil} if there is no such node. - -\subsubsection{\type {node.prev}} - -\startfunctioncall <node> m = node.prev(<node> n) \stopfunctioncall -Returns the node preceding this node, or \type {nil} if there is no such node. +\subsection{\type {current_attr}} + +\libindex{current_attr} -\subsubsection{\type {node.current_attr}} +This returns the currently active list of attributes, if there is one. \startfunctioncall <node> m = node.current_attr() \stopfunctioncall -Returns the currently active list of attributes, if there is one. - The intended usage of \type {current_attr} is as follows: \starttyping @@ -1209,7 +1418,16 @@ attribute list, not a copy thereof. Therefore, changing any of the attributes in the list will change these values for all nodes that have the current attribute list assigned to them. -\subsubsection{\type {node.hpack}} +\subsection{\type {hpack}} + +\libindex {hpack} + +This function creates a new hlist by packaging the list that begins at node \type +{n} into a horizontal box. With only a single argument, this box is created using +the natural width of its components. In the three argument form, \type {info} +must be either \type {additional} or \type {exactly}, and \type {w} is the +additional (\type {\hbox spread}) or exact (\type {\hbox to}) width to be used. +The second return value is the badness of the generated box. \startfunctioncall <node> h, <number> b = @@ -1220,21 +1438,22 @@ list assigned to them. node.hpack(<node> n, <number> w, <string> info, <string> dir) \stopfunctioncall -This function creates a new hlist by packaging the list that begins at node \type -{n} into a horizontal box. With only a single argument, this box is created using -the natural width of its components. In the three argument form, \type {info} -must be either \type {additional} or \type {exactly}, and \type {w} is the -additional (\type {\hbox spread}) or exact (\type {\hbox to}) width to be used. The -second return value is the badness of the generated box. +Caveat: there can be unexpected side|-|effects to this function, like updating +some of the \prm {marks} and \type {\inserts}. Also note that the content of +\type {h} is the original node list \type {n}: if you call \type {node.free(h)} +you will also free the node list itself, unless you explicitly set the \type +{list} field to \type {nil} beforehand. And in a similar way, calling \type +{node.free(n)} will invalidate \type {h} as well! -Caveat: at this moment, there can be unexpected side|-|effects to this function, -like updating some of the \type {\marks} and \type {\inserts}. Also note that the -content of \type {h} is the original node list \type {n}: if you call \type -{node.free(h)} you will also free the node list itself, unless you explicitly set -the \type {list} field to \type {nil} beforehand. And in a similar way, calling -\type {node.free(n)} will invalidate \type {h} as well! +\subsection{\type {vpack}} -\subsubsection{\type {node.vpack}} +\libindex {vpack} + +This function creates a new vlist by packaging the list that begins at node \type +{n} into a vertical box. With only a single argument, this box is created using +the natural height of its components. In the three argument form, \type {info} +must be either \type {additional} or \type {exactly}, and \type {w} is the +additional (\type {\vbox spread}) or exact (\type {\vbox to}) height to be used. \startfunctioncall <node> h, <number> b = @@ -1245,17 +1464,26 @@ the \type {list} field to \type {nil} beforehand. And in a similar way, calling node.vpack(<node> n, <number> w, <string> info, <string> dir) \stopfunctioncall -This function creates a new vlist by packaging the list that begins at node \type -{n} into a vertical box. With only a single argument, this box is created using -the natural height of its components. In the three argument form, \type {info} -must be either \type {additional} or \type {exactly}, and \type {w} is the -additional (\type {\vbox spread}) or exact (\type {\vbox to}) height to be used. +The second return value is the badness of the generated box. See the description +of \type {hpack} for a few memory allocation caveats. -The second return value is the badness of the generated box. +\subsection{\type {prepend_prevdepth}} -See the description of \type {node.hpack()} for a few memory allocation caveats. +\libindex {prepend_prevdepth} -\subsubsection{\type {node.dimensions}, \type {node.rangedimensions}} +This function is somewhat special in the sense that it is an experimental helper +that adds the interlinespace to a line keeping the baselineskip and lineskip into +account. + +\startfunctioncall +<node> n, <number> delta = + node.prepend_prevdepth(<node> n,<number> prevdepth) +\stopfunctioncall + +\subsection{\type {dimensions} and \type {rangedimensions}} + +\libindex{dimensions} +\libindex{rangedimensions} \startfunctioncall <number> w, <number> h, <number> d = @@ -1320,7 +1548,9 @@ cases: node.rangedimensions(<node> parent, <node> first, <node> last) \stopfunctioncall -\subsubsection{\type {node.mlist_to_hlist}} +\subsection{\type {mlist_to_hlist}} + +\libindex {mlist_to_hlist} \startfunctioncall <node> h = @@ -1329,9 +1559,9 @@ cases: This runs the internal mlist to hlist conversion, converting the math list in \type {n} into the horizontal list \type {h}. The interface is exactly the same -as for the callback \type {mlist_to_hlist}. +as for the callback \cbk {mlist_to_hlist}. -\subsubsection{\type {node.slide}} +\subsection{\type {slide}} \startfunctioncall <node> m = @@ -1342,7 +1572,9 @@ Returns the last node of the node list that starts at \type {n}. As a side|-|effect, it also creates a reverse chain of \type {prev} pointers between nodes. -\subsubsection{\type {node.tail}} +\subsection{\type {tail}} + +\libindex {tail} \startfunctioncall <node> m = @@ -1351,7 +1583,10 @@ nodes. Returns the last node of the node list that starts at \type {n}. -\subsubsection{\type {node.length}} +\subsection{\type {length} and type {count}} + +\libindex {length} +\libindex {count} \startfunctioncall <number> i = @@ -1364,8 +1599,6 @@ Returns the number of nodes contained in the node list that starts at \type {n}. If \type {m} is also supplied it stops at \type {m} instead of at the end of the list. The node \type {m} is not counted. -\subsubsection{\type {node.count}} - \startfunctioncall <number> i = node.count(<number> id, <node> n) @@ -1376,14 +1609,29 @@ list. The node \type {m} is not counted. Returns the number of nodes contained in the node list that starts at \type {n} that have a matching \type {id} field. If \type {m} is also supplied, counting stops at \type {m} instead of at the end of the list. The node \type {m} is not -counted. +counted. This function also accept string \type {id}'s. + +\subsection{\type {is_char} and \type {is_glyph}} -This function also accept string \type {id}'s. +\libindex {is_char} +\libindex {is_glyph} -\subsubsection{\type {node.traverse}} +The subtype of a glyph node signals if the glyph is already turned into a character reference +or not. \startfunctioncall -<node> t = +<boolean> b = + node.is_char(<node> n) +<boolean> b = + node.is_glyph(<node> n) +\stopfunctioncall + +\subsection{\type {traverse}} + +\libindex {traverse} + +\startfunctioncall +<node> t, id, subtype = node.traverse(<node> n) \stopfunctioncall @@ -1426,10 +1674,12 @@ pointers remain valid. If the above is unclear to you, see the section \quote {For Statement} in the \LUA\ Reference Manual. -\subsubsection{\type {node.traverse_id}} +\subsection{\type {traverse_id}} + +\libindex {traverse_id} \startfunctioncall -<node> t = +<node> t, subtype = node.traverse_id(<number> id, <node> n) \stopfunctioncall @@ -1454,17 +1704,45 @@ See the previous section for details. The change is in the local function \type end \stoptyping -\subsubsection{\type {node.traverse_char}} +\subsection{\type {traverse_char} and \type {traverse_glyph}} + +\libindex {traverse_char} +\libindex {traverse_glyph} -This iterators loops over the glyph nodes in a list. Only nodes with a subtype -less than 256 are seen. +The \type{traverse_char} iterator loops over the \nod {glyph} nodes in a list. +Only nodes with a subtype less than 256 are seen. \startfunctioncall -<node> n = +<node> n, font, char = node.traverse_char(<node> n) \stopfunctioncall -\subsubsection{\type {node.has_glyph}} +The \type{traverse_glyph} iterator loops over a list and returns the list and +filters all glyphs: + +\startfunctioncall +<node> n, font, char = + node.traverse_glyph(<node> n) +\stopfunctioncall + +\subsection{\type {traverse_list}} + +\libindex {traverse_list} + +This iterator loops over the \nod {hlist} and \nod {vlist} nodes in a list. + +\startfunctioncall +<node> n, id, subtype, list = + node.traverse_list(<node> n) +\stopfunctioncall + +The four return values can save some time compared to fetching these fields but +in practice you seldom need them all. So consider it a (side effect of +experimental) convenience. + +\subsection{\type {has_glyph}} + +\libindex {has_glyph} This function returns the first glyph or disc node in the given list: @@ -1473,7 +1751,9 @@ This function returns the first glyph or disc node in the given list: node.has_glyph(<node> n) \stopfunctioncall -\subsubsection{\type {node.end_of_math}} +\subsection{\type {end_of_math}} + +\libindex {end_of_math} \startfunctioncall <node> t = @@ -1481,11 +1761,13 @@ This function returns the first glyph or disc node in the given list: \stopfunctioncall Looks for and returns the next \type {math_node} following the \type {start}. If -the given node is a math endnode this helper return that node, else it follows -the list and return the next math endnote. If no such node is found nil is +the given node is a math end node this helper returns that node, else it follows +the list and returns the next math endnote. If no such node is found nil is returned. -\subsubsection{\type {node.remove}} +\subsection{\type {remove}} + +\libindex {remove} \startfunctioncall <node> head, current = @@ -1501,7 +1783,9 @@ no such node). The returned \type {head} is more important, because if the function is called with \type {current} equal to \type {head}, it will be changed. -\subsubsection{\type {node.insert_before}} +\subsection{\type {insert_before}} + +\libindex {insert_before} \startfunctioncall <node> head, new = @@ -1515,7 +1799,9 @@ mutated) \type {head} and the node \type {new}, set up to be part of the list (with correct \type {next} field). If \type {head} is initially \type {nil}, it will become \type {new}. -\subsubsection{\type {node.insert_after}} +\subsection{\type {insert_after}} + +\libindex {insert_after} \startfunctioncall <node> head, new = @@ -1528,7 +1814,9 @@ following \type {head}. It is your responsibility to make sure that \type the node \type {new}, set up to be part of the list (with correct \type {next} field). If \type {head} is initially \type {nil}, it will become \type {new}. -\subsubsection{\type {node.first_glyph}} +\subsection{\type {first_glyph}} + +\libindex {first_glyph} \startfunctioncall <node> n = @@ -1542,7 +1830,9 @@ with a subtype indicating it is a glyph, or \type {nil}. If \type {m} is given, processing stops at (but including) that node, otherwise processing stops at the end of the list. -\subsubsection{\type {node.ligaturing}} +\subsection{\type {ligaturing}} + +\libindex {ligaturing} \startfunctioncall <node> h, <node> t, <boolean> success = @@ -1555,7 +1845,9 @@ Apply \TEX-style ligaturing to the specified nodelist. The tail node \type {m} i optional. The two returned nodes \type {h} and \type {t} are the new head and tail (both \type {n} and \type {m} can change into a new ligature). -\subsubsection{\type {node.kerning}} +\subsection{\type {kerning}} + +\libindex {kerning} \startfunctioncall <node> h, <node> t, <boolean> success = @@ -1569,7 +1861,10 @@ optional. The two returned nodes \type {h} and \type {t} are the head and tail (either one of these can be an inserted kern node, because special kernings with word boundaries are possible). -\subsubsection{\type {node.unprotect_glyphs} and \type {node.unprotect_glyph}} +\subsection{\type {unprotect_glyph[s]}} + +\libindex {unprotect_glyphs} +\libindex {unprotect_glyph} \startfunctioncall node.unprotect_glyph(<node> n) @@ -1578,9 +1873,12 @@ node.unprotect_glyphs(<node> n,[<node> n]) Subtracts 256 from all glyph node subtypes. This and the next function are helpers to convert from \type {characters} to \type {glyphs} during node -processing. The second argument is option and indicates the end of a range. +processing. The second argument is optional and indicates the end of a range. -\subsubsection{\type {node.protect_glyphs} and \type {node.protect_glyph}} +\subsection{\type {protect_glyph[s]}} + +\libindex {protect_glyphs} +\libindex {protect_glyph} \startfunctioncall node.protect_glyph(<node> n) @@ -1591,9 +1889,11 @@ Adds 256 to all glyph node subtypes in the node list starting at \type {n}, except that if the value is 1, it adds only 255. The special handling of 1 means that \type {characters} will become \type {glyphs} after subtraction of 256. A single character can be marked by the singular call. The second argument is -option and indicates the end of a range. +optional and indicates the end of a range. + +\subsection{\type {last_node}} -\subsubsection{\type {node.last_node}} +\libindex {last_node} \startfunctioncall <node> n = @@ -1603,17 +1903,21 @@ option and indicates the end of a range. This function pops the last node from \TEX's \quote{current list}. It returns that node, or \type {nil} if the current list is empty. -\subsubsection{\type {node.write}} +\subsection{\type {write}} + +\libindex {write} \startfunctioncall node.write(<node> n) \stopfunctioncall -This is an experimental function that will append a node list to \TEX's \quote -{current list} The node list is not deep|-|copied! There is no error checking -either! +This function that will append a node list to \TEX's \quote {current list}. The +node list is not deep|-|copied! There is no error checking either! You mignt need +to enforce horizontal mode in order for this to work as expected. + +\subsection{\type {protrusion_skippable}} -\subsubsection{\type {node.protrusion_skippable}} +\libindex {protrusion_skippable} \startfunctioncall <boolean> skippable = @@ -1623,9 +1927,13 @@ either! Returns \type {true} if, for the purpose of line boundary discovery when character protrusion is active, this node can be skipped. -\subsection{Glue handling} +\stopsection + +\startsection[title={Glue handling}][library=node] -\subsubsection{\type {node.setglue}} +\subsection{\type {setglue}} + +\libindex {setglue} You can set the properties of a glue in one go. If you pass no values, the glue will become a zero glue. @@ -1645,9 +1953,11 @@ will only adapt the width and shrink. When a list node is passed, you set the glue, order and sign instead. -\subsubsection{\type {node.getglue}} +\subsection{\type {getglue}} + +\libindex {getglue} -The next call will return 5 values (or northing when no glue is passed). +The next call will return 5 values or nothing when no glue is passed. \startfunctioncall <integer> width, <integer> stretch, <integer> shrink, <integer> stretch_order, @@ -1660,7 +1970,9 @@ with \type {tex.get}). When a list node is passed, you get back the glue that is set, the order of that glue and the sign. -\subsubsection{\type {node.is_zero_glue}} +\subsection{\type {is_zero_glue}} + +\libindex {is_zero_glue} This function returns \type {true} when the width, stretch and shrink properties are zero. @@ -1670,13 +1982,61 @@ are zero. node.is_zero_glue(<node> n) \stopfunctioncall -\subsection{Attribute handling} +\stopsection + +\startsection[title={Attribute handling}][library=node] + +\subsection{Attributes} + +\topicindex {attributes} + +The newly introduced attribute registers are non|-|trivial, because the value +that is attached to a node is essentially a sparse array of key|-|value pairs. It +is generally easiest to deal with attribute lists and attributes by using the +dedicated functions in the \type {node} library, but for completeness, here is +the low|-|level interface. Attributes appear as linked list of userdata objects in the \type {attr} field of individual nodes. They can be handled individually, but it is much safer and more efficient to use the dedicated functions associated with them. -\subsubsection{\type {node.has_attribute}} +\subsection{\nod {attribute_list} nodes} + +\topicindex {nodes+attributes} + +An \nod {attribute_list} item is used as a head pointer for a list of attribute +items. It has only one user-visible field: + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{next} \NC node \NC pointer to the first attribute \NC \NR +\LL +\stoptabulate + +\subsection{\nod {attr} nodes} + +A normal node's attribute field will point to an item of type \nod +{attribute_list}, and the \type {next} field in that item will point to the first +defined \quote {attribute} item, whose \type {next} will point to the second +\quote {attribute} item, etc. + +\starttabulate[|l|l|p|] +\DB field \BC type \BC explanation \NC \NR +\TB +\NC \type{next} \NC node \NC pointer to the next attribute \NC \NR +\NC \type{number} \NC number \NC the attribute type id \NC \NR +\NC \type{value} \NC number \NC the attribute value \NC \NR +\LL +\stoptabulate + +As mentioned it's better to use the official helpers rather than edit these +fields directly. For instance the \type {prev} field is used for other purposes +and there is no double linked list. + +\subsection{\type {has_attribute}} + +\libindex {has_attribute} \startfunctioncall <number> v = @@ -1689,7 +2049,9 @@ Tests if a node has the attribute with number \type {id} set. If \type {val} is also supplied, also tests if the value matches \type {val}. It returns the value, or, if no match is found, \type {nil}. -\subsubsection{\type {node.get_attribute}} +\subsection{\type {get_attribute}} + +\libindex {get_attribute} \startfunctioncall <number> v = @@ -1697,9 +2059,12 @@ or, if no match is found, \type {nil}. \stopfunctioncall Tests if a node has an attribute with number \type {id} set. It returns the -value, or, if no match is found, \type {nil}. +value, or, if no match is found, \type {nil}. If no \type {id} is given then the +zero attributes is assumed. + +\subsection{\type {find_attribute}} -\subsubsection{\type {node.find_attribute}} +\libindex {find_attribute} \startfunctioncall <number> v, <node> n = @@ -1709,7 +2074,9 @@ value, or, if no match is found, \type {nil}. Finds the first node that has attribute with number \type {id} set. It returns the value and the node if there is a match and otherwise nothing. -\subsubsection{\type {node.set_attribute}} +\subsection{\type {set_attribute}} + +\libindex {set_attribute} \startfunctioncall node.set_attribute(<node> n, <number> id, <number> val) @@ -1718,7 +2085,9 @@ node.set_attribute(<node> n, <number> id, <number> val) Sets the attribute with number \type {id} to the value \type {val}. Duplicate assignments are ignored. -\subsubsection{\type {node.unset_attribute}} +\subsection{\type {unset_attribute}} + +\libindex {unset_attribute} \startfunctioncall <number> v = @@ -1734,7 +2103,9 @@ attributes or attribute|-|value pairs are ignored. If the attribute was actually deleted, returns its old value. Otherwise, returns \type {nil}. -\subsubsection{\type {node.slide}} +\subsection{\type {slide}} + +\libindex {slide} This helper makes sure that the node lists is double linked and returns the found tail node. @@ -1751,13 +2122,16 @@ pointers but your other callbacks might expect proper \type {prev} pointers too. Future versions of \LUATEX\ can add more checking but this will not influence usage. -\subsubsection{\type {node.check_discretionary} and \type {node.check_discretionaries}} +\subsection{\type {check_discretionary}, \type {check_discretionaries}} + +\libindex{check_discretionary} +\libindex{check_discretionaries} When you fool around with disc nodes you need to be aware of the fact that they have a special internal data structure. As long as you reassign the fields when you have extended the lists it's ok because then the tail pointers get updated, -but when you add to list without reassigning you might end up in troubles when -the linebreak routien kicks in. You can call this function to check the list for +but when you add to list without reassigning you might end up in trouble when +the linebreak routine kicks in. You can call this function to check the list for issues with disc nodes. \startfunctioncall @@ -1768,7 +2142,9 @@ node.check_discretionaries(<node> head) The plural variant runs over all disc nodes in a list, the singular variant checks one node only (it also checks if the node is a disc node). -\subsubsection{\type {node.flatten_discretionaries}} +\subsection{\type {flatten_discretionaries}} + +\libindex {flatten_discretionaries} This function will remove the discretionaries in the list and inject the replace field when set. @@ -1777,45 +2153,51 @@ field when set. <node> head, count = node.flatten_discretionaries(<node> n) \stopfunctioncall -\subsubsection{\type {node.family_font}} +\subsection{\type {family_font}} + +\libindex {family_font} -When you pass it a proper family identifier the next helper will return the font -currently associated with it. You can normally also access the font with the normal -font field or getter because it will resolve the family automatically for noads. +When you pass a proper family identifier the next helper will return the font +currently associated with it. You can normally also access the font with the +normal font field or getter because it will resolve the family automatically for +noads. \startfunctioncall <integer> id = node.family_font(<integer> fam) \stopfunctioncall -\subsubsection{\type {node.set_synctex_fields} and \type {node.get_synctex_fields}} - -You can set and query the synctex fields, a file number aka tag and a line -number, for a glue, kern, hlist, vlist, rule and math nodes as well as glyph -nodes (although this last one are not used in native synctex). +\stopsection -\startfunctioncall -node.set_synctex_fields(<integer> f, <integer> l) -<integer> f, <integer> l = - node.get_synctex_fields(<node> n) -\stopfunctioncall +\startsection[title={Two access models}][library=node] -Of course you need to know what you're doing as no checking on sane values takes -place. Also, the synctex interpreter used in editors is rather peculiar and has -some assumptions (heuristics). +\topicindex{nodes+direct} +\topicindex{direct nodes} -\section{Two access models} +\libindex {todirect} +\libindex {tonode} +\libindex {tostring} -Deep down in \TEX\ a node has a number which is an numeric entry in a memory +Deep down in \TEX\ a node has a number which is a numeric entry in a memory table. In fact, this model, where \TEX\ manages memory is real fast and one of the reasons why plugging in callbacks that operate on nodes is quite fast too. Each node gets a number that is in fact an index in the memory table and that -number often gets reported when you print node related information. +number often is reported when you print node related information. You go from +userdata nodes and there numeric references and back with: + +\startfunctioncall +<integer> d = node.todirect(<node> n)) +<node> n = node.tonode(<integer> d)) +\stopfunctioncall + +The userdata model is rather robust as it is a virtual interface with some +additional checking while the more direct access which uses the node numbers +directly. However, even with userdata you can get into troubles when you free +nodes that are no longer allocated or mess up lists. if you apply \type +{tostring} to a node you see its internal (direct) number and id. -There are two access models, a robust one using a so called user data object that -provides a virtual interface to the internal nodes, and a more direct access which -uses the node numbers directly. The first model provide key based access while -the second always accesses fields via functions: +The first model provides key based access while the second always accesses fields +via functions: \starttyping nodeobject.char @@ -1823,19 +2205,19 @@ getfield(nodenumber,"char") \stoptyping If you use the direct model, even if you know that you deal with numbers, you -should not depend on that property but treat it an abstraction just like +should not depend on that property but treat it as an abstraction just like traditional nodes. In fact, the fact that we use a simple basic datatype has the penalty that less checking can be done, but less checking is also the reason why it's somewhat faster. An important aspect is that one cannot mix both methods, but you can cast both models. So, multiplying a node number makes no sense. So our advice is: use the indexed (table) approach when possible and investigate -the direct one when speed might be an real issue. For that reason we also provide -the \type {get*} and \type {set*} functions in the top level node namespace. -There is a limited set of getters. When implementing this direct approach the -regular index by key variant was also optimized, so direct access only makes -sense when we're accessing nodes millions of times (which happens in some font -processing for instance). +the direct one when speed might be a real issue. For that reason \LUATEX\ also +provide the \type {get*} and \type {set*} functions in the top level node +namespace. There is a limited set of getters. When implementing this direct +approach the regular index by key variant was also optimized, so direct access +only makes sense when nodes are accessed millions of times (which happens in some +font processing for instance). We're talking mostly of getters because setters are less important. Documents have not that many content related nodes and setting many thousands of properties @@ -1873,8 +2255,10 @@ end Some accessors are used frequently and for these we provide more efficient helpers: \starttabulate[|l|p|] +\DB function \BC explanation \NC \NR +\TB \NC \type{getnext} \NC parsing nodelist always involves this one \NC \NR -\NC \type{getprev} \NC used less but is logical companion to \type {getnext} \NC \NR +\NC \type{getprev} \NC used less but a logical companion to \type {getnext} \NC \NR \NC \type{getboth} \NC returns the next and prev pointer of a node \NC \NR \NC \type{getid} \NC consulted a lot \NC \NR \NC \type{getsubtype} \NC consulted less but also a topper \NC \NR @@ -1883,13 +2267,16 @@ Some accessors are used frequently and for these we provide more efficient helpe \NC \type{getwhd} \NC returns the \type {width}, \type {height} and \type {depth} of a list, rule or (unexpanded) glyph as well as glue (its spec is looked at) and unset nodes\NC \NR \NC \type{getdisc} \NC returns the \type {pre}, \type {post} and \type {replace} fields and - optionally when true is passed also the tail fields. \NC \NR + optionally when true is passed also the tail fields \NC \NR \NC \type{getlist} \NC we often parse nested lists so this is a convenient one too \NC \NR \NC \type{getleader} \NC comparable to list, seldom used in \TEX\ (but needs frequent consulting like lists; leaders could have been made a dedicated node type) \NC \NR \NC \type{getfield} \NC generic getter, sufficient for the rest (other field names are often shared so a specific getter makes no sense then) \NC \NR \NC \type{getbox} \NC gets the given box (a list node) \NC \NR +\NC \type{getoffsets} \NC gets the \type {xoffset} and \type {yoffset} of a glyph or + \type {left} and \type {right} values of a rule \NC \NR +\LL \stoptabulate In the direct namespace there are more such helpers and most of them are @@ -1910,145 +2297,157 @@ directmode setter \type {setlink} takes a list of nodes and will link them, thereby ignoring \type {nil} entries. The first valid node is returned (beware: for good reason it assumes single nodes). For rarely used fields no helpers are provided and there are a few that probably are used seldom too but were added for -consistency. You can of course always define additional accessor using \type -{getfield} and \type {setfield} with little overhead. - -% \startcolumns[balance=yes] +consistency. You can of course always define additional accessors using \type +{getfield} and \type {setfield} with little overhead. When the second argument of +\type {setattributelist} is \type {true} the current attribute list is assumed. \def\yes{$+$} \def\nop{$-$} +\def\supported#1#2#3% + {\NC \type{#1} + \NC \ifx#2\yes\libidx{node} {#1}\fi #2 + \NC \ifx#3\yes\libidx{node.direct}{#1}\fi #3 \NC + \NR} + \starttabulate[|l|c|c|] -\HL -\BC function \BC node \BC direct \NC \NR -\HL -%NC \type {do_ligature_n} \NC \yes \NC \yes \NC \NR % was never documented and experimental -\NC \type {check_discretionaries}\NC \yes \NC \yes \NC \NR -\NC \type {copy_list} \NC \yes \NC \yes \NC \NR -\NC \type {copy} \NC \yes \NC \yes \NC \NR -\NC \type {count} \NC \yes \NC \yes \NC \NR -\NC \type {current_attr} \NC \yes \NC \yes \NC \NR -\NC \type {dimensions} \NC \yes \NC \yes \NC \NR -\NC \type {effective_glue} \NC \yes \NC \yes \NC \NR -\NC \type {end_of_math} \NC \yes \NC \yes \NC \NR -\NC \type {family_font} \NC \yes \NC \nop \NC \NR -\NC \type {fields} \NC \yes \NC \nop \NC \NR -\NC \type {find_attribute} \NC \yes \NC \yes \NC \NR -\NC \type {first_glyph} \NC \yes \NC \yes \NC \NR -\NC \type {flush_list} \NC \yes \NC \yes \NC \NR -\NC \type {flush_node} \NC \yes \NC \yes \NC \NR -\NC \type {free} \NC \yes \NC \yes \NC \NR -\NC \type {get_attribute} \NC \yes \NC \yes \NC \NR -\NC \type {getattributelist} \NC \nop \NC \yes \NC \NR -\NC \type {getboth} \NC \yes \NC \yes \NC \NR -\NC \type {getbox} \NC \nop \NC \yes \NC \NR -\NC \type {getchar} \NC \yes \NC \yes \NC \NR -\NC \type {getcomponents} \NC \nop \NC \yes \NC \NR -\NC \type {getdepth} \NC \nop \NC \yes \NC \NR -\NC \type {getdir} \NC \nop \NC \yes \NC \NR -\NC \type {getdisc} \NC \yes \NC \yes \NC \NR -\NC \type {getfam} \NC \nop \NC \yes \NC \NR -\NC \type {getfield} \NC \yes \NC \yes \NC \NR -\NC \type {getfont} \NC \yes \NC \yes \NC \NR -\NC \type {getglue} \NC \yes \NC \yes \NC \NR -\NC \type {getheight} \NC \nop \NC \yes \NC \NR -\NC \type {getid} \NC \yes \NC \yes \NC \NR -\NC \type {getkern} \NC \nop \NC \yes \NC \NR -\NC \type {getlang} \NC \nop \NC \yes \NC \NR -\NC \type {getleader} \NC \yes \NC \yes \NC \NR -\NC \type {getlist} \NC \yes \NC \yes \NC \NR -\NC \type {getnext} \NC \yes \NC \yes \NC \NR -\NC \type {getnucleus} \NC \nop \NC \yes \NC \NR -\NC \type {getoffsets} \NC \nop \NC \yes \NC \NR -\NC \type {getpenalty} \NC \nop \NC \yes \NC \NR -\NC \type {getprev} \NC \yes \NC \yes \NC \NR -\NC \type {getproperty} \NC \yes \NC \yes \NC \NR -\NC \type {getshift} \NC \nop \NC \yes \NC \NR -\NC \type {getwidth} \NC \nop \NC \yes \NC \NR -\NC \type {getwhd} \NC \nop \NC \yes \NC \NR -\NC \type {getsub} \NC \nop \NC \yes \NC \NR -\NC \type {getsubtype} \NC \yes \NC \yes \NC \NR -\NC \type {getsup} \NC \nop \NC \yes \NC \NR -\NC \type {has_attribute} \NC \yes \NC \yes \NC \NR -\NC \type {has_field} \NC \yes \NC \yes \NC \NR -\NC \type {has_glyph} \NC \yes \NC \yes \NC \NR -\NC \type {hpack} \NC \yes \NC \yes \NC \NR -\NC \type {id} \NC \yes \NC \nop \NC \NR -\NC \type {insert_after} \NC \yes \NC \yes \NC \NR -\NC \type {insert_before} \NC \yes \NC \yes \NC \NR -\NC \type {is_char} \NC \yes \NC \yes \NC \NR -\NC \type {is_direct} \NC \nop \NC \yes \NC \NR -\NC \type {is_glue_zero} \NC \yes \NC \yes \NC \NR -\NC \type {is_glyph} \NC \yes \NC \yes \NC \NR -\NC \type {is_node} \NC \yes \NC \yes \NC \NR -\NC \type {kerning} \NC \yes \NC \yes \NC \NR -\NC \type {last_node} \NC \yes \NC \yes \NC \NR -\NC \type {length} \NC \yes \NC \yes \NC \NR -\NC \type {ligaturing} \NC \yes \NC \yes \NC \NR -\NC \type {mlist_to_hlist} \NC \yes \NC \nop \NC \NR -\NC \type {new} \NC \yes \NC \yes \NC \NR -\NC \type {next} \NC \yes \NC \nop \NC \NR -\NC \type {prev} \NC \yes \NC \nop \NC \NR -\NC \type {protect_glyphs} \NC \yes \NC \yes \NC \NR -\NC \type {protect_glyph} \NC \yes \NC \yes \NC \NR -\NC \type {protrusion_skippable} \NC \yes \NC \yes \NC \NR -\NC \type {rangedimensions} \NC \yes \NC \yes \NC \NR -\NC \type {remove} \NC \yes \NC \yes \NC \NR -\NC \type {set_attribute} \NC \nop \NC \yes \NC \NR -\NC \type {setattributelist} \NC \nop \NC \yes \NC \NR -\NC \type {setboth} \NC \nop \NC \yes \NC \NR -\NC \type {setbox} \NC \nop \NC \yes \NC \NR -\NC \type {setchar} \NC \nop \NC \yes \NC \NR -\NC \type {setcomponents} \NC \nop \NC \yes \NC \NR -\NC \type {setdepth} \NC \nop \NC \yes \NC \NR -\NC \type {setdir} \NC \nop \NC \yes \NC \NR -\NC \type {setdisc} \NC \nop \NC \yes \NC \NR -\NC \type {setfield} \NC \yes \NC \yes \NC \NR -\NC \type {setfont} \NC \nop \NC \yes \NC \NR -\NC \type {setglue} \NC \yes \NC \yes \NC \NR -\NC \type {setheight} \NC \nop \NC \yes \NC \NR -\NC \type {setid} \NC \nop \NC \yes \NC \NR -\NC \type {setkern} \NC \nop \NC \yes \NC \NR -\NC \type {setlang} \NC \nop \NC \yes \NC \NR -\NC \type {setleader} \NC \nop \NC \yes \NC \NR -\NC \type {setlist} \NC \nop \NC \yes \NC \NR -\NC \type {setnext} \NC \nop \NC \yes \NC \NR -\NC \type {setnucleus} \NC \nop \NC \yes \NC \NR -\NC \type {setoffsets} \NC \nop \NC \yes \NC \NR -\NC \type {setpenalty} \NC \nop \NC \yes \NC \NR -\NC \type {setprev} \NC \nop \NC \yes \NC \NR -\NC \type {setproperty} \NC \nop \NC \yes \NC \NR -\NC \type {setshift} \NC \nop \NC \yes \NC \NR -\NC \type {setwidth} \NC \nop \NC \yes \NC \NR -\NC \type {setwhd} \NC \nop \NC \yes \NC \NR -\NC \type {setsub} \NC \nop \NC \yes \NC \NR -\NC \type {setsubtype} \NC \nop \NC \yes \NC \NR -\NC \type {setsup} \NC \nop \NC \yes \NC \NR -\NC \type {slide} \NC \yes \NC \yes \NC \NR -\NC \type {subtypes} \NC \yes \NC \nop \NC \NR -\NC \type {subtype} \NC \yes \NC \nop \NC \NR -\NC \type {tail} \NC \yes \NC \yes \NC \NR -\NC \type {todirect} \NC \yes \NC \yes \NC \NR -\NC \type {tonode} \NC \yes \NC \yes \NC \NR -\NC \type {tostring} \NC \yes \NC \yes \NC \NR -\NC \type {traverse_char} \NC \yes \NC \yes \NC \NR -\NC \type {traverse_id} \NC \yes \NC \yes \NC \NR -\NC \type {traverse} \NC \yes \NC \yes \NC \NR -\NC \type {types} \NC \yes \NC \nop \NC \NR -\NC \type {type} \NC \yes \NC \nop \NC \NR -\NC \type {unprotect_glyphs} \NC \yes \NC \yes \NC \NR -\NC \type {unset_attribute} \NC \yes \NC \yes \NC \NR -\NC \type {usedlist} \NC \yes \NC \yes \NC \NR -\NC \type {uses_font} \NC \yes \NC \yes \NC \NR -\NC \type {vpack} \NC \yes \NC \yes \NC \NR -\NC \type {whatsitsubtypes} \NC \yes \NC \nop \NC \NR -\NC \type {whatsits} \NC \yes \NC \nop \NC \NR -\NC \type {write} \NC \yes \NC \yes \NC \NR -\NC \type {set_synctex_fields} \NC \yes \NC \yes \NC \NR -\NC \type {get_synctex_fields} \NC \yes \NC \yes \NC \NR +\DB function \BC node \BC direct \NC \NR +\TB +\supported {check_discretionaries} \yes \yes +\supported {check_discretionary} \yes \yes +\supported {copy_list} \yes \yes +\supported {copy} \yes \yes +\supported {count} \yes \yes +\supported {current_attr} \yes \yes +\supported {dimensions} \yes \yes +\supported {effective_glue} \yes \yes +\supported {end_of_math} \yes \yes +\supported {family_font} \yes \nop +\supported {fields} \yes \nop +\supported {find_attribute} \yes \yes +\supported {first_glyph} \yes \yes +\supported {flatten_discretionaries} \yes \yes +\supported {flush_list} \yes \yes +\supported {flush_node} \yes \yes +\supported {free} \yes \yes +\supported {get_attribute} \yes \yes +\supported {get_synctex_fields} \nop \yes +\supported {getattributelist} \nop \yes +\supported {getboth} \yes \yes +\supported {getbox} \nop \yes +\supported {getchar} \yes \yes +\supported {getcomponents} \nop \yes +\supported {getdepth} \nop \yes +\supported {getdirection} \nop \yes +\supported {getdir} \nop \yes +\supported {getdisc} \yes \yes +\supported {getfam} \nop \yes +\supported {getfield} \yes \yes +\supported {getfont} \yes \yes +\supported {getglue} \yes \yes +\supported {getheight} \nop \yes +\supported {getid} \yes \yes +\supported {getkern} \nop \yes +\supported {getlang} \nop \yes +\supported {getleader} \yes \yes +\supported {getlist} \yes \yes +\supported {getnext} \yes \yes +\supported {getnucleus} \nop \yes +\supported {getoffsets} \nop \yes +\supported {getpenalty} \nop \yes +\supported {getprev} \yes \yes +\supported {getproperty} \yes \yes +\supported {getshift} \nop \yes +\supported {getsubtype} \yes \yes +\supported {getsub} \nop \yes +\supported {getsup} \nop \yes +\supported {getdata} \nop \yes +\supported {getwhd} \yes \yes +\supported {getwidth} \nop \yes +\supported {has_attribute} \yes \yes +\supported {has_field} \yes \yes +\supported {has_glyph} \yes \yes +\supported {hpack} \yes \yes +\supported {id} \yes \nop +\supported {insert_after} \yes \yes +\supported {insert_before} \yes \yes +\supported {is_char} \yes \yes +\supported {is_direct} \nop \yes +\supported {is_glyph} \yes \yes +\supported {is_node} \yes \yes +\supported {is_zero_glue} \yes \yes +\supported {kerning} \yes \yes +\supported {last_node} \yes \yes +\supported {length} \yes \yes +\supported {ligaturing} \yes \yes +\supported {mlist_to_hlist} \yes \nop +\supported {new} \yes \yes +\supported {next} \yes \nop +\supported {prepend_prevdepth} \nop \yes +\supported {prev} \yes \nop +\supported {protect_glyphs} \yes \yes +\supported {protect_glyph} \yes \yes +\supported {protrusion_skippable} \yes \yes +\supported {rangedimensions} \yes \yes +\supported {remove} \yes \yes +\supported {set_attribute} \yes \yes +\supported {set_synctex_fields} \nop \yes +\supported {setattributelist} \nop \yes +\supported {setboth} \nop \yes +\supported {setbox} \nop \yes +\supported {setchar} \nop \yes +\supported {setcomponents} \nop \yes +\supported {setdepth} \nop \yes +\supported {setdirection} \nop \yes +\supported {setdir} \nop \yes +\supported {setdisc} \nop \yes +\supported {setfam} \nop \yes +\supported {setfield} \yes \yes +\supported {setfont} \nop \yes +\supported {setexpansion} \nop \yes +\supported {setglue} \yes \yes +\supported {setheight} \nop \yes +\supported {setkern} \nop \yes +\supported {setlang} \nop \yes +\supported {setleader} \nop \yes +\supported {setlink} \nop \yes +\supported {setlist} \nop \yes +\supported {setnext} \nop \yes +\supported {setnucleus} \nop \yes +\supported {setoffsets} \nop \yes +\supported {setpenalty} \nop \yes +\supported {setprev} \nop \yes +\supported {setproperty} \yes \yes +\supported {setshift} \nop \yes +\supported {setsplit} \nop \yes +\supported {setsubtype} \nop \yes +\supported {setsub} \nop \yes +\supported {setsup} \nop \yes +\supported {setwhd} \nop \yes +\supported {setwidth} \nop \yes +\supported {slide} \yes \yes +\supported {subtypes} \yes \nop +\supported {subtype} \yes \nop +\supported {tail} \yes \yes +\supported {todirect} \yes \yes +\supported {tonode} \yes \yes +\supported {tostring} \yes \yes +\supported {traverse_char} \yes \yes +\supported {traverse_glyph} \yes \yes +\supported {traverse_id} \yes \yes +\supported {traverse} \yes \yes +\supported {types} \yes \nop +\supported {type} \yes \nop +\supported {unprotect_glyphs} \yes \yes +\supported {unprotect_glyph} \yes \yes +\supported {unset_attribute} \yes \yes +\supported {usedlist} \yes \yes +\supported {uses_font} \yes \yes +\supported {vpack} \yes \yes +\supported {whatsits} \yes \nop +\supported {write} \yes \yes +\LL \stoptabulate -% \stopcolumns - The \type {node.next} and \type {node.prev} functions will stay but for consistency there are variants called \type {getnext} and \type {getprev}. We had to use \type {get} because \type {node.id} and \type {node.subtype} are already @@ -2056,6 +2455,13 @@ taken for providing meta information about nodes. Note: The getters do only basi checking for valid keys. You should just stick to the keys mentioned in the sections that describe node properties. +Some of the getters and setters handle multiple node types, given that the field +is relevant. In that case, some field names are considered similar (like \type +{kern} and \type {width}, or \type {data} and \type {value}. In retrospect we +could have normalized field names better but we decided to stick to the original +(internal) names as much as possible. After all, at the \LUA\ end one can easily +create synonyms. + Some nodes have indirect references. For instance a math character refers to a family instead of a font. In that case we provide a virtual font field as accessor. So, \type {getfont} and \type {.font} can be used on them. The same is @@ -2063,6 +2469,196 @@ true for the \type {width}, \type {height} and \type {depth} of glue nodes. Thes actually access the spec node properties, and here we can set as well as get the values. +In some places \LUATEX\ can do a bit of extra checking for valid node lists and +you can enable that with: + +\startfunctioncall +node.fix_node_lists(<boolean> b) +\stopfunctioncall + +You can set and query the \SYNCTEX\ fields, a file number aka tag and a line +number, for a glue, kern, hlist, vlist, rule and math nodes as well as glyph +nodes (although this last one is not used in native \SYNCTEX). + +\startfunctioncall +node.set_synctex_fields(<integer> f, <integer> l) +<integer> f, <integer> l = + node.get_synctex_fields(<node> n) +\stopfunctioncall + +Of course you need to know what you're doing as no checking on sane values takes +place. Also, the synctex interpreter used in editors is rather peculiar and has +some assumptions (heuristics). + +\stopsection + +\startsection[title={Properties}][library=node] + +\topicindex {nodes+properties} +\topicindex {properties} + +\libindex{flush_properties_table} +\libindex{get_properties_table} +\libindex{set_properties_mode} + +Attributes are a convenient way to relate extra information to a node. You can +assign them at the \TEX\ end as well as at the \LUA\ end and and consult them at +the \LUA\ end. One big advantage is that they obey grouping. They are linked +lists and normally checking for them is pretty efficient, even if you use a lot +of them. A macro package has to provide some way to manage these attributes at the +\TEX\ end because otherwise clashes in their usage can occur. + +Each node also can have a properties table and you can assign values to this +table using the \type {setproperty} function and get properties using the \type +{getproperty} function. Managing properties is way more demanding than managing +attributes. + +Take the following example: + +\starttyping +\directlua { + local n = node.new("glyph") + + node.setproperty(n,"foo") + print(node.getproperty(n)) + + node.setproperty(n,"bar") + print(node.getproperty(n)) + + node.free(n) +} +\stoptyping + +This will print \type {foo} and \type {bar} which in itself is not that useful +when multiple mechanisms want to use this feature. A variant is: + +\starttyping +\directlua { + local n = node.new("glyph") + + node.setproperty(n,{ one = "foo", two = "bar" }) + print(node.getproperty(n).one) + print(node.getproperty(n).two) + + node.free(n) +} +\stoptyping + +This time we store two properties with the node. It really makes sense to have a +table as property because that way we can store more. But in order for that to +work well you need to do it this way: + +\starttyping +\directlua { + local n = node.new("glyph") + + local t = node.getproperty(n) + + if not t then + t = { } + node.setproperty(n,t) + end + t.one = "foo" + t.two = "bar" + + print(node.getproperty(n).one) + print(node.getproperty(n).two) + + node.free(n) +} +\stoptyping + +Here our own properties will not overwrite other users properties unless of +course they use the same keys. So, eventually you will end up with something: + +\starttyping +\directlua { + local n = node.new("glyph") + + local t = node.getproperty(n) + + if not t then + t = { } + node.setproperty(n,t) + end + t.myself = { one = "foo", two = "bar" } + + print(node.getproperty(n).myself.one) + print(node.getproperty(n).myself.two) + + node.free(n) +} +\stoptyping + +This assumes that only you use \type {myself} as subtable. The possibilities are +endless but care is needed. For instance, the generic font handler that ships +with \CONTEXT\ uses the \type {injections} subtable and you should not mess with +that one! + +There are a few helper functions that you normally should not touch as user: \typ +{flush_properties_table} will wipe the table (normally a bad idea), \typ +{get_properties_table} and will give the table that stores properties (using +direct entries) and you can best not mess too much with that one either because +\LUATEX\ itself will make sure that entries related to nodes will get wiped when +nodes get freed, so that the \LUA\ garbage collector can do its job. In fact, the +main reason why we have this mechanism is that it saves the user (or macro +package) some work. One can easily write a property mechanism in \LUA\ where +after a shipout properties gets cleaned up but it's not entirely trivial to make +sure that with each freed node also its properties get freed, due to the fact +that there can be nodes left over for a next page. And having a callback bound to +the node deallocator would add way to much overhead. + +Managing properties in the node (de)allocator functions is disabled by default +and is enabled by: + +\starttyping +node.set_properties_mode(true) +\stoptyping + +When we copy a node list that has a table as property, there are several possibilities: we do the same as +a new node, we copy the entry to the table in properties (a reference), we do +a deep copy of a table in the properties, we create a new table and give it +the original one as a metatable. After some experiments (that also included +timing) with these scenarios we decided that a deep copy made no sense, nor +did nilling. In the end both the shallow copy and the metatable variant were +both ok, although the second one is slower. The most important aspect to keep +in mind is that references to other nodes in properties no longer can be +valid for that copy. We could use two tables (one unique and one shared) or +metatables but that only complicates matters. + +When defining a new node, we could already allocate a table but it is rather easy +to do that at the lua end e.g.\ using a metatable \type {__index} method. That +way it is under macro package control. When deleting a node, we could keep the +slot (e.g. setting it to false) but it could make memory consumption raise +unneeded when we have temporary large node lists and after that only small lists. +Both are not done. + +So in the end this is what happens now: when a node is copied, and it has a table +as property, the new node will share that table. If the second argument of \typ +{set_properties_mode} is \type {true} then a metatable approach is chosen: the +copy gets its own table with the original table as metatable. If you use the +generic font loader the mode is enabled that way. + +A few more xperiments were done. For instance: copy attributes to the properties +so that we have fast access at the \LUA\ end. In the end the overhead is not +compensated by speed and convenience, in fact, attributes are not that slow when +it comes to accessing them. So this was rejected. + +Another experiment concerned a bitset in the node but again the gain compared to +attributes was neglectable and given the small amount of available bits it also +demands a pretty strong agreement over what bit represents what, and this is +unlikely to succeed in the \TEX\ community. It doesn't pay off. + +Just in case one wonders why properties make sense: it is not so much speed that +we gain, but more convenience: storing all kind of (temporary) data in attributes +is no fun and this mechanism makes sure that properties are cleaned up when a +node is freed. Also, the advantage of a more or less global properties table is +that we stay at the \LUA\ end. An alternative is to store a reference in the node +itself but that is complicated by the fact that the register has some limitations +(no numeric keys) and we also don't want to mess with it too much. + +\stopsection + \stopchapter \stopcomponent diff --git a/doc/context/sources/general/manuals/luatex/luatex-preamble.tex b/doc/context/sources/general/manuals/luatex/luatex-preamble.tex new file mode 100644 index 000000000..829317977 --- /dev/null +++ b/doc/context/sources/general/manuals/luatex/luatex-preamble.tex @@ -0,0 +1,110 @@ +% language=uk + +\environment luatex-style + +\startcomponent luatex-preamble + +\startchapter[reference=preamble,title={Preamble}] + +\topicindex{nodes} +\topicindex{boxes} +\topicindex{\LUA} + +This is a reference manual, not a tutorial. This means that we discuss changes +relative to traditonal \TEX\ and also present new functionality. As a consequence +we will refer to concepts that we assume to be known or that might be explained +later. + +The average user doesn't need to know much about what is in this manual. For +instance fonts and languages are normally dealt with in the macro package that +you use. Messing around with node lists is also often not really needed at the +user level. If you do mess around, you'd better know what you're dealing with. +Reading \quotation {The \TEX\ Book} by Donald Knuth is a good investment of time +then also because it's good to know where it all started. A more summarizing +overview is given by \quotation {\TEX\ by Topic} by Victor Eijkhout. You might +want to peek in \quotation {The \ETEX\ manual} and documentation about \PDFTEX. + +But \unknown\ if you're here because of \LUA, then all you need to know is that +you can call it from within a run. The macro package that you use probably will +provide a few wrapper mechanisms but the basic \lpr {directlua} command that +does the job is: + +\starttyping +\directlua{tex.print("Hi there")} +\stoptyping + +You can put code between curly braces but if it's a lot you can also put it in a +file and load that file with the usual \LUA\ commands. + +If you still decide to read on, then it's good to know what nodes are, so we do a +quick introduction here. If you input this text: + +\starttyping +Hi There +\stoptyping + +eventually we will get a linked lists of nodes, which in \ASCII\ art looks like: + +\starttyping +H <=> i <=> [glue] <=> T <=> h <=> e <=> r <=> e +\stoptyping + +When we have a paragraph, we actually get something: + +\starttyping +[localpar] <=> H <=> i <=> [glue] <=> T <=> h <=> e <=> r <=> e <=> [glue] +\stoptyping + +Each character becomes a so called glyph node, a record with properties like the +current font, the character code and the current language. Spaces become glue +nodes. There are many node types that we will discuss later. Each node points +back to a previous node or next node, given that these exist. + +It's also good to know beforehand that \TEX\ is basically centered around +creating paragraphs and pages. The par builder takes a list and breaks it into +lines. We turn horizontal material into vertical. Lines are so called boxes and +can be separated by glue, penalties and more. The page builder accumulates lines +and when feasible triggers an output routine that will take the list so far. +Constructing the actual page is not part of \TEX\ but done using primitives that +permit manipulation of boxes. The result is handled back to \TEX\ and flushed to +a (often \PDF) file. + +The \LUATEX\ engine provides hooks for \LUA\ code at nearly every reasonable +point in the process: collecting content, hyphenating, applying font features, +breaking into lines, etc. This means that you can overload \TEX's natural +behaviour, which still is the benchmark. When we refer to \quote {callbacks} we +means these hooks. + +Where plain \TEX\ is basically a basic framework for writing a specific style, +macro packages like \CONTEXT\ and \LATEX\ provide the user a whole lot of +additional tools to make documents look good. They hide the dirty details of font +management, language demands, turning structure into typeset results, wrapping +pages, including images, and so on. You should be aware of the fact that when you +hook in your own code to manipulate lists, this can interfere with the macro +package that you use. + +When you read about nodes in the following chapters it's good to keep in mind their +commands that relate to then. Here are a few: + +\starttabulate[|l|l|p|] +\DB command \BC node \BC explanation \NC \NR +\TB +\NC \prm {hbox} \NC \nod {hlist} \NC horizontal box \NC \NR +\NC \prm {vbox} \NC \nod {vlist} \NC vertical box with the baseline at the bottom \NC \NR +\NC \prm {vtop} \NC \nod {vlist} \NC vertical box with the baseline at the top \NC \NR +\NC \prm {hskip} \NC \nod {glue} \NC horizontal skip with optional stretch and shrink \NC \NR +\NC \prm {vskip} \NC \nod {glue} \NC vertical skip with optional stretch and shrink \NC \NR +\NC \prm {kern} \NC \nod {kern} \NC horizontal or vertical fixed skip \NC \NR +\NC \prm {discretionary} \NC \nod {disc} \NC hyphenation point (pre, post, replace) \NC \NR +\NC \prm {char} \NC \nod {glyph} \NC a character \NC \NR +\NC \prm {hrule} \NC \nod {rule} \NC a horizontal rule \NC \NR +\NC \prm {vrule} \NC \nod {rule} \NC a vertical rule \NC \NR +\NC \prm {textdir(ection)} \NC \nod {dir} \NC a change in text direction \NC \NR +\LL +\stoptabulate + +For now this should be enough to enable you to understand the next chapters. + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/luatex/luatex-registers.tex b/doc/context/sources/general/manuals/luatex/luatex-registers.tex new file mode 100644 index 000000000..36b1ec051 --- /dev/null +++ b/doc/context/sources/general/manuals/luatex/luatex-registers.tex @@ -0,0 +1,47 @@ +\environment luatex-style + +\startcomponent luatex-registers + +\startchapter[title=Topics] + + \placeregister[topicindex] + +\stopchapter + +\startchapter[title=Primitives] + + This register contains the primitives that are mentioned in the manual. There + are of course many more primitives. The \LUATEX\ primitives are typeset in + bold. The primitives from \PDFTEX\ are not supported that way but mentioned + anyway. + + \placeregister[primitiveindex][indicator=no] + +\stopchapter + +\startchapter[title=Callbacks] + + \placeregister[callbackindex] + +\stopchapter + +\startchapter[title=Nodes] + + This register contains the nodes that are known to \LUATEX. The primary nodes + are in bold, whatsits that are determined by their subtype are normal. The + names prefixed by \type {pdf_} are backend specific. + + \placeregister[nodeindex] + +\stopchapter + +\startchapter[title=Libraries] + + This register contains the functions available in libraries. Not all functions + are documented, for instance because they can be experimental or obsolete. + + \placeregister[libraryindex] + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/luatex/luatex-statistics.tex b/doc/context/sources/general/manuals/luatex/luatex-statistics.tex new file mode 100644 index 000000000..efd7f1c75 --- /dev/null +++ b/doc/context/sources/general/manuals/luatex/luatex-statistics.tex @@ -0,0 +1,17 @@ +% language=uk + +\environment luatex-style + +\startcomponent luatex-statistics + +\startchapter[title={Statistics}] + + \topicindex{fonts+used} + + The following fonts are used in this document: + + \showfontusage + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/luatex/luatex-style.tex b/doc/context/sources/general/manuals/luatex/luatex-style.tex index a277a1178..708f83ed6 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-style.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-style.tex @@ -2,7 +2,9 @@ % todo: use \useMPlibrary[lua] -\usemodule[abr-02] +\enabletrackers[fonts.usage] + +\usemodule[fonts-statistics] \setuplayout [height=middle, @@ -31,9 +33,31 @@ [each] [packed] +\definesymbol[1][\Uchar"2023] +\definesymbol[2][\endash] +\definesymbol[3][\wait] % we want to catch it + +\setupitemize + [each] + [headcolor=maincolor, + symbolcolor=maincolor, + color=maincolor] + \setupwhitespace [medium] +\setuptabulate + [blank={small,samepage}, + headstyle=bold, + rulecolor=maincolor, + rulethickness=1pt, + foregroundcolor=white, + foregroundstyle=\ss\bfx\WORD, + backgroundcolor=maincolor] + +\setupcaptions + [headcolor=darkblue] + \startluacode local skipped = table.tohash { 'id', 'subtype', 'next', 'prev' } @@ -89,51 +113,13 @@ \definecolor[keptcolor] [b=.5] \definecolor[othercolor][r=.5,g=.5] -\usebodyfont[lucidaot] -\usebodyfont[pagella] -\usebodyfont[cambria] -%usebodyfont[dejavu] -\usebodyfont[modern] % we need this in examples so we predefine - -% \doifmodeelse {atpragma} { -% -% % \setupbodyfont -% % [lucidaot,10pt] -% -% \setupbodyfont -% [dejavu,10pt] -% -% \setuphead [chapter] [style=\bfd] -% \setuphead [section] [style=\bfb] -% \setuphead [subsection] [style=\bfa] -% \setuphead [subsubsection][style=\bf] -% -% } { -% -% \definetypeface[mainfacenormal] [ss][sans] [iwona] [default] -% \definetypeface[mainfacenormal] [rm][serif][palatino] [default] -% \definetypeface[mainfacenormal] [tt][mono] [modern] [default][rscale=1.1] -% \definetypeface[mainfacenormal] [mm][math] [iwona] [default] -% -% \definetypeface[mainfacemedium] [ss][sans] [iwona-medium][default] -% \definetypeface[mainfacemedium] [rm][serif][palatino] [default] -% \definetypeface[mainfacemedium] [tt][mono] [modern] [default][rscale=1.1] -% \definetypeface[mainfacemedium] [mm][math] [iwona-medium][default] -% -% \setupbodyfont -% [mainfacenormal,10pt] -% -% \setuphead [chapter] [style=\mainfacemedium\bfd] -% \setuphead [section] [style=\mainfacemedium\bfb] -% \setuphead [subsection] [style=\mainfacemedium\bfa] -% \setuphead [subsubsection][style=\mainfacemedium\bf] -% -% } - -\writestatus{luatex manual}{we assume that dejavu math is available} - -\setupbodyfont % assumes dejavu-math - [dejavu,10pt] +\writestatus{luatex manual}{} +\writestatus{luatex manual}{defining lucodaot} \usebodyfont [lucidaot] +\writestatus{luatex manual}{defining pagella} \usebodyfont [pagella] +\writestatus{luatex manual}{defining cambria} \usebodyfont [cambria] +\writestatus{luatex manual}{defining modern} \usebodyfont [modern] +\writestatus{luatex manual}{defining dejavu} \setupbodyfont[dejavu,10pt] +\writestatus{luatex manual}{} \setuphead [chapter] [align={flushleft,broad},style=\bfd] \setuphead [section] [align={flushleft,broad},style=\bfb] @@ -145,6 +131,9 @@ \setuphead [subsection] [color=maincolor] \setuphead [subsubsection][color=maincolor] +\setupfloats + [ntop=4] + \definehead [remark] [subsubsubject] @@ -152,6 +141,10 @@ \setupheadertexts [] +% \setuplayout +% [style=bold, +% color=maincolor] + \definemixedcolumns [twocolumns] [n=2, @@ -207,7 +200,7 @@ fill fullcircle scaled r shifted (d-1/8,d-1/8) withcolor luaholecolor ; luaorbitfactor := .25 ; - ) enddef ; + ) enddef ; \stopMPdefinitions @@ -217,8 +210,15 @@ fill Page withcolor \MPcolor{othercolor} ; luaorbitfactor := 1 ; - picture p ; p := lualogo xsized (3PaperWidth/5) ; - draw p shifted center Page shifted (0,-.85ypart center ulcorner p) ; + + picture p ; p := lualogo ysized (5*\measure{paperheight}/10) ; + draw p + shifted - center p + shifted ( + \measure{spreadwidth} - .5*\measure{paperwidth} + \measure{spinewidth}, + .375*\measure{paperheight} + ) + ; StopPage ; \stopuseMPgraphic @@ -232,7 +232,7 @@ \startuseMPgraphic{luanumber} % luaextraangle := \luaextraangle; - luaextraangle := if (LastPageNumber == 0) : 0 else : (RealPageNumber / LastPageNumber) * 360 fi; + luaextraangle := if (LastPageNumber < 10) : 10 else : (RealPageNumber / LastPageNumber) * 360 fi; luaorbitfactor := 0.25 ; picture p ; p := lualogo ; setbounds p to boundingbox fullcircle ; @@ -252,40 +252,46 @@ [rightpage] [background=page] +\definemeasure[banneroffset][\bottomspace-\footerheight-\footerdistance+2cm] + \startsetups pagenumber:right \setlayerframed [page] - [preset=rightbottom,offset=1cm] + [preset=rightbottom,x=1.0cm,y=\measure{banneroffset}] [frame=off,height=1cm,offset=overlay] - {\useMPgraphic{luanumber}} + {\strut\useMPgraphic{luanumber}} \setlayerframed [page] - [preset=rightbottom,offset=1cm,x=1.5cm] - [frame=off,height=1cm,width=1cm,offset=overlay] - {\pagenumber} + [preset=rightbottom,x=2.5cm,y=\measure{banneroffset}] + [frame=off,height=1cm,width=1cm,offset=overlay, + foregroundstyle=bold,foregroundcolor=maincolor] + {\strut\pagenumber} \setlayerframed [page] - [preset=rightbottom,offset=1cm,x=2.5cm] - [frame=off,height=1cm,offset=overlay] - {\getmarking[chapter]} + [preset=rightbottom,x=3.5cm,y=\measure{banneroffset}] + [frame=off,height=1cm,offset=overlay, + foregroundstyle=bold,foregroundcolor=maincolor] + {\strut\getmarking[chapter]} \stopsetups \startsetups pagenumber:left \setlayerframed [page] - [preset=leftbottom,offset=1cm,x=2.5cm] - [frame=off,height=1cm,offset=overlay] - {\getmarking[chapter]} + [preset=leftbottom,x=3.5cm,y=\measure{banneroffset}] + [frame=off,height=1cm,offset=overlay, + foregroundstyle=bold,foregroundcolor=maincolor] + {\strut\getmarking[chapter]} \setlayerframed [page] - [preset=leftbottom,offset=1cm,x=1.5cm] - [frame=off,height=1cm,width=1cm,offset=overlay] - {\pagenumber} + [preset=leftbottom,x=2.5cm,y=\measure{banneroffset}] + [frame=off,height=1cm,width=1cm,offset=overlay, + foregroundstyle=bold,foregroundcolor=maincolor] + {\strut\pagenumber} \setlayerframed [page] - [preset=leftbottom,offset=1cm] + [preset=leftbottom,x=1.0cm,y=\measure{banneroffset}] [frame=off,height=1cm,offset=overlay] - {\useMPgraphic{luanumber}} + {\strut\useMPgraphic{luanumber}} \stopsetups \unexpanded\def\nonterminal#1>{\mathematics{\langle\hbox{\rm #1}\rangle}} @@ -311,13 +317,15 @@ {\par \tt \startnarrower - \maincolor #1 + % \maincolor + #1 \stopnarrower \par} \unexpanded\def\syntaxbody#1% {\begingroup - \maincolor \tt #1% + % \maincolor + \tt #1% \endgroup} \bgroup \catcodetable\syntaxcodetable @@ -329,7 +337,7 @@ \definetyping [texsyntax] - [color=maincolor] +% [color=maincolor] % end of wave @@ -351,9 +359,14 @@ \setuplist [chapter] [style=bold, + before={\testpage[4]\blank}, color=keptcolor] \setuplist + [section] + [before={\testpage[3]}] + +\setuplist [subsection,subsubsection] [margin=3em, width=5em] @@ -373,4 +386,57 @@ % \setupinteractionscreen % [option=bookmark] +\startbuffer[stylecalculations] + + \normalexpanded{\definemeasure[spinewidth] [0pt]} + \normalexpanded{\definemeasure[paperwidth] [\the\paperwidth ]} + \normalexpanded{\definemeasure[paperheight][\the\paperheight]} + \normalexpanded{\definemeasure[spreadwidth][\measure{paperwidth}]} + +\stopbuffer + +\getbuffer[stylecalculations] + +\dontcomplain + +\environment luatex-logos + +\defineregister[topicindex] +\defineregister[primitiveindex] +\defineregister[callbackindex] +\defineregister[nodeindex] +\defineregister[libraryindex] + +\unexpanded\def\lpr#1{\doifmode{*bodypart}{\primitiveindex[#1]{\bf\tex {#1}}}\tex {#1}} +\unexpanded\def\prm#1{\doifmode{*bodypart}{\primitiveindex[#1]{\tex {#1}}}\tex {#1}} +\unexpanded\def\orm#1{\doifmode{*bodypart}{\primitiveindex[#1]{\tex {#1}}}\tex {#1}} +\unexpanded\def\cbk#1{\doifmode{*bodypart}{\callbackindex [#1]{\type {#1}}}\type{#1}} +\unexpanded\def\nod#1{\doifmode{*bodypart}{\nodeindex [#1]{\bf\type{#1}}}\type{#1}} +\unexpanded\def\whs#1{\doifmode{*bodypart}{\nodeindex [#1]{\type {#1}}}\type{#1}} +\unexpanded\def\noa#1{\doifmode{*bodypart}{\nodeindex [#1]{\type {#1}}}\type{#1}} + +\hyphenation{sub-nodes} + +\def\currentlibraryindex{\namedstructureuservariable{section}{library}} + +\setupregister + [libraryindex] + [indicator=no,before=] + +\setupregister + [libraryindex] + [1] + [textstyle=\ttbf] + +\setupregister + [libraryindex] + [2] + [textstyle=\tttf] + +\unexpanded\def\lib #1{\doifmode{*bodypart}{\expanded{\libraryindex{\currentlibraryindex+#1}}\type{\currentlibraryindex.#1}}} +\unexpanded\def\libindex#1{\doifmode{*bodypart}{\expanded{\libraryindex{\currentlibraryindex+#1}}}} +\unexpanded\def\libidx#1#2{\doifmode{*bodypart}{\expanded{\libraryindex{#1+#2}}\type{#1.#2}}} + +% \setstructurepageregister[][keys:1=,entries:1=] + \stopenvironment diff --git a/doc/context/sources/general/manuals/luatex/luatex-titlepage.tex b/doc/context/sources/general/manuals/luatex/luatex-titlepage.tex index 307741ee1..d9ca4b3f9 100644 --- a/doc/context/sources/general/manuals/luatex/luatex-titlepage.tex +++ b/doc/context/sources/general/manuals/luatex/luatex-titlepage.tex @@ -1,5 +1,4 @@ \environment luatex-style -\environment luatex-logos \startcomponent luatex-titlepage @@ -8,7 +7,7 @@ \switchtobodyfont [mainfacemedium] - \definedfont[Bold*default at \the\dimexpr.08\paperheight\relax] \setupinterlinespace + \definedfont[Bold*default at \the\dimexpr.06\paperheight\relax] \setupinterlinespace \setlayer [page] @@ -16,55 +15,39 @@ \setlayerframed [page] - [preset=middletop, - voffset=.05\paperheight] + [preset=righttop, + location=middletop, + hoffset=.500\measured{paperwidth}, + voffset=.175\measured{paperheight}] [align=middle, - foregroundcolor=blue, + foregroundcolor=white, frame=off] - {Lua\TeX\\Reference Manual} + {\documentvariable{manual}\crlf Reference\crlf Manual} - \definedfont[Bold*default at 18pt] \setupinterlinespace + \definedfont[Bold*default at 14pt] \setupinterlinespace \setlayerframed [page] [preset=rightbottom, - offset=.01\paperheight] + offset=.025\measured{paperheight}] [align=flushright, - foregroundcolor=blue, + foregroundcolor=white, frame=off] {\doifsomething{\documentvariable{status}}{\documentvariable{status}\par} \currentdate[month,space,year]\par Version \documentvariable{version}} -\stopstandardmakeup - -\startstandardmakeup - - \start - \raggedleft - \definedfont[Bold*default at 48pt] - \setupinterlinespace - \blue Lua\TeX \endgraf Reference \endgraf Manual \endgraf - \stop - - \vfill - - \definedfont[Bold*default at 12pt] - - \starttabulate[|l|l|] - \NC copyright \EQ Lua\TeX\ development team \NC \NR - \NC more info \EQ www.luatex.org \NC \NR - \NC version \EQ \currentdate \doifsomething{\documentvariable{snapshot}}{(snapshot \documentvariable{snapshot})} \NC \NR - \stoptabulate + \setlayerframed + [page] + [preset=middle, + hoffset=-.5\dimexpr\measured{paperwidth}-\measured{spinewidth}\relax] + [width=.7\measured{paperwidth}, + align=normal, + foregroundstyle=\bf, + foregroundcolor=white, + frame=off] + {\getbuffer[backpage]} \stopstandardmakeup -\setupbackgrounds - [leftpage] - [setups=pagenumber:left] - -\setupbackgrounds - [rightpage] - [setups=pagenumber:right] - \stopcomponent diff --git a/doc/context/sources/general/manuals/luatex/luatex.tex b/doc/context/sources/general/manuals/luatex/luatex.tex index 3025788bf..1486abd49 100644 --- a/doc/context/sources/general/manuals/luatex/luatex.tex +++ b/doc/context/sources/general/manuals/luatex/luatex.tex @@ -1,11 +1,12 @@ % macros=mkvi +% \nopdfcompression + % \disabledirectives[vspacing.synchronizepage] % wsl /data/context/runcontext.sh --global --path=/mnt/c/data/develop/context/manuals/mkiv/external/luatex luatex.tex % % same runtime as regular context or linux - % author : Hans Hagen with Taco Hoekwater, Luigi Scarso & Hartmut Henkel % copyright : PRAGMA ADE & ConTeXt Development Team % license : Creative Commons Attribution ShareAlike 4.0 International @@ -22,10 +23,6 @@ % comment : Some (parts of) chapters might have been published in TugBoat, the NTG Maps, the % ConTeXt Group journal or otherwise. Thanks to the editors for corrections. Also % thanks to users for testing, feedback and corrections. -% -% 238 pages : 2017-07-06 -% luatex 9.5 sec / luajittex 7.0 sec -% Dell 7600 / i7 3840QM / passmark 1.922 / Windows 10 64 bit % \tex vs \type vs \syntax vs. \luatex % \em \it \/ @@ -33,12 +30,28 @@ % "context --nodates --nocompression luatex" can be used for comparison % runs, not that we do it +% todo: all (sub)section to start/stop + % \enabledirectives[hyphenator.flatten] % \setupsynctex[state=start,method=max] % adds 5 pct overhead +% compoundhyphenmode : looks okay +% endlocalcontrol : very experimental +% fixupboxesmode : very experimental +% mathflattenmode : looks okay +% mathrulethicknessmode : looks okay + +% after adding primitives: context mult-prm.mkiv + \environment luatex-style + +\startmode[book] + \environment luatex-style-book +\stopmode + \environment luatex-logos +\environment luatex-private \startmode[export] @@ -47,14 +60,14 @@ \stopmode -\dontcomplain - \startdocument - [status=experimental, - version=1.08.0] + [manual=Lua\TeX, + status=experimental, + version=1.10] \startnotmode[*export] \component luatex-titlepage + \component luatex-firstpage \stopnotmode \startmode[*export] @@ -67,6 +80,7 @@ \stopfrontmatter \startbodymatter + \component luatex-preamble \component luatex-enhancements \component luatex-modifications \component luatex-lua @@ -81,4 +95,9 @@ \component luatex-backend \stopbodymatter +\startbackmatter + \component luatex-registers + \component luatex-statistics +\stopbackmatter + \stopdocument |