1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
|
% language=uk
\environment luametatex-style
\startcomponent luametatex-preamble
\startchapter[reference=preamble,title={The internals}]
\topicindex{nodes}
\topicindex{boxes}
\topicindex{\LUA}
This is a reference manual, not a tutorial. This means that we discuss changes
relative to traditional \TEX\ and also present new functionality. As a
consequence we will refer to concepts that we assume to be known or that might be
explained later. Because the \LUATEX\ and \LUAMETATEX\ engines open up \TEX\
there's suddenly quite some more to explain, especially about the way a (to be)
typeset stream moves through the machinery. However, discussing all that in
detail makes not much sense, because deep knowledge is only relevant for those
who write code not possible with regular \TEX\ and who are already familiar with
these internals (or willing to spend time on figuring it out).
So, the average user doesn't need to know much about what is in this manual. For
instance fonts and languages are normally dealt with in the macro package that
you use. Messing around with node lists is also often not really needed at the
user level. If you do mess around, you'd better know what you're dealing with.
Reading \quotation {The \TEX\ Book} by Donald Knuth is a good investment of time
then also because it's good to know where it all started. A more summarizing
overview is given by \quotation {\TEX\ by Topic} by Victor Eijkhout. You might
want to peek in \quotation {The \ETEX\ manual} too.
But \unknown\ if you're here because of \LUA, then all you need to know is that
you can call it from within a run. If you want to learn the language, just read
the well written \LUA\ book. The macro package that you use probably will provide
a few wrapper mechanisms but the basic \lpr {directlua} command that does the job
is:
\starttyping
\directlua{tex.print("Hi there")}
\stoptyping
You can put code between curly braces but if it's a lot you can also put it in a
file and load that file with the usual \LUA\ commands. If you don't know what
this means, you definitely need to have a look at the \LUA\ book first.
If you still decide to read on, then it's good to know what nodes are, so we do a
quick introduction here. If you input this text:
\starttyping
Hi There
\stoptyping
eventually we will get a linked lists of nodes, which in \ASCII\ art looks like:
\starttyping
H <=> i <=> [glue] <=> T <=> h <=> e <=> r <=> e
\stoptyping
When we have a paragraph, we actually get something:
\starttyping
[localpar] <=> H <=> i <=> [glue] <=> T <=> h <=> e <=> r <=> e <=> [glue]
\stoptyping
Each character becomes a so called glyph node, a record with properties like the
current font, the character code and the current language. Spaces become glue
nodes. There are many node types that we will discuss later. Each node points
back to a previous node or next node, given that these exist. Sometimes
multiple characters are represented by one glyphs, so one can also get:
\starttyping
[localpar] <=> H <=> i <=> [glue] <=> Th <=> e <=> r <=> e <=> [glue]
\stoptyping
And maybe some characters get positioned relative to each other, so we might
see:
\starttyping
[localpar] <=> H <=> [kern] <=> i <=> [glue] <=> Th <=> e <=> r <=> e <=> [glue]
\stoptyping
It's also good to know beforehand that \TEX\ is basically centered around
creating paragraphs and pages. The par builder takes a list and breaks it into
lines. At some point horizontal blobs are wrapped into vertical ones. Lines are
so called boxes and can be separated by glue, penalties and more. The page
builder accumulates lines and when feasible triggers an output routine that will
take the list so far. Constructing the actual page is not part of \TEX\ but done
using primitives that permit manipulation of boxes. The result is handled back to
\TEX\ and flushed to a (often \PDF) file.
The \LUATEX\ engine provides hooks for \LUA\ code at nearly every reasonable
point in the process: collecting content, hyphenating, applying font features,
breaking into lines, etc. This means that you can overload \TEX's natural
behaviour, which still is the benchmark. When we refer to \quote {callbacks} we
means these hooks. The \TEX\ engine itself is pretty well optimized but when you
kick in much \LUA\ code, you will notices that performance drops. Don't blame and
bother the authors with performance issues. In \CONTEXT\ over 50\% of the time
can be spent in \LUA, but so far we didn't get many complaints about efficiency.
Where plain \TEX\ is basically a basic framework for writing a specific style,
macro packages like \CONTEXT\ and \LATEX\ provide the user a whole lot of
additional tools to make documents look good. They hide the dirty details of font
management, language demands, turning structure into typeset results, wrapping
pages, including images, and so on. You should be aware of the fact that when you
hook in your own code to manipulate lists, this can interfere with the macro
package that you use. Each successive step expects a certain result and if you
mess around to much, the engine eventually might bark and quit. It can even
crash, because testing everywhere for what users can do wrong is no real option.
When you read about nodes in the following chapters it's good to keep in mind
their commands that relate to then. Here are a few:
\starttabulate[|l|l|p|]
\DB command \BC node \BC explanation \NC \NR
\TB
\NC \prm {hbox} \NC \nod {hlist} \NC horizontal box \NC \NR
\NC \prm {vbox} \NC \nod {vlist} \NC vertical box with the baseline at the bottom \NC \NR
\NC \prm {vtop} \NC \nod {vlist} \NC vertical box with the baseline at the top \NC \NR
\NC \prm {hskip} \NC \nod {glue} \NC horizontal skip with optional stretch and shrink \NC \NR
\NC \prm {vskip} \NC \nod {glue} \NC vertical skip with optional stretch and shrink \NC \NR
\NC \prm {kern} \NC \nod {kern} \NC horizontal or vertical fixed skip \NC \NR
\NC \prm {discretionary} \NC \nod {disc} \NC hyphenation point (pre, post, replace) \NC \NR
\NC \prm {char} \NC \nod {glyph} \NC a character \NC \NR
\NC \prm {hrule} \NC \nod {rule} \NC a horizontal rule \NC \NR
\NC \prm {vrule} \NC \nod {rule} \NC a vertical rule \NC \NR
\NC \prm {textdirection} \NC \nod {dir} \NC a change in text direction \NC \NR
\LL
\stoptabulate
Text (interspersed with macros) comes from an input medium. This can be a file,
token list, macro body cq.\ arguments, \ some internal quantity (like a number),
\LUA, etc. Macros get expanded. In the process \TEX\ can enter a group. Inside
the group, changes to registers get saved on a stack, and restored after leaving
the group. When conditionals are encountered, another kind of nesting happens,
and again there is a stack involved. Tokens, expansion, stacks, input levels are
all terms used in the next chapters. Don't worry, they loose their magic once you
use \TEX\ a lot. You have access to most of the internals and when not, at least
it is possible to query some state we're in or level we're at.
When we talk about packing it can mean two things. When \TEX\ has consumed some
tokens that represent text the next can happen. When the text is put into a so
called \type {\hbox} it (normally) first gets hyphenated, next ligatures are
build, and finally kerns are added. Each of that stages can be overloaded using
\LUA\ code. When these three stages are finished, the dimension of the content is
calculated and the box gets its width, height and depth. What happens with the
box depends on what macros do with it.
The other thing that can happen is that the text starts a new paragraph. In that
case some (directional) information is put in front, indentation is prepended and
some skip appended at the end. Again the three stages are applied but this time,
afterwards, the long line is broken into lines and the result is either added to
the content of a box or to the main vertical list (the running text so to say).
This is called par building. At some point \TEX\ decides that enough is enough
and it will trigger the page builder. So, building is another concept we will
encounter. Another example of a builder is the one that turns an intermediate
math list into something typeset.
Wrapping something in a box is called packing. Adding something to a list is
described in terms of contributing. The more complicated processes are wrapped
into builders. For now this should be enough to enable you to understand the next
chapters. The text is not as enlightening and entertaining as Don Knuths books,
sorry.
\stopchapter
\stopcomponent
|
