summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorHans Hagen <pragma@wxs.nl>2019-12-30 20:42:59 +0100
committerContext Git Mirror Bot <phg@phi-gamma.net>2019-12-30 20:42:59 +0100
commit54732448eb933607bdcb11a457756741dc4e0b44 (patch)
treed0f312dd29af54ee85d89f6d6f242be7ee6b5454 /doc
parentede5a2aae42ff502be35d800e97271cf0bdc889b (diff)
downloadcontext-54732448eb933607bdcb11a457756741dc4e0b44.tar.gz
2019-12-30 19:16:00
Diffstat (limited to 'doc')
-rw-r--r--doc/context/documents/general/manuals/bidi.pdfbin88603 -> 121014 bytes
-rw-r--r--doc/context/documents/general/manuals/luametatex.pdfbin1094749 -> 1094746 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-cs.pdfbin888200 -> 887854 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-de.pdfbin889636 -> 889907 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-en.pdfbin894023 -> 893985 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-fr.pdfbin886167 -> 886209 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-it.pdfbin891794 -> 891810 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-mapping-cs.pdfbin383673 -> 383625 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-mapping-de.pdfbin384071 -> 383818 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-mapping-en.pdfbin380465 -> 380544 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-mapping-fr.pdfbin384024 -> 383986 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-mapping-it.pdfbin383605 -> 383506 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-mapping-nl.pdfbin382078 -> 381744 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-mapping-ro.pdfbin620025 -> 620030 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-nl.pdfbin881286 -> 881373 bytes
-rw-r--r--doc/context/documents/general/qrcs/setup-ro.pdfbin886213 -> 886110 bytes
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-contents.tex9
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-fonts.tex99
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-introduction.tex53
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-lua.tex38
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-mixed.tex178
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-numbering.tex19
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-style.tex123
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-titlepage.tex33
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi-vertical.tex18
-rw-r--r--doc/context/sources/general/manuals/bidi/bidi.tex518
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-callbacks.tex820
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-differences.tex209
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-enhancements.tex1781
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-fonts.tex596
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-libraries.tex573
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-lua.tex224
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-math.tex1585
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-metapost.tex479
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-modifications.tex440
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex2527
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-pdf.tex565
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-preamble.tex166
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-primitives.tex392
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-registers.tex47
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-statistics.tex17
-rw-r--r--doc/context/sources/general/manuals/luametatex/luametatex-tex.tex2477
42 files changed, 13488 insertions, 498 deletions
diff --git a/doc/context/documents/general/manuals/bidi.pdf b/doc/context/documents/general/manuals/bidi.pdf
index c9cba69b1..8485f05c7 100644
--- a/doc/context/documents/general/manuals/bidi.pdf
+++ b/doc/context/documents/general/manuals/bidi.pdf
Binary files differ
diff --git a/doc/context/documents/general/manuals/luametatex.pdf b/doc/context/documents/general/manuals/luametatex.pdf
index 3d27d72ce..e2b85f07c 100644
--- a/doc/context/documents/general/manuals/luametatex.pdf
+++ b/doc/context/documents/general/manuals/luametatex.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-cs.pdf b/doc/context/documents/general/qrcs/setup-cs.pdf
index f3c813efa..734846963 100644
--- a/doc/context/documents/general/qrcs/setup-cs.pdf
+++ b/doc/context/documents/general/qrcs/setup-cs.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-de.pdf b/doc/context/documents/general/qrcs/setup-de.pdf
index 398456abc..bca081a31 100644
--- a/doc/context/documents/general/qrcs/setup-de.pdf
+++ b/doc/context/documents/general/qrcs/setup-de.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-en.pdf b/doc/context/documents/general/qrcs/setup-en.pdf
index 3ae23f53b..d67072314 100644
--- a/doc/context/documents/general/qrcs/setup-en.pdf
+++ b/doc/context/documents/general/qrcs/setup-en.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-fr.pdf b/doc/context/documents/general/qrcs/setup-fr.pdf
index 87d636c24..8287b5f1c 100644
--- a/doc/context/documents/general/qrcs/setup-fr.pdf
+++ b/doc/context/documents/general/qrcs/setup-fr.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-it.pdf b/doc/context/documents/general/qrcs/setup-it.pdf
index 47be61e2c..55cadbc1c 100644
--- a/doc/context/documents/general/qrcs/setup-it.pdf
+++ b/doc/context/documents/general/qrcs/setup-it.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-mapping-cs.pdf b/doc/context/documents/general/qrcs/setup-mapping-cs.pdf
index 0296eeb31..5e27abfb4 100644
--- a/doc/context/documents/general/qrcs/setup-mapping-cs.pdf
+++ b/doc/context/documents/general/qrcs/setup-mapping-cs.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-mapping-de.pdf b/doc/context/documents/general/qrcs/setup-mapping-de.pdf
index 307c0c92c..b2a130448 100644
--- a/doc/context/documents/general/qrcs/setup-mapping-de.pdf
+++ b/doc/context/documents/general/qrcs/setup-mapping-de.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-mapping-en.pdf b/doc/context/documents/general/qrcs/setup-mapping-en.pdf
index 8f428e238..5ae76f864 100644
--- a/doc/context/documents/general/qrcs/setup-mapping-en.pdf
+++ b/doc/context/documents/general/qrcs/setup-mapping-en.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-mapping-fr.pdf b/doc/context/documents/general/qrcs/setup-mapping-fr.pdf
index 7b8f577d2..d80856262 100644
--- a/doc/context/documents/general/qrcs/setup-mapping-fr.pdf
+++ b/doc/context/documents/general/qrcs/setup-mapping-fr.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-mapping-it.pdf b/doc/context/documents/general/qrcs/setup-mapping-it.pdf
index 407b87119..f5384869b 100644
--- a/doc/context/documents/general/qrcs/setup-mapping-it.pdf
+++ b/doc/context/documents/general/qrcs/setup-mapping-it.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-mapping-nl.pdf b/doc/context/documents/general/qrcs/setup-mapping-nl.pdf
index bdb1e3a1b..6b58b9ec9 100644
--- a/doc/context/documents/general/qrcs/setup-mapping-nl.pdf
+++ b/doc/context/documents/general/qrcs/setup-mapping-nl.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-mapping-ro.pdf b/doc/context/documents/general/qrcs/setup-mapping-ro.pdf
index 2909492dd..0bf802870 100644
--- a/doc/context/documents/general/qrcs/setup-mapping-ro.pdf
+++ b/doc/context/documents/general/qrcs/setup-mapping-ro.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-nl.pdf b/doc/context/documents/general/qrcs/setup-nl.pdf
index d7cb1d3a6..c9276f270 100644
--- a/doc/context/documents/general/qrcs/setup-nl.pdf
+++ b/doc/context/documents/general/qrcs/setup-nl.pdf
Binary files differ
diff --git a/doc/context/documents/general/qrcs/setup-ro.pdf b/doc/context/documents/general/qrcs/setup-ro.pdf
index 89b593550..3ba6295d2 100644
--- a/doc/context/documents/general/qrcs/setup-ro.pdf
+++ b/doc/context/documents/general/qrcs/setup-ro.pdf
Binary files differ
diff --git a/doc/context/sources/general/manuals/bidi/bidi-contents.tex b/doc/context/sources/general/manuals/bidi/bidi-contents.tex
new file mode 100644
index 000000000..21c318dff
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-contents.tex
@@ -0,0 +1,9 @@
+% language=us
+
+\startcomponent bidi-contents
+
+ \starttitle[title=Table of contents]
+ \placelist[chapter]
+ \stoptitle
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/bidi/bidi-fonts.tex b/doc/context/sources/general/manuals/bidi/bidi-fonts.tex
new file mode 100644
index 000000000..bfedd8f47
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-fonts.tex
@@ -0,0 +1,99 @@
+% language=us
+
+\startcomponent bidi-fonts
+
+\environment bidi-style
+
+\startchapter[title={Setting up fonts}]
+
+So let's see how Arabic and Hebrew come out:
+
+\startbuffer
+The sentence \quotation {I have no clue what this means.} is translated (by
+Google Translate) into \quotation {\ar \righttoleft ليس لدي أي فكرة عما يعنيه هذا.}
+which is then translated back to \quotation {I have no idea what this means.} so
+maybe arabic has no clue what a clue is. The suggested Arabic pronunciation is
+\quotation {lays laday 'ayu fikrat eamaa yaenih hadha}. Hebrew also likes ideas
+more: \quotation {\he \righttoleft אין לי מושג מה זה אומר}.
+\stopbuffer
+
+\typebuffer
+
+\getbuffer
+
+According to Idris Hamid the Arabic should actually be this: \quotation {\ar
+لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا} and the transliteration \quotation {Laysa
+ladayya ayyu fikratin ʿammā yaʿnihi hādhā}.
+
+The \CONTEXT\ (or any \TEX) ecosystem deals with languages and fonts. Languages
+(that relate to scripts) have specific characteristics, like running from right
+to left, and fonts provide a repertoire of glyphs and features. There is no real
+(standard) relationship between these. In for instance browsers, there are
+automatic fallback systems for missing characters in a font: another font is
+taken. These fallbacks are often not easy to tweak.
+
+In this document we use Dejavu and although that font has Arabic shapes in its
+monospace variant, the serifs come without them (at least when I write this
+down). Before we actually define the bodyfont we hook in some fallbacks. The
+typescript for Dejavu has lines like this:
+
+\starttyping
+\definefontsynonym
+ [SerifBoldItalic]
+ [name:dejavuserifbolditalic]
+ [features=default,
+ fallbacks=SerifBoldItalic]
+\stoptyping
+
+This permits us to do this:
+
+\typebuffer[preamble-fonts]
+
+In addition we set up the languages:
+
+\typebuffer[preamble-languages]
+
+The following example demonstrates what the effects of these commands are:
+
+\startbuffer
+{لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
+{אין לי מושג מה זה אומר.}
+{\righttoleft لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
+{\righttoleft אין לי מושג מה זה אומר.}
+{\ar \righttoleft لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
+{\he \righttoleft אין לי מושג מה זה אומר.}
+{\ar لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
+{\he אין לי מושג מה זה אומר.}
+\stopbuffer
+
+\typebuffer
+
+\startlines
+\getbuffer
+\stoplines
+
+In principle you can also rely on automatic direction changes, for instance
+by using the following command:
+
+\starttyping
+\setupdirections
+ [bidi=global,
+ method=three]
+\stoptyping
+
+But that doesn't do a font switch for you, nor does it do any of the other
+language related settings. It really helps if you properly tag your
+document content, as in:
+
+\starttyping
+{\ar لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
+{\he אין לי מושג מה זה אומר.}
+\stoptyping
+
+One reason to set the \type {font} parameter for a language is that it will
+activate the right features in a font. Instead of falling back on some default,
+we can be very specific in what we want to enable.
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/bidi/bidi-introduction.tex b/doc/context/sources/general/manuals/bidi/bidi-introduction.tex
new file mode 100644
index 000000000..5ea0f9879
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-introduction.tex
@@ -0,0 +1,53 @@
+% language=us
+
+\startcomponent bidi-introduction
+
+\environment bidi-style
+
+\startchapter[title=Introduction]
+
+With \CONTEXT\ you can typeset in two directions: from left to right and from
+right to left. In fact you can also combine these two directions, like this:
+
+\startbuffer
+There are many {\righttoleft \maincolor \it scripts in use} and some run into the
+other direction. However, there is {\righttoleft \maincolor \it no fixed relation
+{\lefttoright \black \it between the} direction of the script} and cars being
+driven left or right of the road.
+\stopbuffer
+
+\typebuffer
+
+\getbuffer
+
+Even someone not familiar with right to left typesetting can see what happens
+here, or not? In fact Luigi Scarso pointed out that the \type {fixed} reversed
+into {\righttoleft \type {fixed}} but not in the example where {\bf fixed}
+becomes {\righttoleft \bf fixed}. This signals an important property of the way
+the text gets processed: you input something, at some points font features get
+applied (like ligatures) and in the end the resulting glyph stream is reversed.
+By that time the combination of {\bf f}+{\bf i} has become {\bf fi}! So, be
+prepared for surprises.
+
+This manual is written by a left to right user so don't expect a manual on
+semitic typesetting. Also don't expect a (yet) complete manual. I'll add whatever
+comes to mind. This is not a manual about Hebrew or Arabic, if only because I
+can't read any of those scripts (languages). I leave that to others to cover.
+
+This is work in progress and might always be! So expect errors and typos. As with
+anything related to typesetting the truth about how it should be done and what
+looks best is not absolute. So, the most we can offer is flexibility and the way
+\CONTEXT\ is setup permits that.
+
+Of course this is not possible without input. When we moved to \CONTEXT\ \LMTX,
+the bidi thread was picked up by Mohammad Hossein Bateni, Idris Samawi Hamid,
+Wolfgang Schuster and myself. So, expect more!
+
+\startlines
+Hans Hagen
+Hasselt, NL
+\stoplines
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/bidi/bidi-lua.tex b/doc/context/sources/general/manuals/bidi/bidi-lua.tex
new file mode 100644
index 000000000..beb93b604
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-lua.tex
@@ -0,0 +1,38 @@
+% language=us
+
+\startcomponent bidi-lua
+
+\environment bidi-style
+
+\startchapter[title={The \LUA\ interface}]
+
+We assume that you run \CONTEXT\ \MKIV\ in combination with \LUATEX. Direction
+support in this engine has been improved over time. Originally the \OMEGA\
+(\ALEPH) direction model was used but in the meantime it has been stripped to the
+basics, and what used to be so called whatsits (extension nodes) are now first
+class nodes. Of the many directions only four are kept in \LUATEX\ and they are
+indicated by three letters:
+
+\starttabulate[|l|l|l|]
+\NC 0 \NC \type {TLT} \NC left to right \NC \NR
+\NC 1 \NC \type {TRT} \NC right to left \NC \NR
+\NC 2 \NC \type {LTL} \NC not used in context (obsolete) \NC \NR
+\NC 3 \NC \type {RTT} \NC not used in context (obsolete) \NC \NR
+\stoptabulate
+
+In \LUAMETATEX, and therefore \CONTEXT\ \LMTX\ we only have the first two.
+Therefore in \LMTX\ you normally don't have to worry about checking for them at
+the \LUA\ end because they are irrelevant for calculations (the vertical ones
+swapped the horizontal and vertical progression). Also, when really needed, we
+use the \type {direction} keys with numerical indicators, so zero for \type {l2r}
+and one for \type {r2l}. These values are used for local par nodes as well as
+direction nodes. In addition a direction node has a subtype:
+
+\starttabulate[|l|l|l|]
+\NC 0 \NC \type {normal} \NC comparable to \type {+} \NC \NR
+\NC 1 \NC \type {cancel} \NC comparable to \type {-} \NC \NR
+\stoptabulate
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/bidi/bidi-mixed.tex b/doc/context/sources/general/manuals/bidi/bidi-mixed.tex
new file mode 100644
index 000000000..851d6d8b4
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-mixed.tex
@@ -0,0 +1,178 @@
+% language=us
+
+\startcomponent bidi-mixed
+
+\environment bidi-style
+
+\startchapter[title=A mixed layout]
+
+The typesetting engine normally works from left to right and top to bottom. Going
+from right to left actually involved two decisions:
+
+\startitemize[packed]
+\startitem the direction of the display elements, the paragraphs \stopitem
+\startitem the direction of the inline text, the lines \stopitem
+\stopitemize
+
+The first one is kept track of in a state variable. Every paragraph starts with
+a node that carries, among other information, that state. This node is added
+automatically and does not interfere with the typesetting. The inline direction
+is more intrusive as it is marked by nodes that indicate the beginning and end
+of a reversed strip. This mechanism is rather reliable and normally works out
+well. Take this:
+
+\startbuffer
+left {\righttoleft right} left
+left{ \righttoleft right} left
+left {\righttoleft right }left
+left{ \righttoleft right }left
+\stopbuffer
+
+\typebuffer
+
+You can see that we need to be careful with spaces as they can end up inside or
+outside a substream and by swapping next to each other:
+
+\startlines
+\getbuffer
+\stoplines
+
+We can wrap the lines in boxes as in:
+
+\startbuffer
+\hbox{left\space{\bf\righttoleft right}\space left}
+\hbox{left{\bf\space \righttoleft right}\space left}
+\hbox{left\space{\bf\righttoleft right\space}left}
+\hbox{left{\bf\space\righttoleft right\space}left}
+\stopbuffer
+
+\typebuffer
+
+\definecolor[ShineThrough][s=0,a=1,t=.2]
+
+When visualize the spaces we get this:
+
+\startlines\ShineThrough
+\showmakeup[space,hbox]\getbuffer
+\stoplines
+
+The space of a normal and bold font in the same family normally is the same but
+let's mix with a larger size:
+
+\startbuffer
+\hbox{left {\bfa\righttoleft right} left}
+\hbox{left{\bfa\space \righttoleft right} left}
+\hbox{left {\bfa\righttoleft right }left}
+\hbox{left{\bfa\space\righttoleft right }left}
+\stopbuffer
+
+\typebuffer
+
+Now we get the following. As you can see, it really matters where we put the
+braces.
+
+\startlines
+\ShineThrough\showmakeup[space,hbox]\getbuffer
+\stoplines
+
+Once you are accustomed to tagging and \TEX\ you will probably not fall into
+these traps. In \in {figure} [fig:spaces] we show a large version.
+
+\startplacefigure[location=top,title={Watch your spaces!},reference=fig:spaces]
+ \scale
+ [width=\hsize]
+ {\vbox{\ShineThrough\showmakeup[space,hbox]\getbuffer}}
+\stopplacefigure
+
+The \type {\righttoleft} command actually has two meanings. This can best be seen
+from an example.
+
+\startbuffer
+\righttoleft \bf How will this come out?
+\stopbuffer
+
+\typebuffer
+
+\start \getbuffer \par \stop
+
+\startbuffer
+And \righttoleft \bf how will this come out?
+\stopbuffer
+
+\typebuffer
+
+\start \getbuffer \par \stop
+
+When we start a paragraph (or in \TEX\ speak: when we are still in vertical mode)
+the paragraph direction as well as the inline direction is set. Otherwise only
+the inline direction is set. There are low level \TEX\ commands (primitives) to
+set the direction but you can best {\em not} use these because we need to do a
+bit more than that.
+
+There are quite some low level commands related to changing directions. Some deal
+with the layout, some with boxes. We might provide more in the future.
+
+\starttabulate[|l|p|]
+\FL
+\NC \type {\lefttoright} \NC l2r dir node or paragraph property \NC \NR
+\NC \type {\righttoleft} \NC r2l dir node or paragraph property \NC \NR
+\NC \type {\checkedlefttoright} \NC l2r dir node or paragraph property (unless already set) \NC \NR
+\NC \type {\checkedrighttoleft} \NC r2l dir node or paragraph property (unless already set) \NC \NR
+\ML
+\NC \type {\synchronizeinlinedirection} \NC pickup a (possibly) reset state \NC \NR
+\NC \type {\synchronizelayoutdirection} \NC pickup a (possibly) reset state \NC \NR
+\NC \type {\synchronizedisplaydirection} \NC pickup a (possibly) reset state \NC \NR
+\ML
+\NC \type {\righttolefthbox} \NC r2l \type {\hbox} \NC \NR
+\NC \type {\lefttorighthbox} \NC l2r \type {\hbox} \NC \NR
+\NC \type {\righttoleftvbox} \NC r2l \type {\vbox} \NC \NR
+\NC \type {\lefttorightvbox} \NC l2r \type {\vbox} \NC \NR
+\NC \type {\righttoleftvtop} \NC r2l \type {\vtop} \NC \NR
+\NC \type {\lefttorightvtop} \NC l2r \type {\vtop} \NC \NR
+\ML
+\NC \type {\leftorrighthbox} \NC l2r or r2l \type {\hbox} \NC \NR
+\NC \type {\leftorrightvbox} \NC l2r or r2l \type {\vbox} \NC \NR
+\NC \type {\leftorrightvtop} \NC l2r or r2l \type {\vtop} \NC \NR
+\ML
+\NC \type {\autodirhbox} \NC l2r or r2l \type {\hbox} (a bit more clever) \NC \NR
+\NC \type {\autodirvbox} \NC l2r or r2l \type {\vbox} (a bit more clever) \NC \NR
+\NC \type {\autodirvtop} \NC l2r or r2l \type {\vtop} (a bit more clever) \NC \NR
+\ML
+\NC \type {\bidilre} \NC character \type {U+202A}, enforce l2r state \NC \NR
+\NC \type {\bidirle} \NC character \type {U+202B}, enforce r2l state \NC \NR
+\NC \type {\bidipop} \NC character \type {U+202C}, return to last state \NC \NR
+\NC \type {\bidilro} \NC character \type {U+202D}, override l2r state \NC \NR
+\NC \type {\bidirlo} \NC character \type {U+202E}, override r2l state \NC \NR
+\NC \type {\lefttorightmark} \type {\lrm} \NC character \type {U+200E}, l2r indicator \NC \NR
+\NC \type {\righttoleftmark} \type {\rlm} \NC character \type {U+200F}, r2l indicator \NC \NR
+\ML
+\NC \type {\dirlre} \NC switch to l2r mode using \type {\bidilre} or \lefttoright \NC \NR
+\NC \type {\dirrle} \NC switch to r2l mode using \type {\bidirle} or \righttoleft \NC \NR
+\NC \type {\dirlro} \NC enforce l2r mode using \type {\bidilro} or \lefttoright \NC \NR
+\NC \type {\dirrlo} \NC enforce r2l mode using \type {\bidirlo} or \righttoleft \NC \NR
+\ML
+\NC \type {\naturalhbox} \NC a normal l2r \type {hbox} \NC \NR
+\NC \type {\naturalvbox} \NC a normal l2r \type {vbox} \NC \NR
+\NC \type {\naturalvtop} \NC a normal l2r \type {vtop} \NC \NR
+\NC \type {\naturalhpack} \NC a normal l2r \type {hpack} \NC \NR
+\NC \type {\naturalvpack} \NC a normal l2r \type {vpack} \NC \NR
+\LL
+\stoptabulate
+
+When we talk about layout, we mean the overall layout, concerning the document as
+a whole. We can have a dominantly l2r, dominantly r2l or mixed setup. In a next
+chapter we will give more details on the dominant setup. Here we stick to
+mentioning that the document flow direction is set with
+
+\starttyping
+\setupalign[r2l] % or r2l
+\stoptyping
+
+When a command to setup an environment has a \type {align} parameter, the same
+keywords can be uses as part of the specification. \footnote {We haven't tested
+all situations and possible interferences. Just report anomalies to the mailing
+list.}
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/bidi/bidi-numbering.tex b/doc/context/sources/general/manuals/bidi/bidi-numbering.tex
new file mode 100644
index 000000000..c328d6a1a
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-numbering.tex
@@ -0,0 +1,19 @@
+% language=us
+
+\startcomponent bidi-numbering
+
+\environment bidi-style
+
+\startchapter[title={Numbering and positioning}]
+
+todo: columns (direction key), numbers (conversionsets), margins (begin/end), etc
+
+% \defineconversionset [\s!default] [] [numbers]
+% \defineconversionset [\v!number] [] [numbers]
+% \defineconversionset [\v!pagenumber] [] [numbers]
+% \defineconversionset [\v!appendix:\s!default] [Romannumerals,Characters] [numbers]
+% \defineconversionset [\v!formula] [numbers,characters] % no \v! ?
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/bidi/bidi-style.tex b/doc/context/sources/general/manuals/bidi/bidi-style.tex
new file mode 100644
index 000000000..1ce4e3b79
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-style.tex
@@ -0,0 +1,123 @@
+\startenvironment bidi-style
+
+\usemodule[abr-04]
+
+\dontcomplain
+
+\startbuffer[preamble-fonts]
+\definefontfallback
+ [Serif] [scheherazaderegular*arabic]
+ [arabic] [force=yes,rscale=1.5]
+\definefontfallback
+ [SerifBold] [scheherazadebold*arabic]
+ [arabic] [force=yes,rscale=1.5]
+\definefontfallback
+ [SerifItalic] [scheherazaderegular*arabic]
+ [arabic] [force=yes,rscale=1.5]
+\definefontfallback
+ [SerifBoldItalic] [scheherazadebold*arabic]
+ [arabic] [force=yes,rscale=1.5]
+
+\definefontfallback
+ [Serif] [sileot*hebrew]
+ [hebrew] [force=yes]
+\definefontfallback
+ [SerifBold] [sileot*hebrew]
+ [hebrew] [force=yes]
+\definefontfallback
+ [SerifItalic] [sileot*hebrew]
+ [hebrew] [force=yes]
+\definefontfallback
+ [SerifBoldItalic] [sileot*hebrew]
+ [hebrew] [force=yes]
+
+\definefontfallback
+ [Mono] [almfixed*none]
+ [arabic] [force=yes]
+
+\definefontfallback
+ [Mono] [sileot*none]
+ [hebrew] [force=yes,factor=1] % factor forces a monospace
+
+\setupbodyfont
+ [dejavu,10pt]
+\stopbuffer
+
+\startbuffer[preamble-languages]
+\setuplanguage[ar][font=arabic,bidi=right]
+\setuplanguage[he][font=hebrew,bidi=right]
+\stopbuffer
+
+\getbuffer[preamble-fonts]
+\getbuffer[preamble-languages]
+
+\setuplayout
+ [backspace=15mm,
+ topspace=15mm,
+ footer=0pt,
+ width=middle,
+ height=middle]
+
+% \setuptyping
+% [color=middleblue]
+%
+% \setuptype
+% [color=middleblue]
+
+\usemodule[scite]
+
+\setuptyping
+ [option=TEX]
+
+\setuptype
+ [option=TEX]
+
+\definecolor
+ [maincolor]
+ [middleblue]
+
+\setupwhitespace
+ [big]
+
+\setuphead
+ [color=darkyellow]
+
+\setuphead
+ [chapter]
+ [style=\bfc]
+
+\setuphead
+ [section]
+ [style=\bfb]
+
+\setuphead
+ [subsection]
+ [style=\bfa]
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\startluacode
+ local report = logs.reporter("directions","check")
+ local line = 0
+ function nodes.tracers.checkdirections(head)
+ line = line + 1
+ report("line: %i",line)
+ for n in nodes.traverse_id(nodes.nodecodes.dir,head) do
+ report(" %s (%i,%i)",n.dir,n.subtype,n.direction)
+ end
+ return head, false
+ end
+
+ nodes.tasks.appendaction("contributers","after","nodes.tracers.checkdirections")
+ nodes.tasks.disableaction("contributers","nodes.tracers.checkdirections")
+\stopluacode
+
+\installtextracker
+ {directions.check}
+ {\ctxlua{nodes.tasks.enableaction("contributers","nodes.tracers.checkdirections")}}
+ {\ctxlua{nodes.tasks.disableaction("contributers","nodes.tracers.checkdirections")}}
+
+% \enabletrackers[directions.check]
+% \disabletrackers[directions.check]
+
+\stopenvironment
diff --git a/doc/context/sources/general/manuals/bidi/bidi-titlepage.tex b/doc/context/sources/general/manuals/bidi/bidi-titlepage.tex
new file mode 100644
index 000000000..a80851aea
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-titlepage.tex
@@ -0,0 +1,33 @@
+\startcomponent bidi-titlepage
+
+\environment bidi-style
+
+\startMPpage
+
+ picture p, q, r, s ;
+
+ p := textext("l2r") xsized .9PaperWidth ;
+ q := textext("r2l") xsized .9PaperWidth ;
+ r := textext("a few tips") xsized .9PaperWidth ;
+ s := textext("\bf\ss hans\quad\space\quad hagen") xsized .5bbheight(p);
+
+ p := p shifted - llcorner p ;
+ q := q shifted - llcorner q ;
+ r := r shifted - llcorner r ;
+ s := s shifted - llcorner s ;
+
+ fill Page withcolor "darkyellow" ;
+
+ p := p shifted (.05PaperWidth,ypart .5[ulcorner Page, urcorner Page]-1.1bbheight(p)) ;
+ q := q shifted (.05PaperWidth,ypart .5[ulcorner Page, urcorner Page]-1.1bbheight(p)-1.15bbheight(q)) ;
+ r := r shifted (.05PaperWidth,ypart .5[llcorner Page, lrcorner Page]+0.3bbheight(r)) ;
+ s := s shifted (.66PaperWidth,ypart .5[llcorner Page, lrcorner Page]+1.5bbheight(s)) ;
+
+ draw p withcolor "lightgray" ;
+ draw q withcolor "lightgray" ;
+ draw r withcolor "middleblue" ;
+ draw s withcolor "lightgray" ;
+
+\stopMPpage
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/bidi/bidi-vertical.tex b/doc/context/sources/general/manuals/bidi/bidi-vertical.tex
new file mode 100644
index 000000000..824d7a033
--- /dev/null
+++ b/doc/context/sources/general/manuals/bidi/bidi-vertical.tex
@@ -0,0 +1,18 @@
+% language=us
+
+\startcomponent bidi-vertical
+
+\environment bidi-style
+
+\startchapter[title={Going vertical}]
+
+Normally the term bidi is reserved for horizontal direction swapping but there is
+no reason to limit our view to that. So, here I will spend some words on how we
+can deal with vertical directions.
+
+{\em I will move some (not yet public) explanation from elsewhere to here in due
+time.}
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/bidi/bidi.tex b/doc/context/sources/general/manuals/bidi/bidi.tex
index fb3705a37..c6fedacdf 100644
--- a/doc/context/sources/general/manuals/bidi/bidi.tex
+++ b/doc/context/sources/general/manuals/bidi/bidi.tex
@@ -1,506 +1,28 @@
-% language=uk
+% Because we cross a threshold in the amount of content the sources
+% have been split into components. This is typical a manual that
+% grows. Feel free to send suggestions and improvements.
-% \showglyphs
+\environment bidi-style
-\usemodule[abr-04]
+\startdocument
-\dontcomplain
+ \component bidi-titlepage
-\startbuffer[preamble-fonts]
-\definefontfallback
- [Serif] [scheherazaderegular*arabic]
- [arabic] [force=yes,rscale=1.5]
-\definefontfallback
- [SerifBold] [scheherazadebold*arabic]
- [arabic] [force=yes,rscale=1.5]
-\definefontfallback
- [SerifItalic] [scheherazaderegular*arabic]
- [arabic] [force=yes,rscale=1.5]
-\definefontfallback
- [SerifBoldItalic] [scheherazadebold*arabic]
- [arabic] [force=yes,rscale=1.5]
+ \startfrontmatter
+ \component bidi-contents
+ \component bidi-introduction
+ \stopfrontmatter
-\definefontfallback
- [Serif] [sileot*hebrew]
- [hebrew] [force=yes]
-\definefontfallback
- [SerifBold] [sileot*hebrew]
- [hebrew] [force=yes]
-\definefontfallback
- [SerifItalic] [sileot*hebrew]
- [hebrew] [force=yes]
-\definefontfallback
- [SerifBoldItalic] [sileot*hebrew]
- [hebrew] [force=yes]
+ \startbodymatter
+ \component bidi-fonts
+ \component bidi-mixed
+ \component bidi-numbering
-\definefontfallback
- [Mono] [almfixed*none]
- [arabic] [force=yes]
+ % work in progress / dev:
+ \component bidi-tables
-\definefontfallback
- [Mono] [sileot*none]
- [hebrew] [force=yes,factor=1] % factor forces a monospace
+ \component bidi-lua
+ \component bidi-vertical
+ \stopbodymatter
-\setupbodyfont
- [dejavu,10pt]
-\stopbuffer
-
-\startbuffer[preamble-languages]
-\setuplanguage[ar][font=arabic,bidi=right]
-\setuplanguage[he][font=hebrew,bidi=right]
-\stopbuffer
-
-\getbuffer[preamble-fonts]
-\getbuffer[preamble-languages]
-
-\setuplayout
- [backspace=15mm,
- topspace=15mm,
- footer=0pt,
- width=middle,
- height=middle]
-
-\setuptyping
- [color=middleblue]
-
-\setuptype
- [color=middleblue]
-
-\definecolor
- [maincolor]
- [middleblue]
-
-\setupwhitespace
- [big]
-
-\setuphead
- [color=darkyellow]
-
-\setuphead
- [chapter]
- [style=\bfc]
-
-\setuphead
- [section]
- [style=\bfb]
-
-\setuphead
- [subsection]
- [style=\bfa]
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-\startluacode
- local report = logs.reporter("directions","check")
- local line = 0
- function nodes.tracers.checkdirections(head)
- line = line + 1
- report("line: %i",line)
- for n in nodes.traverse_id(nodes.nodecodes.dir,head) do
- report(" %s (%i,%i)",n.dir,n.subtype,n.direction)
- end
- return head, false
- end
-
- nodes.tasks.appendaction("contributers","after","nodes.tracers.checkdirections")
- nodes.tasks.disableaction("contributers","nodes.tracers.checkdirections")
-\stopluacode
-
-\installtextracker
- {directions.check}
- {\ctxlua{nodes.tasks.enableaction("contributers","nodes.tracers.checkdirections")}}
- {\ctxlua{nodes.tasks.disableaction("contributers","nodes.tracers.checkdirections")}}
-
-% \enabletrackers[directions.check]
-% \disabletrackers[directions.check]
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-\starttext
-
-\startMPpage
-
- picture p, q, r, s ;
-
- p := textext("l2r") xsized .9PaperWidth ;
- q := textext("r2l") xsized .9PaperWidth ;
- r := textext("a few tips") xsized .9PaperWidth ;
- s := textext("\bf\ss hans\quad\space\quad hagen") xsized .5bbheight(p);
-
- p := p shifted - llcorner p ;
- q := q shifted - llcorner q ;
- r := r shifted - llcorner r ;
- s := s shifted - llcorner s ;
-
- fill Page withcolor "darkyellow" ;
-
- p := p shifted (.05PaperWidth,ypart .5[ulcorner Page, urcorner Page]-1.1bbheight(p)) ;
- q := q shifted (.05PaperWidth,ypart .5[ulcorner Page, urcorner Page]-1.1bbheight(p)-1.15bbheight(q)) ;
- r := r shifted (.05PaperWidth,ypart .5[llcorner Page, lrcorner Page]+0.3bbheight(r)) ;
- s := s shifted (.66PaperWidth,ypart .5[llcorner Page, lrcorner Page]+1.5bbheight(s)) ;
-
- draw p withcolor "lightgray" ;
- draw q withcolor "lightgray" ;
- draw r withcolor "middleblue" ;
- draw s withcolor "lightgray" ;
-
-\stopMPpage
-
-\startchapter[title=Introduction]
-
-With \CONTEXT\ you can typeset in two directions: from left to right and from
-right to left. In fact you can also combine these two directions, like this:
-
-\startbuffer
-There are many {\righttoleft \maincolor \it scripts in use} and some run into the
-other direction. However, there is {\righttoleft \maincolor \it no fixed relation
-{\lefttoright \black \it between the} direction of the script} and cars being
-driven left or right of the road.
-\stopbuffer
-
-\typebuffer
-
-\getbuffer
-
-Even someone not familiar with right to left typesetting can see what happens
-here, or not? In fact Luigi Scarso pointed out that the \type {fixed} reversed
-into {\righttoleft \type {fixed}} but not in the example where {\bf fixed}
-becomes {\righttoleft \bf fixed}. This signals an important property of the way
-the text gets processed: you input something, at some points font features get
-applied (like ligatures) and in the end the resulting glyph stream is reversed.
-By that time the combination of {\bf f}+{\bf i} has become {\bf fi}! So, be
-prepared for surprises.
-
-This manual is written by a left to right user so don't expect a manual on
-semitic typesetting. Also don't expect a (yet) complete manual. I'll add whatever
-comes to mind. This is not a manual about Hebrew or Arabic, if only because I
-can't read any of those scripts (languages). I leave that to others to cover.
-
-{\em This is work in progress! So expect errors and typos.}
-
-\startlines
-Hans Hagen
-Hasselt, NL
-\stoplines
-
-\stopchapter
-
-\startchapter[title={Setting up fonts}]
-
-So let's see how Arabic and Hebrew come out:
-
-\startbuffer
-The sentence \quotation {I have no clue what this means.} is translated (by
-Google Translate) into \quotation {\ar \righttoleft ليس لدي أي فكرة عما يعنيه هذا.}
-which is then translated back to \quotation {I have no idea what this means.} so
-maybe arabic has no clue what a clue is. The suggested Arabic pronunciation is
-\quotation {lays laday 'ayu fikrat eamaa yaenih hadha}. Hebrew also likes ideas
-more: \quotation {\he \righttoleft אין לי מושג מה זה אומר}.
-\stopbuffer
-
-\typebuffer
-
-\getbuffer
-
-According to Idris Hamid the Arabic should actually be this: \quotation {\ar
-لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا} and the transliteration \quotation {Laysa
-ladayya ayyu fikratin ʿammā yaʿnihi hādhā}.
-
-The \CONTEXT\ (or any \TEX) ecosystem deals with languages and fonts. Languages
-(that relate to scripts) have specific characteristics, like running from right
-to left, and fonts provide a repertoire of glyphs and features. There is no real
-(standard) relationship between these. In for instance browsers, there are
-automatic fallback systems for missing characters in a font: another font is
-taken. These fallbacks are often not easy to tweak.
-
-In this document we use Dejavu and although that font has Arabic shapes in its
-monospace variant, the serifs come without them (at least when I write this
-down). Before we actually define the bodyfont we hook in some fallbacks. The
-typescript for Dejavu has lines like this:
-
-\starttyping
-\definefontsynonym
- [SerifBoldItalic]
- [name:dejavuserifbolditalic]
- [features=default,
- fallbacks=SerifBoldItalic]
-\stoptyping
-
-This permits us to do this:
-
-\typebuffer[preamble-fonts]
-
-In addition we set up the languages:
-
-\typebuffer[preamble-languages]
-
-The following example demonstrates what the effects of these commands are:
-
-\startbuffer
-{لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
-{אין לי מושג מה זה אומר.}
-{\righttoleft لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
-{\righttoleft אין לי מושג מה זה אומר.}
-{\ar \righttoleft لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
-{\he \righttoleft אין לי מושג מה זה אומר.}
-{\ar لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
-{\he אין לי מושג מה זה אומר.}
-\stopbuffer
-
-\typebuffer
-
-\startlines
-\getbuffer
-\stoplines
-
-In principle you can also rely on automatic direction changes, for instance
-by using the following command:
-
-\starttyping
-\setupdirections
- [bidi=global,
- method=three]
-\stoptyping
-
-But that doesn't do a font switch for you, nor does it do any of the other
-language related settings. It really helps if you properly tag your
-document content, as in:
-
-\starttyping
-{\ar لَيْسَ لَدَيَّ أَيُّ فِكْرَةٍ عَمَّا يَعْنِيهِ هٰذَا.}
-{\he אין לי מושג מה זה אומר.}
-\stoptyping
-
-One reason to set the \type {font} parameter for a language is that it will
-activate the right features in a font. Instead of falling back on some default,
-we can be very specific in what we want to enable.
-
-\stopchapter
-
-\startchapter[title=A mixed layout]
-
-The typesetting engine normally works from left to right and top to bottom. Going
-from right to left actually involved two decisions:
-
-\startitemize[packed]
-\startitem the direction of the display elements, the paragraphs \stopitem
-\startitem the direction of the inline text, the lines \stopitem
-\stopitemize
-
-The first one is kept track of in a state variable. Every paragraph starts with
-a node that carries, among other information, that state. This node is added
-automatically and does not interfere with the typesetting. The inline direction
-is more intrusive as it is marked by nodes that indicate the beginning and end
-of a reversed strip. This mechanism is rather reliable and normally works out
-well. Take this:
-
-\startbuffer
-left {\righttoleft right} left
-left{ \righttoleft right} left
-left {\righttoleft right }left
-left{ \righttoleft right }left
-\stopbuffer
-
-\typebuffer
-
-You can see that we need to be careful with spaces as they can end up inside or
-outside a substream and by swapping next to each other:
-
-\startlines
-\getbuffer
-\stoplines
-
-We can wrap the lines in boxes as in:
-
-\startbuffer
-\hbox{left\space{\bf\righttoleft right}\space left}
-\hbox{left{\bf\space \righttoleft right}\space left}
-\hbox{left\space{\bf\righttoleft right\space}left}
-\hbox{left{\bf\space\righttoleft right\space}left}
-\stopbuffer
-
-\typebuffer
-
-\definecolor[ShineThrough][s=0,a=1,t=.2]
-
-When visualize the spaces we get this:
-
-\startlines\ShineThrough
-\showmakeup[space,hbox]\getbuffer
-\stoplines
-
-The space of a normal and bold font in the same family normally is the same but
-let's mix with a larger size:
-
-\startbuffer
-\hbox{left {\bfa\righttoleft right} left}
-\hbox{left{\bfa\space \righttoleft right} left}
-\hbox{left {\bfa\righttoleft right }left}
-\hbox{left{\bfa\space\righttoleft right }left}
-\stopbuffer
-
-\typebuffer
-
-Now we get the following. As you can see, it really matters where we put the
-braces.
-
-\startlines
-\ShineThrough\showmakeup[space,hbox]\getbuffer
-\stoplines
-
-Once you are accustomed to tagging and \TEX\ you will probably not fall into
-these traps. In \in {figure} [fig:spaces] we show a large version.
-
-\startplacefigure[location=top,title={Watch your spaces!},reference=fig:spaces]
- \scale
- [width=\hsize]
- {\vbox{\ShineThrough\showmakeup[space,hbox]\getbuffer}}
-\stopplacefigure
-
-
-The \type {\righttoleft} command actually has two meanings. This can best be seen
-from an example.
-
-\startbuffer
-\righttoleft \bf How will this come out?
-\stopbuffer
-
-\typebuffer
-
-\start \getbuffer \par \stop
-
-\startbuffer
-And \righttoleft \bf how will this come out?
-\stopbuffer
-
-\typebuffer
-
-\start \getbuffer \par \stop
-
-When we start a paragraph (or in \TEX\ speak: when we are still in vertical mode)
-the paragraph direction as well as the inline direction is set. Otherwise only
-the inline direction is set. There are low level \TEX\ commands (primitives) to
-set the direction but you can best {\em not} use these because we need to do a
-bit more than that.
-
-There are quite some low level commands related to changing directions. Some deal
-with the layout, some with boxes. We might provide more in the future.
-
-\starttabulate[|l|p|]
-\FL
-\NC \type {\lefttoright} \NC l2r dir node or paragraph property \NC \NR
-\NC \type {\righttoleft} \NC r2l dir node or paragraph property \NC \NR
-\NC \type {\checkedlefttoright} \NC l2r dir node or paragraph property (unless already set) \NC \NR
-\NC \type {\checkedrighttoleft} \NC r2l dir node or paragraph property (unless already set) \NC \NR
-\ML
-\NC \type {\synchronizeinlinedirection} \NC pickup a (possibly) reset state \NC \NR
-\NC \type {\synchronizelayoutdirection} \NC pickup a (possibly) reset state \NC \NR
-\NC \type {\synchronizedisplaydirection} \NC pickup a (possibly) reset state \NC \NR
-\ML
-\NC \type {\righttolefthbox} \NC r2l \type {\hbox} \NC \NR
-\NC \type {\lefttorighthbox} \NC l2r \type {\hbox} \NC \NR
-\NC \type {\righttoleftvbox} \NC r2l \type {\vbox} \NC \NR
-\NC \type {\lefttorightvbox} \NC l2r \type {\vbox} \NC \NR
-\NC \type {\righttoleftvtop} \NC r2l \type {\vtop} \NC \NR
-\NC \type {\lefttorightvtop} \NC l2r \type {\vtop} \NC \NR
-\ML
-\NC \type {\leftorrighthbox} \NC l2r or r2l \type {\hbox} \NC \NR
-\NC \type {\leftorrightvbox} \NC l2r or r2l \type {\vbox} \NC \NR
-\NC \type {\leftorrightvtop} \NC l2r or r2l \type {\vtop} \NC \NR
-\ML
-\NC \type {\autodirhbox} \NC l2r or r2l \type {\hbox} (a bit more clever) \NC \NR
-\NC \type {\autodirvbox} \NC l2r or r2l \type {\vbox} (a bit more clever) \NC \NR
-\NC \type {\autodirvtop} \NC l2r or r2l \type {\vtop} (a bit more clever) \NC \NR
-\ML
-\NC \type {\bidilre} \NC character \type {U+202A}, enforce l2r state \NC \NR
-\NC \type {\bidirle} \NC character \type {U+202B}, enforce r2l state \NC \NR
-\NC \type {\bidipop} \NC character \type {U+202C}, return to last state \NC \NR
-\NC \type {\bidilro} \NC character \type {U+202D}, override l2r state \NC \NR
-\NC \type {\bidirlo} \NC character \type {U+202E}, override r2l state \NC \NR
-\NC \type {\lefttorightmark} \type {\lrm} \NC character \type {U+200E}, l2r indicator \NC \NR
-\NC \type {\righttoleftmark} \type {\rlm} \NC character \type {U+200F}, r2l indicator \NC \NR
-\ML
-\NC \type {\dirlre} \NC switch to l2r mode using \type {\bidilre} or \lefttoright \NC \NR
-\NC \type {\dirrle} \NC switch to r2l mode using \type {\bidirle} or \righttoleft \NC \NR
-\NC \type {\dirlro} \NC enforce l2r mode using \type {\bidilro} or \lefttoright \NC \NR
-\NC \type {\dirrlo} \NC enforce r2l mode using \type {\bidirlo} or \righttoleft \NC \NR
-\ML
-\NC \type {\naturalhbox} \NC a normal l2r \type {hbox} \NC \NR
-\NC \type {\naturalvbox} \NC a normal l2r \type {vbox} \NC \NR
-\NC \type {\naturalvtop} \NC a normal l2r \type {vtop} \NC \NR
-\NC \type {\naturalhpack} \NC a normal l2r \type {hpack} \NC \NR
-\NC \type {\naturalvpack} \NC a normal l2r \type {vpack} \NC \NR
-\LL
-\stoptabulate
-
-When we talk about layout, we mean the overall layout, concerning the document as
-a whole. We can have a dominantly l2r, dominantly r2l or mixed setup. In a next
-chapter we will give more details on the dominant setup. Here we stick to
-mentioning that the document flow direction is set with
-
-\starttyping
-\setupalign[r2l] % or r2l
-\stoptyping
-
-When a command to setup an environment has a \type {align} parameter, the same
-keywords can be uses as part of the specification. \footnote {We haven't tested
-all situations and possible interferences. Just report anomalies to the mailing
-list.}
-
-\stopchapter
-
-\startchapter[title={Numbering and positioning}]
-
-todo: columns (direction key), numbers (conversionsets), margins (begin/end), etc
-
-\stopchapter
-
-\startchapter[title={The \LUA\ interface}]
-
-We assume that you run \CONTEXT\ \MKIV\ in combination with \LUATEX. Direction
-support in this engine has been improved over time. Originally the \OMEGA\
-(\ALEPH) direction model was used but in the meantime it has been stripped to the
-basics, and what used to be so called whatsits (extension nodes) are now first
-class nodes. Of the many directions only four are kept in \LUATEX\ and they are
-indicated by three letters:
-
-\starttabulate[|l|l|l|]
-\NC 0 \NC \type {TLT} \NC left to right \NC \NR
-\NC 1 \NC \type {TRT} \NC right to left \NC \NR
-\NC 2 \NC \type {LTL} \NC not used in context (obsolete) \NC \NR
-\NC 3 \NC \type {RTT} \NC not used in context (obsolete) \NC \NR
-\stoptabulate
-
-In \LUAMETATEX, and therefore \CONTEXT\ \LMTX\ we only have the first two.
-Therefore in \LMTX\ you normally don't have to worry about checking for them at
-the \LUA\ end because they are irrelevant for calculations (the vertical ones
-swapped the horizontal and vertical progression). Also, when really needed, we
-use the \type {direction} keys with numerical indicators, so zero for \type {l2r}
-and one for \type {r2l}. These values are used for local par nodes as well as
-direction nodes. In addition a direction node has a subtype:
-
-\starttabulate[|l|l|l|]
-\NC 0 \NC \type {normal} \NC comparable to \type {+} \NC \NR
-\NC 1 \NC \type {cancel} \NC comparable to \type {-} \NC \NR
-\stoptabulate
-
-\stopchapter
-
-\startchapter[title={Going vertical}]
-
-Normally the term bidi is reserved for horizontal direction swapping but there is
-no reason to limit our view to that. So, here I will spend some words on how we
-can deal with vertical directions.
-
-{\em I will move some (not yet public) explanation from elsewhere to here in due
-time.}
-
-\stopchapter
-
-\stoptext
-
-% \defineconversionset [\s!default] [] [numbers]
-% \defineconversionset [\v!number] [] [numbers]
-% \defineconversionset [\v!pagenumber] [] [numbers]
-% \defineconversionset [\v!appendix:\s!default] [Romannumerals,Characters] [numbers]
-% \defineconversionset [\v!formula] [numbers,characters] % no \v! ?
+\stopdocument
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-callbacks.tex b/doc/context/sources/general/manuals/luametatex/luametatex-callbacks.tex
new file mode 100644
index 000000000..e713d13c3
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-callbacks.tex
@@ -0,0 +1,820 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-callbacks
+
+\startchapter[reference=callbacks,title={\LUA\ callbacks}]
+
+\startsection[title={Registering callbacks}][library=callback]
+
+\topicindex{callbacks}
+
+\libindex{register}
+\libindex{list}
+\libindex{find}
+
+This library has functions that register, find and list callbacks. Callbacks are
+\LUA\ functions that are called in well defined places. There are two kind of
+callbacks: those that mix with existing functionality, and those that (when
+enabled) replace functionality. In mosty cases the second category is expected to
+behave similar to the built in functionality because in a next step specific
+data is expected. For instance, you can replace the hyphenation routine. The
+function gets a list that can be hyphenated (or not). The final list should be
+valid and is (normally) used for constructing a paragraph. Another function can
+replace the ligature builder and|/|or kerner. Doing something else is possible
+but in the end might not give the user the expected outcome.
+
+The first thing you need to do is registering a callback:
+
+\startfunctioncall
+id, error =
+ callback.register(<string> callback_name, <function> func)
+id, error =
+ callback.register(<string> callback_name, nil)
+id, error =
+ callback.register(<string> callback_name, false)
+\stopfunctioncall
+
+Here the \syntax {callback_name} is a predefined callback name, see below. The
+function returns the internal \type {id} of the callback or \type {nil}, if the
+callback could not be registered. In the latter case, \type {error} contains an
+error message, otherwise it is \type {nil}.
+
+\LUATEX\ internalizes the callback function in such a way that it does not matter
+if you redefine a function accidentally.
+
+Callback assignments are always global. You can use the special value \type {nil}
+instead of a function for clearing the callback.
+
+For some minor speed gain, you can assign the boolean \type {false} to the
+non|-|file related callbacks, doing so will prevent \LUATEX\ from executing
+whatever it would execute by default (when no callback function is registered at
+all). Be warned: this may cause all sorts of grief unless you know \notabene
+{exactly} what you are doing!
+
+\startfunctioncall
+<table> info =
+ callback.list()
+\stopfunctioncall
+
+The keys in the table are the known callback names, the value is a boolean where
+\type {true} means that the callback is currently set (active).
+
+\startfunctioncall
+<function> f = callback.find(callback_name)
+\stopfunctioncall
+
+If the callback is not set, \type {find} returns \type {nil}.
+
+\stopsection
+
+\startsection[title={File related callbacks}][library=callback]
+
+The behaviour documented in this subsection is considered stable in the sense that
+there will not be backward|-|incompatible changes any more.
+
+\subsection{\cbk {find_read_file}}
+
+Your callback function should have the following conventions:
+
+\startfunctioncall
+<string> actual_name =
+ function (<number> id_number, <string> asked_name)
+\stopfunctioncall
+
+Arguments:
+
+\startitemize
+
+\sym{id_number}
+
+This number is zero for the log or \prm {input} files. For \TEX's \prm {read}
+or \prm {write} the number is incremented by one, so \type {\read0} becomes~1.
+
+\sym{asked_name}
+
+This is the user|-|supplied filename, as found by \prm {input}, \prm {openin}
+or \prm {openout}.
+
+\stopitemize
+
+Return value:
+
+\startitemize
+
+\sym{actual_name}
+
+This is the filename used. For the very first file that is read in by \TEX, you
+have to make sure you return an \type {actual_name} that has an extension and
+that is suitable for use as \type {jobname}. If you don't, you will have to
+manually fix the name of the log file and output file after \LUATEX\ is finished,
+and an eventual format filename will become mangled. That is because these file
+names depend on the jobname.
+
+You have to return \type {nil} if the file cannot be found.
+
+\stopitemize
+
+\subsection{\cbk {find_data_file}}
+
+\topicindex{callbacks+data files}
+
+Your callback function should have the following conventions:
+
+\startfunctioncall
+<string> actual_name =
+ function (<string> asked_name)
+\stopfunctioncall
+
+Return \type {nil} if the file cannot be found.
+
+\subsection{\cbk {find_format_file}}
+
+\topicindex{callbacks+format file}
+
+Your callback function should have the following conventions:
+
+\startfunctioncall
+<string> actual_name =
+ function (<string> asked_name)
+\stopfunctioncall
+
+The \type {asked_name} is a format file for reading (the format file for writing
+is always opened in the current directory).
+
+\subsection{\cbk {open_read_file}}
+
+\topicindex{callbacks+opening files}
+
+Your callback function should have the following conventions:
+
+\startfunctioncall
+<table> env =
+ function (<string> file_name)
+\stopfunctioncall
+
+Argument:
+
+\startitemize
+
+\sym{file_name}
+
+The filename returned by a previous \cbk {find_read_file} or the return value of
+\type {kpse.find_file()} if there was no such callback defined.
+
+\stopitemize
+
+Return value:
+
+\startitemize
+
+\sym{env}
+
+This is a table containing at least one required and one optional callback
+function for this file. The required field is \type {reader} and the associated
+function will be called once for each new line to be read, the optional one is
+\type {close} that will be called once when \LUATEX\ is done with the file.
+
+\LUATEX\ never looks at the rest of the table, so you can use it to store your
+private per|-|file data. Both the callback functions will receive the table as
+their only argument.
+
+\stopitemize
+
+\subsubsection{\type {reader}}
+
+\topicindex{callbacks+reader}
+
+\LUATEX\ will run this function whenever it needs a new input line from the file.
+
+\startfunctioncall
+function(<table> env)
+ return <string> line
+end
+\stopfunctioncall
+
+Your function should return either a string or \type {nil}. The value \type {nil}
+signals that the end of file has occurred, and will make \TEX\ call the optional
+\type {close} function next.
+
+\subsubsection{\type {close}}
+
+\topicindex{callbacks+closing files}
+
+\LUATEX\ will run this optional function when it decides to close the file.
+
+\startfunctioncall
+function(<table> env)
+end
+\stopfunctioncall
+
+Your function should not return any value.
+
+\stopsection
+
+\startsection[title={Data processing callbacks}][library=callback]
+
+\subsection{\cbk {process_jobname}}
+
+\topicindex{callbacks+jobname}
+
+This callback allows you to change the jobname given by \prm {jobname} in \TEX\
+and \type {tex.jobname} in Lua. It does not affect the internal job name or the
+name of the output or log files.
+
+\startfunctioncall
+function(<string> jobname)
+ return <string> adjusted_jobname
+end
+\stopfunctioncall
+
+The only argument is the actual job name; you should not use \type {tex.jobname}
+inside this function or infinite recursion may occur. If you return \type {nil},
+\LUATEX\ will pretend your callback never happened. This callback does not
+replace any internal code.
+
+\stopsection
+
+\startsection[title={Node list processing callbacks}][library=callback]
+
+The description of nodes and node lists is in~\in{chapter}[nodes].
+
+\subsection{\cbk {contribute_filter}}
+
+\topicindex{callbacks+contributions}
+
+This callback is called when \LUATEX\ adds contents to list:
+
+\startfunctioncall
+function(<string> extrainfo)
+end
+\stopfunctioncall
+
+The string reports the group code. From this you can deduce from
+what list you can give a treat.
+
+\starttabulate[|l|p|]
+\DB value \BC explanation \NC \NR
+\TB
+\NC \type{pre_box} \NC interline material is being added \NC \NR
+\NC \type{pre_adjust} \NC \prm {vadjust} material is being added \NC \NR
+\NC \type{box} \NC a typeset box is being added (always called) \NC \NR
+\NC \type{adjust} \NC \prm {vadjust} material is being added \NC \NR
+\LL
+\stoptabulate
+
+\subsection{\cbk {buildpage_filter}}
+
+\topicindex{callbacks+building pages}
+
+This callback is called whenever \LUATEX\ is ready to move stuff to the main
+vertical list. You can use this callback to do specialized manipulation of the
+page building stage like imposition or column balancing.
+
+\startfunctioncall
+function(<string> extrainfo)
+end
+\stopfunctioncall
+
+The string \type {extrainfo} gives some additional information about what \TEX's
+state is with respect to the \quote {current page}. The possible values for the
+\cbk {buildpage_filter} callback are:
+
+\starttabulate[|l|p|]
+\DB value \BC explanation \NC \NR
+\TB
+\NC \type{alignment} \NC a (partial) alignment is being added \NC \NR
+\NC \type{after_output} \NC an output routine has just finished \NC \NR
+\NC \type{new_graf} \NC the beginning of a new paragraph \NC \NR
+\NC \type{vmode_par} \NC \prm {par} was found in vertical mode \NC \NR
+\NC \type{hmode_par} \NC \prm {par} was found in horizontal mode \NC \NR
+\NC \type{insert} \NC an insert is added \NC \NR
+\NC \type{penalty} \NC a penalty (in vertical mode) \NC \NR
+\NC \type{before_display} \NC immediately before a display starts \NC \NR
+\NC \type{after_display} \NC a display is finished \NC \NR
+\NC \type{end} \NC \LUATEX\ is terminating (it's all over) \NC \NR
+\LL
+\stoptabulate
+
+\subsection{\cbk {build_page_insert}}
+
+\topicindex{callbacks+inserts}
+
+This callback is called when the pagebuilder adds an insert. There is not much
+control over this mechanism but this callback permits some last minute
+manipulations of the spacing before an insert, something that might be handy when
+for instance multiple inserts (types) are appended in a row.
+
+\startfunctioncall
+function(<number> n, <number> i)
+ return <number> register
+end
+\stopfunctioncall
+
+with
+
+\starttabulate[|l|p|]
+\DB value \BC explanation \NC \NR
+\TB
+\NC \type{n} \NC the insert class \NC \NR
+\NC \type{i} \NC the order of the insert \NC \NR
+\LL
+\stoptabulate
+
+The return value is a number indicating the skip register to use for the
+prepended spacing. This permits for instance a different top space (when \type
+{i} equals one) and intermediate space (when \type {i} is larger than one). Of
+course you can mess with the insert box but you need to make sure that \LUATEX\
+is happy afterwards.
+
+\subsection{\cbk {pre_linebreak_filter}}
+
+\topicindex{callbacks+linebreaks}
+
+This callback is called just before \LUATEX\ starts converting a list of nodes
+into a stack of \prm {hbox}es, after the addition of \prm {parfillskip}.
+
+\startfunctioncall
+function(<node> head, <string> groupcode)
+ return true | false | <node> newhead
+end
+\stopfunctioncall
+
+The string called \type {groupcode} identifies the nodelist's context within
+\TEX's processing. The range of possibilities is given in the table below, but
+not all of those can actually appear in \cbk {pre_linebreak_filter}, some are
+for the \cbk {hpack_filter} and \cbk {vpack_filter} callbacks that will be
+explained in the next two paragraphs.
+
+\starttabulate[|l|p|]
+\DB value \BC explanation \NC \NR
+\TB
+\NC \type{<empty>} \NC main vertical list \NC \NR
+\NC \type{hbox} \NC \prm {hbox} in horizontal mode \NC \NR
+\NC \type{adjusted_hbox} \NC \prm {hbox} in vertical mode \NC \NR
+\NC \type{vbox} \NC \prm {vbox} \NC \NR
+\NC \type{vtop} \NC \prm {vtop} \NC \NR
+\NC \type{align} \NC \prm {halign} or \prm {valign} \NC \NR
+\NC \type{disc} \NC discretionaries \NC \NR
+\NC \type{insert} \NC packaging an insert \NC \NR
+\NC \type{vcenter} \NC \prm {vcenter} \NC \NR
+\NC \type{local_box} \NC \lpr {localleftbox} or \lpr {localrightbox} \NC \NR
+\NC \type{split_off} \NC top of a \prm {vsplit} \NC \NR
+\NC \type{split_keep} \NC remainder of a \prm {vsplit} \NC \NR
+\NC \type{align_set} \NC alignment cell \NC \NR
+\NC \type{fin_row} \NC alignment row \NC \NR
+\LL
+\stoptabulate
+
+As for all the callbacks that deal with nodes, the return value can be one of
+three things:
+
+\startitemize
+\startitem
+ boolean \type {true} signals successful processing
+\stopitem
+\startitem
+ \type {<node>} signals that the \quote {head} node should be replaced by the
+ returned node
+\stopitem
+\startitem
+ boolean \type {false} signals that the \quote {head} node list should be
+ ignored and flushed from memory
+\stopitem
+\stopitemize
+
+This callback does not replace any internal code.
+
+\subsection{\cbk {linebreak_filter}}
+
+\topicindex{callbacks+linebreaks}
+
+This callback replaces \LUATEX's line breaking algorithm.
+
+\startfunctioncall
+function(<node> head, <boolean> is_display)
+ return <node> newhead
+end
+\stopfunctioncall
+
+The returned node is the head of the list that will be added to the main vertical
+list, the boolean argument is true if this paragraph is interrupted by a
+following math display.
+
+If you return something that is not a \type {<node>}, \LUATEX\ will apply the
+internal linebreak algorithm on the list that starts at \type {<head>}.
+Otherwise, the \type {<node>} you return is supposed to be the head of a list of
+nodes that are all allowed in vertical mode, and at least one of those has to
+represent a hbox. Failure to do so will result in a fatal error.
+
+Setting this callback to \type {false} is possible, but dangerous, because it is
+possible you will end up in an unfixable \quote {deadcycles loop}.
+
+\subsection{\type {append_to_vlist_filter}}
+
+\topicindex{callbacks+contributions}
+
+This callback is called whenever \LUATEX\ adds a box to a vertical list:
+
+\startfunctioncall
+function(<node> box, <string> locationcode, <number prevdepth>,
+ <boolean> mirrored)
+ return list, prevdepth
+end
+\stopfunctioncall
+
+It is ok to return nothing in which case you also need to flush the box or deal
+with it yourself. The prevdepth is also optional. Locations are \type {box},
+\type {alignment}, \type {equation}, \type {equation_number} and \type
+{post_linebreak}.
+
+\subsection{\cbk {post_linebreak_filter}}
+
+\topicindex{callbacks+linebreaks}
+
+This callback is called just after \LUATEX\ has converted a list of nodes into a
+stack of \prm {hbox}es.
+
+\startfunctioncall
+function(<node> head, <string> groupcode)
+ return true | false | <node> newhead
+end
+\stopfunctioncall
+
+This callback does not replace any internal code.
+
+\subsection{\cbk {hpack_filter}}
+
+\topicindex{callbacks+packing}
+
+This callback is called when \TEX\ is ready to start boxing some horizontal mode
+material. Math items and line boxes are ignored at the moment.
+
+\startfunctioncall
+function(<node> head, <string> groupcode, <number> size,
+ <string> packtype [, <string> direction] [, <node> attributelist])
+ return true | false | <node> newhead
+end
+\stopfunctioncall
+
+The \type {packtype} is either \type {additional} or \type {exactly}. If \type
+{additional}, then the \type {size} is a \type {\hbox spread ...} argument. If
+\type {exactly}, then the \type {size} is a \type {\hbox to ...}. In both cases,
+the number is in scaled points.
+
+The \type {direction} is either one of the three-letter direction specifier
+strings, or \type {nil}.
+
+This callback does not replace any internal code.
+
+\subsection{\cbk {vpack_filter}}
+
+\topicindex{callbacks+packing}
+
+This callback is called when \TEX\ is ready to start boxing some vertical mode
+material. Math displays are ignored at the moment.
+
+This function is very similar to the \cbk {hpack_filter}. Besides the fact
+that it is called at different moments, there is an extra variable that matches
+\TEX's \prm {maxdepth} setting.
+
+\startfunctioncall
+function(<node> head, <string> groupcode, <number> size, <string> packtype,
+ <number> maxdepth [, <string> direction] [, <node> attributelist]))
+ return true | false | <node> newhead
+end
+\stopfunctioncall
+
+This callback does not replace any internal code.
+
+\subsection{\type {hpack_quality}}
+
+\topicindex{callbacks+packing}
+
+This callback can be used to intercept the overfull messages that can result from
+packing a horizontal list (as happens in the par builder). The function takes a
+few arguments:
+
+\startfunctioncall
+function(<string> incident, <number> detail, <node> head, <number> first,
+ <number> last)
+ return <node> whatever
+end
+\stopfunctioncall
+
+The incident is one of \type {overfull}, \type {underfull}, \type {loose} or
+\type {tight}. The detail is either the amount of overflow in case of \type
+{overfull}, or the badness otherwise. The head is the list that is constructed
+(when protrusion or expansion is enabled, this is an intermediate list).
+Optionally you can return a node, for instance an overfull rule indicator. That
+node will be appended to the list (just like \TEX's own rule would).
+
+\subsection{\type {vpack_quality}}
+
+\topicindex{callbacks+packing}
+
+This callback can be used to intercept the overfull messages that can result from
+packing a vertical list (as happens in the page builder). The function takes a
+few arguments:
+
+\startfunctioncall
+function(<string> incident, <number> detail, <node> head, <number> first,
+ <number> last)
+end
+\stopfunctioncall
+
+The incident is one of \type {overfull}, \type {underfull}, \type {loose} or
+\type {tight}. The detail is either the amount of overflow in case of \type
+{overfull}, or the badness otherwise. The head is the list that is constructed.
+
+\subsection{\cbk {process_rule}}
+
+\topicindex{callbacks+rules}
+
+This is an experimental callback. It can be used with rules of subtype~4
+(user). The callback gets three arguments: the node, the width and the
+height. The callback can use \type {pdf.print} to write code to the \PDF\
+file but beware of not messing up the final result. No checking is done.
+
+\subsection{\type {pre_output_filter}}
+
+\topicindex{callbacks+output}
+
+This callback is called when \TEX\ is ready to start boxing the box 255 for \prm
+{output}.
+
+\startfunctioncall
+function(<node> head, <string> groupcode, <number> size, <string> packtype,
+ <number> maxdepth [, <string> direction])
+ return true | false | <node> newhead
+end
+\stopfunctioncall
+
+This callback does not replace any internal code.
+
+\subsection{\cbk {hyphenate}}
+
+\topicindex{callbacks+hyphenation}
+
+\startfunctioncall
+function(<node> head, <node> tail)
+end
+\stopfunctioncall
+
+No return values. This callback has to insert discretionary nodes in the node
+list it receives.
+
+Setting this callback to \type {false} will prevent the internal discretionary
+insertion pass.
+
+\subsection{\cbk {ligaturing}}
+
+\topicindex{callbacks+ligature building}
+
+\startfunctioncall
+function(<node> head, <node> tail)
+end
+\stopfunctioncall
+
+No return values. This callback has to apply ligaturing to the node list it
+receives.
+
+You don't have to worry about return values because the \type {head} node that is
+passed on to the callback is guaranteed not to be a glyph_node (if need be, a
+temporary node will be prepended), and therefore it cannot be affected by the
+mutations that take place. After the callback, the internal value of the \quote
+{tail of the list} will be recalculated.
+
+The \type {next} of \type {head} is guaranteed to be non-nil.
+
+The \type {next} of \type {tail} is guaranteed to be nil, and therefore the
+second callback argument can often be ignored. It is provided for orthogonality,
+and because it can sometimes be handy when special processing has to take place.
+
+Setting this callback to \type {false} will prevent the internal ligature
+creation pass.
+
+You must not ruin the node list. For instance, the head normally is a local par node,
+and the tail a glue. Messing too much can push \LUATEX\ into panic mode.
+
+\subsection{\cbk {kerning}}
+
+\topicindex{callbacks+kerning}
+
+\startfunctioncall
+function(<node> head, <node> tail)
+end
+\stopfunctioncall
+
+No return values. This callback has to apply kerning between the nodes in the
+node list it receives. See \cbk {ligaturing} for calling conventions.
+
+Setting this callback to \type {false} will prevent the internal kern insertion
+pass.
+
+You must not ruin the node list. For instance, the head normally is a local par node,
+and the tail a glue. Messing too much can push \LUATEX\ into panic mode.
+
+\subsection{\type {insert_local_par}}
+
+Each paragraph starts with a local par node that keeps track of for instance
+the direction. You can hook a callback into the creator:
+
+\startfunctioncall
+function(<node> local_par, <string> location)
+end
+\stopfunctioncall
+
+There is no return value and you should make sure that the node stays valid
+as otherwise \TEX\ can get confused.
+
+\subsection{\cbk {mlist_to_hlist}}
+
+\topicindex{callbacks+math}
+
+This callback replaces \LUATEX's math list to node list conversion algorithm.
+
+\startfunctioncall
+function(<node> head, <string> display_type, <boolean> need_penalties)
+ return <node> newhead
+end
+\stopfunctioncall
+
+The returned node is the head of the list that will be added to the vertical or
+horizontal list, the string argument is either \quote {text} or \quote {display}
+depending on the current math mode, the boolean argument is \type {true} if
+penalties have to be inserted in this list, \type {false} otherwise.
+
+Setting this callback to \type {false} is bad, it will almost certainly result in
+an endless loop.
+
+\stopsection
+
+\startsection[title={Information reporting callbacks}][library=callback]
+
+\subsection{\cbk {pre_dump}}
+
+\topicindex{callbacks+dump}
+
+\startfunctioncall
+function()
+end
+\stopfunctioncall
+
+This function is called just before dumping to a format file starts. It does not
+replace any code and there are neither arguments nor return values.
+
+\subsection{\cbk {start_run}}
+
+\topicindex{callbacks+job run}
+
+\startfunctioncall
+function()
+end
+\stopfunctioncall
+
+This callback replaces the code that prints \LUATEX's banner. Note that for
+successful use, this callback has to be set in the \LUA\ initialization script,
+otherwise it will be seen only after the run has already started.
+
+\subsection{\cbk {stop_run}}
+
+\topicindex{callbacks+job run}
+
+\startfunctioncall
+function()
+end
+\stopfunctioncall
+
+This callback replaces the code that prints \LUATEX's statistics and \quote
+{output written to} messages. The engine can still do housekeeping and therefore
+you should not rely on this hook for postprocessing the \PDF\ or log file.
+
+\subsection{\cbk {show_error_hook}}
+
+\topicindex{callbacks+errors}
+
+\startfunctioncall
+function()
+end
+\stopfunctioncall
+
+This callback is run from inside the \TEX\ error function, and the idea is to
+allow you to do some extra reporting on top of what \TEX\ already does (none of
+the normal actions are removed). You may find some of the values in the \type
+{status} table useful. This callback does not replace any internal code.
+
+\subsection{\cbk {show_error_message}}
+
+\topicindex{callbacks+errors}
+
+\startfunctioncall
+function()
+end
+\stopfunctioncall
+
+This callback replaces the code that prints the error message. The usual
+interaction after the message is not affected.
+
+\subsection{\cbk {show_lua_error_hook}}
+
+\topicindex{callbacks+errors}
+
+\startfunctioncall
+function()
+end
+\stopfunctioncall
+
+This callback replaces the code that prints the extra \LUA\ error message.
+
+\subsection{\cbk {start_file}}
+
+\topicindex{callbacks+files}
+
+\startfunctioncall
+function(category,filename)
+end
+\stopfunctioncall
+
+This callback replaces the code that prints \LUATEX's when a file is opened like
+\type {(filename} for regular files. The category is a number:
+
+\starttabulate[|c|l|]
+\DB value \BC meaning \NC \NR
+\TB
+\NC 1 \NC a normal data file, like a \TEX\ source \NC \NR
+\NC 2 \NC a font map coupling font names to resources \NC \NR
+\NC 3 \NC an image file (\type {png}, \type {pdf}, etc) \NC \NR
+\NC 4 \NC an embedded font subset \NC \NR
+\NC 5 \NC a fully embedded font \NC \NR
+\LL
+\stoptabulate
+
+\subsection{\cbk {stop_file}}
+
+\topicindex{callbacks+files}
+
+\startfunctioncall
+function(category)
+end
+\stopfunctioncall
+
+This callback replaces the code that prints \LUATEX's when a file is closed like
+the \type {)} for regular files.
+
+\subsection{\cbk {wrapup_run}}
+
+\topicindex{callbacks+wrapping up}
+
+This callback is called after the \PDF\ and log files are closed. Use it at your own
+risk.
+
+\stopsection
+
+\startsection[title={Font-related callbacks}][library=callback]
+
+\subsection{\cbk {define_font}}
+
+\topicindex{callbacks+fonts}
+
+\startfunctioncall
+function(<string> name, <number> size)
+ return <number> id
+end
+\stopfunctioncall
+
+The string \type {name} is the filename part of the font specification, as given
+by the user.
+
+The number \type {size} is a bit special:
+
+\startitemize[packed]
+\startitem
+ If it is positive, it specifies an \quote{at size} in scaled points.
+\stopitem
+\startitem
+ If it is negative, its absolute value represents a \quote {scaled} setting
+ relative to the design size of the font.
+\stopitem
+\stopitemize
+
+The font can be defined with \type {font.define} which returns a font identifier
+that can be returned in the callback. So, contrary to \LUATEX, in \LUAMETATEX\
+we only accept a number.
+
+The internal structure of the \type {font} table that is passed to \type
+{font.define} is explained in \in {chapter} [fonts]. That table is saved
+internally, so you can put extra fields in the table for your later \LUA\ code to
+use. In alternative, \type {retval} can be a previously defined fontid. This is
+useful if a previous definition can be reused instead of creating a whole new
+font structure.
+
+Setting this callback to \type {false} is pointless as it will prevent font
+loading completely but will nevertheless generate errors.
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-differences.tex b/doc/context/sources/general/manuals/luametatex/luametatex-differences.tex
new file mode 100644
index 000000000..3e164c711
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-differences.tex
@@ -0,0 +1,209 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-differences
+
+\startchapter[reference=differences,title={Differences with \LUATEX}]
+
+As \LUAMETATEX\ is a leaner and meaner \LUATEX, this chapter will discuss
+what is gone. We start with the primitives that were dropped.
+
+\starttabulate[|l|pl|]
+\NC fonts \NC \type {\letterspacefont}
+ \type {\copyfont}
+ \type {\expandglyphsinfont}
+ \type {\ignoreligaturesinfont}
+ \type {\tagcode}
+ \NC \NR
+\NC backend \NC \type {\dviextension}
+ \type {\dvivariable }
+ \type {\dvifeedback}
+ \type {\pdfextension}
+ \type {\pdfvariable }
+ \type {\pdffeedback}
+ \type {\dviextension}
+ \type {\draftmode}
+ \type {\outputmode}
+ \NC \NR
+\NC dimensions \NC \type {\pageleftoffset}
+ \type {\pagerightoffset}
+ \type {\pagetopoffset}
+ \type {\pagebottomoffset}
+ \type {\pageheight}
+ \type {\pagewidth}
+ \NC \NR
+\NC resources \NC \type {\saveboxresource}
+ \type {\useboxresource}
+ \type {\lastsavedboxresourceindex}
+ \type {\saveimageresource}
+ \type {\useimageresource}
+ \type {\lastsavedimageresourceindex}
+ \type {\lastsavedimageresourcepages}
+ \NC \NR
+\NC positioning \NC \type {\savepos}
+ \type {\lastxpos}
+ \type {\lastypos}
+ \NC \NR
+\NC directions \NC \type {\textdir}
+ \type {\linedir}
+ \type {\mathdir}
+ \type {\pardir}
+ \type {\pagedir}
+ \type {\bodydir}
+ \type {\pagedirection}
+ \type {\bodydirection}
+ \NC \NR
+\NC randomizer \NC \type {\randomseed}
+ \type {\setrandomseed}
+ \type {\normaldeviate}
+ \type {\uniformdeviate}
+ \NC \NR
+\NC utilities \NC \type {\synctex}
+ \NC \NR
+\NC extensions \NC \type {\latelua}
+ \type {\lateluafunction}
+ \type {\immediate}
+ \type {\openout}
+ \type {\write}
+ \type {\closeout}
+ \NC \NR
+\NC control \NC \type {\suppressfontnotfounderror}
+ \type {\suppresslongerror}
+ \type {\suppressprimitiveerror}
+ \type {\suppressmathparerror}
+ \type {\suppressifcsnameerror}
+ \type {\suppressoutererror}
+ \type {\mathoption}
+ \NC \NR
+\NC whatever \NC \type {\primitive}
+ \type {\ifprimitive}
+ \NC \NR
+\NC ignored \NC \type {\long}
+ \type {\outer}
+ \type {\mag}
+ \NC \NR
+\stoptabulate
+
+The resources and positioning primitives are actually useful but can be defined
+as macros that (via \LUA) inject nodes in the input that suit the macro package
+and backend. The three||letter direction primitives are gone and the numeric
+variants are now leading. There is no need for page and body related directions
+and they don't work well in \LUATEX\ anyway. We only have two directions left.
+
+The primitive related extensions were not that useful and reliable so they have
+been removed. There are some new variants that will be discussed later. The \type
+{\outer} and \type {\long} prefixes are gone as they don't make much sense
+nowadays and them becoming dummies opened the way to something new, again to be
+discussed elsewhere. I don't think that (\CONTEXT) users will notice it. The
+\type {\suppress..} features are now default.
+
+The \type {\shipout} primitive does no ship out but just erases the content of
+the box, if that hasn't happened already in another way.
+
+The extension primitives relate to the backend (when not immediate) and can be
+implemented as part of a backend design using generic whatsits. There is only one
+type of whatsit now. In fact we're now closer to original \TEX\ with respect to
+the extensions.
+
+The \type {img} library has been removed as it's rather bound to the backend. The
+\type {slunicode} library is also gone. There are some helpers in the string
+library that can be used instead and one can write additional \LUA\ code if
+needed. There is no longer a \type {pdf} backend library.
+
+In the \type {node}, \type {tex} and \type {status} library we no longer have
+helpers and variables that relate to the backend. The \LUAMETATEX\ engine is in
+principle \DVI\ and \PDF\ unaware. There are only generic whatsit nodes that can
+be used for some management related tasks. For instance you can use them to
+implement user nodes.
+
+The \KPSE\ library is no longer built|-|in. Because there is no backend, quite
+some file related callbacks could go away. The following file related callbacks
+remained (till now):
+
+\starttyping
+find_write_file find_data_file find_format_file
+open_data_file read_data_file
+\stoptyping
+
+Also callbacks related to errors stay:
+
+\starttyping
+show_error_hook show_lua_error_hook,
+show_error_message show_warning_message
+\stoptyping
+
+The (job) management hooks are kept:
+
+\starttyping
+process_jobname
+start_run stop_run wrapup_run
+pre_dump
+start_file stop_file
+\stoptyping
+
+Because we use a more generic whatsit model, there is a new callback:
+
+\starttyping
+show_whatsit
+\stoptyping
+
+Being the core of extensibility, the typesetting callbacks of course stayed. This
+is what we ended up with:
+
+\starttyping
+find_log_file, find_data_file, find_format_file, open_data_file, read_data_file,
+process_jobname, start_run, stop_run, define_font, pre_output_filter,
+buildpage_filter, hpack_filter, vpack_filter, hyphenate, ligaturing, kerning,
+pre_linebreak_filter, linebreak_filter, post_linebreak_filter,
+append_to_vlist_filter, mlist_to_hlist, pre_dump, start_file, stop_file,
+handle_error_hook, show_error_hook, show_lua_error_hook, show_error_message,
+show_warning_message, hpack_quality, vpack_quality, insert_local_par,
+contribute_filter, build_page_insert, wrapup_run, new_graf, make_extensible,
+show_whatsit, terminal_input,
+\stoptyping
+
+As in \LUATEX\ font loading happens with the following callback. This time it
+really needs to be set because there is no built|-|in font loader.
+
+\starttyping
+define_font
+\stoptyping
+
+There are all kind of subtle differences in the implementation, for instance we
+no longer intercept \type {*} and \type {&} as these were already replaced long
+ago in \TEX\ engines by command line options. Talking of options, only a few are
+left.
+
+We took our time for reaching a stable state in \LUATEX. Among the reasons is the
+fact that most was experimented with in \CONTEXT. It took many man|-|years to
+decide what to keep and how to do things. Of course there are places when things
+can be improved and it might happen in \LUAMETATEX. Contrary to what is sometimes
+suggested, the \LUATEX|-|\CONTEXT\ \MKIV\ combination (assuming matched versions)
+has been quite stable. It made no sense otherwise. Most \CONTEXT\ functionality
+didn't change much at the user level. Of course there have been issues, as is
+natural with everything new and beta, but we have a fast update cycle.
+
+The same is true for \LUAMETATEX\ and \CONTEXT\ \LMTX: it can be used for
+production as usual and in practice \CONTEXT\ users tend to use the beta
+releases, which proves this. Of course, of you use low level features that are
+experimental you're on your own. Also, as with \LUATEX\ it might take many years
+before a long term stable is defined. The good news is that, the source code
+being part of the \CONTEXT\ distribution, there is always a properly working,
+more of less long term stable, snapshot.
+
+The error reporting subsystem has been redone a little but is still fundamentally
+the same. We don't really assume interactive usage but if someone uses it, it
+might be noticed that it is not possible to backtrack or inject something. Of
+course it is no big deal to implement all that in \LUA\ if needed. It removes a
+system dependency and makes for a bit cleaner code.
+
+There are new primitives too as well as some extensions to existing primitive
+functionality. These are described in following chapters but there might be
+hidden treasures in the binary. If you locate them, don't automatically assume
+them to stay, some might be part of experiments!
+
+\stopchapter
+
+\stopcomponent
+
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-enhancements.tex b/doc/context/sources/general/manuals/luametatex/luametatex-enhancements.tex
new file mode 100644
index 000000000..34e717a72
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-enhancements.tex
@@ -0,0 +1,1781 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-enhancements
+
+\startchapter[reference=enhancements,title={Basic \TEX\ enhancements}]
+
+\startsection[title={Introduction}]
+
+\startsubsection[title={Primitive behaviour}]
+
+From day one, \LUATEX\ has offered extra features compared to the superset of
+\PDFTEX, which includes \ETEX, and \ALEPH. This has not been limited to the
+possibility to execute \LUA\ code via \prm {directlua}, but \LUATEX\ also adds
+functionality via new \TEX|-|side primitives or extensions to existing ones. The
+same is true fir \LUAMETATEX. Some primitives have \type {luatex} in their name
+and there will be no \type {luametatex} variants. This is because we consider
+\LUAMETATEX\ to be \LUATEX 2\high{+}.
+
+Contrary to the \LUATEX\ engine \LUAMETATEX\ enables all its primitives. You can
+clone (a selection of) primitives with a different prefix, like:
+
+\starttyping
+\directlua { tex.enableprimitives('normal',tex.extraprimitives()) }
+\stoptyping
+
+The \type {extraprimitives} function returns the whole list or a subset,
+specified by one or more keywords \type {core}, \type {tex}, \type {etex} or
+\type {luatex}. \footnote {At some point this function might be changed to return
+the whole list always}.
+
+But be aware that the curly braces may not have the proper \prm {catcode}
+assigned to them at this early time (giving a \quote {Missing number} error), so
+it may be needed to put these assignments before the above line:
+
+\starttyping
+\catcode `\{=1
+\catcode `\}=2
+\stoptyping
+
+More fine|-|grained primitives control is possible and you can look up the
+details in \in {section} [luaprimitives]. For simplicity's sake, this manual
+assumes that you have executed the \prm {directlua} command as given above.
+
+The startup behaviour documented above is considered stable in the sense that
+there will not be backward|-|incompatible changes any more. We have promoted some
+rather generic \PDFTEX\ primitives to core \LUATEX\ ones, and a few that we
+inherited from \ALEPH\ (\OMEGA) are also promoted. Effectively this means that we
+now only have the \type {tex}, \type {etex} and \type {luatex} sets left.
+
+\stopsubsection
+
+\startsubsection[title={Experiments}]
+
+There are a few extensions to the engine regarding the macro machinery. Some are
+already well tested but others are (still) experimental. Although they are likely
+to stay, their exact behaviour might evolve. Because \LUAMETATEX\ is also used
+for experiments, this is not a problem. We can always decide to also add some of
+what is discussed here to \LUATEX, but it will happen with a delay.
+
+There are all kind of small improvements that might find their way into stock
+\LUATEX: a few more helpers, some cleanup of code, etc. We'll see. In any case,
+if you play with these before they are declared stable, unexpected side effects
+are what you have to accept.
+
+\stopsubsection
+
+\startsubsection[title={Version information}]
+
+\startsubsubsection[title={\lpr {luatexbanner}, \lpr {luatexversion} and \lpr {luatexrevision}}]
+
+\topicindex{version}
+\topicindex{banner}
+
+There are three primitives to test the version of \LUATEX\ (and \LUAMETATEX):
+
+\unexpanded\def\VersionHack#1% otherwise different luatex and luajittex runs
+ {\ctxlua{%
+ local banner = "\luatexbanner"
+ local banner = string.match(banner,"(.+)\letterpercent(") or banner
+ context(string.gsub(banner ,"jit",""))%
+ }}
+
+\starttabulate[|l|l|pl|]
+\DB primitive \BC value
+ \BC explanation \NC \NR
+\TB
+\NC \lpr {luatexbanner} \NC \VersionHack{\luatexbanner}
+ \NC the banner reported on the command line \NC \NR
+\NC \lpr {luatexversion} \NC \the\luatexversion
+ \NC a combination of major and minor number \NC \NR
+\NC \lpr {luatexrevision} \NC \luatexrevision
+ \NC the revision number, the current value is \NC \NR
+\LL
+\stoptabulate
+
+A version is defined as follows:
+
+\startitemize
+\startitem
+ The major version is the integer result of \lpr {luatexversion} divided by
+ 100. The primitive is an \quote {internal variable}, so you may need to prefix
+ its use with \prm {the} depending on the context.
+\stopitem
+\startitem
+ The minor version is the two|-|digit result of \lpr {luatexversion} modulo 100.
+\stopitem
+\startitem
+ The revision is reported by \lpr {luatexrevision}. This primitive expands to
+ a positive integer.
+\stopitem
+\startitem
+ The full version number consists of the major version, minor version and
+ revision, separated by dots.
+\stopitem
+\stopitemize
+
+\stopsubsubsection
+
+The \LUAMETATEX\ version number starts at 2 in order to prevent a clash with
+\LUATEX, and the version commands are the same. This is a way to indicate that
+these projects are related.
+
+\startsubsubsection[title={\lpr {formatname}}]
+
+\topicindex{format}
+
+The \lpr {formatname} syntax is identical to \prm {jobname}. In \INITEX, the
+expansion is empty. Otherwise, the expansion is the value that \prm {jobname} had
+during the \INITEX\ run that dumped the currently loaded format. You can use this
+token list to provide your own version info.
+
+\stopsubsubsection
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={\UNICODE\ text support}]
+
+\startsubsection[title={Extended ranges}]
+
+\topicindex{\UNICODE}
+
+Text input and output is now considered to be \UNICODE\ text, so input characters
+can use the full range of \UNICODE\ ($2^{20}+2^{16}-1 = \hbox{0x10FFFF}$). Later
+chapters will talk of characters and glyphs. Although these are not
+interchangeable, they are closely related. During typesetting, a character is
+always converted to a suitable graphic representation of that character in a
+specific font. However, while processing a list of to|-|be|-|typeset nodes, its
+contents may still be seen as a character. Inside the engine there is no clear
+separation between the two concepts. Because the subtype of a glyph node can be
+changed in \LUA\ it is up to the user. Subtypes larger than 255 indicate that
+font processing has happened.
+
+A few primitives are affected by this, all in a similar fashion: each of them has
+to accommodate for a larger range of acceptable numbers. For instance, \prm
+{char} now accepts values between~0 and $1{,}114{,}111$. This should not be a
+problem for well|-|behaved input files, but it could create incompatibilities for
+input that would have generated an error when processed by older \TEX|-|based
+engines. The affected commands with an altered initial (left of the equal sign)
+or secondary (right of the equal sign) value are: \prm {char}, \prm {lccode},
+\prm {uccode}, \lpr {hjcode}, \prm {catcode}, \prm {sfcode}, \lpr {efcode}, \lpr
+{lpcode}, \lpr {rpcode}, \prm {chardef}.
+
+As far as the core engine is concerned, all input and output to text files is
+\UTF-8 encoded. Input files can be pre|-|processed using the \type {reader}
+callback. This will be explained in \in {section} [iocallback]. Normalization of
+the \UNICODE\ input is on purpose not built|-|in and can be handled by a macro
+package during callback processing. We have made some practical choices and the
+user has to live with those.
+
+Output in byte|-|sized chunks can be achieved by using characters just outside of
+the valid \UNICODE\ range, starting at the value $1{,}114{,}112$ (0x110000). When
+the time comes to print a character $c>=1{,}114{,}112$, \LUATEX\ will actually
+print the single byte corresponding to $c$ minus 1{,}114{,}112.
+
+Contrary to other \TEX\ engines, the output to the terminal is as|-|is so there
+is no escaping with \type {^^}. We operate in a \UTF\ universe.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {Uchar}}]
+
+\topicindex{\UNICODE}
+
+The expandable command \lpr {Uchar} reads a number between~0 and $1{,}114{,}111$
+and expands to the associated \UNICODE\ character.
+
+\stopsubsection
+
+\startsubsection[title={Extended tables}]
+
+All traditional \TEX\ and \ETEX\ registers can be 16-bit numbers. The affected
+commands are:
+
+\startfourcolumns
+\startlines
+\prm {count}
+\prm {dimen}
+\prm {skip}
+\prm {muskip}
+\prm {marks}
+\prm {toks}
+\prm {countdef}
+\prm {dimendef}
+\prm {skipdef}
+\prm {muskipdef}
+\prm {toksdef}
+\prm {insert}
+\prm {box}
+\prm {unhbox}
+\prm {unvbox}
+\prm {copy}
+\prm {unhcopy}
+\prm {unvcopy}
+\prm {wd}
+\prm {ht}
+\prm {dp}
+\prm {setbox}
+\prm {vsplit}
+\stoplines
+\stopfourcolumns
+
+Fonts are loaded via \LUA\ and a minimal amount of information is kept at the
+\TEX\ end. Sharing resources is up to the loaders. The engine doesn't really care
+about what a character (or glyph) number represents (an \UNICODE\ or index) as it
+only is interested in dimensions.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Attributes}]
+
+\startsubsection[title={Nodes}]
+
+\topicindex {nodes}
+
+When \TEX\ reads input it will interpret the stream according to the properties
+of the characters. Some signal a macro name and trigger expansion, others open
+and close groups, trigger math mode, etc. What's left over becomes the typeset
+text. Internally we get a linked list of nodes. Characters become \nod {glyph}
+nodes that have for instance a \type {font} and \type {char} property and \typ
+{\kern 10pt} becomes a \nod {kern} node with a \type {width} property. Spaces are
+alien to \TEX\ as they are turned into \nod {glue} nodes. So, a simple paragraph
+is mostly a mix of sequences of \nod {glyph} nodes (words) and \nod {glue} nodes
+(spaces). A node can have a subtype to that it can be recognized as for instance
+a space related glue.
+
+The sequences of characters at some point are extended with \nod {disc} nodes
+that relate to hyphenation. After that font logic can be applied and we get a
+list where some characters can be replaced, for instance multiple characters can
+become one ligature, and font kerns can be injected. This is driven by the
+font properties.
+
+Boxes (like \prm {hbox} and \prm {vbox}) become \nod {hlist} or \nod {vlist}
+nodes with \type {width}, \type {height}, \type {depth} and \type {shift}
+properties and a pointer \type {list} to its actual content. Boxes can be
+constructed explicitly or can be the result of subprocesses. For instance, when
+lines are broken into paragraphs, the lines are a linked list of \nod {hlist}
+nodes, possibly with glue and penalties in between.
+
+Internally nodes have a number. This number is actually an index in the memory
+used to store nodes.
+
+So, to summarize: all that you enter as content eventually becomes a node, often
+as part of a (nested) list structure. They have a relative small memory footprint
+and carry only the minimal amount of information needed. In traditional \TEX\ a
+character node only held the font and slot number, in \LUATEX\ we also store some
+language related information, the expansion factor, etc. Now that we have access
+to these nodes from \LUA\ it makes sense to be able to carry more information
+with an node and this is where attributes kick in.
+
+\stopsubsection
+
+\startsubsection[title={Attribute registers}]
+
+\topicindex {attributes}
+
+Attributes are a completely new concept in \LUATEX. Syntactically, they behave a
+lot like counters: attributes obey \TEX's nesting stack and can be used after
+\prm {the} etc.\ just like the normal \prm {count} registers.
+
+\startsyntax
+\attribute <16-bit number> <optional equals> <32-bit number>!crlf
+\attributedef <csname> <optional equals> <16-bit number>
+\stopsyntax
+
+Conceptually, an attribute is either \quote {set} or \quote {unset}. Unset
+attributes have a special negative value to indicate that they are unset, that
+value is the lowest legal value: \type {-"7FFFFFFF} in hexadecimal, a.k.a.
+$-2147483647$ in decimal. It follows that the value \type {-"7FFFFFFF} cannot be
+used as a legal attribute value, but you {\it can\/} assign \type {-"7FFFFFFF} to
+\quote {unset} an attribute. All attributes start out in this \quote {unset}
+state in \INITEX.
+
+Attributes can be used as extra counter values, but their usefulness comes mostly
+from the fact that the numbers and values of all \quote {set} attributes are
+attached to all nodes created in their scope. These can then be queried from any
+\LUA\ code that deals with node processing. Further information about how to use
+attributes for node list processing from \LUA\ is given in~\in {chapter}[nodes].
+
+Attributes are stored in a sorted (sparse) linked list that are shared when
+possible. This permits efficient testing and updating. You can define many
+thousands of attributes but normally such a large number makes no sense and is
+also not that efficient because each node carries a (possibly shared) link to a
+list of currently set attributes. But they are a convenient extension and one of
+the first extensions we implemented in \LUATEX.
+
+In \LUAMETATEX\ we try to minimize the memory footprint and creation of these
+attribute lists more aggressive sharing them. This feature is still somewhat
+experimental.
+
+\stopsubsection
+
+\startsubsection[title={Box attributes}]
+
+\topicindex {attributes}
+\topicindex {boxes}
+
+Nodes typically receive the list of attributes that is in effect when they are
+created. This moment can be quite asynchronous. For example: in paragraph
+building, the individual line boxes are created after the \prm {par} command has
+been processed, so they will receive the list of attributes that is in effect
+then, not the attributes that were in effect in, say, the first or third line of
+the paragraph.
+
+Similar situations happen in \LUATEX\ regularly. A few of the more obvious
+problematic cases are dealt with: the attributes for nodes that are created
+during hyphenation, kerning and ligaturing borrow their attributes from their
+surrounding glyphs, and it is possible to influence box attributes directly.
+
+When you assemble a box in a register, the attributes of the nodes contained in
+the box are unchanged when such a box is placed, unboxed, or copied. In this
+respect attributes act the same as characters that have been converted to
+references to glyphs in fonts. For instance, when you use attributes to implement
+color support, each node carries information about its eventual color. In that
+case, unless you implement mechanisms that deal with it, applying a color to
+already boxed material will have no effect. Keep in mind that this
+incompatibility is mostly due to the fact that separate specials and literals are
+a more unnatural approach to colors than attributes.
+
+It is possible to fine-tune the list of attributes that are applied to a \type
+{hbox}, \type {vbox} or \type {vtop} by the use of the keyword \type {attr}. The
+\type {attr} keyword(s) should come before a \type {to} or \type {spread}, if
+that is also specified. An example is:
+
+\startbuffer[tex]
+\attribute997=123
+\attribute998=456
+\setbox0=\hbox {Hello}
+\setbox2=\hbox attr 999 = 789 attr 998 = -"7FFFFFFF{Hello}
+\stopbuffer
+
+\startbuffer[lua]
+ for b=0,2,2 do
+ for a=997, 999 do
+ tex.sprint("box ", b, " : attr ",a," : ",tostring(tex.box[b] [a]))
+ tex.sprint("\\quad\\quad")
+ tex.sprint("list ",b, " : attr ",a," : ",tostring(tex.box[b].list[a]))
+ tex.sprint("\\par")
+ end
+ end
+\stopbuffer
+
+\typebuffer[tex]
+
+Box 0 now has attributes 997 and 998 set while box 2 has attributes 997 and 999
+set while the nodes inside that box will all have attributes 997 and 998 set.
+Assigning the maximum negative value causes an attribute to be ignored.
+
+To give you an idea of what this means at the \LUA\ end, take the following
+code:
+
+\typebuffer[lua]
+
+Later we will see that you can access properties of a node. The boxes here are so
+called \nod {hlist} nodes that have a field \type {list} that points to the
+content. Because the attributes are a list themselves you can access them by
+indexing the node (here we do that with \type {[a]}. Running this snippet gives:
+
+\start
+ \getbuffer[tex]
+ \startpacked \tt
+ \ctxluabuffer[lua]
+ \stoppacked
+\stop
+
+Because some values are not set we need to apply the \type {tostring} function
+here so that we get the word \type {nil}.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={\LUA\ related primitives}]
+
+\startsubsection[title={\prm {directlua}}]
+
+In order to merge \LUA\ code with \TEX\ input, a few new primitives are needed.
+The primitive \prm {directlua} is used to execute \LUA\ code immediately. The
+syntax is
+
+\startsyntax
+\directlua <general text>
+\stopsyntax
+
+The \syntax {<general text>} is expanded fully, and then fed into the \LUA\
+interpreter. After reading and expansion has been applied to the \syntax
+{<general text>}, the resulting token list is converted to a string as if it was
+displayed using \type {\the\toks}. On the \LUA\ side, each \prm {directlua} block
+is treated as a separate chunk. In such a chunk you can use the \type {local}
+directive to keep your variables from interfering with those used by the macro
+package.
+
+The conversion to and from a token list means that you normally can not use \LUA\
+line comments (starting with \type {--}) within the argument. As there typically
+will be only one \quote {line} the first line comment will run on until the end
+of the input. You will either need to use \TEX|-|style line comments (starting
+with \%), or change the \TEX\ category codes locally. Another possibility is to
+say:
+
+\starttyping
+\begingroup
+\endlinechar=10
+\directlua ...
+\endgroup
+\stoptyping
+
+Then \LUA\ line comments can be used, since \TEX\ does not replace line endings
+with spaces. Of course such an approach depends on the macro package that you
+use.
+
+The \prm {directlua} command is expandable. Since it passes \LUA\ code to the
+\LUA\ interpreter its expansion from the \TEX\ viewpoint is usually empty.
+However, there are some \LUA\ functions that produce material to be read by \TEX,
+the so called print functions. The most simple use of these is \type
+{tex.print(<string> s)}. The characters of the string \type {s} will be placed on
+the \TEX\ input buffer, that is, \quote {before \TEX's eyes} to be read by \TEX\
+immediately. For example:
+
+\startbuffer
+\count10=20
+a\directlua{tex.print(tex.count[10]+5)}b
+\stopbuffer
+
+\typebuffer
+
+expands to
+
+\getbuffer
+
+Here is another example:
+
+\startbuffer
+$\pi = \directlua{tex.print(math.pi)}$
+\stopbuffer
+
+\typebuffer
+
+will result in
+
+\getbuffer
+
+Note that the expansion of \prm {directlua} is a sequence of characters, not of
+tokens, contrary to all \TEX\ commands. So formally speaking its expansion is
+null, but it places material on a pseudo-file to be immediately read by \TEX, as
+\ETEX's \prm {scantokens}. For a description of print functions look at \in
+{section} [sec:luaprint].
+
+Because the \syntax {<general text>} is a chunk, the normal \LUA\ error handling
+is triggered if there is a problem in the included code. The \LUA\ error messages
+should be clear enough, but the contextual information is still pretty bad.
+Often, you will only see the line number of the right brace at the end of the
+code.
+
+While on the subject of errors: some of the things you can do inside \LUA\ code
+can break up \LUAMETATEX\ pretty bad. If you are not careful while working with
+the node list interface, you may even end up with assertion errors from within
+the \TEX\ portion of the executable.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {luaescapestring}}]
+
+\topicindex {escaping}
+
+This primitive converts a \TEX\ token sequence so that it can be safely used as
+the contents of a \LUA\ string: embedded backslashes, double and single quotes,
+and newlines and carriage returns are escaped. This is done by prepending an
+extra token consisting of a backslash with category code~12, and for the line
+endings, converting them to \type {n} and \type {r} respectively. The token
+sequence is fully expanded.
+
+\startsyntax
+\luaescapestring <general text>
+\stopsyntax
+
+Most often, this command is not actually the best way to deal with the
+differences between \TEX\ and \LUA. In very short bits of \LUA\ code it is often
+not needed, and for longer stretches of \LUA\ code it is easier to keep the code
+in a separate file and load it using \LUA's \type {dofile}:
+
+\starttyping
+\directlua { dofile("mysetups.lua") }
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\lpr {luafunction}, \lpr {luafunctioncall} and \lpr {luadef}}]
+
+The \prm {directlua} commands involves tokenization of its argument (after
+picking up an optional name or number specification). The tokenlist is then
+converted into a string and given to \LUA\ to turn into a function that is
+called. The overhead is rather small but when you have millions of calls it can
+have some impact. For this reason there is a variant call available: \lpr
+{luafunction}. This command is used as follows:
+
+\starttyping
+\directlua {
+ local t = lua.get_functions_table()
+ t[1] = function() tex.print("!") end
+ t[2] = function() tex.print("?") end
+}
+
+\luafunction1
+\luafunction2
+\stoptyping
+
+Of course the functions can also be defined in a separate file. There is no limit
+on the number of functions apart from normal \LUA\ limitations. Of course there
+is the limitation of no arguments but that would involve parsing and thereby give
+no gain. The function, when called in fact gets one argument, being the index, so
+in the following example the number \type {8} gets typeset.
+
+\starttyping
+\directlua {
+ local t = lua.get_functions_table()
+ t[8] = function(slot) tex.print(slot) end
+}
+\stoptyping
+
+The \lpr {luafunctioncall} primitive does the same but is unexpandable, for
+instance in an \prm {edef}. In addition \LUATEX\ provides a definer:
+
+\starttyping
+ \luadef\MyFunctionA 1
+ \global\luadef\MyFunctionB 2
+\protected\global\luadef\MyFunctionC 3
+\stoptyping
+
+You should really use these commands with care. Some references get stored in
+tokens and assume that the function is available when that token expands. On the
+other hand, as we have tested this functionality in relative complex situations
+normal usage should not give problems.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {luabytecode} and \lpr {luabytecodecall}}]
+
+Analogue to the function callers discussed in the previous section we have byte
+code callers. Again the call variant is unexpandable.
+
+\starttyping
+\directlua {
+ lua.bytecode[9998] = function(s)
+ tex.sprint(s*token.scan_int())
+ end
+ lua.bytecode[5555] = function(s)
+ tex.sprint(s*token.scan_dimen())
+ end
+}
+\stoptyping
+
+This works with:
+
+\starttyping
+\luabytecode 9998 5 \luabytecode 5555 5sp
+\luabytecodecall9998 5 \luabytecodecall5555 5sp
+\stoptyping
+
+The variable \type {s} in the code is the number of the byte code register that
+can be used for diagnostic purposes. The advantage of bytecode registers over
+function calls is that they are stored in the format (but without upvalues).
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Catcode tables}]
+
+\startsubsection[title={Catcodes}]
+
+\topicindex {catcodes}
+
+Catcode tables are a new feature that allows you to switch to a predefined
+catcode regime in a single statement. You can have lots of different tables, but
+if you need a dozen you might wonder what you're doing. . This subsystem is
+backward compatible: if you never use the following commands, your document will
+not notice any difference in behaviour compared to traditional \TEX. The contents
+of each catcode table is independent from any other catcode table, and its
+contents is stored and retrieved from the format file.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {catcodetable}}]
+
+\startsyntax
+\catcodetable <15-bit number>
+\stopsyntax
+
+The primitive \lpr {catcodetable} switches to a different catcode table. Such a
+table has to be previously created using one of the two primitives below, or it
+has to be zero. Table zero is initialized by \INITEX.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {initcatcodetable}}]
+
+\startsyntax
+\initcatcodetable <15-bit number>
+\stopsyntax
+
+The primitive \lpr {initcatcodetable} creates a new table with catcodes
+identical to those defined by \INITEX. The new catcode table is allocated
+globally: it will not go away after the current group has ended. If the supplied
+number is identical to the currently active table, an error is raised. The
+initial values are:
+
+\starttabulate[|c|c|l|l|]
+\DB catcode \BC character \BC equivalent \BC category \NC \NR
+\TB
+\NC 0 \NC \tttf \letterbackslash \NC \NC \type {escape} \NC \NR
+\NC 5 \NC \tttf \letterhat\letterhat M \NC return \NC \type {car_ret} \NC \NR
+\NC 9 \NC \tttf \letterhat\letterhat @ \NC null \NC \type {ignore} \NC \NR
+\NC 10 \NC \tttf <space> \NC space \NC \type {spacer} \NC \NR
+\NC 11 \NC {\tttf a} \endash\ {\tttf z} \NC \NC \type {letter} \NC \NR
+\NC 11 \NC {\tttf A} \endash\ {\tttf Z} \NC \NC \type {letter} \NC \NR
+\NC 12 \NC everything else \NC \NC \type {other} \NC \NR
+\NC 14 \NC \tttf \letterpercent \NC \NC \type {comment} \NC \NR
+\NC 15 \NC \tttf \letterhat\letterhat ? \NC delete \NC \type {invalid_char} \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsection
+
+\startsubsection[title={\lpr {savecatcodetable}}]
+
+\startsyntax
+\savecatcodetable <15-bit number>
+\stopsyntax
+
+\lpr {savecatcodetable} copies the current set of catcodes to a new table with
+the requested number. The definitions in this new table are all treated as if
+they were made in the outermost level.
+
+The new table is allocated globally: it will not go away after the current group
+has ended. If the supplied number is the currently active table, an error is
+raised.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Tokens, commands and strings}]
+
+\startsubsection[title={\lpr {scantextokens}}]
+
+\topicindex {tokens+scanning}
+
+The syntax of \lpr {scantextokens} is identical to \prm {scantokens}. This
+primitive is a slightly adapted version of \ETEX's \prm {scantokens}. The
+differences are:
+
+\startitemize
+\startitem
+ The last (and usually only) line does not have a \prm {endlinechar}
+ appended.
+\stopitem
+\startitem
+ \lpr {scantextokens} never raises an EOF error, and it does not execute
+ \prm {everyeof} tokens.
+\stopitem
+\startitem
+ There are no \quote {\unknown\ while end of file \unknown} error tests
+ executed. This allows the expansion to end on a different grouping level or
+ while a conditional is still incomplete.
+\stopitem
+\stopitemize
+
+\stopsubsection
+
+\startsubsection[title={\lpr {toksapp}, \lpr {tokspre}, \lpr {etoksapp}, \lpr {etokspre},
+\lpr {gtoksapp}, \lpr {gtokspre}, \lpr {xtoksapp}, \lpr {xtokspre}}]
+
+Instead of:
+
+\starttyping
+\toks0\expandafter{\the\toks0 foo}
+\stoptyping
+
+you can use:
+
+\starttyping
+\etoksapp0{foo}
+\stoptyping
+
+The \type {pre} variants prepend instead of append, and the \type {e} variants
+expand the passed general text. The \type {g} and \type {x} variants are global.
+
+\stopsubsection
+
+\startsubsection[title={\prm {csstring}, \lpr {begincsname} and \lpr {lastnamedcs}}]
+
+These are somewhat special. The \prm {csstring} primitive is like
+\prm {string} but it omits the leading escape character. This can be
+somewhat more efficient than stripping it afterwards.
+
+The \lpr {begincsname} primitive is like \prm {csname} but doesn't create
+a relaxed equivalent when there is no such name. It is equivalent to
+
+\starttyping
+\ifcsname foo\endcsname
+ \csname foo\endcsname
+\fi
+\stoptyping
+
+The advantage is that it saves a lookup (don't expect much speedup) but more
+important is that it avoids using the \prm {if} test. The \lpr {lastnamedcs}
+is one that should be used with care. The above example could be written as:
+
+\starttyping
+\ifcsname foo\endcsname
+ \lastnamedcs
+\fi
+\stoptyping
+
+This is slightly more efficient than constructing the string twice (deep down in
+\LUATEX\ this also involves some \UTF8 juggling), but probably more relevant is
+that it saves a few tokens and can make code a bit more readable.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {clearmarks}}]
+
+\topicindex {marks}
+
+This primitive complements the \ETEX\ mark primitives and clears a mark class
+completely, resetting all three connected mark texts to empty. It is an
+immediate command.
+
+\startsyntax
+\clearmarks <16-bit number>
+\stopsyntax
+
+\stopsubsection
+
+\startsubsection[title={\lpr {alignmark} and \lpr {aligntab}}]
+
+The primitive \lpr {alignmark} duplicates the functionality of \type {#} inside
+alignment preambles, while \lpr {aligntab} duplicates the functionality of \type
+{&}.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {letcharcode}}]
+
+This primitive can be used to assign a meaning to an active character, as in:
+
+\starttyping
+\def\foo{bar} \letcharcode123=\foo
+\stoptyping
+
+This can be a bit nicer than using the uppercase tricks (using the property of
+\prm {uppercase} that it treats active characters special).
+
+\stopsubsection
+
+\startsubsection[title={\lpr {glet}}]
+
+This primitive is similar to:
+
+\starttyping
+\protected\def\glet{\global\let}
+\stoptyping
+
+but faster (only measurable with millions of calls) and probably more convenient
+(after all we also have \type {\gdef}).
+
+\stopsubsection
+
+\startsubsection[title={\lpr {expanded}, \lpr {immediateassignment} and \lpr {immediateassigned}}]
+
+\topicindex {expansion}
+
+The \lpr {expanded} primitive takes a token list and expands it content which can
+come in handy: it avoids a tricky mix of \prm {expandafter} and \prm {noexpand}.
+You can compare it with what happens inside the body of an \prm {edef}. But this
+kind of expansion it still doesn't expand some primitive operations.
+
+\startbuffer
+\newcount\NumberOfCalls
+
+\def\TestMe{\advance\NumberOfCalls1 }
+
+\edef\Tested{\TestMe foo:\the\NumberOfCalls}
+\edef\Tested{\TestMe foo:\the\NumberOfCalls}
+\edef\Tested{\TestMe foo:\the\NumberOfCalls}
+
+\meaning\Tested
+\stopbuffer
+
+\typebuffer
+
+The result is a macro that has the not expanded code in its body:
+
+\getbuffer
+
+Instead we can define \tex {TestMe} in a way that expands the assignment
+immediately. You need of course to be aware of preventing look ahead interference
+by using a space or \tex {relax} (often an expression works better as it doesn't
+leave an \tex {relax}).
+
+\startbuffer
+\def\TestMe{\immediateassignment\advance\NumberOfCalls1 }
+
+\edef\Tested{\TestMe foo:\the\NumberOfCalls}
+\edef\Tested{\TestMe foo:\the\NumberOfCalls}
+\edef\Tested{\TestMe foo:\the\NumberOfCalls}
+
+\meaning\Tested
+\stopbuffer
+
+\typebuffer
+
+This time the counter gets updates and we don't see interference in the
+resulting \tex {Tested} macro:
+
+\getbuffer
+
+Here is a somewhat silly example of expanded comparison:
+
+\startbuffer
+\def\expandeddoifelse#1#2#3#4%
+ {\immediateassignment\edef\tempa{#1}%
+ \immediateassignment\edef\tempb{#2}%
+ \ifx\tempa\tempb
+ \immediateassignment\def\next{#3}%
+ \else
+ \immediateassignment\def\next{#4}%
+ \fi
+ \next}
+
+\edef\Tested
+ {(\expandeddoifelse{abc}{def}{yes}{nop}/%
+ \expandeddoifelse{abc}{abc}{yes}{nop})}
+
+\meaning\Tested
+\stopbuffer
+
+\typebuffer
+
+It gives:
+
+\getbuffer
+
+A variant is:
+
+\starttyping
+\def\expandeddoifelse#1#2#3#4%
+ {\immediateassigned{
+ \edef\tempa{#1}%
+ \edef\tempb{#2}%
+ }%
+ \ifx\tempa\tempb
+ \immediateassignment\def\next{#3}%
+ \else
+ \immediateassignment\def\next{#4}%
+ \fi
+ \next}
+\stoptyping
+
+The possible error messages are the same as using assignments in preambles of
+alignments and after the \prm {accent} command. The supported assignments are the
+so called prefixed commands (except box assignments).
+
+\stopsubsection
+
+\startsubsection[title={\lpr {ignorepars}}]
+
+This primitives is like \prm {ignorespaces} but also skips paragraph ending
+commands (normally \prm {par} and empty lines).
+
+\stopsubsection
+
+\startsubsection[title={\lpr {futureexpand}, \lpr {futureexpandis}, \lpr {futureexpandisap}}]
+
+These commands are use as:
+
+\starttyping
+\futureexpand\sometoken\whenfound\whennotfound
+\stoptyping
+
+When there is no match and a space was gobbled a space will be put back. The
+\type {is} variant doesn't do that while the \type {isap} even skips \type
+{\pars}, These characters stand for \quote {ignorespaces} and \quote
+{ignorespacesandpars}.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {aftergrouped}}]
+
+There is a new experimental feature that can inject multiple tokens to after the group
+ends. An example demonstrate its use:
+
+\startbuffer
+{
+ \aftergroup A \aftergroup B \aftergroup C
+test 1 : }
+
+{
+ \aftergrouped{What comes next 1}
+ \aftergrouped{What comes next 2}
+ \aftergrouped{What comes next 3}
+test 2 : }
+
+
+{
+ \aftergroup A \aftergrouped{What comes next 1}
+ \aftergroup B \aftergrouped{What comes next 2}
+ \aftergroup C \aftergrouped{What comes next 3}
+test 3 : }
+
+{
+ \aftergrouped{What comes next 1} \aftergroup A
+ \aftergrouped{What comes next 2} \aftergroup B
+ \aftergrouped{What comes next 3} \aftergroup C
+test 4 : }
+\stopbuffer
+
+\typebuffer
+
+This gives:
+
+\startpacked\getbuffer\stoppacked
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title=Conditions]
+
+\startsubsection[title={\lpr{ifabsnum} and \lpr {ifabsdim}}]
+
+There are two tests that we took from \PDFTEX:
+
+\startbuffer
+\ifabsnum -10 = 10
+ the same number
+\fi
+\ifabsdim -10pt = 10pt
+ the same dimension
+\fi
+\stopbuffer
+
+\typebuffer
+
+This gives
+
+\blank {\tt \getbuffer} \blank
+
+\stopsubsection
+
+\startsubsection[title={\lpr{ifcmpnum}, \lpr {ifcmpdim}, \lpr {ifnumval}, \lpr
+{ifdimval}, \lpr {ifchknum} and \lpr {ifchkdim}}]
+
+\topicindex {conditions+numbers}
+\topicindex {conditions+dimensions}
+\topicindex {numbers}
+\topicindex {dimensions}
+
+New are the ones that compare two numbers or dimensions:
+
+\startbuffer
+\ifcmpnum 5 8 less \or equal \else more \fi
+\ifcmpnum 5 5 less \or equal \else more \fi
+\ifcmpnum 8 5 less \or equal \else more \fi
+\stopbuffer
+
+\typebuffer \blank {\tt \getbuffer} \blank
+
+and
+
+\startbuffer
+\ifcmpdim 5pt 8pt less \or equal \else more \fi
+\ifcmpdim 5pt 5pt less \or equal \else more \fi
+\ifcmpdim 8pt 5pt less \or equal \else more \fi
+\stopbuffer
+
+\typebuffer \blank {\tt \getbuffer} \blank
+
+There are also some number and dimension tests. All four expose the \type {\else}
+branch when there is an error, but two also report if the number is less, equal
+or more than zero.
+
+\startbuffer
+\ifnumval -123 \or < \or = \or > \or ! \else ? \fi
+\ifnumval 0 \or < \or = \or > \or ! \else ? \fi
+\ifnumval 123 \or < \or = \or > \or ! \else ? \fi
+\ifnumval abc \or < \or = \or > \or ! \else ? \fi
+
+\ifdimval -123pt \or < \or = \or > \or ! \else ? \fi
+\ifdimval 0pt \or < \or = \or > \or ! \else ? \fi
+\ifdimval 123pt \or < \or = \or > \or ! \else ? \fi
+\ifdimval abcpt \or < \or = \or > \or ! \else ? \fi
+\stopbuffer
+
+\typebuffer \blank {\tt \getbuffer} \blank
+
+\startbuffer
+\ifchknum -123 \or okay \else bad \fi
+\ifchknum 0 \or okay \else bad \fi
+\ifchknum 123 \or okay \else bad \fi
+\ifchknum abc \or okay \else bad \fi
+
+\ifchkdim -123pt \or okay \else bad \fi
+\ifchkdim 0pt \or okay \else bad \fi
+\ifchkdim 123pt \or okay \else bad \fi
+\ifchkdim abcpt \or okay \else bad \fi
+\stopbuffer
+
+\typebuffer \blank {\tt \getbuffer} \blank
+
+\stopsubsection
+
+\startsubsection[title={\lpr {iftok} and \lpr {ifcstok}}]
+
+\topicindex {conditions+tokens}
+\topicindex {tokens}
+
+Comparing tokens and macros can be done with \type {\ifx}. Two extra test are
+provided in \LUAMETATEX:
+
+\startbuffer
+\def\ABC{abc} \def\DEF{def} \def\PQR{abc} \newtoks\XYZ \XYZ {abc}
+
+\iftok{abc}{def}\relax (same) \else [different] \fi
+\iftok{abc}{abc}\relax [same] \else (different) \fi
+\iftok\XYZ {abc}\relax [same] \else (different) \fi
+
+\ifcstok\ABC \DEF\relax (same) \else [different] \fi
+\ifcstok\ABC \PQR\relax [same] \else (different) \fi
+\ifcstok{abc}\ABC\relax [same] \else (different) \fi
+\stopbuffer
+
+\typebuffer \startpacked[blank] {\tt\nospacing\getbuffer} \stoppacked
+
+You can check if a macro is is defined as protected with \type {\ifprotected}
+while frozen macros can be tested with \type {\iffrozen}. A provisional \type
+{\ifusercmd} tests will check if a command is defined at the user level (and this
+one might evolve).
+
+\stopsubsection
+
+\startsubsection[title={\lpr {ifcondition}}]
+
+\topicindex {conditions}
+
+This is a somewhat special one. When you write macros conditions need to be
+properly balanced in order to let \TEX's fast branch skipping work well. This new
+primitive is basically a no||op flagged as a condition so that the scanner can
+recognize it as an if|-|test. However, when a real test takes place the work is
+done by what follows, in the next example \tex {something}.
+
+\starttyping
+\unexpanded\def\something#1#2%
+ {\edef\tempa{#1}%
+ \edef\tempb{#2}
+ \ifx\tempa\tempb}
+
+\ifcondition\something{a}{b}%
+ \ifcondition\something{a}{a}%
+ true 1
+ \else
+ false 1
+ \fi
+\else
+ \ifcondition\something{a}{a}%
+ true 2
+ \else
+ false 2
+ \fi
+\fi
+\stoptyping
+
+If you are familiar with \METAPOST, this is a bit like \type {vardef} where the macro
+has a return value. Here the return value is a test.
+
+Experiments with something \type {\ifdef} actually worked ok but were rejected
+because in the end it gave no advantage so this generic one has to do. The \type
+{\ifcondition} test is basically is a no|-|op except when branches are skipped.
+However, when a test is expected, the scanner gobbles it and the next test result
+is used. Here is an other example:
+
+\startbuffer
+\def\mytest#1%
+ {\ifabsdim#1>0pt\else
+ \expandafter \unless
+ \fi
+ \iftrue}
+
+\ifcondition\mytest{10pt}\relax non-zero \else zero \fi
+\ifcondition\mytest {0pt}\relax non-zero \else zero \fi
+\stopbuffer
+
+\typebuffer \blank {\tt \getbuffer} \blank
+
+The last expansion in a macro like \type {\mytest} has to be a condition and here
+we use \type {\unless} to negate the result.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {orelse}}]
+
+Sometimes you have successive tests that, when laid out in the source lead to
+deep trees. The \type {\ifcase} test is an exception. Experiments with \type
+{\ifcasex} worked out fine but eventually were rejected because we have many
+tests so it would add a lot. As \LUAMETATEX\ permitted more experiments,
+eventually an alternative was cooked up, one that has some restrictions but is
+relative lightweight. It goes like this:
+
+\starttyping
+\ifnum\count0<10
+ less
+\orelse\ifnum\count0=10
+ equal
+\else
+ more
+\fi
+\stoptyping
+
+The \type {\orelse} has to be followed by one of the if test commands, except
+\type {\ifcondition}, and there can be an \type {\unless} in front of such a
+command. These restrictions make it possible to stay in the current condition
+(read: at the same level). If you need something more complex, using \type
+{\orelse} is probably unwise anyway. In case you wonder about performance, there
+is a little more checking needed when skipping branches but that can be
+neglected. There is some gain due to staying at the same level but that is only
+measurable when you runs tens of millions of complex tests and in that case it is
+very likely to drown in the real action. It's a convenience mechanism, in the
+sense that it can make your code look a bit easier to follow.
+
+There is an nice side effect of this mechanism. When you define:
+
+\starttyping
+\def\quitcondition{\orelse\iffalse}
+\stoptyping
+
+you can do this:
+
+\starttyping
+\ifnum\count0<10
+ less
+\orelse\ifnum\count0=10
+ equal
+ \quitcondition
+ indeed
+\else
+ more
+\fi
+\stoptyping
+
+Of course it is only useful at the right level, so you might end up with cases like
+
+\starttyping
+\ifnum\count0<10
+ less
+\orelse\ifnum\count0=10
+ equal
+ \ifnum\count2=30
+ \expandafter\quitcondition
+ \fi
+ indeed
+\else
+ more
+\fi
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\lpr {ifprotected}, \lpr {frozen}, \lpr {iffrozen} and \lpr {ifusercmd}}]
+
+These checkers deal with control sequences. You can check if a command is a
+protected one, that is, defined with the \type {\protected} prefix. A command is
+frozen when it has been defined with the \type {\frozen} prefix. Beware: only
+macros can be frozen. A user command is a command that is not part of the
+predefined set of commands. This is an experimental command.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Boxes, rules and leaders}]
+
+\startsubsection[title={\lpr {outputbox}}]
+
+\topicindex {output}
+
+This integer parameter allows you to alter the number of the box that will be
+used to store the page sent to the output routine. Its default value is 255, and
+the acceptable range is from 0 to 65535.
+
+\startsyntax
+\outputbox = 12345
+\stopsyntax
+
+\stopsubsection
+
+\startsubsection[title={\prm {vpack}, \prm {hpack} and \prm {tpack}}]
+
+These three primitives are like \prm {vbox}, \prm {hbox} and \prm {vtop}
+but don't apply the related callbacks.
+
+\stopsubsection
+
+\startsubsection[title={\prm {vsplit}}]
+
+\topicindex {splitting}
+
+The \prm {vsplit} primitive has to be followed by a specification of the required
+height. As alternative for the \type {to} keyword you can use \type {upto} to get
+a split of the given size but result has the natural dimensions then.
+
+\stopsubsection
+
+\startsubsection[title={Images and reused box objects},reference=sec:imagedandforms]
+
+In original \TEX\ image support is dealt with via specials. It's not a native
+feature of the engine. All that \TEX\ cares about is dimensions, so in practice
+that meant: using a box with known dimensions that wraps a special that instructs
+the backend to include an image. The wrapping is needed because a special itself
+is a whatsit and as such has no dimensions.
+
+In \PDFTEX\ a special whatsit for images was introduced and that one {\em has}
+dimensions. As a consequence, in several places where the engine to deal with the
+dimensions of nodes, it now has to check the details of whatsits. By inheriting
+code from \PDFTEX, the \LUATEX\ engine also had that property. However, at some
+point this approach was abandoned and a more natural trick was used: images (and
+box resources) became a special kind of rules, and as rules already have
+dimensions, the code could be simplified.
+
+When direction nodes and localpar nodes also became first class nodes, whatsits
+again became just that: nodes representing whatever you want, but without
+dimensions, and therefore they could again be ignored when dimensions mattered.
+And, because images were disguised as rules, as mentioned ,their dimensions
+automatically were taken into account. This seperation between front and backend
+cleaned up the code base already quite a bit.
+
+In \LUAMETATEX\ we still have the image specific subtypes for rules, but the
+engine never looks at subtypes of rules. That was up to the backend. This means
+that image support is not present in \LUAMETATEX. When an image specification was
+parsed the special properties, like the filename, or additional attributes, were
+stored in the backend and all that \LUATEX\ does is registering a reference to an
+image s specification in the rule node. But, having no backend means nothing is
+stored, which in turn would make the image inclusion primitives kind of weird.
+
+Therefore you need to realize that contrary to \LUATEX, {\em in \LUAMETATEX\
+support for images and box reuse is not built in}! However, we can assume that
+an implementation uses rules in a similar fashion as \LUATEX\ does. So, you can
+still consider images and box reuse to be core concepts. Here we just mention the
+primitives that \LUATEX\ provides. They are not available in the engine but can
+of course be implemented in \LUA.
+
+\starttabulate[|l|p|]
+\DB command \BC explanation \NC \NR
+\TB
+\NC \lpr {saveboxresource} \NC save the box as an object to be included later \NC \NR
+\NC \lpr {saveimageresource} \NC save the image as an object to be included later \NC \NR
+\NC \lpr {useboxresource} \NC include the saved box object here (by index) \NC \NR
+\NC \lpr {useimageresource} \NC include the saved image object here (by index) \NC \NR
+\NC \lpr {lastsavedboxresourceindex} \NC the index of the last saved box object \NC \NR
+\NC \lpr {lastsavedimageresourceindex} \NC the index of the last saved image object \NC \NR
+\NC \lpr {lastsavedimageresourcepages} \NC the number of pages in the last saved image object \NC \NR
+\LL
+\stoptabulate
+
+An implementation probably should accepts the usual optional dimension parameters
+for \type {\use...resource} in the same format as for rules. With images, these
+dimensions are then used instead of the ones given to \lpr {useimageresource} but
+the original dimensions are not overwritten, so that a \lpr {useimageresource}
+without dimensions still provides the image with dimensions defined by \lpr
+{saveimageresource}. These optional parameters are not implemented for \lpr
+{saveboxresource}.
+
+\starttyping
+\useimageresource width 20mm height 10mm depth 5mm \lastsavedimageresourceindex
+\useboxresource width 20mm height 10mm depth 5mm \lastsavedboxresourceindex
+\stoptyping
+
+Examples or optional entries are \type {attr} and \type {resources} that accept a
+token list, and the \type {type} key. When set to non|-|zero the \type {/Type}
+entry is omitted. A value of 1 or 3 still writes a \type {/BBox}, while 2 or 3
+will write a \type {/Matrix}. But, as said: this is entirely up to the backend.
+Generic macro packages (like \type {tikz}) can use these assumed primitives so
+one can best provide them. It is probably, for historic reasons, the only more or
+less standardized image inclusion interface one can expect to work in all macro
+packages.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {hpack}, \lpr {vpack} and \lpr {tpack}}]
+
+These three primitives are the equivalents of \type {\hbox}, \type {\vbox} and
+\type {\vtop} but they don't trigger the packaging related callbacks. Of course
+one never know if content needs a treatment so using them should be done with
+care.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {nohrule} and \lpr {novrule}}]
+
+\topicindex {rules}
+
+Because introducing a new keyword can cause incompatibilities, two new primitives
+were introduced: \lpr {nohrule} and \lpr {novrule}. These can be used to
+reserve space. This is often more efficient than creating an empty box with fake
+dimensions.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {gleaders}},reference=sec:gleaders]
+
+\topicindex {leaders}
+
+This type of leaders is anchored to the origin of the box to be shipped out. So
+they are like normal \prm {leaders} in that they align nicely, except that the
+alignment is based on the {\it largest\/} enclosing box instead of the {\it
+smallest\/}. The \type {g} stresses this global nature.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Languages}]
+
+\startsubsection[title={\lpr {hyphenationmin}}]
+
+\topicindex {languages}
+\topicindex {hyphenation}
+
+This primitive can be used to set the minimal word length, so setting it to a value
+of~$5$ means that only words of 6 characters and more will be hyphenated, of course
+within the constraints of the \prm {lefthyphenmin} and \prm {righthyphenmin}
+values (as stored in the glyph node). This primitive accepts a number and stores
+the value with the language.
+
+\stopsubsection
+
+\startsubsection[title={\prm {boundary}, \prm {noboundary}, \prm {protrusionboundary} and \prm {wordboundary}}]
+
+The \prm {noboundary} command is used to inject a whatsit node but now injects a normal
+node with type \nod {boundary} and subtype~0. In addition you can say:
+
+\starttyping
+x\boundary 123\relax y
+\stoptyping
+
+This has the same effect but the subtype is now~1 and the value~123 is stored.
+The traditional ligature builder still sees this as a cancel boundary directive
+but at the \LUA\ end you can implement different behaviour. The added benefit of
+passing this value is a side effect of the generalization. The subtypes~2 and~3
+are used to control protrusion and word boundaries in hyphenation and have
+related primitives.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Control and debugging}]
+
+\startsubsection[title={Tracing}]
+
+\topicindex {tracing}
+
+If \prm {tracingonline} is larger than~2, the node list display will also print
+the node number of the nodes.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {lastnodetype}, \lpr {lastnodesubtype}, \lpr
+{currentiftype} and \lpr {internalcodesmode}.}]
+
+The \ETEX\ command \type {\lastnodetype} is limited to some nodes. When the
+parameter \type {\internalcodesmode} is set to a non|-|zero value the normal
+(internally used) numbers are reported. The same is true for \type
+{\currentiftype}, as we have more conditionals and also use a different order.
+The \type {\lastnodesubtype} is a bonus.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Files}]
+
+\startsubsection[title={File syntax}]
+
+\topicindex {files+names}
+
+\LUAMETATEX\ will accept a braced argument as a file name:
+
+\starttyping
+\input {plain}
+\openin 0 {plain}
+\stoptyping
+
+This allows for embedded spaces, without the need for double quotes. Macro
+expansion takes place inside the argument.
+
+The \lpr {tracingfonts} primitive that has been inherited from \PDFTEX\ has
+been adapted to support variants in reporting the font. The reason for this
+extension is that a csname not always makes sense. The zero case is the default.
+
+\starttabulate[|l|l|]
+\DB value \BC reported \NC \NR
+\TB
+\NC \type{0} \NC \type{\foo xyz} \NC \NR
+\NC \type{1} \NC \type{\foo (bar)} \NC \NR
+\NC \type{2} \NC \type{<bar> xyz} \NC \NR
+\NC \type{3} \NC \type{<bar @ ..pt> xyz} \NC \NR
+\NC \type{4} \NC \type{<id>} \NC \NR
+\NC \type{5} \NC \type{<id: bar>} \NC \NR
+\NC \type{6} \NC \type{<id: bar @ ..pt> xyz} \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsection
+
+\startsubsection[title={Writing to file}]
+
+\topicindex {files+writing}
+
+You can now open upto 127 files with \prm {openout}. When no file is open writes
+will go to the console and log. The \type {write} related primitives have to be
+implemented as part of a backend! As a consequence a system command is no longer
+possible but one can use \type {os.execute} to do the same.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Math}]
+
+\topicindex {math}
+
+We will cover math extensions in its own chapter because not only the font
+subsystem and spacing model have been enhanced (thereby introducing many new
+primitives) but also because some more control has been added to existing
+functionality. Much of this relates to the different approaches of traditional
+\TEX\ fonts and \OPENTYPE\ math.
+
+\stopsection
+
+\startsection[title={Fonts}]
+
+\topicindex {fonts}
+
+Like math, we will cover fonts extensions in its own chapter. Here we stick to
+mentioning that loading fonts is different in \LUAMETATEX. As in \LUATEX\ we have
+the extra primitives \type {\fontid} and \type {\setfontid}, \type {\noligs} and
+\type {\nokerns}, and \type {\nospaces}. The other new primitives in \LUATEX\
+have been dropped.
+
+\stopsection
+
+\startsection[title=Directions]
+
+\topicindex {\OMEGA}
+\topicindex {\ALEPH}
+\topicindex {directions}
+
+\startsubsection[title={Two directions}]
+
+The directional model in \LUAMETATEX\ is a simplified version the the model used
+in \LUATEX. In fact, not much is happening at all: we only register a change in
+direction.
+
+\stopsubsection
+
+\startsubsection[title={How it works}]
+
+The approach is that we try to make node lists balanced but also try to avoid
+some side effects. What happens is quite intuitive if we forget about spaces
+(turned into glue) but even there what happens makes sense if you look at it in
+detail. However that logic makes in|-|group switching kind of useless when no
+proper nested grouping is used: switching from right to left several times
+nested, results in spacing ending up after each other due to nested mirroring. Of
+course a sane macro package will manage this for the user but here we are
+discussing the low level injection of directional information.
+
+This is what happens:
+
+\starttyping
+\textdirection 1 nur {\textdirection 0 run \textdirection 1 NUR} nur
+\stoptyping
+
+This becomes stepwise:
+
+\startnarrower
+\starttyping
+injected: [push 1]nur {[push 0]run [push 1]NUR} nur
+balanced: [push 1]nur {[push 0]run [pop 0][push 1]NUR[pop 1]} nur[pop 0]
+result : run {RUNrun } run
+\stoptyping
+\stopnarrower
+
+And this:
+
+\starttyping
+\textdirection 1 nur {nur \textdirection 0 run \textdirection 1 NUR} nur
+\stoptyping
+
+becomes:
+
+\startnarrower
+\starttyping
+injected: [+TRT]nur {nur [+TLT]run [+TRT]NUR} nur
+balanced: [+TRT]nur {nur [+TLT]run [-TLT][+TRT]NUR[-TRT]} nur[-TRT]
+result : run {run RUNrun } run
+\stoptyping
+\stopnarrower
+
+Now, in the following examples watch where we put the braces:
+
+\startbuffer
+\textdirection 1 nur {{\textdirection 0 run} {\textdirection 1 NUR}} nur
+\stopbuffer
+
+\typebuffer
+
+This becomes:
+
+\startnarrower
+\getbuffer
+\stopnarrower
+
+Compare this to:
+
+\startbuffer
+\textdirection 1 nur {{\textdirection 0 run }{\textdirection 1 NUR}} nur
+\stopbuffer
+
+\typebuffer
+
+Which renders as:
+
+\startnarrower
+\getbuffer
+\stopnarrower
+
+So how do we deal with the next?
+
+\startbuffer
+\def\ltr{\textdirection 0\relax}
+\def\rtl{\textdirection 1\relax}
+
+run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}
+run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}
+\stopbuffer
+
+\typebuffer
+
+It gets typeset as:
+
+\startnarrower
+\startlines
+\getbuffer
+\stoplines
+\stopnarrower
+
+We could define the two helpers to look back, pick up a skip, remove it and
+inject it after the dir node. But that way we loose the subtype information that
+for some applications can be handy to be kept as|-|is. This is why we now have a
+variant of \lpr {textdirection} which injects the balanced node before the skip.
+Instead of the previous definition we can use:
+
+\startbuffer[def]
+\def\ltr{\linedirection 0\relax}
+\def\rtl{\linedirection 1\relax}
+\stopbuffer
+
+\typebuffer[def]
+
+and this time:
+
+\startbuffer[txt]
+run {\rtl nur {\ltr run \rtl NUR \ltr run \rtl NUR} nur}
+run {\ltr run {\rtl nur \ltr RUN \rtl nur \ltr RUN} run}
+\stopbuffer
+
+\typebuffer[txt]
+
+comes out as a properly spaced:
+
+\startnarrower
+\startlines
+\getbuffer[def,txt]
+\stoplines
+\stopnarrower
+
+Anything more complex that this, like combination of skips and penalties, or
+kerns, should be handled in the input or macro package because there is no way we
+can predict the expected behaviour. In fact, the \lpr {linedir} is just a
+convenience extra which could also have been implemented using node list parsing.
+
+\stopsubsection
+
+\startsubsection[title={Controlling glue with \lpr {breakafterdirmode}}]
+
+Glue after a dir node is ignored in the linebreak decision but you can bypass that
+by setting \lpr {breakafterdirmode} to~\type {1}. The following table shows the
+difference. Watch your spaces.
+
+\def\ShowSome#1{%
+ \BC \type{#1}
+ \NC \breakafterdirmode\zerocount\hsize\zeropoint#1
+ \NC
+ \NC \breakafterdirmode\plusone\hsize\zeropoint#1
+ \NC
+ \NC \NR
+}
+
+\starttabulate[|l|Tp(1pt)|w(5em)|Tp(1pt)|w(5em)|]
+ \DB
+ \BC \type{0}
+ \NC
+ \BC \type{1}
+ \NC
+ \NC \NR
+ \TB
+ \ShowSome{pre {\textdirection 0 xxx} post}
+ \ShowSome{pre {\textdirection 0 xxx }post}
+ \ShowSome{pre{ \textdirection 0 xxx} post}
+ \ShowSome{pre{ \textdirection 0 xxx }post}
+ \ShowSome{pre { \textdirection 0 xxx } post}
+ \ShowSome{pre {\textdirection 0\relax\space xxx} post}
+ \LL
+\stoptabulate
+
+\stopsubsection
+
+\startsubsection[title={Controling parshapes with \lpr {shapemode}}]
+
+Another adaptation to the \ALEPH\ directional model is control over shapes driven
+by \prm {hangindent} and \prm {parshape}. This is controlled by a new parameter
+\lpr {shapemode}:
+
+\starttabulate[|c|l|l|]
+\DB value \BC \prm {hangindent} \BC \prm {parshape} \NC \NR
+\TB
+\BC \type{0} \NC normal \NC normal \NC \NR
+\BC \type{1} \NC mirrored \NC normal \NC \NR
+\BC \type{2} \NC normal \NC mirrored \NC \NR
+\BC \type{3} \NC mirrored \NC mirrored \NC \NR
+\LL
+\stoptabulate
+
+The value is reset to zero (like \prm {hangindent} and \prm {parshape})
+after the paragraph is done with. You can use negative values to prevent
+this. In \in {figure} [fig:shapemode] a few examples are given.
+
+\startplacefigure[reference=fig:shapemode,title={The effect of \type {shapemode}.}]
+ \startcombination[2*3]
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \pardirection 0 \textdirection 0
+ \hangindent 40pt \hangafter -3
+ \leftskip10pt \input tufte \par
+ \egroup} {TLT: hangindent}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \pardirection 0 \textdirection 0
+ \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
+ \input tufte \par
+ \egroup} {TLT: parshape}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \pardirection 1 \textdirection 1
+ \hangindent 40pt \hangafter -3
+ \leftskip10pt \input tufte \par
+ \egroup} {TRT: hangindent mode 0}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \pardirection 1 \textdirection 1
+ \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
+ \input tufte \par
+ \egroup} {TRT: parshape mode 0}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \shapemode=3
+ \pardirection 1 \textdirection 1
+ \hangindent 40pt \hangafter -3
+ \leftskip10pt \input tufte \par
+ \egroup} {TRT: hangindent mode 1 & 3}
+ {\ruledvbox \bgroup \setuptolerance[verytolerant]
+ \hsize .45\textwidth \switchtobodyfont[6pt]
+ \shapemode=3
+ \pardirection 1 \textdirection 1
+ \parshape 4 0pt .8\hsize 10pt .8\hsize 20pt .8\hsize 0pt \hsize
+ \input tufte \par
+ \egroup} {TRT: parshape mode 2 & 3}
+ \stopcombination
+\stopplacefigure
+
+We have \type {\pardirection}, \type {\textdirection}, \type {\mathdirection} and
+\type {\linedirection} that is like \type {\textdirection} but with some
+additional (inline) glue checking.
+
+\stopsubsection
+
+\startsubsection[title=Orientations]
+
+As mentioned, the difference with \LUATEX\ is that we only have numeric
+directions and that there are only two: left|-|to|-|right (\type {0}) and
+right|-|to|-|left (\type {1}). The direction of a box is set with \type
+{direction}.
+
+In addition to that boxes can now have an \type {orientation} keyword followed by
+optional \type {xoffset} and|/|or \type {yoffset} keywords. The offsets don't
+have consequences for the dimensions. The alternatives \type {xmove} and \type
+{ymove} on the contrary are reflected in the dimensions. Just play with them. The
+offsets and moves only are accepted when there is also an orientation, so no time
+is wasted on testing for these rarely used keywords. There are related primitives
+\type {\box...} that set these properties.
+
+As these are experimental it will not be explained here (yet). They are covered
+in the descriptions of the development of \LUAMETATEX: articles and|/|or
+documents in the \CONTEXT\ distribution. For now it is enough to know that the
+orientation can be up, down, left or right (rotated) and that it has some
+anchoring variants. Combined with the offsets this permits macro writers to
+provide solutions for top|-|down and bottom|-|up writing directions, something
+that is rather macro package specific and used for scripts that need
+manipulations anyway. The \quote {old} vertical directions were never okay and
+therefore not used.
+
+There are a couple of properties in boxes that you can set and query but that
+only really take effect when the backend supports them. When usage on \CONTEXT\
+shows that is't okay, they will become official, so we just mention them: \type
+{\boxdirection}, \type {\boxattr}, \type {\boxorientation}, \type {\boxxoffset},
+\type {\boxyoffset}, \type {\boxxmove}, \type {\boxymove} and \type {\boxtotal}.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title=Expressions]
+
+The \type {*expr} parsers now accept \type {:} as operator for integer division
+(the \type {/} operators does rounding. This can be used for division compatible
+with \type {\divide}. I'm still wondering if adding a couple of bit operators
+makes sense (for integers).
+
+\stopsection
+
+\startsection[title=Nodes]
+
+The \ETEX\ primitive \type {\lastnodetype} is not honest in reporting the
+internal numbers as it uses its own values. But you can set \type
+{\internalcodesmode} to a non|-|zero value to get the real id's instead. In
+addition there is \type {\lastnodesubtype}.
+
+Another last one is \type {\lastnamedcs} which holds the last match but this one
+should be used with care because one never knows if in the meantime something
+else \quote {last} has been seen.
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-fonts.tex b/doc/context/sources/general/manuals/luametatex/luametatex-fonts.tex
new file mode 100644
index 000000000..077d9e51e
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-fonts.tex
@@ -0,0 +1,596 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-fonts
+
+\startchapter[reference=fonts,title={Fonts}]
+
+\startsection[title={Introduction}]
+
+Only traditional font support is built in, anything more needs to be implemented
+in \LUA. This is conform the \LUATEX\ philosophy. When you pass a font to the
+frontend only the dimensions matter, as these are used in typesetting, and
+optionally ligatures and kerns when you rely on the built|-|in font handler. For
+math some extra data is needed, like information about extensibles and next in
+size glyphs. You can of course put more information in your \LUA\ tables because
+when such a table is passed to \TEX\ only that what is needed is filtered from
+it.
+
+Because there is no built|-|in backend, virtual font information is not used. If
+you want to be compatible you'd better make sure that your tables are okay, and
+in that case you can best consult the \LUATEX\ manual. For instance, parameters
+like \type {extend} are backend related and the standard \LUATEX\ backend sets
+the standard here.
+
+\stopsection
+
+\startsection[title={Defining fonts}]
+
+All \TEX\ fonts are represented to \LUA\ code as tables, and internally as
+\CCODE\ structures. All keys in the table below are saved in the internal font
+structure if they are present in the table passed to \type {font.define}. When
+the callback is set, which is needed for \type {\font} to work, its function
+gets the name and size passed, and it has to return a valid font identifier (a
+positive number).
+
+For the engine to work well, the following information has to be present at
+the font level:
+
+\starttabulate[|l|l|pl|]
+\DB key \BC value type \BC description \NC \NR
+\TB
+\NC \type {name} \NC string \NC metric (file) name \NC \NR
+\NC \type {characters} \NC table \NC the defined glyphs of this font \NC \NR
+\NC \type {designsize} \NC number \NC expected size (default: 655360 == 10pt) \NC \NR
+\NC \type {fonts} \NC table \NC locally used fonts \NC \NR
+\NC \type {hyphenchar} \NC number \NC default: \TEX's \prm {hyphenchar} \NC \NR
+\NC \type {parameters} \NC hash \NC default: 7 parameters, all zero \NC \NR
+\NC \type {size} \NC number \NC the required scaling (by default the same as designsize) \NC \NR
+\NC \type {skewchar} \NC number \NC default: \TEX's \prm {skewchar} \NC \NR
+\NC \type {stretch} \NC number \NC the \quote {stretch} \NC \NR
+\NC \type {shrink} \NC number \NC the \quote {shrink} \NC \NR
+\NC \type {step} \NC number \NC the \quote {step} \NC \NR
+\NC \type {nomath} \NC boolean \NC this key allows a minor speedup for text fonts; if it
+ is present and true, then \LUATEX\ will not check the
+ character entries for math|-|specific keys \NC \NR
+\NC \type {oldmath} \NC boolean \NC this key flags a font as representing an old school \TEX\
+ math font and disables the \OPENTYPE\ code path \NC \NR
+\LL
+\stoptabulate
+
+The \type {parameters} is a hash with mixed key types. There are seven possible
+string keys, as well as a number of integer indices (these start from 8 up). The
+seven strings are actually used instead of the bottom seven indices, because that
+gives a nicer user interface.
+
+The names and their internal remapping are:
+
+\starttabulate[|l|c|]
+\DB name \BC remapping \NC \NR
+\TB
+\NC \type {slant} \NC 1 \NC \NR
+\NC \type {space} \NC 2 \NC \NR
+\NC \type {space_stretch} \NC 3 \NC \NR
+\NC \type {space_shrink} \NC 4 \NC \NR
+\NC \type {x_height} \NC 5 \NC \NR
+\NC \type {quad} \NC 6 \NC \NR
+\NC \type {extra_space} \NC 7 \NC \NR
+\LL
+\stoptabulate
+
+The \type {characters} table is a list of character hashes indexed by an integer
+number. The number is the \quote {internal code} \TEX\ knows this character by.
+For proper paragraph building and math rendering the following fields can be
+present in en entry in the \type {characters} table. You can of course add all
+kind of extra fields. The engine only uses those that it needs for typesetting
+a paragraph or formula.
+
+Each character hash itself is a hash. For example, here is the character \quote
+{f} (decimal 102) in the font \type {cmr10 at 10pt}. The numbers that represent
+dimensions are in scaled points.
+
+\starttyping
+[102] = {
+ ["width"] = 200250,
+ ["height"] = 455111,
+ ["depth"] = 0,
+ ["italic"] = 50973,
+ ["kerns"] = {
+ [63] = 50973,
+ [93] = 50973,
+ [39] = 50973,
+ [33] = 50973,
+ [41] = 50973
+ },
+ ["ligatures"] = {
+ [102] = { ["char"] = 11, ["type"] = 0 },
+ [108] = { ["char"] = 13, ["type"] = 0 },
+ [105] = { ["char"] = 12, ["type"] = 0 }
+ }
+}
+\stoptyping
+
+Providing ligatures and kerns this way permits \TEX\ to construct ligatures and
+add inter|-|character kerning. However, normally you will use an \OPENTYPE\ font
+in combination with \LUA\ code that does this. In \CONTEXT\ we have base mode
+that uses the engine, and node mode that uses \LUA. A monospaced font normally
+has no ligatures and kerns and is normally not processed at all.
+
+\starttabulate[|l|l|pl|]
+\DB key \BC type \BC description \NC\NR
+\TB
+\NC \type {width} \NC number \NC width in sp (default 0) \NC\NR
+\NC \type {height} \NC number \NC height in sp (default 0) \NC\NR
+\NC \type {depth} \NC number \NC depth in sp (default 0) \NC\NR
+\NC \type {italic} \NC number \NC italic correction in sp (default 0) \NC\NR
+\NC \type {top_accent} \NC number \NC top accent alignment place in sp (default zero) \NC\NR
+\NC \type {bot_accent} \NC number \NC bottom accent alignment place, in sp (default zero) \NC\NR
+\NC \type {left_protruding} \NC number \NC left protruding factor (\lpr {lpcode}) \NC\NR
+\NC \type {right_protruding} \NC number \NC right protruding factor (\lpr {rpcode}) \NC\NR
+\NC \type {expansion_factor} \NC number \NC expansion factor (\lpr {efcode}) \NC\NR
+\NC \type {next} \NC number \NC \quote {next larger} character index \NC\NR
+\NC \type {extensible} \NC table \NC constituent parts of an extensible recipe \NC\NR
+\NC \type {vert_variants} \NC table \NC constituent parts of a vertical variant set \NC \NR
+\NC \type {horiz_variants} \NC table \NC constituent parts of a horizontal variant set \NC \NR
+\NC \type {kerns} \NC table \NC kerning information \NC\NR
+\NC \type {ligatures} \NC table \NC ligaturing information \NC\NR
+\NC \type {mathkern} \NC table \NC math cut-in specifications \NC\NR
+\LL
+\stoptabulate
+
+Two very special string indexes can be used also: \type {left_boundary} is a
+virtual character whose ligatures and kerns are used to handle word boundary
+processing. \type {right_boundary} is similar but not actually used for anything
+(yet).
+
+The values of \type {top_accent}, \type {bot_accent} and \type {mathkern} are
+used only for math accent and superscript placement, see \at {page} [math] in
+this manual for details. The values of \type {left_protruding} and \type
+{right_protruding} are used only when \lpr {protrudechars} is non-zero. Whether
+or not \type {expansion_factor} is used depends on the font's global expansion
+settings, as well as on the value of \lpr {adjustspacing}.
+
+A math character can have a \type {next} field that points to a next larger
+shape. However, the presence of \type {extensible} will overrule \type {next}, if
+that is also present. The \type {extensible} field in turn can be overruled by
+\type {vert_variants}, the \OPENTYPE\ version. The \type {extensible} table is
+very simple:
+
+\starttabulate[|l|l|p|]
+\DB key \BC type \BC description \NC\NR
+\TB
+\NC \type{top} \NC number \NC top character index \NC\NR
+\NC \type{mid} \NC number \NC middle character index \NC\NR
+\NC \type{bot} \NC number \NC bottom character index \NC\NR
+\NC \type{rep} \NC number \NC repeatable character index \NC\NR
+\LL
+\stoptabulate
+
+The \type {horiz_variants} and \type {vert_variants} are arrays of components.
+Each of those components is itself a hash of up to five keys:
+
+\starttabulate[|l|l|p|]
+\DB key \BC type \BC explanation \NC \NR
+\TB
+\NC \type{glyph} \NC number \NC The character index. Note that this is an encoding number, not a name. \NC \NR
+\NC \type{extender} \NC number \NC One (1) if this part is repeatable, zero (0) otherwise. \NC \NR
+\NC \type{start} \NC number \NC The maximum overlap at the starting side (in scaled points). \NC \NR
+\NC \type{end} \NC number \NC The maximum overlap at the ending side (in scaled points). \NC \NR
+\NC \type{advance} \NC number \NC The total advance width of this item. It can be zero or missing,
+ then the natural size of the glyph for character \type {component}
+ is used. \NC \NR
+\LL
+\stoptabulate
+
+The \type {kerns} table is a hash indexed by character index (and \quote
+{character index} is defined as either a non|-|negative integer or the string
+value \type {right_boundary}), with the values of the kerning to be applied, in
+scaled points.
+
+The \type {ligatures} table is a hash indexed by character index (and \quote
+{character index} is defined as either a non|-|negative integer or the string
+value \type {right_boundary}), with the values being yet another small hash, with
+two fields:
+
+\starttabulate[|l|l|p|]
+\DB key \BC type \BC description \NC \NR
+\TB
+\NC \type{type} \NC number \NC the type of this ligature command, default 0 \NC \NR
+\NC \type{char} \NC number \NC the character index of the resultant ligature \NC \NR
+\LL
+\stoptabulate
+
+The \type {char} field in a ligature is required. The \type {type} field inside a
+ligature is the numerical or string value of one of the eight possible ligature
+types supported by \TEX. When \TEX\ inserts a new ligature, it puts the new glyph
+in the middle of the left and right glyphs. The original left and right glyphs
+can optionally be retained, and when at least one of them is kept, it is also
+possible to move the new \quote {insertion point} forward one or two places. The
+glyph that ends up to the right of the insertion point will become the next
+\quote {left}.
+
+\starttabulate[|l|c|l|l|]
+\DB textual (Knuth) \BC number \BC string \BC result \NC\NR
+\TB
+\NC \type{l + r =: n} \NC 0 \NC \type{=:} \NC \type{|n} \NC\NR
+\NC \type{l + r =:| n} \NC 1 \NC \type{=:|} \NC \type{|nr} \NC\NR
+\NC \type{l + r |=: n} \NC 2 \NC \type{|=:} \NC \type{|ln} \NC\NR
+\NC \type{l + r |=:| n} \NC 3 \NC \type{|=:|} \NC \type{|lnr} \NC\NR
+\NC \type{l + r =:|> n} \NC 5 \NC \type{=:|>} \NC \type{n|r} \NC\NR
+\NC \type{l + r |=:> n} \NC 6 \NC \type{|=:>} \NC \type{l|n} \NC\NR
+\NC \type{l + r |=:|> n} \NC 7 \NC \type{|=:|>} \NC \type{l|nr} \NC\NR
+\NC \type{l + r |=:|>> n} \NC 11 \NC \type{|=:|>>} \NC \type{ln|r} \NC\NR
+\LL
+\stoptabulate
+
+The default value is~0, and can be left out. That signifies a \quote {normal}
+ligature where the ligature replaces both original glyphs. In this table the~\type {|}
+indicates the final insertion point.
+
+\stopsection
+
+\startsection[reference=virtualfonts,title={Virtual fonts}]
+
+% \topicindex {fonts+virtual}
+
+Virtual fonts have been introduced to overcome limitations of good old \TEX. They
+were mostly use for providing a direct mapping from for instance accented
+characters onto a glyph. The backend was responsible for turning a reference to a
+character slot into a real glyph, possibly constructed from other glyphs. In our
+case there is no backend so there is also no need to pass this information
+through \TEX. But it can of course be part of the font information and because it is
+a kind of standard, we describe it here.
+
+A character is virtual when it has a \type {commands} array as part of the data.
+A virtual character can itself point to virtual characters but be careful with
+nesting as you can create loops and overflow the stack (which often indicates an
+error anyway).
+
+At the font level there can be a \type {fonts} an (indexed) \LUA\ table. The
+values are one- or two|-|key hashes themselves, each entry indicating one of the
+base fonts in a virtual font. In case your font is referring to itself in for
+instance a virtual font, you can use the \type {slot} command with a zero font
+reference, which indicates that the font itself is used. So, a table looks like
+this:
+
+\starttyping
+fonts = {
+ { name = "ptmr8a", size = 655360 },
+ { name = "psyr", size = 600000 },
+ { id = 38 }
+}
+\stoptyping
+
+The first referenced font (at index~1) in this virtual font is \type {ptrmr8a}
+loaded at 10pt, and the second is \type {psyr} loaded at a little over 9pt. The
+third one is a previously defined font that is known to \LUATEX\ as font id~38.
+The array index numbers are used by the character command definitions that are
+part of each character.
+
+The \type {commands} array is a hash where each item is another small array,
+with the first entry representing a command and the extra items being the
+parameters to that command. The allowed commands and their arguments are:
+
+\starttabulate[|l|l|l|p|]
+\DB command \BC arguments \BC type \BC description \NC \NR
+\TB
+\NC \type{font} \NC 1 \NC number \NC select a new font from the local \type {fonts} table \NC \NR
+\NC \type{char} \NC 1 \NC number \NC typeset this character number from the current font,
+ and move right by the character's width \NC \NR
+\NC \type{node} \NC 1 \NC node \NC output this node (list), and move right
+ by the width of this list\NC \NR
+\NC \type{slot} \NC 2 \NC 2 numbers \NC a shortcut for the combination of a font and char command\NC \NR
+\NC \type{push} \NC 0 \NC \NC save current position\NC \NR
+\NC \type{nop} \NC 0 \NC \NC do nothing \NC \NR
+\NC \type{pop} \NC 0 \NC \NC pop position \NC \NR
+\NC \type{rule} \NC 2 \NC 2 numbers \NC output a rule $ht*wd$, and move right. \NC \NR
+\NC \type{down} \NC 1 \NC number \NC move down on the page \NC \NR
+\NC \type{right} \NC 1 \NC number \NC move right on the page \NC \NR
+\NC \type{special} \NC 1 \NC string \NC output a \prm {special} command \NC \NR
+\NC \type{pdf} \NC 2 \NC 2 strings \NC output a \PDF\ literal, the first string is one of \type {origin},
+ \type {page}, \type {text}, \type {font}, \type {direct} or \type {raw}; if you
+ have one string only \type {origin} is assumed \NC \NR
+\NC \type{lua} \NC 1 \NC string,
+ function \NC execute a \LUA\ script when the glyph is embedded; in case of a
+ function it gets the font id and character code passed \NC \NR
+\NC \type{image} \NC 1 \NC image \NC output an image (the argument can be either an \type {<image>} variable or an \type {image_spec} table) \NC \NR
+\NC \type{comment} \NC any \NC any \NC the arguments of this command are ignored \NC \NR
+\LL
+\stoptabulate
+
+When a font id is set to~0 then it will be replaced by the currently assigned
+font id. This prevents the need for hackery with future id's.
+
+The \type {pdf} option also accepts a \type {mode} keyword in which case the
+third argument sets the mode. That option will change the mode in an efficient
+way (passing an empty string would result in an extra empty lines in the \PDF\
+file. This option only makes sense for virtual fonts. The \type {font} mode only
+makes sense in virtual fonts. Modes are somewhat fuzzy and partially inherited
+from \PDFTEX.
+
+\starttabulate[|l|p|]
+\DB mode \BC description \NC \NR
+\TB
+\NC \type {origin} \NC enter page mode and set the position \NC \NR
+\NC \type {page} \NC enter page mode \NC \NR
+\NC \type {text} \NC enter text mode \NC \NR
+\NC \type {font} \NC enter font mode (kind of text mode, only in virtual fonts) \NC \NR
+\NC \type {always} \NC finish the current string and force a transform if needed \NC \NR
+\NC \type {raw} \NC finish the current string \NC \NR
+\LL
+\stoptabulate
+
+You always need to check what \PDF\ code is generated because there can be all
+kind of interferences with optimization in the backend and fonts are complicated
+anyway. Here is a rather elaborate glyph commands example using such keys:
+
+\starttyping
+...
+commands = {
+ { "push" }, -- remember where we are
+ { "right", 5000 }, -- move right about 0.08pt
+ { "font", 3 }, -- select the fonts[3] entry
+ { "char", 97 }, -- place character 97 (ASCII 'a')
+ -- { "slot", 2, 97 }, -- an alternative for the previous two
+ { "pop" }, -- go all the way back
+ { "down", -200000 }, -- move upwards by about 3pt
+ { "special", "pdf: 1 0 0 rg" } -- switch to red color
+ -- { "pdf", "origin", "1 0 0 rg" } -- switch to red color (alternative)
+ { "rule", 500000, 20000 } -- draw a bar
+ { "special", "pdf: 0 g" } -- back to black
+ -- { "pdf", "origin", "0 g" } -- back to black (alternative)
+}
+...
+\stoptyping
+
+The default value for \type {font} is always~1 at the start of the
+\type {commands} array. Therefore, if the virtual font is essentially only a
+re|-|encoding, then you do usually not have created an explicit \quote {font}
+command in the array.
+
+Rules inside of \type {commands} arrays are built up using only two dimensions:
+they do not have depth. For correct vertical placement, an extra \type {down}
+command may be needed.
+
+Regardless of the amount of movement you create within the \type {commands}, the
+output pointer will always move by exactly the width that was given in the \type
+{width} key of the character hash. Any movements that take place inside the \type
+{commands} array are ignored on the upper level.
+
+The special can have a \type {pdf:}, \type {pdf:origin:}, \type {pdf:page:},
+\type {pdf:direct:} or \type {pdf:raw:} prefix. When you have to concatenate
+strings using the \type {pdf} command might be more efficient.
+
+The fields mentioned above can be found in external fonts. It is good to keep in
+mind that we can extend this model, given that the backend knows what to do with
+it.
+
+\stopsection
+
+\startsection[title={Additional \TEX\ commands}]
+
+\startsubsection[title={Font syntax}]
+
+\topicindex {fonts}
+
+\LUATEX\ will accept a braced argument as a font name:
+
+\starttyping
+\font\myfont = {cmr10}
+\stoptyping
+
+This allows for embedded spaces, without the need for double quotes. Macro
+expansion takes place inside the argument.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {fontid} and \lpr {setfontid}}]
+
+\startsyntax
+\fontid\font
+\stopsyntax
+
+This primitive expands into a number. It is not a register so there is no need to
+prefix with \prm {number} (and using \prm {the} gives an error). The currently
+used font id is \fontid\font. Here are some more:
+
+\starttabulate[|l|c|c|]
+\DB style \BC command \BC font id \NC \NR
+\TB
+\NC normal \NC \type {\tf} \NC \tf \fontid\font \NC \NR
+\NC bold \NC \type {\bf} \NC \bf \fontid\font \NC \NR
+\NC italic \NC \type {\it} \NC \it \fontid\font \NC \NR
+\NC bold italic \NC \type {\bi} \NC \bi \fontid\font \NC \NR
+\LL
+\stoptabulate
+
+These numbers depend on the macro package used because each one has its own way
+of dealing with fonts. They can also differ per run, as they can depend on the
+order of loading fonts. For instance, when in \CONTEXT\ virtual math \UNICODE\
+fonts are used, we can easily get over a hundred ids in use. Not all ids have to
+be bound to a real font, after all it's just a number.
+
+The primitive \lpr {setfontid} can be used to enable a font with the given id,
+which of course needs to be a valid one.
+
+\stopsubsection
+
+\startsubsection[title={\lpr {noligs} and \lpr {nokerns}}]
+
+\topicindex {ligatures+suppress}
+\topicindex {kerns+suppress}
+
+These primitives prohibit ligature and kerning insertion at the time when the
+initial node list is built by \LUATEX's main control loop. You can enable these
+primitives when you want to do node list processing of \quote {characters}, where
+\TEX's normal processing would get in the way.
+
+\startsyntax
+\noligs <integer>!crlf
+\nokerns <integer>
+\stopsyntax
+
+These primitives can also be implemented by overloading the ligature building and
+kerning functions, i.e.\ by assigning dummy functions to their associated
+callbacks. Keep in mind that when you define a font (using \LUA) you can also
+omit the kern and ligature tables, which has the same effect as the above.
+
+\stopsubsection
+
+\startsubsection[title={\type{\nospaces}}]
+
+\topicindex {spaces+suppress}
+
+This new primitive can be used to overrule the usual \prm {spaceskip} related
+heuristics when a space character is seen in a text flow. The value~\type{1}
+triggers no injection while \type{2} results in injection of a zero skip. In \in
+{figure} [fig:nospaces] we see the results for four characters separated by a
+space.
+
+\startplacefigure[reference=fig:nospaces,title={The \lpr {nospaces} options.}]
+\startcombination[3*2]
+ {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 10mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 10mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 10mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 1mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 1mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 1mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 1mm}}
+ {\ruledhbox to 5cm{\vtop{\hsize 1mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 1mm}}
+\stopcombination
+\stopplacefigure
+
+\stopsubsection
+
+\startsubsection[title={\type{\protrusionboundary}}]
+
+\topicindex {protrusion}
+\topicindex {boundaries}
+
+The protrusion detection mechanism is enhanced a bit to enable a bit more complex
+situations. When protrusion characters are identified some nodes are skipped:
+
+\startitemize[packed,columns,two]
+\startitem zero glue \stopitem
+\startitem penalties \stopitem
+\startitem empty discretionaries \stopitem
+\startitem normal zero kerns \stopitem
+\startitem rules with zero dimensions \stopitem
+\startitem math nodes with a surround of zero \stopitem
+\startitem dir nodes \stopitem
+\startitem empty horizontal lists \stopitem
+\startitem local par nodes \stopitem
+\startitem inserts, marks and adjusts \stopitem
+\startitem boundaries \stopitem
+\startitem whatsits \stopitem
+\stopitemize
+
+Because this can not be enough, you can also use a protrusion boundary node to
+make the next node being ignored. When the value is~1 or~3, the next node will be
+ignored in the test when locating a left boundary condition. When the value is~2
+or~3, the previous node will be ignored when locating a right boundary condition
+(the search goes from right to left). This permits protrusion combined with for
+instance content moved into the margin:
+
+\starttyping
+\protrusionboundary1\llap{!\quad}«Who needs protrusion?»
+\stoptyping
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={The \LUA\ font library}][library=font]
+
+\startsubsection[title={Introduction}]
+
+The \LUA\ font library is reduced to a few commands. Contrary to \LUATEX\ there
+is no loading of \TFM\ or \VF\ files. The explanation of the following commands
+is in the \LUATEX\ manual.
+
+\starttabulate[|l|pl|]
+\DB function \BC description \NC\NR
+\TB
+\NC \type {current} \NC returns the id of the currently active font \NC \NR
+\NC \type {max} \NC returns the last assigned font identifier \NC \NR
+\NC \type {setfont} \NC enables a font setfont (sets the current font id) \NC \NR
+\NC \type {addcharacters} \NC adds characters to a font \NC \NR
+\NC \type {define} \NC defined a font \NC \NR
+\NC \type {id} \NC returns the id that relates to a command name \NC \NR
+\LL
+\stoptabulate
+
+For practical reasons the management of font identifiers is still done by \TEX\
+but it can become an experiment to delegate that to \LUA\ as well.
+
+\stopsubsection
+
+\startsubsection[title={Defining a font with \type {define}, \type {addcharacters} and \type
+{setfont}}]
+
+\topicindex {fonts+define}
+\topicindex {fonts+extend}
+
+Normally you will use a callback to define a font but there's also a \LUA\
+function that does the job.
+
+\startfunctioncall
+id = font.define(<table> f)
+\stopfunctioncall
+
+Within reasonable bounds you can extend a font after it has been defined. Because
+some properties are best left unchanged this is limited to adding characters.
+
+\startfunctioncall
+font.addcharacters(<number n>, <table> f)
+\stopfunctioncall
+
+The table passed can have the fields \type {characters} which is a (sub)table
+like the one used in define, and for virtual fonts a \type {fonts} table can be
+added. The characters defined in the \type {characters} table are added (when not
+yet present) or replace an existing entry. Keep in mind that replacing can have
+side effects because a character already can have been used. Instead of posing
+restrictions we expect the user to be careful. The \type {setfont} helper is
+a more drastic replacer and only works when a font has not been used yet.
+
+\stopsubsection
+
+\startsubsection[title={Font ids: \type {id}, \type {max} and \type {current}}]
+
+\topicindex {fonts+id}
+\topicindex {fonts+current}
+
+\startfunctioncall
+<number> i = font.id(<string> csname)
+\stopfunctioncall
+
+This returns the font id associated with \type {csname}, or $-1$ if \type
+{csname} is not defined.
+
+\startfunctioncall
+<number> i = font.max()
+\stopfunctioncall
+
+This is the largest used index so far. The currently active font id can be
+queried or set with:
+
+\startfunctioncall
+<number> i = font.current()
+font.current(<number> i)
+\stopfunctioncall
+
+\stopsubsection
+
+% \startsubsection[title={Glyph data: \lpr {glyphdata}]
+%
+% This primitive can be used to set an additional glyph property. Of course it's very
+% macro package dependant what is done with that. Consider it an experiment (we had
+% some room left in the glyphs data structure). It's basically an single attribute.
+%
+% \stopsubsection
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
+
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-libraries.tex b/doc/context/sources/general/manuals/luametatex/luametatex-libraries.tex
new file mode 100644
index 000000000..d8210ac48
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-libraries.tex
@@ -0,0 +1,573 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-libraries
+
+\startchapter[reference=libraries,title={Extra libraries}]
+
+\startsection[title=Introduction]
+
+The libraries can be grouped in categories like fonts, languages, \TEX,
+\METAPOST, \PDF, etc. There are however also some that are more general puspose
+and these are discussed here.
+
+\stopsection
+
+\startsection[title=File and string readers: \type {fio} and type {sio}]
+
+This library provides a set of functions for reading numbers from a file and
+in addition to the regular \type {io} library functions. The following
+work on normal \LUA\ file handles.
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC readcardinal1 \NC (f) \NC a 1 byte unsigned integer \NC \NR
+\NC readcardinal2 \NC (f) \NC a 2 byte unsigned integer \NC \NR
+\NC readcardinal3 \NC (f) \NC a 3 byte unsigned integer \NC \NR
+\NC readcardinal4 \NC (f) \NC a 4 byte unsigned integer \NC \NR
+\NC readcardinaltable \NC (f,n,b) \NC \type {n} cardinals of \type {b} bytes \NC \NR
+\NC readinteger1 \NC (f) \NC a 1 byte signed integer \NC \NR
+\NC readinteger2 \NC (f) \NC a 2 byte signed integer \NC \NR
+\NC readinteger3 \NC (f) \NC a 3 byte signed integer \NC \NR
+\NC readinteger4 \NC (f) \NC a 4 byte signed integer \NC \NR
+\NC readintegertable \NC (f,n,b) \NC \type {n} integers of \type {b} bytes \NC \NR
+\NC readfixed2 \NC (f) \NC a 2 byte float (used in font files) \NC \NR
+\NC readfixed4 \NC (f) \NC a 4 byte float (used in font files) \NC \NR
+\NC read2dot14 \NC (f) \NC a 2 byte float (used in font files) \NC \NR
+\NC setposition \NC (f,p) \NC goto position \type {p} \NC \NR
+\NC getposition \NC (f) \NC get the current position \NC \NR
+\NC skipposition \NC (f,n) \NC skip \type {n} positions \NC \NR
+\NC readbytes \NC (f,n) \NC \type {n} bytes \NC \NR
+\NC readbytetable \NC (f,n) \NC \type {n} bytes\NC \NR
+\LL
+\stoptabulate
+
+When relevant there are also variants that end with \type {le} that do it the
+little endian way.
+
+A similar set of function as in the \type {fio} library is available in the \type
+{sio} library: \libidx {sio} {readcardinal1}, \libidx {sio} {readcardinal2},
+\libidx {sio} {readcardinal3}, \libidx {sio} {readcardinal4}, \libidx {sio}
+{readcardinaltable}, \libidx {sio} {readinteger1}, \libidx {sio} {readinteger2},
+\libidx {sio} {readinteger3}, \libidx {sio} {readinteger4}, \libidx {sio}
+{readintegertable}, \libidx {sio} {readfixed2}, \libidx {sio} {readfixed4},
+\libidx {sio} {read2dot14}, \libidx {sio} {setposition}, \libidx {sio}
+{getposition}, \libidx {sio} {skipposition}, \libidx {sio} {readbytes} and
+\libidx {sio} {readbytetable}. Here the first argument is a string instead of a
+file handle.
+
+\stopsection
+
+\startsection[title=\type{md5}]
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC sum \NC \NC \NC \NR
+\NC hex \NC \NC \NC \NR
+\NC HEX \NC \NC \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title=\type{sha2}]
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC digest256 \NC \NC \NC \NR
+\NC digest384 \NC \NC \NC \NR
+\NC digest512 \NC \NC \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+% \startsection[title=\type{flate}]
+%
+% \starttabulate[|Tw(12em)|T|T|]
+% \DB name \BC arguments \BC results \NC \NR
+% \TB
+% \NC flate_compress \NC \NC \NC \NR
+% \NC flate_decompress \NC \NC \NC \NR
+% \NC zip_compress \NC \NC \NC \NR
+% \NC zip_decompress \NC \NC \NC \NR
+% \NC gz_compress \NC \NC \NC \NR
+% \NC gz_decompress \NC \NC \NC \NR
+% \NC update_adler32 \NC \NC \NC \NR
+% \NC update_crc32 \NC \NC \NC \NR
+% \LL
+% \stoptabulate
+%
+% \stopsection
+
+\startsection[title=\type{xzip}]
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC compress \NC \NC \NC \NR
+\NC decompress \NC \NC \NC \NR
+\NC adler32 \NC \NC \NC \NR
+\NC crc32 \NC \NC \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title=\type{xmath}]
+
+This library just opens up standard \CCODE\ math library and the main reason for
+it being there is that it permits advanced graphics in \METAPOST\ (via the \LUA\
+interface). There are three constant values:
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC inf \NC \emdash \NC \cldcontext{xmath.inf} \NC \NR
+\NC nan \NC \emdash \NC \cldcontext{xmath.nan} \NC \NR
+\NC pi \NC \emdash \NC \cldcontext{xmath.pi} \NC \NR
+\LL
+\stoptabulate
+
+and a lot of functions:
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC acos \NC (a) \NC \NC \NR
+\NC acosh \NC (a) \NC \NC \NR
+\NC asin \NC (a) \NC \NC \NR
+\NC asinh \NC (a) \NC \NC \NR
+\NC atan \NC (a[,b]) \NC \NC \NR
+\NC atan2 \NC (a[,b]) \NC \NC \NR
+\NC atanh \NC (a) \NC \NC \NR
+\NC cbrt \NC (a) \NC \NC \NR
+\NC ceil \NC (a) \NC \NC \NR
+\NC copysign \NC (a,b) \NC \NC \NR
+\NC cos \NC (a) \NC \NC \NR
+\NC cosh \NC (a) \NC \NC \NR
+\NC deg \NC (a) \NC \NC \NR
+\NC erf \NC (a) \NC \NC \NR
+\NC erfc \NC (a) \NC \NC \NR
+\NC exp \NC (a) \NC \NC \NR
+\NC exp2 \NC (a) \NC \NC \NR
+\NC expm1 \NC (a) \NC \NC \NR
+\NC fabs \NC (a) \NC \NC \NR
+\NC fdim \NC (a,b) \NC \NC \NR
+\NC floor \NC (a) \NC \NC \NR
+\NC fma \NC (a,b,c) \NC \NC \NR
+\NC fmax \NC (...) \NC \NC \NR
+\NC fmin \NC (...) \NC \NC \NR
+\NC fmod \NC (a,b) \NC \NC \NR
+\NC frexp \NC (a,b) \NC \NC \NR
+\NC gamma \NC (a) \NC \NC \NR
+\NC hypot \NC (a,b) \NC \NC \NR
+\NC isfinite \NC (a) \NC \NC \NR
+\NC isinf \NC (a) \NC \NC \NR
+\NC isnan \NC (a) \NC \NC \NR
+\NC isnormal \NC (a) \NC \NC \NR
+\NC j0 \NC (a) \NC \NC \NR
+\NC j1 \NC (a) \NC \NC \NR
+\NC jn \NC (a,b) \NC \NC \NR
+\NC ldexp \NC (a,b) \NC \NC \NR
+\NC lgamma \NC (a) \NC \NC \NR
+\NC l0 \NC (a) \NC \NC \NR
+\NC l1 \NC (a) \NC \NC \NR
+\NC ln \NC (a,b) \NC \NC \NR
+\NC log \NC (a[,b]) \NC \NC \NR
+\NC log10 \NC (a) \NC \NC \NR
+\NC log1p \NC (a) \NC \NC \NR
+\NC log2 \NC (a) \NC \NC \NR
+\NC logb \NC (a) \NC \NC \NR
+\NC modf \NC (a,b) \NC \NC \NR
+\NC nearbyint \NC (a) \NC \NC \NR
+\NC nextafter \NC (a,b) \NC \NC \NR
+\NC pow \NC (a,b) \NC \NC \NR
+\NC rad \NC (a) \NC \NC \NR
+\NC remainder \NC (a,b) \NC \NC \NR
+\NC remquo \NC (a,b) \NC \NC \NR
+\NC round \NC (a) \NC \NC \NR
+\NC scalbn \NC (a,b) \NC \NC \NR
+\NC sin \NC (a) \NC \NC \NR
+\NC sinh \NC (a) \NC \NC \NR
+\NC sqrt \NC (a) \NC \NC \NR
+\NC tan \NC (a) \NC \NC \NR
+\NC tanh \NC (a) \NC \NC \NR
+\NC tgamma \NC (a) \NC \NC \NR
+\NC trunc \NC (a) \NC \NC \NR
+\NC y0 \NC (a) \NC \NC \NR
+\NC y1 \NC (a) \NC \NC \NR
+\NC yn \NC (a) \NC \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title=\type{xcomplex}]
+
+\LUAMETATEX\ also provides a complex library \type {xcomplex}. The complex
+number is a userdatum:
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC new \NC (r,i) \NC a complex userdata type \NC \NR
+\NC tostring \NC (z) \NC a string representation \NC \NR
+\NC topair \NC (z) \NC two numbers \NC \NR
+\LL
+\stoptabulate
+
+There is a bunch of functions that take a complex number:
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC abs \NC (a) \NC \NC \NR
+\NC arg \NC (a) \NC \NC \NR
+\NC imag \NC (a) \NC \NC \NR
+\NC real \NC (a) \NC \NC \NR
+\NC onj \NC (a) \NC \NC \NR
+\NC proj \NC (a) \NC \NC \NR
+\NC exp" \NC (a) \NC \NC \NR
+\NC log \NC (a) \NC \NC \NR
+\NC sqrt \NC (a) \NC \NC \NR
+\NC pow \NC (a,b) \NC \NC \NR
+\NC sin \NC (a) \NC \NC \NR
+\NC cos \NC (a) \NC \NC \NR
+\NC tan \NC (a) \NC \NC \NR
+\NC asin \NC (a) \NC \NC \NR
+\NC acos \NC (a) \NC \NC \NR
+\NC atan \NC (a) \NC \NC \NR
+\NC sinh \NC (a) \NC \NC \NR
+\NC cosh \NC (a) \NC \NC \NR
+\NC tanh \NC (a) \NC \NC \NR
+\NC asinh \NC (a) \NC \NC \NR
+\NC acosh \NC (a) \NC \NC \NR
+\NC atanh \NC (a) \NC \NC \NR
+\LL
+\stoptabulate
+
+These are accompanied by \type {libcerf} functions:
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC erf \NC (a) \NC The complex error function erf(z) \NC \NR
+\NC erfc \NC (a) \NC The complex complementary error function erfc(z) = 1 - erf(z) \NC \NR
+\NC erfcx \NC (a) \NC The underflow-compensating function erfcx(z) = exp(z^2) erfc(z) \NC \NR
+\NC erfi \NC (a) \NC The imaginary error function erfi(z) = -i erf(iz) \NC \NR
+\NC dawson \NC (a) \NC Dawson's integral D(z) = sqrt(pi)/2 * exp(-z^2) * erfi(z) \NC \NR
+\NC voigt \NC (a,b,c) \NC The convolution of a Gaussian and a Lorentzian \NC \NR
+\NC voigt_hwhm \NC (a,b) \NC The half width at half maximum of the Voigt profile \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title=\type{lfs}]
+
+The original \type {lfs} module has been adapted a bit to our needs but for
+practical reasons we kept the namespace. This module will probably evolve a bit
+over time.
+
+\starttabulate[|Tw(12em)|T|Tp|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC attributes \NC (name) \NC \NC \NR
+\NC chdir \NC (name) \NC \NC \NR
+\NC currentdir \NC () \NC \NC \NR
+\NC dir \NC (name) \NC \type {name}, \type {mode}, \type {size} and \type {mtime} \NC \NR
+\NC mkdir \NC (name) \NC \NC \NR
+\NC rmdir \NC (name) \NC \NC \NR
+\NC touch \NC (name) \NC \NC \NR
+\NC link \NC (name) \NC \NC \NR
+\NC symlinkattributes \NC (name) \NC \NC \NR
+\NC isdir \NC (name) \NC \NC \NR
+\NC isfile \NC (name) \NC \NC \NR
+\NC iswriteabledir \NC (name) \NC \NC \NR
+\NC iswriteablefile \NC (name) \NC \NC \NR
+\NC isreadabledir \NC (name) \NC \NC \NR
+\NC isreadablefile \NC (name) \NC \NC \NR
+\LL
+\stoptabulate
+
+The \type {dir} function is a traverser which in addition to the name returns
+some more properties. Keep in mind that the traverser loops over a directory and
+that it doesn't run well when used nested. This is a side effect of the operating
+system. It is also the reason why we return some properties because querying them
+via \type {attributes} would interfere badly.
+
+The following attributes are returned by \type {attributes}:
+
+\starttabulate[|Tw(12em)|T|]
+\DB name \BC value \NC \NR
+\TB
+\NC mode \NC \NC \NR
+\NC size \NC \NC \NR
+\NC modification \NC \NC \NR
+\NC access \NC \NC \NR
+\NC change \NC \NC \NR
+\NC permissions \NC \NC \NR
+\NC nlink \NC \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title=\type{pngdecode}]
+
+This module is experimental and used in image inclusion. It is not some general
+purpose module and is supposed to be used in a very controlled way. The
+interfaces might evolve.
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC applyfilter \NC (str,nx,ny,slice) \NC string \NC \NR
+\NC splitmask \NC (str,nx,ny,bpp,bytes) \NC string \NC \NR
+\NC interlace \NC (str,nx,ny,slice,pass) \NC string \NC \NR
+\NC expand \NC (str,nx,ny,parts,xline,factor) \NC string \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title=\type{basexx}]
+
+Some more experimental helpers:
+
+\starttabulate[|Tw(12em)|T|T|]
+\DB name \BC arguments \BC results \NC \NR
+\TB
+\NC encode16 \NC (str[,newline]) \NC string \NC \NR
+\NC decode16 \NC (str) \NC string \NC \NR
+\NC encode64 \NC (str[,newline]) \NC string \NC \NR
+\NC decode64 \NC (str) \NC string \NC \NR
+\NC encode85 \NC (str[,newline]) \NC string \NC \NR
+\NC decode85 \NC (str) \NC string \NC \NR
+\NC encodeRL \NC (str) \NC string \NC \NR
+\NC decodeRL \NC (str) \NC string \NC \NR
+\NC encodeLZW \NC (str[,defaults]) \NC string \NC \NR
+\NC decodeLZW \NC (str[,defaults]) \NC string \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title={Multibyte \type {string} functions}]
+
+The \type {string} library has a few extra functions, for example \libidx
+{string} {explode}. This function takes upto two arguments: \type
+{string.explode(s[,m])} and returns an array containing the string argument \type
+{s} split into sub-strings based on the value of the string argument \type {m}.
+The second argument is a string that is either empty (this splits the string into
+characters), a single character (this splits on each occurrence of that
+character, possibly introducing empty strings), or a single character followed by
+the plus sign \type {+} (this special version does not create empty sub-strings).
+The default value for \type {m} is \quote {\type { +}} (multiple spaces). Note:
+\type {m} is not hidden by surrounding braces as it would be if this function was
+written in \TEX\ macros.
+
+The \type {string} library also has six extra iterators that return strings
+piecemeal: \libidx {string} {utfvalues}, \libidx {string} {utfcharacters},
+\libidx {string} {characters}, \libidx {string} {characterpairs}, \libidx
+{string} {bytes} and \libidx {string} {bytepairs}.
+
+\startitemize
+\startitem
+ \type {string.utfvalues(s)}: an integer value in the \UNICODE\ range
+\stopitem
+\startitem
+ \type {string.utfcharacters(s)}: a string with a single \UTF-8 token in it
+\stopitem
+\startitem
+ \type {string.cWharacters(s)}: a string containing one byte
+\stopitem
+\startitem
+ \type {string.characterpairs(s)}: two strings each containing one byte or an
+ empty second string if the string length was odd
+\stopitem
+\startitem
+ \type {string.bytes(s)}: a single byte value
+\stopitem
+\startitem
+ \type {string.bytepairs(s)}: two byte values or nil instead of a number as
+ its second return value if the string length was odd
+\stopitem
+\stopitemize
+
+The \type {string.characterpairs()} and \type {string.bytepairs()} iterators
+are useful especially in the conversion of \UTF16 encoded data into \UTF8.
+
+There is also a two|-|argument form of \type {string.dump()}. The second argument
+is a boolean which, if true, strips the symbols from the dumped data. This
+matches an extension made in \type {luajit}. This is typically a function that
+gets adapted as \LUA\ itself progresses.
+
+The \type {string} library functions \type {len}, \type {lower}, \type {sub}
+etc.\ are not \UNICODE|-|aware. For strings in the \UTF8 encoding, i.e., strings
+containing characters above code point 127, the corresponding functions from the
+\type {slnunicode} library can be used, e.g., \type {unicode.utf8.len}, \type
+{unicode.utf8.lower} etc.\ The exceptions are \type {unicode.utf8.find}, that
+always returns byte positions in a string, and \type {unicode.utf8.match} and
+\type {unicode.utf8.gmatch}. While the latter two functions in general {\it
+are} \UNICODE|-|aware, they fall|-|back to non|-|\UNICODE|-|aware behavior when
+using the empty capture \type {()} but other captures work as expected. For the
+interpretation of character classes in \type {unicode.utf8} functions refer to
+the library sources at \hyphenatedurl {http://luaforge.net/projects/sln}.
+
+Version 5.3 of \LUA\ provides some native \UTF8 support but we have added a few
+similar helpers too: \libidx {string} {utfvalue}, \libidx {string} {utfcharacter}
+and \libidx {string} {utflength}.
+
+\startitemize
+\startitem
+ \type {string.utfvalue(s)}: returns the codepoints of the characters in the
+ given string
+\stopitem
+\startitem
+ \type {string.utfcharacter(c,...)}: returns a string with the characters of
+ the given code points
+\stopitem
+\startitem
+ \type {string.utflength(s)}: returns the length of the given string
+\stopitem
+\stopitemize
+
+These three functions are relative fast and don't do much checking. They can be
+used as building blocks for other helpers.
+
+\stopsection
+
+\startsection[title={Extra \type {os} library functions}]
+
+The \type {os} library has a few extra functions and variables: \libidx {os}
+{selfdir}, \libidx {os} {selfarg}, \libidx {os} {setenv}, \libidx {os} {env}, \libidx {os}
+{gettimeofday}, \libidx {os} {type}, \libidx {os} {name} and \libidx {os}
+{uname}, that we will discuss here. There are also some time related helpers in
+the \type {lua} namespace.
+
+\startitemize
+
+% selfbin
+% selfpath
+% selfdir
+% selfbase
+% selfname
+% selfcore
+
+\startitem
+ \type {os.selfdir} is a variable that holds the directory path of the
+ actual executable. For example: \type {\directlua {tex.sprint(os.selfdir)}}.
+\stopitem
+
+\startitem
+ \type {os.selfarg} is a table with the command line arguments.
+\stopitem
+
+\startitem
+ \type {os.setenv(key,value)} sets a variable in the environment. Passing
+ \type {nil} instead of a value string will remove the variable.
+\stopitem
+
+\startitem
+ \type {os.env} is a hash table containing a dump of the variables and
+ values in the process environment at the start of the run. It is writeable,
+ but the actual environment is \notabene {not} updated automatically.
+\stopitem
+
+\startitem
+ \type {os.gettimeofday} returns the current \quote {\UNIX\ time}, but as a
+ float. Keep in mind that there might be platforms where this function is
+ available.
+\stopitem
+
+\startitem
+ \type {os.type} is a string that gives a global indication of the class of
+ operating system. The possible values are currently \type {windows}, \type
+ {unix}, and \type {msdos} (you are unlikely to find this value \quote {in the
+ wild}).
+\stopitem
+
+\startitem
+ \type {os.name} is a string that gives a more precise indication of the
+ operating system. These possible values are not yet fixed, and for \type
+ {os.type} values \type {windows} and \type {msdos}, the \type {os.name}
+ values are simply \type {windows} and \type {msdos}
+
+ The list for the type \type {unix} is more precise: \type {linux}, \type
+ {freebsd}, \type {kfreebsd}, \type {cygwin}, \type {openbsd}, \type
+ {solaris}, \type {sunos} (pre-solaris), \type {hpux}, \type {irix}, \type
+ {macosx}, \type {gnu} (hurd), \type {bsd} (unknown, but \BSD|-|like), \type
+ {sysv}, \type {generic} (unknown). But \unknown\ we only provide \LUAMETATEX\
+ binaries for the mainstream variants.
+
+ Officially we only support mainstream systems: \MSWINDOWS, \LINUX, \FREEBSD\
+ and \OSX. Of course one can build \LUAMETATEX\ for other systems, in which
+ case on has to check the above.
+\stopitem
+
+\startitem
+ \type {os.uname} returns a table with specific operating system
+ information acquired at runtime. The keys in the returned table are all
+ string values, and their names are: \type {sysname}, \type {machine}, \type
+ {release}, \type {version}, and \type {nodename}.
+\stopitem
+
+\stopitemize
+
+\stopsection
+
+\startsection[title={The \type {lua} library functions}]
+
+The \type {lua} library provide some general helpers.
+
+\startitemize
+
+\startitem
+ The \type {newtable} and \type {newindex} functions can be used to create
+ tables with space reserved beforehand for the given amount of entries.
+\stopitem
+
+\startitem
+ The \type {getstacktop} function returns a number that can be used for
+ diagnostic purposes.
+\stopitem
+
+\startitem
+ The functions \type {getruntime}, \type {getcurrenttime}, \type
+ {getpreciseticks} and \type {getpreciseseconds} return what their name
+ suggests.
+\stopitem
+
+\startitem
+ On \MSWINDOWS\ the \type {getcodepage} function returns two numbers, one
+ for the command handler and one for the graphical user interface.
+\stopitem
+
+\startitem
+ The name of the startup file is reported by \type {getstartupfile}.
+\stopitem
+
+\startitem
+ The \LUA\ version is reported by \type {getversion}.
+\stopitem
+
+\startitem
+ The \type {lua.openfile} function can be used instead of \type {io.open}. On
+ \MSWINDOWS\ it will convert the filename to a so called wide one which means
+ that filenames in \UTF8 encoding will work ok. On the other hand, names given
+ in the codepage won't.
+\stopitem
+
+\stopitemize
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-lua.tex b/doc/context/sources/general/manuals/luametatex/luametatex-lua.tex
new file mode 100644
index 000000000..50db9593b
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-lua.tex
@@ -0,0 +1,224 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-lua
+
+\startchapter[reference=lua,title={Using \LUAMETATEX}]
+
+\startsection[title={Initialization},reference=init]
+
+\startsubsection[title={\LUAMETATEX\ as a \LUA\ interpreter}]
+
+\topicindex {initialization}
+\topicindex {\LUA+interpreter}
+
+Although \LUAMETATEX\ is primarily meant as a \TEX\ engine, it can also serve as
+a stand alone \LUA\ interpreter. There are two ways to make \LUAMETATEX\ behave
+like a standalone \LUA\ interpreter:
+
+\startitemize[packed]
+\startitem
+ if a \type {--luaonly} option is given on the commandline, or
+\stopitem
+\startitem
+ if the only non|-|option argument (file) on the commandline has the extension
+ \type {lua} or \type {luc}.
+\stopitem
+\stopitemize
+
+In this mode, it will set \LUA's \type {arg[0]} to the found script name, pushing
+preceding options in negative values and the rest of the command line in the
+positive values, just like the \LUA\ interpreter does.
+
+\LUAMETATEX\ will exit immediately after executing the specified \LUA\ script and is,
+in effect, a somewhat bulky stand alone \LUA\ interpreter with a bunch of extra
+preloaded libraries.
+
+When no argument is given, \LUAMETATEX\ will look for a \LUA\ file with the same
+name as the binary and run that one when present. This makes it possible to use
+the engine as a stub. For instance, in \CONTEXT\ a symlink from \type {mtxrun} to
+type {luametatex} will run the \type {mtxrun.lua} script when present in the same
+path as the binary itself
+
+\stopsubsection
+
+\startsubsection[title={Other commandline processing}]
+
+\topicindex {command line}
+
+When the \LUAMETATEX\ executable starts, it looks for the \type {--lua} command line
+option. If there is no \type {--lua} option, the command line is interpreted in a
+similar fashion as the other \TEX\ engines. All options are accepted but only some
+have are understood by \LUAMETATEX\ itself:
+
+\starttabulate[|l|p|]
+\DB commandline argument \BC explanation \NC \NR
+\TB
+\NC \type{--credits} \NC display credits and exit \NC \NR
+\NC \type{--fmt=FORMAT} \NC load the format file \type {FORMAT} \NC\NR
+\NC \type{--help} \NC display help and exit \NC\NR
+\NC \type{--ini} \NC be \type {iniluatex}, for dumping formats \NC\NR
+\NC \type{--jobname=STRING} \NC set the job name to \type {STRING} \NC \NR
+\NC \type{--lua=FILE} \NC load and execute a \LUA\ initialization script \NC\NR
+\NC \type{--version} \NC display version and exit \NC \NR
+\LL
+\stoptabulate
+
+There are less options than with \LUATEX, because one has to deal with them in
+\LUA\ anyway. There are no options to enter a safer mode or control executing
+programs. This can easily be achieved with a startup \LUA\ script.
+
+The value to use for \prm {jobname} is decided as follows:
+
+\startitemize
+\startitem
+ If \type {--jobname} is given on the command line, its argument will be the
+ value for \prm {jobname}, without any changes. The argument will not be
+ used for actual input so it need not exist. The \type {--jobname} switch only
+ controls the \prm {jobname} setting.
+\stopitem
+\startitem
+ Otherwise, \prm {jobname} will be the name of the first file that is read
+ from the file system, with any path components and the last extension (the
+ part following the last \type {.}) stripped off.
+\stopitem
+\startitem
+ There is an exception to the previous point: if the command line goes into
+ interactive mode (by starting with a command) and there are no files input
+ via \prm {everyjob} either, then the \prm {jobname} is set to \type
+ {texput} as a last resort.
+\stopitem
+\stopitemize
+
+Next the initialization script is loaded and executed. From within the script,
+the entire command line is available in the \LUA\ table \type {arg}, beginning
+with \type {arg[0]}, containing the name of the executable. As consequence
+warnings about unrecognized options are suppressed.
+
+Command line processing happens very early on. So early, in fact, that none of
+\TEX's initializations have taken place yet. The \LUA\ libraries that don't deal
+with \TEX\ are initialized early.
+
+\LUAMETATEX\ allows some of the command line options to be overridden by reading
+values from the \type {texconfig} table at the end of script execution (see the
+description of the \type {texconfig} table later on in this document for more
+details on which ones exactly).
+
+So let's summarize this. The handling of when is called jobname is a bit complex.
+There can be explicit names set on the command line but when not set they can be
+taken from the \type {texconfig} table.
+
+\starttabulate[|l|T|T|T|]
+\NC startup filename \NC --lua \NC a \LUA\ file \NC \NC \NR
+\NC startup jobname \NC --jobname \NC a \TEX\ tex \NC texconfig.jobname \NC \NR
+\NC startup dumpname \NC --fmt \NC a format file \NC texconfig.formatname \NC \NR
+\stoptabulate
+
+These names are initialized according to \type {--luaonly} or the first filename
+seen in the list of options. Special treatment of \type {&} and \type {*} as well
+as interactive startup is gone.
+
+When we are in \TEX\ mode at some point the engine needs a filename, for instance
+for opening a log file. At that moment the set jobname becomes the internal one
+and when it has not been set which internalized to jobname but when not set
+becomes \type {texput}. When you see a \type {texput.log} file someplace on your
+system it normally indicates a bad run.
+
+When running on \MSWINDOWS\ the command line, filenames, environment variable
+access etc.\ internally uses the current code page but to the user is exposed as
+\UTF8. Normally users won't notice this.
+
+% fileio_state .jobname : a tex string (set when a (log) file is opened)
+% engine_state .startup_jobname : handles by option parser
+% environment_state.input_name : temporary interceptor
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={\LUA\ behaviour}]
+
+\startsubsection[title={The \LUA\ version}]
+
+\topicindex {\LUA+libraries}
+\topicindex {\LUA+extensions}
+
+We currently use \LUA\ 5.4 and will follow developments of the language but
+normally with some delay. Therefore the user needs to keep an eye on (subtle)
+differences in successive versions of the language. Here is an example of one
+aspect.
+
+\LUA s \type {tostring} function (and \type {string.format} may return values in
+scientific notation, thereby confusing the \TEX\ end of things when it is used as
+the right|-|hand side of an assignment to a \prm {dimen} or \prm {count}. The
+output of these serializers also depend on the \LUA\ version, so in \LUA\ 5.3 you
+can get different output than from 5.2. It is best not to depend the automatic
+cast from string to number and vise versa as this can change in future versions.
+
+\stopsubsection
+
+\startsubsection[title={Locales}]
+
+\index {locales}
+
+In stock \LUA, many things depend on the current locale. In \LUAMETATEX, we can't
+do that, because it makes documents unportable. While \LUAMETATEX\ is running if
+forces the following locale settings:
+
+\starttyping
+LC_CTYPE=C
+LC_COLLATE=C
+LC_NUMERIC=C
+\stoptyping
+
+There is no way to change that as it would interfere badly with the often
+language specific conversions needed at the \TEX\ end.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={\LUA\ modules}]
+
+\topicindex {\LUA+libraries}
+\topicindex {\LUA+modules}
+
+Of course the regular \LUA\ modules are present. In addition we provide the \type
+{lpeg} library by Roberto Ierusalimschy, This library is not \UNICODE|-|aware,
+but interprets strings on a byte|-|per|-|byte basis. This mainly means that \type
+{lpeg.S} cannot be used with \UTF8 characters encoded in more than two bytes, and
+thus \type {lpeg.S} will look for one of those two bytes when matching, not the
+combination of the two. The same is true for \type {lpeg.R}, although the latter
+will display an error message if used with multibyte characters. Therefore \type
+{lpeg.R('aä')} results in the message \type {bad argument #1 to 'R' (range must
+have two characters)}, since to \type {lpeg}, \type {ä} is two 'characters'
+(bytes), so \type {aä} totals three. In practice this is no real issue and with
+some care you can deal with \UNICODE\ just fine.
+
+There are some more libraries present. These are discussed on a later chapter.
+For instance we embed \type {luasocket} but contrary to \LUATEX\ don't embed the
+related \LUA\ code. An adapted version of \type {luafilesystem} is also included.
+There is a more extensive math library and there are libraries that deal with
+encryption and compression.
+
+\stopsection
+
+\startsection[title={Testing}]
+
+\topicindex {testing}
+
+For development reasons you can influence the used startup date and time. By
+setting the \type {start_time} variable in the \type {texconfig} table; as with
+other variables we use the internal name there. When Universal Time is needed,
+set the entry \type {use_utc_time} in the \type {texconfig} table.
+
+In \CONTEXT\ we provide the command line argument \type {--nodates} that does
+a bit more than disabling dates; it avoids time dependent information in the
+output file for instance.
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-math.tex b/doc/context/sources/general/manuals/luametatex/luametatex-math.tex
new file mode 100644
index 000000000..111a6cf03
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-math.tex
@@ -0,0 +1,1585 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-math
+
+\startchapter[reference=math,title={Math}]
+
+\startsection[title={Traditional alongside \OPENTYPE}]
+
+\topicindex {math}
+
+At this point there is no difference between \LUAMETATEX\ and \LUATEX\ with
+respect to math. The handling of mathematics in \LUATEX\ differs quite a bit from
+how \TEX82 (and therefore \PDFTEX) handles math. First, \LUATEX\ adds primitives
+and extends some others so that \UNICODE\ input can be used easily. Second, all
+of \TEX82's internal special values (for example for operator spacing) have been
+made accessible and changeable via control sequences. Third, there are extensions
+that make it easier to use \OPENTYPE\ math fonts. And finally, there are some
+extensions that have been proposed or considered in the past that are now added
+to the engine.
+
+\stopsection
+
+\startsection[title={Unicode math characters}]
+
+\topicindex {math+\UNICODE}
+\topicindex {\UNICODE+math}
+
+Character handling is now extended up to the full \UNICODE\ range (the \type {\U}
+prefix), which is compatible with \XETEX.
+
+The math primitives from \TEX\ are kept as they are, except for the ones that
+convert from input to math commands: \type {mathcode}, and \type {delcode}. These
+two now allow for a 21-bit character argument on the left hand side of the equals
+sign.
+
+Some of the new \LUATEX\ primitives read more than one separate value. This is
+shown in the tables below by a plus sign.
+
+The input for such primitives would look like this:
+
+\starttyping
+\def\overbrace{\Umathaccent 0 1 "23DE }
+\stoptyping
+
+The altered \TEX82 primitives are:
+
+\starttabulate[|l|l|r|c|l|r|]
+\DB primitive \BC min \BC max \BC \kern 2em \BC min \BC max \NC \NR
+\TB
+\NC \prm {mathcode} \NC 0 \NC 10FFFF \NC = \NC 0 \NC 8000 \NC \NR
+\NC \prm {delcode} \NC 0 \NC 10FFFF \NC = \NC 0 \NC FFFFFF \NC \NR
+\LL
+\stoptabulate
+
+The unaltered ones are:
+
+\starttabulate[|l|l|r|]
+\DB primitive \BC min \BC max \NC \NR
+\TB
+\NC \prm {mathchardef} \NC 0 \NC 8000 \NC \NR
+\NC \prm {mathchar} \NC 0 \NC 7FFF \NC \NR
+\NC \prm {mathaccent} \NC 0 \NC 7FFF \NC \NR
+\NC \prm {delimiter} \NC 0 \NC 7FFFFFF \NC \NR
+\NC \prm {radical} \NC 0 \NC 7FFFFFF \NC \NR
+\LL
+\stoptabulate
+
+For practical reasons \prm {mathchardef} will silently accept values larger
+that \type {0x8000} and interpret it as \lpr {Umathcharnumdef}. This is needed
+to satisfy older macro packages.
+
+The following new primitives are compatible with \XETEX:
+
+% somewhat fuzzy:
+
+\starttabulate[|l|l|r|c|l|r|]
+\DB primitive \BC min \BC max \BC \kern 2em \BC min \BC max \NC \NR
+\TB
+\NC \lpr {Umathchardef} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR
+\NC \lpr {Umathcharnumdef}\rlap{\high{5}} \NC -80000000 \NC 7FFFFFFF \NC \NC \NC \NC \NR
+\NC \lpr {Umathcode} \NC 0 \NC 10FFFF \NC = \NC 0+0+0 \NC 7+FF+10FFFF \NC \NR
+\NC \lpr {Udelcode} \NC 0 \NC 10FFFF \NC = \NC 0+0 \NC FF+10FFFF \NC \NR
+\NC \lpr {Umathchar} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR
+\NC \lpr {Umathaccent} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR
+\NC \lpr {Udelimiter} \NC 0+0+0 \NC 7+FF+10FFFF \NC \NC \NC \NC \NR
+\NC \lpr {Uradical} \NC 0+0 \NC FF+10FFFF \NC \NC \NC \NC \NR
+\NC \lpr {Umathcharnum} \NC -80000000 \NC 7FFFFFFF \NC \NC \NC \NC \NR
+\NC \lpr {Umathcodenum} \NC 0 \NC 10FFFF \NC = \NC -80000000 \NC 7FFFFFFF \NC \NR
+\NC \lpr {Udelcodenum} \NC 0 \NC 10FFFF \NC = \NC -80000000 \NC 7FFFFFFF \NC \NR
+\LL
+\stoptabulate
+
+Specifications typically look like:
+
+\starttyping
+\Umathchardef\xx="1"0"456
+\Umathcode 123="1"0"789
+\stoptyping
+
+The new primitives that deal with delimiter|-|style objects do not set up a
+\quote {large family}. Selecting a suitable size for display purposes is expected
+to be dealt with by the font via the \lpr {Umathoperatorsize} parameter.
+
+For some of these primitives, all information is packed into a single signed
+integer. For the first two (\lpr {Umathcharnum} and \lpr {Umathcodenum}), the
+lowest 21 bits are the character code, the 3 bits above that represent the math
+class, and the family data is kept in the topmost bits. This means that the values
+for math families 128--255 are actually negative. For \lpr {Udelcodenum} there
+is no math class. The math family information is stored in the bits directly on
+top of the character code. Using these three commands is not as natural as using
+the two- and three|-|value commands, so unless you know exactly what you are
+doing and absolutely require the speedup resulting from the faster input
+scanning, it is better to use the verbose commands instead.
+
+The \lpr {Umathaccent} command accepts optional keywords to control various
+details regarding math accents. See \in {section} [mathacc] below for details.
+
+There are more new primitives and all of these will be explained in following
+sections:
+
+\starttabulate[|l|l|]
+\DB primitive \BC value range (in hex) \NC \NR
+\TB
+\NC \lpr {Uroot} \NC 0 + 0--FF + 10FFFF \NC \NR
+\NC \lpr {Uoverdelimiter} \NC 0 + 0--FF + 10FFFF \NC \NR
+\NC \lpr {Uunderdelimiter} \NC 0 + 0--FF + 10FFFF \NC \NR
+\NC \lpr {Udelimiterover} \NC 0 + 0--FF + 10FFFF \NC \NR
+\NC \lpr {Udelimiterunder} \NC 0 + 0--FF + 10FFFF \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title={Math styles}]
+
+\subsection{\lpr {mathstyle}}
+
+\topicindex {math+styles}
+
+It is possible to discover the math style that will be used for a formula in an
+expandable fashion (while the math list is still being read). To make this
+possible, \LUATEX\ adds the new primitive: \lpr {mathstyle}. This is a \quote
+{convert command} like e.g. \prm {romannumeral}: its value can only be read,
+not set.
+
+The returned value is between 0 and 7 (in math mode), or $-1$ (all other modes).
+For easy testing, the eight math style commands have been altered so that they can
+be used as numeric values, so you can write code like this:
+
+\starttyping
+\ifnum\mathstyle=\textstyle
+ \message{normal text style}
+\else \ifnum\mathstyle=\crampedtextstyle
+ \message{cramped text style}
+\fi \fi
+\stoptyping
+
+Sometimes you won't get what you expect so a bit of explanation might help to
+understand what happens. When math is parsed and expanded it gets turned into a
+linked list. In a second pass the formula will be build. This has to do with the
+fact that in order to determine the automatically chosen sizes (in for instance
+fractions) following content can influence preceding sizes. A side effect of this
+is for instance that one cannot change the definition of a font family (and
+thereby reusing numbers) because the number that got used is stored and used in
+the second pass (so changing \type {\fam 12} mid|-|formula spoils over to
+preceding use of that family).
+
+The style switching primitives like \prm {textstyle} are turned into nodes so the
+styles set there are frozen. The \prm {mathchoice} primitive results in four
+lists being constructed of which one is used in the second pass. The fact that
+some automatic styles are not yet known also means that the \lpr {mathstyle}
+primitive expands to the current style which can of course be different from the
+one really used. It's a snapshot of the first pass state. As a consequence in the
+following example you get a style number (first pass) typeset that can actually
+differ from the used style (second pass). In the case of a math choice used
+ungrouped, the chosen style is used after the choice too, unless you group.
+
+\startbuffer[1]
+ [a:\mathstyle]\quad
+ \bgroup
+ \mathchoice
+ {\bf \scriptstyle (x:d :\mathstyle)}
+ {\bf \scriptscriptstyle (x:t :\mathstyle)}
+ {\bf \scriptscriptstyle (x:s :\mathstyle)}
+ {\bf \scriptscriptstyle (x:ss:\mathstyle)}
+ \egroup
+ \quad[b:\mathstyle]\quad
+ \mathchoice
+ {\bf \scriptstyle (y:d :\mathstyle)}
+ {\bf \scriptscriptstyle (y:t :\mathstyle)}
+ {\bf \scriptscriptstyle (y:s :\mathstyle)}
+ {\bf \scriptscriptstyle (y:ss:\mathstyle)}
+ \quad[c:\mathstyle]\quad
+ \bgroup
+ \mathchoice
+ {\bf \scriptstyle (z:d :\mathstyle)}
+ {\bf \scriptscriptstyle (z:t :\mathstyle)}
+ {\bf \scriptscriptstyle (z:s :\mathstyle)}
+ {\bf \scriptscriptstyle (z:ss:\mathstyle)}
+ \egroup
+ \quad[d:\mathstyle]
+\stopbuffer
+
+\startbuffer[2]
+ [a:\mathstyle]\quad
+ \begingroup
+ \mathchoice
+ {\bf \scriptstyle (x:d :\mathstyle)}
+ {\bf \scriptscriptstyle (x:t :\mathstyle)}
+ {\bf \scriptscriptstyle (x:s :\mathstyle)}
+ {\bf \scriptscriptstyle (x:ss:\mathstyle)}
+ \endgroup
+ \quad[b:\mathstyle]\quad
+ \mathchoice
+ {\bf \scriptstyle (y:d :\mathstyle)}
+ {\bf \scriptscriptstyle (y:t :\mathstyle)}
+ {\bf \scriptscriptstyle (y:s :\mathstyle)}
+ {\bf \scriptscriptstyle (y:ss:\mathstyle)}
+ \quad[c:\mathstyle]\quad
+ \begingroup
+ \mathchoice
+ {\bf \scriptstyle (z:d :\mathstyle)}
+ {\bf \scriptscriptstyle (z:t :\mathstyle)}
+ {\bf \scriptscriptstyle (z:s :\mathstyle)}
+ {\bf \scriptscriptstyle (z:ss:\mathstyle)}
+ \endgroup
+ \quad[d:\mathstyle]
+\stopbuffer
+
+\typebuffer[1]
+
+% \typebuffer[2]
+
+This gives:
+
+\blank $\displaystyle \getbuffer[1]$ \blank
+\blank $\textstyle \getbuffer[1]$ \blank
+
+Using \prm {begingroup} \unknown\ \prm {endgroup} instead gives:
+
+\blank $\displaystyle \getbuffer[2]$ \blank
+\blank $\textstyle \getbuffer[2]$ \blank
+
+This might look wrong but it's just a side effect of \lpr {mathstyle} expanding
+to the current (first pass) style and the number being injected in the list that
+gets converted in the second pass. It all makes sense and it illustrates the
+importance of grouping. In fact, the math choice style being effective afterwards
+has advantages. It would be hard to get it otherwise.
+
+\subsection{\lpr {Ustack}}
+
+\topicindex {math+stacks}
+
+There are a few math commands in \TEX\ where the style that will be used is not
+known straight from the start. These commands (\prm {over}, \prm {atop},
+\prm {overwithdelims}, \prm {atopwithdelims}) would therefore normally return
+wrong values for \lpr {mathstyle}. To fix this, \LUATEX\ introduces a special
+prefix command: \lpr {Ustack}:
+
+\starttyping
+$\Ustack {a \over b}$
+\stoptyping
+
+The \lpr {Ustack} command will scan the next brace and start a new math group
+with the correct (numerator) math style.
+
+\subsection{The new \type {\cramped...style} commands}
+
+\topicindex {math+styles}
+\topicindex {math+spacing}
+\topicindex {math+cramped}
+
+\LUATEX\ has four new primitives to set the cramped math styles directly:
+
+\starttyping
+\crampeddisplaystyle
+\crampedtextstyle
+\crampedscriptstyle
+\crampedscriptscriptstyle
+\stoptyping
+
+These additional commands are not all that valuable on their own, but they come
+in handy as arguments to the math parameter settings that will be added shortly.
+
+In Eijkhouts \quotation {\TEX\ by Topic} the rules for handling styles in scripts
+are described as follows:
+
+\startitemize
+\startitem
+ In any style superscripts and subscripts are taken from the next smaller style.
+ Exception: in display style they are in script style.
+\stopitem
+\startitem
+ Subscripts are always in the cramped variant of the style; superscripts are only
+ cramped if the original style was cramped.
+\stopitem
+\startitem
+ In an \type {..\over..} formula in any style the numerator and denominator are
+ taken from the next smaller style.
+\stopitem
+\startitem
+ The denominator is always in cramped style; the numerator is only in cramped
+ style if the original style was cramped.
+\stopitem
+\startitem
+ Formulas under a \type {\sqrt} or \prm {overline} are in cramped style.
+\stopitem
+\stopitemize
+
+In \LUATEX\ one can set the styles in more detail which means that you sometimes
+have to set both normal and cramped styles to get the effect you want. (Even) if
+we force styles in the script using \prm {scriptstyle} and \lpr
+{crampedscriptstyle} we get this:
+
+\startbuffer[demo]
+\starttabulate
+\DB style \BC example \NC \NR
+\TB
+\NC default \NC $b_{x=xx}^{x=xx}$ \NC \NR
+\NC script \NC $b_{\scriptstyle x=xx}^{\scriptstyle x=xx}$ \NC \NR
+\NC crampedscript \NC $b_{\crampedscriptstyle x=xx}^{\crampedscriptstyle x=xx}$ \NC \NR
+\LL
+\stoptabulate
+\stopbuffer
+
+\getbuffer[demo]
+
+Now we set the following parameters
+
+\startbuffer[setup]
+\Umathordrelspacing\scriptstyle=30mu
+\Umathordordspacing\scriptstyle=30mu
+\stopbuffer
+
+\typebuffer[setup]
+
+This gives a different result:
+
+\start\getbuffer[setup,demo]\stop
+
+But, as this is not what is expected (visually) we should say:
+
+\startbuffer[setup]
+\Umathordrelspacing\scriptstyle=30mu
+\Umathordordspacing\scriptstyle=30mu
+\Umathordrelspacing\crampedscriptstyle=30mu
+\Umathordordspacing\crampedscriptstyle=30mu
+\stopbuffer
+
+\typebuffer[setup]
+
+Now we get:
+
+\start\getbuffer[setup,demo]\stop
+
+\stopsection
+
+\startsection[title={Math parameter settings}]
+
+\subsection {Many new \lpr {Umath*} primitives}
+
+\topicindex {math+parameters}
+
+In \LUATEX, the font dimension parameters that \TEX\ used in math typesetting are
+now accessible via primitive commands. In fact, refactoring of the math engine
+has resulted in many more parameters than were not accessible before.
+
+\starttabulate
+\DB primitive name \BC description \NC \NR
+\TB
+\NC \lpr {Umathquad} \NC the width of 18 mu's \NC \NR
+\NC \lpr {Umathaxis} \NC height of the vertical center axis of
+ the math formula above the baseline \NC \NR
+\NC \lpr {Umathoperatorsize} \NC minimum size of large operators in display mode \NC \NR
+\NC \lpr {Umathoverbarkern} \NC vertical clearance above the rule \NC \NR
+\NC \lpr {Umathoverbarrule} \NC the width of the rule \NC \NR
+\NC \lpr {Umathoverbarvgap} \NC vertical clearance below the rule \NC \NR
+\NC \lpr {Umathunderbarkern} \NC vertical clearance below the rule \NC \NR
+\NC \lpr {Umathunderbarrule} \NC the width of the rule \NC \NR
+\NC \lpr {Umathunderbarvgap} \NC vertical clearance above the rule \NC \NR
+\NC \lpr {Umathradicalkern} \NC vertical clearance above the rule \NC \NR
+\NC \lpr {Umathradicalrule} \NC the width of the rule \NC \NR
+\NC \lpr {Umathradicalvgap} \NC vertical clearance below the rule \NC \NR
+\NC \lpr {Umathradicaldegreebefore}\NC the forward kern that takes place before placement of
+ the radical degree \NC \NR
+\NC \lpr {Umathradicaldegreeafter} \NC the backward kern that takes place after placement of
+ the radical degree \NC \NR
+\NC \lpr {Umathradicaldegreeraise} \NC this is the percentage of the total height and depth of
+ the radical sign that the degree is raised by; it is
+ expressed in \type {percents}, so 60\% is expressed as the
+ integer $60$ \NC \NR
+\NC \lpr {Umathstackvgap} \NC vertical clearance between the two
+ elements in a \prm {atop} stack \NC \NR
+\NC \lpr {Umathstacknumup} \NC numerator shift upward in \prm {atop} stack \NC \NR
+\NC \lpr {Umathstackdenomdown} \NC denominator shift downward in \prm {atop} stack \NC \NR
+\NC \lpr {Umathfractionrule} \NC the width of the rule in a \prm {over} \NC \NR
+\NC \lpr {Umathfractionnumvgap} \NC vertical clearance between the numerator and the rule \NC \NR
+\NC \lpr {Umathfractionnumup} \NC numerator shift upward in \prm {over} \NC \NR
+\NC \lpr {Umathfractiondenomvgap} \NC vertical clearance between the denominator and the rule \NC \NR
+\NC \lpr {Umathfractiondenomdown} \NC denominator shift downward in \prm {over} \NC \NR
+\NC \lpr {Umathfractiondelsize} \NC minimum delimiter size for \type {\...withdelims} \NC \NR
+\NC \lpr {Umathlimitabovevgap} \NC vertical clearance for limits above operators \NC \NR
+\NC \lpr {Umathlimitabovebgap} \NC vertical baseline clearance for limits above operators \NC \NR
+\NC \lpr {Umathlimitabovekern} \NC space reserved at the top of the limit \NC \NR
+\NC \lpr {Umathlimitbelowvgap} \NC vertical clearance for limits below operators \NC \NR
+\NC \lpr {Umathlimitbelowbgap} \NC vertical baseline clearance for limits below operators \NC \NR
+\NC \lpr {Umathlimitbelowkern} \NC space reserved at the bottom of the limit \NC \NR
+\NC \lpr {Umathoverdelimitervgap} \NC vertical clearance for limits above delimiters \NC \NR
+\NC \lpr {Umathoverdelimiterbgap} \NC vertical baseline clearance for limits above delimiters \NC \NR
+\NC \lpr {Umathunderdelimitervgap} \NC vertical clearance for limits below delimiters \NC \NR
+\NC \lpr {Umathunderdelimiterbgap} \NC vertical baseline clearance for limits below delimiters \NC \NR
+\NC \lpr {Umathsubshiftdrop} \NC subscript drop for boxes and subformulas \NC \NR
+\NC \lpr {Umathsubshiftdown} \NC subscript drop for characters \NC \NR
+\NC \lpr {Umathsupshiftdrop} \NC superscript drop (raise, actually) for boxes and subformulas \NC \NR
+\NC \lpr {Umathsupshiftup} \NC superscript raise for characters \NC \NR
+\NC \lpr {Umathsubsupshiftdown} \NC subscript drop in the presence of a superscript \NC \NR
+\NC \lpr {Umathsubtopmax} \NC the top of standalone subscripts cannot be higher than this
+ above the baseline \NC \NR
+\NC \lpr {Umathsupbottommin} \NC the bottom of standalone superscripts cannot be less than
+ this above the baseline \NC \NR
+\NC \lpr {Umathsupsubbottommax} \NC the bottom of the superscript of a combined super- and subscript
+ be at least as high as this above the baseline \NC \NR
+\NC \lpr {Umathsubsupvgap} \NC vertical clearance between super- and subscript \NC \NR
+\NC \lpr {Umathspaceafterscript} \NC additional space added after a super- or subscript \NC \NR
+\NC \lpr {Umathconnectoroverlapmin}\NC minimum overlap between parts in an extensible recipe \NC \NR
+\LL
+\stoptabulate
+
+Each of the parameters in this section can be set by a command like this:
+
+\starttyping
+\Umathquad\displaystyle=1em
+\stoptyping
+
+they obey grouping, and you can use \type {\the\Umathquad\displaystyle} if
+needed.
+
+\subsection{Font|-|based math parameters}
+
+\topicindex {math+parameters}
+
+While it is nice to have these math parameters available for tweaking, it would
+be tedious to have to set each of them by hand. For this reason, \LUATEX\
+initializes a bunch of these parameters whenever you assign a font identifier to
+a math family based on either the traditional math font dimensions in the font
+(for assignments to math family~2 and~3 using \TFM|-|based fonts like \type
+{cmsy} and \type {cmex}), or based on the named values in a potential \type
+{MathConstants} table when the font is loaded via Lua. If there is a \type
+{MathConstants} table, this takes precedence over font dimensions, and in that
+case no attention is paid to which family is being assigned to: the \type
+{MathConstants} tables in the last assigned family sets all parameters.
+
+In the table below, the one|-|letter style abbreviations and symbolic tfm font
+dimension names match those used in the \TeX book. Assignments to \prm
+{textfont} set the values for the cramped and uncramped display and text styles,
+\prm {scriptfont} sets the script styles, and \prm {scriptscriptfont} sets the
+scriptscript styles, so we have eight parameters for three font sizes. In the
+\TFM\ case, assignments only happen in family~2 and family~3 (and of course only
+for the parameters for which there are font dimensions).
+
+Besides the parameters below, \LUATEX\ also looks at the \quote {space} font
+dimension parameter. For math fonts, this should be set to zero.
+
+\def\MathLine#1#2#3#4#5%
+ {\TB
+ \NC \llap{\high{\tx #2\enspace}}\ttbf \string #1 \NC \tt #5 \NC \NR
+ \NC \tx #3 \NC \tt #4 \NC \NR}
+
+\starttabulate[|l|l|]
+\DB variable / style \BC tfm / opentype \NC \NR
+\MathLine{\Umathaxis} {} {} {AxisHeight} {axis_height}
+\MathLine{\Umathoperatorsize} {6} {D, D'} {DisplayOperatorMinHeight} {\emdash}
+\MathLine{\Umathfractiondelsize} {9} {D, D'} {FractionDelimiterDisplayStyleSize} {delim1}
+\MathLine{\Umathfractiondelsize} {9} {T, T', S, S', SS, SS'}{FractionDelimiterSize} {delim2}
+\MathLine{\Umathfractiondenomdown} {} {D, D'} {FractionDenominatorDisplayStyleShiftDown}{denom1}
+\MathLine{\Umathfractiondenomdown} {} {T, T', S, S', SS, SS'}{FractionDenominatorShiftDown} {denom2}
+\MathLine{\Umathfractiondenomvgap} {} {D, D'} {FractionDenominatorDisplayStyleGapMin} {3*default_rule_thickness}
+\MathLine{\Umathfractiondenomvgap} {} {T, T', S, S', SS, SS'}{FractionDenominatorGapMin} {default_rule_thickness}
+\MathLine{\Umathfractionnumup} {} {D, D'} {FractionNumeratorDisplayStyleShiftUp} {num1}
+\MathLine{\Umathfractionnumup} {} {T, T', S, S', SS, SS'}{FractionNumeratorShiftUp} {num2}
+\MathLine{\Umathfractionnumvgap} {} {D, D'} {FractionNumeratorDisplayStyleGapMin} {3*default_rule_thickness}
+\MathLine{\Umathfractionnumvgap} {} {T, T', S, S', SS, SS'}{FractionNumeratorGapMin} {default_rule_thickness}
+\MathLine{\Umathfractionrule} {} {} {FractionRuleThickness} {default_rule_thickness}
+\MathLine{\Umathskewedfractionhgap} {} {} {SkewedFractionHorizontalGap} {math_quad/2}
+\MathLine{\Umathskewedfractionvgap} {} {} {SkewedFractionVerticalGap} {math_x_height}
+\MathLine{\Umathlimitabovebgap} {} {} {UpperLimitBaselineRiseMin} {big_op_spacing3}
+\MathLine{\Umathlimitabovekern} {1} {} {0} {big_op_spacing5}
+\MathLine{\Umathlimitabovevgap} {} {} {UpperLimitGapMin} {big_op_spacing1}
+\MathLine{\Umathlimitbelowbgap} {} {} {LowerLimitBaselineDropMin} {big_op_spacing4}
+\MathLine{\Umathlimitbelowkern} {1} {} {0} {big_op_spacing5}
+\MathLine{\Umathlimitbelowvgap} {} {} {LowerLimitGapMin} {big_op_spacing2}
+\MathLine{\Umathoverdelimitervgap} {} {} {StretchStackGapBelowMin} {big_op_spacing1}
+\MathLine{\Umathoverdelimiterbgap} {} {} {StretchStackTopShiftUp} {big_op_spacing3}
+\MathLine{\Umathunderdelimitervgap} {} {} {StretchStackGapAboveMin} {big_op_spacing2}
+\MathLine{\Umathunderdelimiterbgap} {} {} {StretchStackBottomShiftDown} {big_op_spacing4}
+\MathLine{\Umathoverbarkern} {} {} {OverbarExtraAscender} {default_rule_thickness}
+\MathLine{\Umathoverbarrule} {} {} {OverbarRuleThickness} {default_rule_thickness}
+\MathLine{\Umathoverbarvgap} {} {} {OverbarVerticalGap} {3*default_rule_thickness}
+\MathLine{\Umathquad} {1} {} {<font_size(f)>} {math_quad}
+\MathLine{\Umathradicalkern} {} {} {RadicalExtraAscender} {default_rule_thickness}
+\MathLine{\Umathradicalrule} {2} {} {RadicalRuleThickness} {<not set>}
+\MathLine{\Umathradicalvgap} {3} {D, D'} {RadicalDisplayStyleVerticalGap} {default_rule_thickness+abs(math_x_height)/4}
+\MathLine{\Umathradicalvgap} {3} {T, T', S, S', SS, SS'}{RadicalVerticalGap} {default_rule_thickness+abs(default_rule_thickness)/4}
+\MathLine{\Umathradicaldegreebefore}{2} {} {RadicalKernBeforeDegree} {<not set>}
+\MathLine{\Umathradicaldegreeafter} {2} {} {RadicalKernAfterDegree} {<not set>}
+\MathLine{\Umathradicaldegreeraise} {2,7}{} {RadicalDegreeBottomRaisePercent} {<not set>}
+\MathLine{\Umathspaceafterscript} {4} {} {SpaceAfterScript} {script_space}
+\MathLine{\Umathstackdenomdown} {} {D, D'} {StackBottomDisplayStyleShiftDown} {denom1}
+\MathLine{\Umathstackdenomdown} {} {T, T', S, S', SS, SS'}{StackBottomShiftDown} {denom2}
+\MathLine{\Umathstacknumup} {} {D, D'} {StackTopDisplayStyleShiftUp} {num1}
+\MathLine{\Umathstacknumup} {} {T, T', S, S', SS, SS'}{StackTopShiftUp} {num3}
+\MathLine{\Umathstackvgap} {} {D, D'} {StackDisplayStyleGapMin} {7*default_rule_thickness}
+\MathLine{\Umathstackvgap} {} {T, T', S, S', SS, SS'}{StackGapMin} {3*default_rule_thickness}
+\MathLine{\Umathsubshiftdown} {} {} {SubscriptShiftDown} {sub1}
+\MathLine{\Umathsubshiftdrop} {} {} {SubscriptBaselineDropMin} {sub_drop}
+\MathLine{\Umathsubsupshiftdown} {8} {} {SubscriptShiftDownWithSuperscript} {\emdash}
+\MathLine{\Umathsubtopmax} {} {} {SubscriptTopMax} {abs(math_x_height*4)/5}
+\MathLine{\Umathsubsupvgap} {} {} {SubSuperscriptGapMin} {4*default_rule_thickness}
+\MathLine{\Umathsupbottommin} {} {} {SuperscriptBottomMin} {abs(math_x_height/4)}
+\MathLine{\Umathsupshiftdrop} {} {} {SuperscriptBaselineDropMax} {sup_drop}
+\MathLine{\Umathsupshiftup} {} {D} {SuperscriptShiftUp} {sup1}
+\MathLine{\Umathsupshiftup} {} {T, S, SS,} {SuperscriptShiftUp} {sup2}
+\MathLine{\Umathsupshiftup} {} {D', T', S', SS'} {SuperscriptShiftUpCramped} {sup3}
+\MathLine{\Umathsupsubbottommax} {} {} {SuperscriptBottomMaxWithSubscript} {abs(math_x_height*4)/5}
+\MathLine{\Umathunderbarkern} {} {} {UnderbarExtraDescender} {default_rule_thickness}
+\MathLine{\Umathunderbarrule} {} {} {UnderbarRuleThickness} {default_rule_thickness}
+\MathLine{\Umathunderbarvgap} {} {} {UnderbarVerticalGap} {3*default_rule_thickness}
+\MathLine{\Umathconnectoroverlapmin}{5} {} {MinConnectorOverlap} {0}
+\LL
+\stoptabulate
+
+Note 1: \OPENTYPE\ fonts set \lpr {Umathlimitabovekern} and \lpr
+{Umathlimitbelowkern} to zero and set \lpr {Umathquad} to the font size of the
+used font, because these are not supported in the \type {MATH} table,
+
+Note 2: Traditional \TFM\ fonts do not set \lpr {Umathradicalrule} because
+\TEX82\ uses the height of the radical instead. When this parameter is indeed not
+set when \LUATEX\ has to typeset a radical, a backward compatibility mode will
+kick in that assumes that an oldstyle \TEX\ font is used. Also, they do not set
+\lpr {Umathradicaldegreebefore}, \lpr {Umathradicaldegreeafter}, and \lpr
+{Umathradicaldegreeraise}. These are then automatically initialized to
+$5/18$quad, $-10/18$quad, and 60.
+
+Note 3: If \TFM\ fonts are used, then the \lpr {Umathradicalvgap} is not set
+until the first time \LUATEX\ has to typeset a formula because this needs
+parameters from both family~2 and family~3. This provides a partial backward
+compatibility with \TEX82, but that compatibility is only partial: once the \lpr
+{Umathradicalvgap} is set, it will not be recalculated any more.
+
+Note 4: When \TFM\ fonts are used a similar situation arises with respect to \lpr
+{Umathspaceafterscript}: it is not set until the first time \LUATEX\ has to
+typeset a formula. This provides some backward compatibility with \TEX82. But
+once the \lpr {Umathspaceafterscript} is set, \prm {scriptspace} will never be
+looked at again.
+
+Note 5: Traditional \TFM\ fonts set \lpr {Umathconnectoroverlapmin} to zero
+because \TEX82\ always stacks extensibles without any overlap.
+
+Note 6: The \lpr {Umathoperatorsize} is only used in \prm {displaystyle}, and is
+only set in \OPENTYPE\ fonts. In \TFM\ font mode, it is artificially set to one
+scaled point more than the initial attempt's size, so that always the \quote
+{first next} will be tried, just like in \TEX82.
+
+Note 7: The \lpr {Umathradicaldegreeraise} is a special case because it is the
+only parameter that is expressed in a percentage instead of a number of scaled
+points.
+
+Note 8: \type {SubscriptShiftDownWithSuperscript} does not actually exist in the
+\quote {standard} \OPENTYPE\ math font Cambria, but it is useful enough to be
+added.
+
+Note 9: \type {FractionDelimiterDisplayStyleSize} and \type
+{FractionDelimiterSize} do not actually exist in the \quote {standard} \OPENTYPE\
+math font Cambria, but were useful enough to be added.
+
+\stopsection
+
+\startsection[title={Math spacing}]
+
+\subsection{Setting inline surrounding space with \lpr {mathsurround[skip]}}
+
+\topicindex {math+spacing}
+
+Inline math is surrounded by (optional) \prm {mathsurround} spacing but that is a fixed
+dimension. There is now an additional parameter \lpr {mathsurroundskip}. When set to a
+non|-|zero value (or zero with some stretch or shrink) this parameter will replace
+\prm {mathsurround}. By using an additional parameter instead of changing the nature
+of \prm {mathsurround}, we can remain compatible. In the meantime a bit more
+control has been added via \lpr {mathsurroundmode}. This directive can take 6 values
+with zero being the default behaviour.
+
+\start
+
+\def\OneLiner#1#2%
+ {\NC \type{#1}
+ \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x$x$x}
+ \NC \dontleavehmode\inframed[align=normal,offset=0pt,frame=off]{\mathsurroundmode#1\relax\hsize 100pt x $x$ x}
+ \NC #2
+ \NC \NR}
+
+\startbuffer
+\mathsurround 10pt
+\mathsurroundskip20pt
+\stopbuffer
+
+\typebuffer \getbuffer
+
+\starttabulate[|c|c|c|pl|]
+\DB mode \BC x\$x\$x \BC x \$x\$ x \BC effect \NC \NR
+\TB
+\OneLiner{0}{obey \prm {mathsurround} when \lpr {mathsurroundskip} is 0pt}
+\OneLiner{1}{only add skip to the left}
+\OneLiner{2}{only add skip to the right}
+\OneLiner{3}{add skip to the left and right}
+\OneLiner{4}{ignore the skip setting, obey \prm {mathsurround}}
+\OneLiner{5}{disable all spacing around math}
+\OneLiner{6}{only apply \lpr {mathsurroundskip} when also spacing}
+\OneLiner{7}{only apply \lpr {mathsurroundskip} when no spacing}
+\LL
+\stoptabulate
+
+\stop
+
+Method six omits the surround glue when there is (x)spacing glue present while
+method seven does the opposite, the glue is only applied when there is (x)space
+glue present too. Anything more fancy, like checking the begining or end of a
+paragraph (or edges of a box) would not be robust anyway. If you want that you
+can write a callback that runs over a list and analyzes a paragraph. Actually, in
+that case you could also inject glue (or set the properties of a math node)
+explicitly. So, these modes are in practice mostly useful for special purposes
+and experiments (they originate in a tracker item). Keep in mind that this glue
+is part of the math node and not always treated as normal glue: it travels with
+the begin and end math nodes. Also, method 6 and 7 will zero the skip related
+fields in a node when applicable in the first occasion that checks them
+(linebreaking or packaging).
+
+\subsection{Pairwise spacing and \lpr {Umath...spacing} commands}
+
+\topicindex {math+spacing}
+
+Besides the parameters mentioned in the previous sections, there are also 64 new
+primitives to control the math spacing table (as explained in Chapter~18 of the
+\TEX book). The primitive names are a simple matter of combining two math atom
+types, but for completeness' sake, here is the whole list:
+
+\starttwocolumns
+\startlines
+\lpr {Umathordordspacing}
+\lpr {Umathordopspacing}
+\lpr {Umathordbinspacing}
+\lpr {Umathordrelspacing}
+\lpr {Umathordopenspacing}
+\lpr {Umathordclosespacing}
+\lpr {Umathordpunctspacing}
+\lpr {Umathordinnerspacing}
+\lpr {Umathopordspacing}
+\lpr {Umathopopspacing}
+\lpr {Umathopbinspacing}
+\lpr {Umathoprelspacing}
+\lpr {Umathopopenspacing}
+\lpr {Umathopclosespacing}
+\lpr {Umathoppunctspacing}
+\lpr {Umathopinnerspacing}
+\lpr {Umathbinordspacing}
+\lpr {Umathbinopspacing}
+\lpr {Umathbinbinspacing}
+\lpr {Umathbinrelspacing}
+\lpr {Umathbinopenspacing}
+\lpr {Umathbinclosespacing}
+\lpr {Umathbinpunctspacing}
+\lpr {Umathbininnerspacing}
+\lpr {Umathrelordspacing}
+\lpr {Umathrelopspacing}
+\lpr {Umathrelbinspacing}
+\lpr {Umathrelrelspacing}
+\lpr {Umathrelopenspacing}
+\lpr {Umathrelclosespacing}
+\lpr {Umathrelpunctspacing}
+\lpr {Umathrelinnerspacing}
+\lpr {Umathopenordspacing}
+\lpr {Umathopenopspacing}
+\lpr {Umathopenbinspacing}
+\lpr {Umathopenrelspacing}
+\lpr {Umathopenopenspacing}
+\lpr {Umathopenclosespacing}
+\lpr {Umathopenpunctspacing}
+\lpr {Umathopeninnerspacing}
+\lpr {Umathcloseordspacing}
+\lpr {Umathcloseopspacing}
+\lpr {Umathclosebinspacing}
+\lpr {Umathcloserelspacing}
+\lpr {Umathcloseopenspacing}
+\lpr {Umathcloseclosespacing}
+\lpr {Umathclosepunctspacing}
+\lpr {Umathcloseinnerspacing}
+\lpr {Umathpunctordspacing}
+\lpr {Umathpunctopspacing}
+\lpr {Umathpunctbinspacing}
+\lpr {Umathpunctrelspacing}
+\lpr {Umathpunctopenspacing}
+\lpr {Umathpunctclosespacing}
+\lpr {Umathpunctpunctspacing}
+\lpr {Umathpunctinnerspacing}
+\lpr {Umathinnerordspacing}
+\lpr {Umathinneropspacing}
+\lpr {Umathinnerbinspacing}
+\lpr {Umathinnerrelspacing}
+\lpr {Umathinneropenspacing}
+\lpr {Umathinnerclosespacing}
+\lpr {Umathinnerpunctspacing}
+\lpr {Umathinnerinnerspacing}
+\stoplines
+\stoptwocolumns
+
+These parameters are of type \prm {muskip}, so setting a parameter can be done
+like this:
+
+\starttyping
+\Umathopordspacing\displaystyle=4mu plus 2mu
+\stoptyping
+
+They are all initialized by \type {initex} to the values mentioned in the table
+in Chapter~18 of the \TEX book.
+
+Note 1: for ease of use as well as for backward compatibility, \prm {thinmuskip},
+\prm {medmuskip} and \prm {thickmuskip} are treated specially. In their case a
+pointer to the corresponding internal parameter is saved, not the actual \prm
+{muskip} value. This means that any later changes to one of these three
+parameters will be taken into account.
+
+Note 2: Careful readers will realise that there are also primitives for the items
+marked \type {*} in the \TEX book. These will not actually be used as those
+combinations of atoms cannot actually happen, but it seemed better not to break
+orthogonality. They are initialized to zero.
+
+\subsection{Local settings}
+
+Math is processed in two passes. The first pass is needed to intercept for
+instance \type {\over}, one of the few \TEX\ commands that actually has a
+preceding argument. There are often lots of curly braces used in math and these
+can result in a nested run of the math sub engine. However, you need to be aware
+of the fact that some properties are kind of global to a formula and the last
+setting (for instance a family switch) wins. This also means that a change (or
+again, the last one) in math parameters affects the whole formula. In
+\LUAMETATEX\ we have changed this model a bit. One can argue that this introduces
+an incompatibility but it's hard to imagine a reason for setting the parameters
+at the end of a formula run and assume that they also influence what goes in
+front.
+
+\startbuffer
+$
+ x \Usubscript {-}
+ \frozen\Umathsubshiftdown\textstyle 0pt x \Usubscript {0}
+ {\frozen\Umathsubshiftdown\textstyle 5pt x \Usubscript {5}}
+ x \Usubscript {0}
+ {\frozen\Umathsubshiftdown\textstyle 15pt x \Usubscript {15}}
+ x \Usubscript {0}
+ {\frozen\Umathsubshiftdown\textstyle 20pt x \Usubscript {20}}
+ x \Usubscript {0}
+ \frozen\Umathsubshiftdown\textstyle 10pt x \Usubscript {10}
+ x \Usubscript {0}
+$
+\stopbuffer
+
+\typebuffer
+
+The \type {\frozen} prefix does the magic: it injects information in the
+math list about the set parameter.
+
+In \LUATEX\ 1.10 the last setting, the \type {10pt} drop wins, but in
+\LUAMETATEX\ you will see each local setting taking effect. The implementation
+uses a new node type, parameters nodes, so you you might encounter these in an
+unprocessed math list. The result looks as follows:
+
+\blank \getbuffer \blank
+
+
+
+\subsection{Skips around display math and \lpr {mathdisplayskipmode}}
+
+\topicindex {math+spacing}
+
+The injection of \prm {abovedisplayskip} and \prm {belowdisplayskip} is not
+symmetrical. An above one is always inserted, also when zero, but the below is
+only inserted when larger than zero. Especially the latter makes it sometimes hard
+to fully control spacing. Therefore \LUATEX\ comes with a new directive: \lpr
+{mathdisplayskipmode}. The following values apply:
+
+\starttabulate[|c|l|]
+\DB value \BC meaning \NC \NR
+\TB
+\NC 0 \NC normal \TEX\ behaviour \NC \NR
+\NC 1 \NC always (same as 0) \NC \NR
+\NC 2 \NC only when not zero \NC \NR
+\NC 3 \NC never, not even when not zero \NC \NR
+\LL
+\stoptabulate
+
+\subsection {Nolimit correction with \lpr {mathnolimitsmode}}
+
+\topicindex {math+limits}
+
+There are two extra math parameters \lpr {Umathnolimitsupfactor} and \lpr
+{Umathnolimitsubfactor} that were added to provide some control over how limits
+are spaced (for example the position of super and subscripts after integral
+operators). They relate to an extra parameter \lpr {mathnolimitsmode}. The half
+corrections are what happens when scripts are placed above and below. The
+problem with italic corrections is that officially that correction italic is used
+for above|/|below placement while advanced kerns are used for placement at the
+right end. The question is: how often is this implemented, and if so, do the
+kerns assume correction too. Anyway, with this parameter one can control it.
+
+\starttabulate[|l|ck1|ck1|ck1|ck1|ck1|ck1|]
+ \NC
+ \NC \mathnolimitsmode0 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode1 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode2 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode3 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode4 $\displaystyle\int\nolimits^0_1$
+ \NC \mathnolimitsmode8000 $\displaystyle\int\nolimits^0_1$
+ \NC \NR
+ \TB
+ \BC mode
+ \NC \tttf 0
+ \NC \tttf 1
+ \NC \tttf 2
+ \NC \tttf 3
+ \NC \tttf 4
+ \NC \tttf 8000
+ \NC \NR
+ \BC superscript
+ \NC 0
+ \NC font
+ \NC 0
+ \NC 0
+ \NC +ic/2
+ \NC 0
+ \NC \NR
+ \BC subscript
+ \NC -ic
+ \NC font
+ \NC 0
+ \NC -ic/2
+ \NC -ic/2
+ \NC 8000ic/1000
+ \NC \NR
+\stoptabulate
+
+When the mode is set to one, the math parameters are used. This way a macro
+package writer can decide what looks best. Given the current state of fonts in
+\CONTEXT\ we currently use mode 1 with factor 0 for the superscript and 750 for
+the subscripts. Positive values are used for both parameters but the subscript
+shifts to the left. A \lpr {mathnolimitsmode} larger that 15 is considered to
+be a factor for the subscript correction. This feature can be handy when
+experimenting.
+
+\subsection {Controlling math italic mess with \lpr {mathitalicsmode}}
+
+\topicindex {math+italics}
+
+The \lpr {mathitalicsmode} parameter can be set to~1 to force italic correction
+before noads that represent some more complex structure (read: everything
+that is not an ord, bin, rel, open, close, punct or inner). We show a Cambria
+example.
+
+\starttexdefinition Whatever #1
+ \NC \type{\mathitalicsmode = #1}
+ \NC \mathitalicsmode#1\ruledhbox{$\left|T^1\right|$}
+ \NC \mathitalicsmode#1\ruledhbox{$\left|T\right|$}
+ \NC \mathitalicsmode#1\ruledhbox{$T+1$}
+ \NC \mathitalicsmode#1\ruledhbox{$T{1\over2}$}
+ \NC \mathitalicsmode#1\ruledhbox{$T\sqrt{1}$}
+ \NC \NR
+\stoptexdefinition
+
+\start
+ \switchtobodyfont[cambria]
+ \starttabulate[|c|c|c|c|c|c|]
+ \Whatever{0}%
+ \Whatever{1}%
+ \stoptabulate
+\stop
+
+This kind of parameters relate to the fact that italic correction in \OPENTYPE\
+math is bound to fuzzy rules. So, control is the solution.
+
+\subsection {Influencing script kerning with \lpr {mathscriptboxmode}}
+
+\topicindex {math+kerning}
+\topicindex {math+scripts}
+
+If you want to typeset text in math macro packages often provide something \type
+{\text} which obeys the script sizes. As the definition can be anything there is
+a good chance that the kerning doesn't come out well when used in a script. Given
+that the first glyph ends up in a \prm {hbox} we have some control over this.
+And, as a bonus we also added control over the normal sublist kerning. The \lpr
+{mathscriptboxmode} parameter defaults to~1.
+
+\starttabulate[|c|l|]
+\DB value \BC meaning \NC \NR
+\TB
+\NC \type {0} \NC forget about kerning \NC \NR
+\NC \type {1} \NC kern math sub lists with a valid glyph \NC \NR
+\NC \type {2} \NC also kern math sub boxes that have a valid glyph \NC \NR
+\NC \type {2} \NC only kern math sub boxes with a boundary node present\NC \NR
+\LL
+\stoptabulate
+
+Here we show some examples. Of course this doesn't solve all our problems, if
+only because some fonts have characters with bounding boxes that compensate for
+italics, while other fonts can lack kerns.
+
+\startbuffer[1]
+ $T_{\tf fluff}$
+\stopbuffer
+
+\startbuffer[2]
+ $T_{\text{fluff}}$
+\stopbuffer
+
+\startbuffer[3]
+ $T_{\text{\boundary1 fluff}}$
+\stopbuffer
+
+\unexpanded\def\Show#1#2#3%
+ {\doifelsenothing{#3}
+ {\small\tx\typeinlinebuffer[#1]}
+ {\doifelse{#3}{-}
+ {\small\bf\tt mode #2}
+ {\switchtobodyfont[#3]\showfontkerns\showglyphs\mathscriptboxmode#2\relax\inlinebuffer[#1]}}}
+
+\starttabulate[|lBT|c|c|c|c|c|]
+ \NC \NC \Show{1}{0}{} \NC\Show{1}{1}{} \NC \Show{2}{1}{} \NC \Show{2}{2}{} \NC \Show{3}{3}{} \NC \NR
+ \NC \NC \Show{1}{0}{-} \NC\Show{1}{1}{-} \NC \Show{2}{1}{-} \NC \Show{2}{2}{-} \NC \Show{3}{3}{-} \NC \NR
+ \NC modern \NC \Show{1}{0}{modern} \NC\Show{1}{1}{modern} \NC \Show{2}{1}{modern} \NC \Show{2}{2}{modern} \NC \Show{3}{3}{modern} \NC \NR
+ \NC lucidaot \NC \Show{1}{0}{lucidaot} \NC\Show{1}{1}{lucidaot} \NC \Show{2}{1}{lucidaot} \NC \Show{2}{2}{lucidaot} \NC \Show{3}{3}{lucidaot} \NC \NR
+ \NC pagella \NC \Show{1}{0}{pagella} \NC\Show{1}{1}{pagella} \NC \Show{2}{1}{pagella} \NC \Show{2}{2}{pagella} \NC \Show{3}{3}{pagella} \NC \NR
+ \NC cambria \NC \Show{1}{0}{cambria} \NC\Show{1}{1}{cambria} \NC \Show{2}{1}{cambria} \NC \Show{2}{2}{cambria} \NC \Show{3}{3}{cambria} \NC \NR
+ \NC dejavu \NC \Show{1}{0}{dejavu} \NC\Show{1}{1}{dejavu} \NC \Show{2}{1}{dejavu} \NC \Show{2}{2}{dejavu} \NC \Show{3}{3}{dejavu} \NC \NR
+\stoptabulate
+
+Kerning between a character subscript is controlled by \lpr {mathscriptcharmode}
+which also defaults to~1.
+
+Here is another example. Internally we tag kerns as italic kerns or font kerns
+where font kerns result from the staircase kern tables. In 2018 fonts like Latin
+Modern and Pagella rely on cheats with the boundingbox, Cambria uses staircase
+kerns and Lucida a mixture. Depending on how fonts evolve we might add some more
+control over what one can turn on and off.
+
+\def\MathSample#1#2#3%
+ {\NC
+ #1 \NC
+ #2 \NC
+ \showglyphdata \switchtobodyfont[#2,17.3pt]$#3T_{f}$ \NC
+ \showglyphdata \switchtobodyfont[#2,17.3pt]$#3\gamma_{e}$ \NC
+ \showglyphdata \switchtobodyfont[#2,17.3pt]$#3\gamma_{ee}$ \NC
+ \showglyphdata \switchtobodyfont[#2,17.3pt]$#3T_{\tf fluff}$ \NC
+ \NR}
+
+\starttabulate[|Tl|Tl|l|l|l|l|]
+ \FL
+ \MathSample{normal}{modern} {\mr}
+ \MathSample{} {pagella} {\mr}
+ \MathSample{} {cambria} {\mr}
+ \MathSample{} {lucidaot}{\mr}
+ \ML
+ \MathSample{bold} {modern} {\mb}
+ \MathSample{} {pagella} {\mb}
+ \MathSample{} {cambria} {\mb}
+ \MathSample{} {lucidaot}{\mb}
+ \LL
+\stoptabulate
+
+\subsection{Forcing fixed scripts with \lpr {mathscriptsmode}}
+
+We have three parameters that are used for this fixed anchoring:
+
+\starttabulate[|c|l|]
+\DB parameter \BC register \NC \NR
+\NC $d$ \NC \lpr {Umathsubshiftdown} \NC \NR
+\NC $u$ \NC \lpr {Umathsupshiftup} \NC \NR
+\NC $s$ \NC \lpr {Umathsubsupshiftdown} \NC \NR
+\LL
+\stoptabulate
+
+When we set \lpr {mathscriptsmode} to a value other than zero these are used
+for calculating fixed positions. This is something that is needed for instance
+for chemistry. You can manipulate the mentioned variables to achieve different
+effects.
+
+\def\SampleMath#1%
+ {$\mathscriptsmode#1\mathupright CH_2 + CH^+_2 + CH^2_2$}
+
+\starttabulate[|c|c|c|p|]
+\DB mode \BC down \BC up \BC example \NC \NR
+\TB
+\NC 0 \NC dynamic \NC dynamic \NC \SampleMath{0} \NC \NR
+\NC 1 \NC $d$ \NC $u$ \NC \SampleMath{1} \NC \NR
+\NC 2 \NC $s$ \NC $u$ \NC \SampleMath{2} \NC \NR
+\NC 3 \NC $s$ \NC $u + s - d$ \NC \SampleMath{3} \NC \NR
+\NC 4 \NC $d + (s-d)/2$ \NC $u + (s-d)/2$ \NC \SampleMath{4} \NC \NR
+\NC 5 \NC $d$ \NC $u + s - d$ \NC \SampleMath{5} \NC \NR
+\LL
+\stoptabulate
+
+The value of this parameter obeys grouping but applies to the whole current
+formula.
+
+% if needed we can put the value in stylenodes but maybe more should go there
+
+\subsection{Penalties: \lpr {mathpenaltiesmode}}
+
+\topicindex {math+penalties}
+
+Only in inline math penalties will be added in a math list. You can force
+penalties (also in display math) by setting:
+
+\starttyping
+\mathpenaltiesmode = 1
+\stoptyping
+
+This primnitive is not really needed in \LUATEX\ because you can use the callback
+\cbk {mlist_to_hlist} to force penalties by just calling the regular routine
+with forced penalties. However, as part of opening up and control this primitive
+makes sense. As a bonus we also provide two extra penalties:
+
+\starttyping
+\prebinoppenalty = -100 % example value
+\prerelpenalty = 900 % example value
+\stoptyping
+
+They default to inifinite which signals that they don't need to be inserted. When
+set they are injected before a binop or rel noad. This is an experimental feature.
+
+\subsection{Equation spacing: \lpr {matheqnogapstep}}
+
+By default \TEX\ will add one quad between the equation and the number. This is
+hard coded. A new primitive can control this:
+
+\startsyntax
+\matheqnogapstep = 1000
+\stopsyntax
+
+Because a math quad from the math text font is used instead of a dimension, we
+use a step to control the size. A value of zero will suppress the gap. The step
+is divided by 1000 which is the usual way to mimmick floating point factors in
+\TEX.
+
+\stopsection
+
+\startsection[title={Math constructs}]
+
+\subsection {Unscaled fences and \lpr{mathdelimitersmode}}
+
+\topicindex {math+fences}
+
+The \lpr {mathdelimitersmode} primitive is experimental and deals with the
+following (potential) problems. Three bits can be set. The first bit prevents an
+unwanted shift when the fence symbol is not scaled (a cambria side effect). The
+second bit forces italic correction between a preceding character ordinal and the
+fenced subformula, while the third bit turns that subformula into an ordinary so
+that the same spacing applies as with unfenced variants. Here we show Cambria
+(with \lpr {mathitalicsmode} enabled).
+
+\starttexdefinition Whatever #1
+ \NC \type{\mathdelimitersmode = #1}
+ \NC \mathitalicsmode1\mathdelimitersmode#1\ruledhbox{\showglyphs\showfontkerns\showfontitalics$f(x)$}
+ \NC \mathitalicsmode1\mathdelimitersmode#1\ruledhbox{\showglyphs\showfontkerns\showfontitalics$f\left(x\right)$}
+ \NC \NR
+\stoptexdefinition
+
+\start
+ \switchtobodyfont[cambria]
+ \starttabulate[|l|l|l|]
+ \Whatever{0}\Whatever{1}\Whatever{2}\Whatever{3}%
+ \Whatever{4}\Whatever{5}\Whatever{6}\Whatever{7}%
+ \stoptabulate
+\stop
+
+So, when set to 7 fenced subformulas with unscaled delimiters come out the same
+as unfenced ones. This can be handy for cases where one is forced to use \prm
+{left} and \prm {right} always because of unpredictable content. As said, it's an
+experimental feature (which somehow fits in the exceptional way fences are dealt
+with in the engine). The full list of flags is given in the next table:
+
+\starttabulate[|c|l|]
+\DB value \BC meaning \NC \NR
+\TB
+\NC \type{"01} \NC don't apply the usual shift \NC \NR
+\NC \type{"02} \NC apply italic correction when possible \NC \NR
+\NC \type{"04} \NC force an ordinary subformula \NC \NR
+\NC \type{"08} \NC no shift when a base character \NC \NR
+\NC \type{"10} \NC only shift when an extensible \NC \NR
+\LL
+\stoptabulate
+
+The effect can depend on the font (and for Cambria one can use for instance \type {"16}).
+
+\subsection[mathacc]{Accent handling with \lpr {Umathaccent}}
+
+\topicindex {math+accents}
+
+\LUATEX\ supports both top accents and bottom accents in math mode, and math
+accents stretch automatically (if this is supported by the font the accent comes
+from, of course). Bottom and combined accents as well as fixed-width math accents
+are controlled by optional keywords following \lpr {Umathaccent}.
+
+The keyword \type {bottom} after \lpr {Umathaccent} signals that a bottom accent
+is needed, and the keyword \type {both} signals that both a top and a bottom
+accent are needed (in this case two accents need to be specified, of course).
+
+Then the set of three integers defining the accent is read. This set of integers
+can be prefixed by the \type {fixed} keyword to indicate that a non-stretching
+variant is requested (in case of both accents, this step is repeated).
+
+A simple example:
+
+\starttyping
+\Umathaccent both fixed 0 0 "20D7 fixed 0 0 "20D7 {example}
+\stoptyping
+
+If a math top accent has to be placed and the accentee is a character and has a
+non-zero \type {top_accent} value, then this value will be used to place the
+accent instead of the \prm {skewchar} kern used by \TEX82.
+
+The \type {top_accent} value represents a vertical line somewhere in the
+accentee. The accent will be shifted horizontally such that its own \type
+{top_accent} line coincides with the one from the accentee. If the \type
+{top_accent} value of the accent is zero, then half the width of the accent
+followed by its italic correction is used instead.
+
+The vertical placement of a top accent depends on the \type {x_height} of the
+font of the accentee (as explained in the \TEX book), but if a value turns out
+to be zero and the font had a \type {MathConstants} table, then \type
+{AccentBaseHeight} is used instead.
+
+The vertical placement of a bottom accent is straight below the accentee, no
+correction takes place.
+
+Possible locations are \type {top}, \type {bottom}, \type {both} and \type
+{center}. When no location is given \type {top} is assumed. An additional
+parameter \nod {fraction} can be specified followed by a number; a value of for
+instance 1200 means that the criterium is 1.2 times the width of the nucleus. The
+fraction only applies to the stepwise selected shapes and is mostly meant for the
+\type {overlay} location. It also works for the other locations but then it
+concerns the width.
+
+\subsection{Building radicals with \lpr {Uradical} and \lpr {Uroot}}
+
+\topicindex {math+radicals}
+
+The new primitive \lpr {Uroot} allows the construction of a radical noad
+including a degree field. Its syntax is an extension of \lpr {Uradical}:
+
+\starttyping
+\Uradical <fam integer> <char integer> <radicand>
+\Uroot <fam integer> <char integer> <degree> <radicand>
+\stoptyping
+
+The placement of the degree is controlled by the math parameters \lpr
+{Umathradicaldegreebefore}, \lpr {Umathradicaldegreeafter}, and \lpr
+{Umathradicaldegreeraise}. The degree will be typeset in \prm
+{scriptscriptstyle}.
+
+\subsection{Super- and subscripts}
+
+The character fields in a \LUA|-|loaded \OPENTYPE\ math font can have a \quote
+{mathkern} table. The format of this table is the same as the \quote {mathkern}
+table that is returned by the \type {fontloader} library, except that all height
+and kern values have to be specified in actual scaled points.
+
+When a super- or subscript has to be placed next to a math item, \LUATEX\ checks
+whether the super- or subscript and the nucleus are both simple character items.
+If they are, and if the fonts of both character items are \OPENTYPE\ fonts (as
+opposed to legacy \TEX\ fonts), then \LUATEX\ will use the \OPENTYPE\ math
+algorithm for deciding on the horizontal placement of the super- or subscript.
+
+This works as follows:
+
+\startitemize
+ \startitem
+ The vertical position of the script is calculated.
+ \stopitem
+ \startitem
+ The default horizontal position is flat next to the base character.
+ \stopitem
+ \startitem
+ For superscripts, the italic correction of the base character is added.
+ \stopitem
+ \startitem
+ For a superscript, two vertical values are calculated: the bottom of the
+ script (after shifting up), and the top of the base. For a subscript, the two
+ values are the top of the (shifted down) script, and the bottom of the base.
+ \stopitem
+ \startitem
+ For each of these two locations:
+ \startitemize
+ \startitem
+ find the math kern value at this height for the base (for a subscript
+ placement, this is the bottom_right corner, for a superscript
+ placement the top_right corner)
+ \stopitem
+ \startitem
+ find the math kern value at this height for the script (for a
+ subscript placement, this is the top_left corner, for a superscript
+ placement the bottom_left corner)
+ \stopitem
+ \startitem
+ add the found values together to get a preliminary result.
+ \stopitem
+ \stopitemize
+ \stopitem
+ \startitem
+ The horizontal kern to be applied is the smallest of the two results from
+ previous step.
+ \stopitem
+\stopitemize
+
+The math kern value at a specific height is the kern value that is specified by the
+next higher height and kern pair, or the highest one in the character (if there is no
+value high enough in the character), or simply zero (if the character has no math kern
+pairs at all).
+
+\subsection{Scripts on extensibles: \lpr {Uunderdelimiter}, \lpr {Uoverdelimiter},
+\lpr {Udelimiterover}, \lpr {Udelimiterunder} and \lpr {Uhextensible}}
+
+\topicindex {math+scripts}
+\topicindex {math+delimiters}
+\topicindex {math+extensibles}
+
+The primitives \lpr {Uunderdelimiter} and \lpr {Uoverdelimiter} allow the
+placement of a subscript or superscript on an automatically extensible item and
+\lpr {Udelimiterunder} and \lpr {Udelimiterover} allow the placement of an
+automatically extensible item as a subscript or superscript on a nucleus. The
+input:
+
+% these produce radical noads .. in fact the code base has the numbers wrong for
+% quite a while, so no one seems to use this
+
+\startbuffer
+$\Uoverdelimiter 0 "2194 {\hbox{\strut overdelimiter}}$
+$\Uunderdelimiter 0 "2194 {\hbox{\strut underdelimiter}}$
+$\Udelimiterover 0 "2194 {\hbox{\strut delimiterover}}$
+$\Udelimiterunder 0 "2194 {\hbox{\strut delimiterunder}}$
+\stopbuffer
+
+\typebuffer will render this:
+
+\blank \startnarrower \getbuffer \stopnarrower \blank
+
+The vertical placements are controlled by \lpr {Umathunderdelimiterbgap}, \lpr
+{Umathunderdelimitervgap}, \lpr {Umathoverdelimiterbgap}, and \lpr
+{Umathoverdelimitervgap} in a similar way as limit placements on large operators.
+The superscript in \lpr {Uoverdelimiter} is typeset in a suitable scripted style,
+the subscript in \lpr {Uunderdelimiter} is cramped as well.
+
+These primitives accepts an option \type {width} specification. When used the
+also optional keywords \type {left}, \type {middle} and \type {right} will
+determine what happens when a requested size can't be met (which can happen when
+we step to successive larger variants).
+
+An extra primitive \lpr {Uhextensible} is available that can be used like this:
+
+\startbuffer
+$\Uhextensible width 10cm 0 "2194$
+\stopbuffer
+
+\typebuffer This will render this:
+
+\blank \startnarrower \getbuffer \stopnarrower \blank
+
+Here you can also pass options, like:
+
+\startbuffer
+$\Uhextensible width 1pt middle 0 "2194$
+\stopbuffer
+
+\typebuffer This gives:
+
+\blank \startnarrower \getbuffer \stopnarrower \blank
+
+\LUATEX\ internally uses a structure that supports \OPENTYPE\ \quote
+{MathVariants} as well as \TFM\ \quote {extensible recipes}. In most cases where
+font metrics are involved we have a different code path for traditional fonts end
+\OPENTYPE\ fonts.
+
+\subsection{Fractions and the new \lpr {Uskewed} and \lpr {Uskewedwithdelims}}
+
+\topicindex {math+fractions}
+
+The \prm {abovewithdelims} command accepts a keyword \type {exact}. When issued
+the extra space relative to the rule thickness is not added. One can of course
+use the \type {\Umathfraction..gap} commands to influence the spacing. Also the
+rule is still positioned around the math axis.
+
+\starttyping
+$$ { {a} \abovewithdelims() exact 4pt {b} }$$
+\stoptyping
+
+The math parameter table contains some parameters that specify a horizontal and
+vertical gap for skewed fractions. Of course some guessing is needed in order to
+implement something that uses them. And so we now provide a primitive similar to the
+other fraction related ones but with a few options so that one can influence the
+rendering. Of course a user can also mess around a bit with the parameters
+\lpr {Umathskewedfractionhgap} and \lpr {Umathskewedfractionvgap}.
+
+The syntax used here is:
+
+\starttyping
+{ {1} \Uskewed / <options> {2} }
+{ {1} \Uskewedwithdelims / () <options> {2} }
+\stoptyping
+
+where the options can be \type {noaxis} and \type {exact}. By default we add half
+the axis to the shifts and by default we zero the width of the middle character.
+For Latin Modern the result looks as follows:
+
+\def\ShowA#1#2#3{$x + { {#1} \Uskewed / #3 {#2} } + x$}
+\def\ShowB#1#2#3{$x + { {#1} \Uskewedwithdelims / () #3 {#2} } + x$}
+
+\start
+ \switchtobodyfont[modern]
+ \starttabulate[||||||]
+ \NC \NC
+ \ShowA{a}{b}{} \NC
+ \ShowA{1}{2}{} \NC
+ \ShowB{a}{b}{} \NC
+ \ShowB{1}{2}{} \NC
+ \NR
+ \NC \type{exact} \NC
+ \ShowA{a}{b}{exact} \NC
+ \ShowA{1}{2}{exact} \NC
+ \ShowB{a}{b}{exact} \NC
+ \ShowB{1}{2}{exact} \NC
+ \NR
+ \NC \type{noaxis} \NC
+ \ShowA{a}{b}{noaxis} \NC
+ \ShowA{1}{2}{noaxis} \NC
+ \ShowB{a}{b}{noaxis} \NC
+ \ShowB{1}{2}{noaxis} \NC
+ \NR
+ \NC \type{exact noaxis} \NC
+ \ShowA{a}{b}{exact noaxis} \NC
+ \ShowA{1}{2}{exact noaxis} \NC
+ \ShowB{a}{b}{exact noaxis} \NC
+ \ShowB{1}{2}{exact noaxis} \NC
+ \NR
+ \stoptabulate
+\stop
+
+\subsection {Delimiters: \type{\Uleft}, \prm {Umiddle} and \prm {Uright}}
+
+\topicindex {math+delimiters}
+
+Normally you will force delimiters to certain sizes by putting an empty box or
+rule next to it. The resulting delimiter will either be a character from the
+stepwise size range or an extensible. The latter can be quite differently
+positioned than the characters as it depends on the fit as well as the fact if
+the used characters in the font have depth or height. Commands like (plain \TEX
+s) \type {\big} need use this feature. In \LUATEX\ we provide a bit more control
+by three variants that support optional parameters \type {height}, \type {depth}
+and \type {axis}. The following example uses this:
+
+\startbuffer
+\Uleft height 30pt depth 10pt \Udelimiter "0 "0 "000028
+\quad x\quad
+\Umiddle height 40pt depth 15pt \Udelimiter "0 "0 "002016
+\quad x\quad
+\Uright height 30pt depth 10pt \Udelimiter "0 "0 "000029
+\quad \quad \quad
+\Uleft height 30pt depth 10pt axis \Udelimiter "0 "0 "000028
+\quad x\quad
+\Umiddle height 40pt depth 15pt axis \Udelimiter "0 "0 "002016
+\quad x\quad
+\Uright height 30pt depth 10pt axis \Udelimiter "0 "0 "000029
+\stopbuffer
+
+\typebuffer
+
+\startlinecorrection
+\ruledhbox{\mathematics{\getbuffer}}
+\stoplinecorrection
+
+The keyword \type {exact} can be used as directive that the real dimensions
+should be applied when the criteria can't be met which can happen when we're
+still stepping through the successively larger variants. When no dimensions are
+given the \type {noaxis} command can be used to prevent shifting over the axis.
+
+You can influence the final class with the keyword \type {class} which will
+influence the spacing. The numbers are the same as for character classes.
+
+\subsection {Accents: \type{\mathlimitsmode}}
+
+\topicindex {math+accents}
+
+When you use \type {\limits} or \type {\nolimits} without scripts spacing might
+get messed up. This can be prevented by setting \type {\mathlimitsmode} to a
+non|-|zero value.
+
+\stopsection
+
+\startsection[title={Extracting values}]
+
+\subsection{Codes and using \lpr {Umathcode}, \lpr {Umathcharclass}, \lpr
+{Umathcharfam} and \lpr {Umathcharslot}}
+
+\topicindex {math+codes}
+
+You can extract the components of a math character. Say that we have defined:
+
+\starttyping
+\Umathcode 1 2 3 4
+\stoptyping
+
+then
+
+\starttyping
+[\Umathcharclass1] [\Umathcharfam1] [\Umathcharslot1]
+\stoptyping
+
+will return:
+
+\starttyping
+[2] [3] [4]
+\stoptyping
+
+These commands are provides as convenience. Before they come available you could
+do the following:
+
+\starttyping
+\def\Umathcharclass{\directlua{tex.print(tex.getmathcode(token.scan_int())[1])}}
+\def\Umathcharfam {\directlua{tex.print(tex.getmathcode(token.scan_int())[2])}}
+\def\Umathcharslot {\directlua{tex.print(tex.getmathcode(token.scan_int())[3])}}
+\stoptyping
+
+\subsection {Last lines and \lpr{predisplaygapfactor}}
+
+\topicindex {math+last line}
+
+There is a new primitive to control the overshoot in the calculation of the
+previous line in mid|-|paragraph display math. The default value is 2 times
+the em width of the current font:
+
+\starttyping
+\predisplaygapfactor=2000
+\stoptyping
+
+If you want to have the length of the last line independent of math i.e.\ you don't
+want to revert to a hack where you insert a fake display math formula in order to
+get the length of the last line, the following will often work too:
+
+\starttyping
+\def\lastlinelength{\dimexpr
+ \directlua {tex.sprint (
+ (nodes.dimensions(node.tail(tex.lists.page_head).list))
+ )}sp
+\relax}
+\stoptyping
+
+\stopsection
+
+\startsection[title={Math mode}]
+
+\subsection {Verbose versions of single|-|character math commands like \lpr {Usuperscript}
+and \lpr {Usubscript}}
+
+\topicindex {math+styles}
+
+\LUATEX\ defines six new primitives that have the same function as
+\type {^}, \type {_}, \type {$}, and \type {$$}:
+
+\starttabulate[|l|l|]
+\DB primitive \BC explanation \NC \NR
+\TB
+\NC \lpr {Usuperscript} \NC duplicates the functionality of \type {^} \NC \NR
+\NC \lpr {Usubscript} \NC duplicates the functionality of \type {_} \NC \NR
+\NC \lpr {Ustartmath} \NC duplicates the functionality of \type {$}, % $
+ when used in non-math mode. \NC \NR
+\NC \lpr {Ustopmath} \NC duplicates the functionality of \type {$}, % $
+ when used in inline math mode. \NC \NR
+\NC \lpr {Ustartdisplaymath} \NC duplicates the functionality of \type {$$}, % $$
+ when used in non-math mode. \NC \NR
+\NC \lpr {Ustopdisplaymath} \NC duplicates the functionality of \type {$$}, % $$
+ when used in display math mode. \NC \NR
+\LL
+\stoptabulate
+
+The \lpr {Ustopmath} and \lpr {Ustopdisplaymath} primitives check if the current
+math mode is the correct one (inline vs.\ displayed), but you can freely intermix
+the four mathon|/|mathoff commands with explicit dollar sign(s).
+
+\subsection{Script commands \lpr {Unosuperscript} and \lpr {Unosubscript}}
+
+\topicindex {math+styles}
+\topicindex {math+scripts}
+
+These two commands result in super- and subscripts but with the current style (at the
+time of rendering). So,
+
+\startbuffer[script]
+$
+ x\Usuperscript {1}\Usubscript {2} =
+ x\Unosuperscript{1}\Unosubscript{2} =
+ x\Usuperscript {1}\Unosubscript{2} =
+ x\Unosuperscript{1}\Usubscript {2}
+$
+\stopbuffer
+
+\typebuffer
+
+results in \inlinebuffer[script].
+
+\subsection{Allowed math commands in non|-|math modes}
+
+\topicindex {math+text}
+\topicindex {text+math}
+
+The commands \prm {mathchar}, and \lpr {Umathchar} and control sequences that are
+the result of \prm {mathchardef} or \lpr {Umathchardef} are also acceptable in
+the horizontal and vertical modes. In those cases, the \prm {textfont} from the
+requested math family is used.
+
+\stopsection
+
+\startsection[title={Goodies}]
+
+\subsection {Flattening: \lpr {mathflattenmode}}
+
+\topicindex {math+flattening}
+
+The \TEX\ math engine collapses \type {ord} noads without sub- and superscripts
+and a character as nucleus. and which has the side effect that in \OPENTYPE\ mode
+italic corrections are applied (given that they are enabled).
+
+\startbuffer[sample]
+\switchtobodyfont[modern]
+$V \mathbin{\mathbin{v}} V$\par
+$V \mathord{\mathord{v}} V$\par
+\stopbuffer
+
+\typebuffer[sample]
+
+This renders as:
+
+\blank \start \mathflattenmode\plusone \getbuffer[sample] \stop \blank
+
+When we set \lpr {mathflattenmode} to 31 we get:
+
+\blank \start \mathflattenmode\numexpr1+2+4+8+16\relax \getbuffer[sample] \stop \blank
+
+When you see no difference, then the font probably has the proper character
+dimensions and no italic correction is needed. For Latin Modern (at least till
+2018) there was a visual difference. In that respect this parameter is not always
+needed unless of course you want efficient math lists anyway.
+
+You can influence flattening by adding the appropriate number to the value of the
+mode parameter. The default value is~1.
+
+\starttabulate[|Tc|c|]
+\DB mode \BC class \NC \NR
+\TB
+\NC 1 \NC ord \NC \NR
+\NC 2 \NC bin \NC \NR
+\NC 4 \NC rel \NC \NR
+\NC 8 \NC punct \NC \NR
+\NC 16 \NC inner \NC \NR
+\LL
+\stoptabulate
+
+\subsection {Less Tracing}
+
+\topicindex {math+tracing}
+
+Because there are quite some math related parameters and values, it is possible
+to limit tracing. Only when \type {tracingassigns} and|/|or \type
+{tracingrestores} are set to~2 or more they will be traced.
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-metapost.tex b/doc/context/sources/general/manuals/luametatex/luametatex-metapost.tex
new file mode 100644
index 000000000..3f0d0a945
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-metapost.tex
@@ -0,0 +1,479 @@
+% language=uk
+
+% lua.newtable
+
+\environment luametatex-style
+
+\startcomponent luametatex-metapost
+
+\startchapter[reference=metapost,title={The \METAPOST\ library \type {mplib}}]
+
+\startsection[title={Process management}][library=mplib]
+
+\topicindex {\METAPOST}
+\topicindex {\METAPOST+mplib}
+\topicindex {images+mplib}
+\topicindex {images+\METAPOST}
+
+\libindex{version}
+
+The \METAPOST\ library interface registers itself in the table \type {mplib}. It
+is based on \MPLIB\ version \ctxlua {context(mplib.version())}.
+
+\subsection{\type {new}}
+
+\libindex{new}
+
+To create a new \METAPOST\ instance, call
+
+\startfunctioncall
+<mpinstance> mp = mplib.new({...})
+\stopfunctioncall
+
+This creates the \type {mp} instance object. The argument hash can have a number
+of different fields, as follows:
+
+\starttabulate[|l|l|pl|pl|]
+\DB name \BC type \BC description \BC default \NC \NR
+\TB
+\NC \type{error_line} \NC number \NC error line width \NC 79 \NC \NR
+\NC \type{print_line} \NC number \NC line length in ps output \NC 100 \NC \NR
+\NC \type{random_seed} \NC number \NC the initial random seed \NC variable \NC \NR
+\NC \type{math_mode} \NC string \NC the number system to use:
+ \type {scaled},
+ \type {double} or
+ % \type {binary} or
+ \type {decimal} \NC \type {scaled} \NC \NR
+\NC \type{interaction} \NC string \NC the interaction mode:
+ \type {batch},
+ \type {nonstop},
+ \type {scroll} or
+ \type {errorstop} \NC \type {errorstop} \NC \NR
+\NC \type{job_name} \NC string \NC a compatibility value \NC \type {mpout} \NC \NR
+\NC \type{find_file} \NC function \NC a function to find files \NC only local files \NC \NR
+\NC \type{utf8} \NC boolean \NC permit characters in the
+ range 128 upto 255 to be
+ part of names \NC \type {false} \NC \NR
+\LL
+\stoptabulate
+
+The binary mode is no longer available in the \LUATEX\ version of \MPLIB. It
+offers no real advantage and brings a ton of extra libraries with platform
+specific properties that we can now avoid. We might introduce a high resolution
+scaled variant at some point but only when it pays of performance wise.
+
+The \type {find_file} function should be of this form:
+
+\starttyping
+<string> found = finder (<string> name, <string> mode, <string> type)
+\stoptyping
+
+with:
+
+\starttabulate[|l|p|]
+\DB name \BC the requested file \NC \NR
+\TB
+\NC \type{mode} \NC the file mode: \type {r} or \type {w} \NC \NR
+\NC \type{type} \NC the kind of file, one of: \type {mp}, \type {tfm}, \type {map},
+ \type {pfb}, \type {enc} \NC \NR
+\LL
+\stoptabulate
+
+Return either the full path name of the found file, or \type {nil} if the file
+cannot be found.
+
+Note that the new version of \MPLIB\ no longer uses binary mem files, so the way
+to preload a set of macros is simply to start off with an \type {input} command
+in the first \type {execute} call.
+
+When you are processing a snippet of text starting with \type {btex} and
+ending with either \type {etex} or \type {verbatimtex}, the \METAPOST\
+\type {texscriptmode} parameter controls how spaces and newlines get honoured.
+The default value is~1. Possible values are:
+
+\starttabulate[|l|p|]
+\DB name \BC meaning \NC \NR
+\TB
+\NC \type {0} \NC no newlines \NC \NR
+\NC \type {1} \NC newlines in \type {verbatimtex} \NC \NR
+\NC \type {2} \NC newlines in \type {verbatimtex} and \type {etex} \NC \NR
+\NC \type {3} \NC no leading and trailing strip in \type {verbatimtex} \NC \NR
+\NC \type {4} \NC no leading and trailing strip in \type {verbatimtex} and \type {btex} \NC \NR
+\LL
+\stoptabulate
+
+That way the \LUA\ handler (assigned to \type {make_text}) can do what it likes.
+An \type {etex} has to be followed by a space or \type {;} or be at the end of a
+line and preceded by a space or at the beginning of a line.
+
+\subsection{\type {statistics}}
+
+\libindex{statistics}
+
+You can request statistics with:
+
+\startfunctioncall
+<table> stats = mp:statistics()
+\stopfunctioncall
+
+This function returns the vital statistics for an \MPLIB\ instance. There are
+four fields, giving the maximum number of used items in each of four allocated
+object classes:
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{main_memory} \NC number \NC memory size \NC \NR
+\NC \type{hash_size} \NC number \NC hash size\NC \NR
+\NC \type{param_size} \NC number \NC simultaneous macro parameters\NC \NR
+\NC \type{max_in_open} \NC number \NC input file nesting levels\NC \NR
+\LL
+\stoptabulate
+
+Note that in the new version of \MPLIB, this is informational only. The objects
+are all allocated dynamically, so there is no chance of running out of space
+unless the available system memory is exhausted.
+
+\subsection{\type {execute}}
+
+\libindex{execute}
+
+You can ask the \METAPOST\ interpreter to run a chunk of code by calling
+
+\startfunctioncall
+<table> rettable = execute(mp,"metapost code")
+\stopfunctioncall
+
+for various bits of \METAPOST\ language input. Be sure to check the \type
+{rettable.status} (see below) because when a fatal \METAPOST\ error occurs the
+\MPLIB\ instance will become unusable thereafter.
+
+Generally speaking, it is best to keep your chunks small, but beware that all
+chunks have to obey proper syntax, like each of them is a small file. For
+instance, you cannot split a single statement over multiple chunks.
+
+In contrast with the normal stand alone \type {mpost} command, there is
+\notabene {no} implied \quote{input} at the start of the first chunk.
+
+\subsection{\type {finish}}
+
+\libindex{finish}
+
+\startfunctioncall
+<table> rettable = finish(mp)
+\stopfunctioncall
+
+If for some reason you want to stop using an \MPLIB\ instance while processing is
+not yet actually done, you can call \type {finish}. Eventually, used memory
+will be freed and open files will be closed by the \LUA\ garbage collector, but
+an explicit \type {finish} is the only way to capture the final part of the
+output streams.
+
+\stopsection
+
+\startsection[title={The end result}]
+
+\libindex {fields}
+
+The return value of \type {execute} and \type {finish} is a table with a
+few possible keys (only \type {status} is always guaranteed to be present).
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{log} \NC string \NC output to the \quote {log} stream \NC \NR
+\NC \type{term} \NC string \NC output to the \quote {term} stream \NC \NR
+\NC \type{error} \NC string \NC output to the \quote {error} stream
+ (only used for \quote {out of memory}) \NC \NR
+\NC \type{status} \NC number \NC the return value:
+ \type {0} = good,
+ \type {1} = warning,
+ \type {2} = errors,
+ \type {3} = fatal error \NC \NR
+\NC \type{fig} \NC table \NC an array of generated figures (if any) \NC \NR
+\LL
+\stoptabulate
+
+When \type {status} equals~3, you should stop using this \MPLIB\ instance
+immediately, it is no longer capable of processing input.
+
+If it is present, each of the entries in the \type {fig} array is a userdata
+representing a figure object, and each of those has a number of object methods
+you can call:
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{boundingbox} \NC function \NC returns the bounding box, as an array of 4
+ values \NC \NR
+\NC \type{postscript} \NC function \NC returns a string that is the ps output of the
+ \type {fig}. this function accepts two optional
+ integer arguments for specifying the values of
+ \type {prologues} (first argument) and \type
+ {procset} (second argument) \NC \NR
+\NC \type{svg} \NC function \NC returns a string that is the svg output of the
+ \type {fig}. This function accepts an optional
+ integer argument for specifying the value of
+ \type {prologues} \NC \NR
+\NC \type{objects} \NC function \NC returns the actual array of graphic objects in
+ this \type {fig} \NC \NR
+\NC \type{copy_objects} \NC function \NC returns a deep copy of the array of graphic
+ objects in this \type {fig} \NC \NR
+\NC \type{filename} \NC function \NC the filename this \type {fig}'s \POSTSCRIPT\
+ output would have written to in stand alone
+ mode \NC \NR
+\NC \type{width} \NC function \NC the \type {fontcharwd} value \NC \NR
+\NC \type{height} \NC function \NC the \type {fontcharht} value \NC \NR
+\NC \type{depth} \NC function \NC the \type {fontchardp} value \NC \NR
+\NC \type{italcorr} \NC function \NC the \type {fontcharit} value \NC \NR
+\NC \type{charcode} \NC function \NC the (rounded) \type {charcode} value \NC \NR
+\LL
+\stoptabulate
+
+Note: you can call \type {fig:objects()} only once for any one \type {fig}
+object!
+
+When the boundingbox represents a \quote {negated rectangle}, i.e.\ when the
+first set of coordinates is larger than the second set, the picture is empty.
+
+Graphical objects come in various types that each has a different list of
+accessible values. The types are: \type {fill}, \type {outline}, \type {text},
+\type {start_clip}, \type {stop_clip}, \type {start_bounds}, \type {stop_bounds},
+\type {special}.
+
+There is a helper function (\type {mplib.fields(obj)}) to get the list of
+accessible values for a particular object, but you can just as easily use the
+tables given below.
+
+All graphical objects have a field \type {type} that gives the object type as a
+string value; it is not explicit mentioned in the following tables. In the
+following, \type {number}s are \POSTSCRIPT\ points represented as a floating
+point number, unless stated otherwise. Field values that are of type \type
+{table} are explained in the next section.
+
+\subsection{fill}
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{path} \NC table \NC the list of knots \NC \NR
+\NC \type{htap} \NC table \NC the list of knots for the reversed trajectory \NC \NR
+\NC \type{pen} \NC table \NC knots of the pen \NC \NR
+\NC \type{color} \NC table \NC the object's color \NC \NR
+\NC \type{linejoin} \NC number \NC line join style (bare number)\NC \NR
+\NC \type{miterlimit} \NC number \NC miterlimit\NC \NR
+\NC \type{prescript} \NC string \NC the prescript text \NC \NR
+\NC \type{postscript} \NC string \NC the postscript text \NC \NR
+\LL
+\stoptabulate
+
+The entries \type {htap} and \type {pen} are optional.
+
+\subsection{outline}
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{path} \NC table \NC the list of knots \NC \NR
+\NC \type{pen} \NC table \NC knots of the pen \NC \NR
+\NC \type{color} \NC table \NC the object's color \NC \NR
+\NC \type{linejoin} \NC number \NC line join style (bare number) \NC \NR
+\NC \type{miterlimit} \NC number \NC miterlimit \NC \NR
+\NC \type{linecap} \NC number \NC line cap style (bare number) \NC \NR
+\NC \type{dash} \NC table \NC representation of a dash list \NC \NR
+\NC \type{prescript} \NC string \NC the prescript text \NC \NR
+\NC \type{postscript} \NC string \NC the postscript text \NC \NR
+\LL
+\stoptabulate
+
+The entry \type {dash} is optional.
+
+\subsection{text}
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{text} \NC string \NC the text \NC \NR
+\NC \type{font} \NC string \NC font tfm name \NC \NR
+\NC \type{dsize} \NC number \NC font size \NC \NR
+\NC \type{color} \NC table \NC the object's color \NC \NR
+\NC \type{width} \NC number \NC \NC \NR
+\NC \type{height} \NC number \NC \NC \NR
+\NC \type{depth} \NC number \NC \NC \NR
+\NC \type{transform} \NC table \NC a text transformation \NC \NR
+\NC \type{prescript} \NC string \NC the prescript text \NC \NR
+\NC \type{postscript} \NC string \NC the postscript text \NC \NR
+\LL
+\stoptabulate
+
+\subsection{special}
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{prescript} \NC string \NC special text \NC \NR
+\LL
+\stoptabulate
+
+\subsection{start_bounds, start_clip}
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{path} \NC table \NC the list of knots \NC \NR
+\LL
+\stoptabulate
+
+\subsubsection{stop_bounds, stop_clip}
+
+Here are no fields available.
+
+\stopsection
+
+\startsection[title={Subsidiary table formats}]
+
+\subsection{Paths and pens}
+
+Paths and pens (that are really just a special type of paths as far as \MPLIB\ is
+concerned) are represented by an array where each entry is a table that
+represents a knot.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{left_type} \NC string \NC when present: endpoint, but usually absent \NC \NR
+\NC \type{right_type} \NC string \NC like \type {left_type} \NC \NR
+\NC \type{x_coord} \NC number \NC X coordinate of this knot \NC \NR
+\NC \type{y_coord} \NC number \NC Y coordinate of this knot \NC \NR
+\NC \type{left_x} \NC number \NC X coordinate of the precontrol point of this knot \NC \NR
+\NC \type{left_y} \NC number \NC Y coordinate of the precontrol point of this knot \NC \NR
+\NC \type{right_x} \NC number \NC X coordinate of the postcontrol point of this knot \NC \NR
+\NC \type{right_y} \NC number \NC Y coordinate of the postcontrol point of this knot \NC \NR
+\LL
+\stoptabulate
+
+There is one special case: pens that are (possibly transformed) ellipses have an
+extra string-valued key \type {type} with value \type {elliptical} besides the
+array part containing the knot list.
+
+\subsection{Colors}
+
+A color is an integer array with 0, 1, 3 or 4 values:
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{0} \NC marking only \NC no values \NC \NR
+\NC \type{1} \NC greyscale \NC one value in the range $(0,1)$, \quote {black} is $0$ \NC \NR
+\NC \type{3} \NC \RGB \NC three values in the range $(0,1)$, \quote {black} is $0,0,0$ \NC \NR
+\NC \type{4} \NC \CMYK \NC four values in the range $(0,1)$, \quote {black} is $0,0,0,1$ \NC \NR
+\LL
+\stoptabulate
+
+If the color model of the internal object was \type {uninitialized}, then it was
+initialized to the values representing \quote {black} in the colorspace \type
+{defaultcolormodel} that was in effect at the time of the \type {shipout}.
+
+\subsection{Transforms}
+
+Each transform is a six|-|item array.
+
+\starttabulate[|l|l|p|]
+\DB index \BC type \BC explanation \NC \NR
+\TB
+\NC \type{1} \NC number \NC represents x \NC \NR
+\NC \type{2} \NC number \NC represents y \NC \NR
+\NC \type{3} \NC number \NC represents xx \NC \NR
+\NC \type{4} \NC number \NC represents yx \NC \NR
+\NC \type{5} \NC number \NC represents xy \NC \NR
+\NC \type{6} \NC number \NC represents yy \NC \NR
+\LL
+\stoptabulate
+
+Note that the translation (index 1 and 2) comes first. This differs from the
+ordering in \POSTSCRIPT, where the translation comes last.
+
+\subsection{Dashes}
+
+Each \type {dash} is two-item hash, using the same model as \POSTSCRIPT\ for the
+representation of the dashlist. \type {dashes} is an array of \quote {on} and
+\quote {off}, values, and \type {offset} is the phase of the pattern.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{dashes} \NC hash \NC an array of on-off numbers \NC \NR
+\NC \type{offset} \NC number \NC the starting offset value \NC \NR
+\LL
+\stoptabulate
+
+\subsection{Pens and \type {pen_info}}
+
+\libindex{pen_info}
+
+There is helper function (\type {pen_info(obj)}) that returns a table containing
+a bunch of vital characteristics of the used pen (all values are floats):
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{width} \NC number \NC width of the pen \NC \NR
+\NC \type{sx} \NC number \NC $x$ scale \NC \NR
+\NC \type{rx} \NC number \NC $xy$ multiplier \NC \NR
+\NC \type{ry} \NC number \NC $yx$ multiplier \NC \NR
+\NC \type{sy} \NC number \NC $y$ scale \NC \NR
+\NC \type{tx} \NC number \NC $x$ offset \NC \NR
+\NC \type{ty} \NC number \NC $y$ offset \NC \NR
+\LL
+\stoptabulate
+
+\stopsection
+
+\startsection[title=Acessors]
+
+\subsection[title={Character size information}]
+
+\libindex{char_width}
+\libindex{char_height}
+\libindex{char_depth}
+
+These functions find the size of a glyph in a defined font. The \type {fontname}
+is the same name as the argument to \type {infont}; the \type {char} is a glyph
+id in the range 0 to 255; the returned \type {w} is in AFM units.
+
+
+\startfunctioncall
+<number> w = char_width(mp,<string> fontname, <number> char)
+<number> h = char_height(mp,<string> fontname, <number> char)
+<number> d = char_depth(mp,<string> fontname, <number> char)
+\stopfunctioncall
+
+These are rather useless and might become obsolete.
+
+\subsection{\type {get_[boolean|numeric|string|path]}}
+
+\libindex{get_boolean}
+\libindex{get_numeric}
+\libindex{get_path}
+\libindex{get_string}
+
+When a script call brings you from the \METAPOST\ run (temporarily) back to
+\LUA\ you can access variables, but only if they are known (so for instance
+anonymous capsules like loop variables are not accessible).
+
+\startfunctioncall
+<boolean> w = get_boolean(mp,<string> name)
+<number> n = get_numeric(mp,<string> name)
+<string> s = get_string (mp,<string> name)
+<table> p = get_path (mp,<string> name)
+\stopfunctioncall
+
+The path is returned a a table with subtables that have six numbers: the
+coordinates of the point, pre- and postcontrol. A \type {cycle} fields indicates
+if a path is cyclic.
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-modifications.tex b/doc/context/sources/general/manuals/luametatex/luametatex-modifications.tex
new file mode 100644
index 000000000..dca6c3781
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-modifications.tex
@@ -0,0 +1,440 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-modifications
+
+\startchapter[reference=modifications,title={The original engines}]
+
+\startsection[title=The merged engines]
+
+\startsubsection[title=The rationale]
+
+\topicindex {engines}
+\topicindex {history}
+
+The first version of \LUATEX, made by Hartmut after we discussed the possibility
+of an extension language, only had a few extra primitives and it was largely the
+same as \PDFTEX. It was presented to the public in 2005. As part of the Oriental
+\TEX\ project, Taco merged substantial parts of \ALEPH\ into the code and some
+more primitives were added. Then we started more fundamental experiments. After
+many years, when the engine had become more stable, the decision was made to
+clean up the rather hybrid nature of the program. This means that some primitives
+were promoted to core primitives, often with a different name, and that others
+were removed. This also made it possible to start cleaning up the code base. In
+\in {chapter} [enhancements] we discussed some new primitives, here we will cover
+most of the adapted ones.
+
+During more than a decade stepwise new functionality was added and after 10 years
+the more of less stable version 1.0 was presented. But we continued and after
+some 15 years the \LUAMETATEX\ follow up entered its first testing stage. But
+before details about the engine are discussed in successive chapters, we first
+summarize where we started from. Keep in mind that in \LUAMETATEX\ we have a bit
+less than in \LUATEX, so this section differs from the one in the \LUATEX\
+manual.
+
+Besides the expected changes caused by new functionality, there are a number of
+not|-|so|-|expected changes. These are sometimes a side|-|effect of a new
+(conflicting) feature, or, more often than not, a change necessary to clean up
+the internal interfaces. These will also be mentioned.
+
+\stopsubsection
+
+\startsubsection[title=Changes from \TEX\ 3.1415926]
+
+\topicindex {\TEX}
+
+Of course it all starts with traditional \TEX. Even if we started with \PDFTEX,
+most still comes from original Knuthian \TEX. But we divert a bit.
+
+\startitemize
+
+\startitem
+ The current code base is written in \CCODE, not \PASCAL. The original \CWEB\
+ documentation is kept when possible and not wrapped in tagged comments. As a
+ consequence instead of one large file plus change files, we now have multiple
+ files organized in categories like \type {tex}, \type {luaf}, \type
+ {languages}, \type {fonts}, \type {libraries}, etc. There are some artifacts
+ of the conversion to \CCODE, but these got (and get) removed stepwise. The
+ documentation, which is actually comes from the mix of engines (via so called
+ change files) is kept as much as possible. Of course we want to stay as close
+ as possible to the original so that the documentation of the fundamentals
+ behind \TEX\ by Don Knuth still applies. However, because we use \CCODE, some
+ documentation is a bit off. Also, most global variables are now collected in
+ structures, but the original names were kept. There are lots of so called
+ macros too.
+\stopitem
+
+\startitem
+ See \in {chapter} [languages] for many small changes related to paragraph
+ building, language handling and hyphenation. The most important change is
+ that adding a brace group in the middle of a word (like in \type {of{}fice})
+ does not prevent ligature creation. Also, the hyphenation, ligature building
+ and kerning has been split so that we can hook in alternative or extra code
+ wherever we like. There are various options to control discretionary
+ injection and related penalties are now integrated in these nodes. Language
+ information is now bound to glyphs. The number of languages in \LUAMETATEX\
+ is smaller than in \LUATEX.
+\stopitem
+
+\startitem
+ There is no pool file, all strings are embedded during compilation. This also
+ removed some memory constraints. We kept token and node memory management
+ because it is convenient and efficient but parts were reimplemented in order
+ to remove some constraints. Token memory management is largely the same.
+\stopitem
+
+\startitem
+ The specifier \type {plus 1 fillll} does not generate an error. The extra
+ \quote {l} is simply typeset.
+\stopitem
+
+\startitem
+ The upper limit to \prm {endlinechar} and \prm {newlinechar} is 127.
+\stopitem
+
+\startitem
+ Because the backend is not built|-|in, the magnification (\prm {mag})
+ primitive is not doing nothing. A \type {shipout} just discards the content
+ of the given box. The write related primitives have to be implemented in the
+ used macro package using \LUA. None of the \PDFTEX\ derived primitives is
+ present.
+\stopitem
+
+\startitem
+ There is more control over some (formerly hard|-|coded) math properties. In fact,
+ there is a whole extra bit of math related code because we need to deal with
+ \OPENTYPE\ fonts.
+\stopitem
+
+\startitem
+ The \type {\outer} and \type {\long} prefixed are silently ignored. It is
+ permitted to use \type {\par} in math.
+\stopitem
+
+\startitem
+ Because there is no font loader, a \LUA\ variant is free to either support or
+ not the \OMEGA\ \type {ofm} file format. As there are hardly any such fonts
+ it probably makes no sense.
+\stopitem
+
+\startitem
+ The lack of a backend means that some primitives related to it are not
+ implemented. This is no big deal because it is possible to use the scanner
+ library to implement them as needed, which depends on the macro package and
+ backend.
+\stopitem
+
+\startitem
+ When detailed logging is enabled more detail is output with respect to what
+ nodes are involved. This is a side effect of the core nodes having more
+ detailed subtype information. The benefit of more detail wins from any wish
+ to be byte compatible in the logging. One can always write additional logging
+ in \LUA.
+\stopitem
+
+\stopitemize
+
+\stopsubsection
+
+\startsubsection[title=Changes from \ETEX\ 2.2]
+
+\topicindex {\ETEX}
+
+Being the de|-|facto standard extension of course we provide the \ETEX\
+features, but with a few small adaptations.
+
+\startitemize
+
+\startitem
+ The \ETEX\ functionality is always present and enabled so the prepended
+ asterisk or \type {-etex} switch for \INITEX\ is not needed.
+\stopitem
+
+\startitem
+ The \TEXXET\ extension is not present, so the primitives \type
+ {\TeXXeTstate}, \type {\beginR}, \type {\beginL}, \type {\endR} and \type
+ {\endL} are missing. Instead we used the \OMEGA|/|\ALEPH\ approach to
+ directionality as starting point, albeit it has been changed quite a bit,
+ so that we're probably not that far from \TEXXET.
+\stopitem
+
+\startitem
+ Some of the tracing information that is output by \ETEX's \prm
+ {tracingassigns} and \prm {tracingrestores} is not there. Also keep in mind
+ that tracing doesn't involve what \LUA\ does.
+\stopitem
+
+\startitem
+ Register management in \LUAMETATEX\ uses the \OMEGA|/|\ALEPH\ model, so the
+ maximum value is 65535 and the implementation uses a flat array instead of
+ the mixed flat & sparse model from \ETEX.
+\stopitem
+
+\startitem
+ Because we don't use change files on top of original \TEX, the integration of
+ \ETEX\ functionality is bit more natural, code wise.
+\stopitem
+
+\stopitemize
+
+\stopsubsection
+
+\startsubsection[title=Changes from \PDFTEX\ 1.40]
+
+\topicindex {\PDFTEX}
+
+Because we want to produce \PDF\ the most natural starting point was the popular
+\PDFTEX\ program. We inherit the stable features, dropped most of the
+experimental code and promoted some functionality to core \LUATEX\ functionality
+which in turn triggered renaming primitives. However, as the backend was dropped,
+not that much from \PDFTEX\ is present any more. Basically all we now inherit
+from \PDFTEX\ is expansion and protrusion but even that has been adapted. So
+don't expect \LUAMETATEX\ to be compatible.
+
+\startitemize
+
+\startitem
+ The experimental primitives \lpr {ifabsnum} and \lpr {ifabsdim} have been
+ promoted to core primitives.
+\stopitem
+
+\startitem
+ The primitives \lpr {ifincsname}, \lpr {expanded} and \lpr {quitvmode}
+ have become core primitives.
+\stopitem
+
+\startitem
+ As the hz (expansion) and protrusion mechanism are part of the core the
+ related primitives \lpr {lpcode}, \lpr {rpcode}, \lpr {efcode}, \lpr
+ {leftmarginkern}, \lpr {rightmarginkern} are promoted to core primitives. The
+ two commands \lpr {protrudechars} and \lpr {adjustspacing} control these
+ processes.
+\stopitem
+
+\startitem
+ In \LUAMETATEX\ three extra primitives can be used to overload the font
+ specific settings: \lpr {adjustspacingstep} (max: 100), \lpr
+ {adjustspacingstretch} (max: 1000) and \lpr {adjustspacingshrink} (max: 500).
+\stopitem
+
+\startitem
+ The hz optimization code has been partially redone so that we no longer need
+ to create extra font instances. The front- and backend have been decoupled
+ and the glyph and kern nodes carry the used values. In \LUATEX\ that made a
+ more efficient generation of \PDF\ code possible. It also resulted in much
+ cleaner code. The backend code is gone, but of course the information is
+ still carried around.
+\stopitem
+
+\startitem
+ When \lpr {adjustspacing} has value~2, hz optimization will be applied to
+ glyphs and kerns. When the value is~3, only glyphs will be treated. A value
+ smaller than~2 disables this feature. With value of~1, font expansion is
+ applied after \TEX's normal paragraph breaking routines have broken the
+ paragraph into lines. In this case, line breaks are identical to standard
+ \TEX\ behavior (as with \PDFTEX). But \unknown\ this is a left|-|over from
+ the early days of \PDFTEX\ when this feature was part of a research topic. At
+ some point level~1 might be dropped from \LUAMETATEX.
+\stopitem
+
+\startitem
+ When \lpr {protrudechars} has a value larger than zero characters at the edge
+ of a line can be made to hang out. A value of~2 will take the protrusion into
+ account when breaking a paragraph into lines. A value of~3 will try to deal
+ with right|-|to|-|left rendering; this is a still experimental feature.
+\stopitem
+
+\startitem
+ The pixel multiplier dimension \lpr {pxdimen} has be inherited as core
+ primitive.
+\stopitem
+
+\startitem
+ The primitive \lpr {tracingfonts} is now a core primitive but doesn't relate
+ to the backend.
+\stopitem
+
+\stopitemize
+
+\stopsubsection
+
+\startsubsection[title=Changes from \ALEPH\ RC4]
+
+\topicindex {\ALEPH}
+
+In \LUATEX\ we took the 32 bit aspects and much of the directional mechanisms and
+merged it into the \PDFTEX\ code base as starting point for further development.
+Then we simplified directionality, fixed it and opened it up. In \LUAMETATEX\ not
+that much of the later is left. We only have two horizontal directions. Instead
+of vertical directions we introduce an orientation model bound to boxes.
+
+The already reduced|-|to|-|four set of directions now only has two members:
+left|-|to|-|right and right|-|to|-|left. They don't do much as it is the backend
+that has to deal with them. When paragraphs are constructed a change in
+horizontal direction is irrelevant for calculating the dimensions. So, basically
+most that we do is registering state and passing that on till the backend can do
+something with it.
+
+Here is a summary of inherited functionality:
+
+\startitemize
+
+\startitem
+ The \type {^^} notation has been extended: after \type {^^^^} four
+ hexadecimal characters are expected and after \type {^^^^^^} six hexadecimal
+ characters have to be given. The original \TEX\ interpretation is still valid
+ for the \type {^^} case but the four and six variants do no backtracking,
+ i.e.\ when they are not followed by the right number of hexadecimal digits
+ they issue an error message. Because \type {^^^} is a normal \TEX\ case, we
+ don't support the odd number of \type {^^^^^} either.
+\stopitem
+
+\startitem
+ Glues {\it immediately after} direction change commands are not legal
+ breakpoints. There is a bit more sanity testing for the direction state.
+\stopitem
+
+\startitem
+ The placement of math formula numbers is direction aware and adapts
+ accordingly. Boxes carry directional information but rules don't.
+\stopitem
+
+\startitem
+ There are no direction related primitives for page and body directions. The
+ paragraph, text and math directions are specified using primitives that
+ take a number.
+\stopitem
+
+\stopitemize
+
+\stopsubsection
+
+\startsubsection[title=Changes from standard \WEBC]
+
+\topicindex {\WEBC}
+
+The \LUAMETATEX\ codebase is not dependent on the \WEBC\ framework. The
+interaction with the file system and \TDS\ is up to \LUA. There still might be
+traces but eventually the code base should be lean and mean. The \METAPOST\
+library is coded in \CWEB\ and in order to be independent from related tools,
+conversion to \CCODE\ is done with a \LUA\ script ran by, surprise, \LUAMETATEX.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title=Implementation notes]
+
+\startsubsection[title=Memory allocation]
+
+\topicindex {memory}
+
+The single internal memory heap that traditional \TEX\ used for tokens and nodes
+is split into two separate arrays. Each of these will grow dynamically when
+needed. Internally a token or node is an index into these arrays. This permits
+for an efficient implementation and is also responsible for the performance of
+the core. The original documentation in \TEX\ The Program mostly applies!
+
+\stopsubsection
+
+\startsubsection[title=Sparse arrays]
+
+The \prm {mathcode}, \prm {delcode}, \prm {catcode}, \prm {sfcode}, \prm {lccode}
+and \prm {uccode} (and the new \lpr {hjcode}) tables are now sparse arrays that
+are implemented in~\CCODE. They are no longer part of the \TEX\ \quote
+{equivalence table} and because each had 1.1 million entries with a few memory
+words each, this makes a major difference in memory usage. Performance is not
+really hurt by this.
+
+The \prm {catcode}, \prm {sfcode}, \prm {lccode}, \prm {uccode} and \lpr {hjcode}
+assignments don't show up when using the \ETEX\ tracing routines \prm
+{tracingassigns} and \prm {tracingrestores} but we don't see that as a real
+limitation. It also saves a lot of clutter.
+
+A side|-|effect of the current implementation is that \prm {global} is now more
+expensive in terms of processing than non|-|global assignments but not many users
+will notice that.
+
+The glyph ids within a font are also managed by means of a sparse array as glyph
+ids can go up to index $2^{21}-1$ but these are never accessed directly so again
+users will not notice this.
+
+\stopsubsection
+
+\startsubsection[title=Simple single|-|character csnames]
+
+\topicindex {csnames}
+
+Single|-|character commands are no longer treated specially in the internals,
+they are stored in the hash just like the multiletter csnames.
+
+The code that displays control sequences explicitly checks if the length is one
+when it has to decide whether or not to add a trailing space.
+
+Active characters are internally implemented as a special type of multi|-|letter
+control sequences that uses a prefix that is otherwise impossible to obtain.
+
+\stopsubsection
+
+\startsubsection[title=Binary file reading]
+
+\topicindex {files+binary}
+
+All of the internal code is changed in such a way that if one of the \type
+{read_xxx_file} callbacks is not set, then the file is read by a \CCODE\ function
+using basically the same convention as the callback: a single read into a buffer
+big enough to hold the entire file contents. While this uses more memory than the
+previous code (that mostly used \type {getc} calls), it can be quite a bit faster
+(depending on your \IO\ subsystem). So far we never had issues with this approach.
+
+\stopsubsection
+
+\startsubsection[title=Tabs and spaces]
+
+\topicindex {space}
+\topicindex {newline}
+
+We conform to the way other \TEX\ engines handle trailing tabs and spaces. For
+decades trailing tabs and spaces (before a newline) were removed from the input
+but this behaviour was changed in September 2017 to only handle spaces. We are
+aware that this can introduce compatibility issues in existing workflows but
+because we don't want too many differences with upstream \TEXLIVE\ we just follow
+up on that patch (which is a functional one and not really a fix). It is up to
+macro packages maintainers to deal with possible compatibility issues and in
+\LUAMETATEX\ they can do so via the callbacks that deal with reading from files.
+
+The previous behaviour was a known side effect and (as that kind of input
+normally comes from generated sources) it was normally dealt with by adding a
+comment token to the line in case the spaces and|/|or tabs were intentional and
+to be kept. We are aware of the fact that this contradicts some of our other
+choices but consistency with other engines. We still stick to our view that at
+the log level we can (and might be) more incompatible. We already expose some
+more details anyway.
+
+\stopsubsection
+
+\startsubsection[title=Logging]
+
+The information that goes into the log file can be different from \LUATEX, and
+might even differ a bit more in the future. The main reason is that inside the
+engine we have more granularity, which for instance means that we output subtype
+related information when nodes are printed. Of course we could have offered a
+compatibility mode but it serves no purpose. Over time there have been many
+subtle changes to control logs in the \TEX\ ecosystems so another one is
+bearable.
+
+In a similar fashion, there is a bit different behaviour when \TEX\ expects
+input, which in turn is a side effect of removing the interception of \type {*}
+and \type {&} which made for cleaner code (quite a bit had accumulated as side
+effect of continuous adaptations in the \TEX\ ecosystems). There was already code
+that was never executed, simply as side effect of the way \LUATEX\ initializes
+itself (one needs to enable classes of primitives for instance).
+
+\stopsubsection
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex b/doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex
new file mode 100644
index 000000000..1d408ba6c
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-nodes.tex
@@ -0,0 +1,2527 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-nodes
+
+\startchapter[reference=nodes,title={Nodes}]
+
+\startsection[title={\LUA\ node representation}][library=node]
+
+\topicindex {nodes}
+
+\libindex {fields}
+\libindex {subtypes}
+\libindex {values}
+
+\TEX's nodes are represented in \LUA\ as userdata objects with a variable set of
+fields or by a numeric identifier when requested. When you print a node userdata
+object you will see these numbers. In the following syntax tables, such as the
+type of such a userdata object is represented as \syntax {<node>}.
+
+\blank
+\dontleavehmode {\bf The return values of \type {node.types()} are:} \showtypes
+\blank
+
+In \ETEX\ the \prm {lastnodetype} primitive has been introduced. With this
+primitive the valid range of numbers is still $[-1,15]$ and glyph nodes (formerly
+known as char nodes) have number~0. That way macro packages can use the same
+symbolic names as in traditional \ETEX. But you need to keep in mind that these
+\ETEX\ node numbers are different from the real internal ones. When you set \prm
+{internalcodesmode} to a non|-|zero value, the internal codes will be used in
+the \ETEX\ introspection commands \prm {lastnodetype} and \prm {currentiftype}.
+
+You can ask for a list of fields with \type {node.fields} and for valid subtypes
+with \type {node.subtypes}. The \type {node.values} function reports some used
+values. Valid arguments are \type {glue}, \type {style} and \type {math}. Keep in
+mind that the setters normally expect a number, but this helper gives you a list
+of what numbers matter. For practical reason the \type {pagestate} values are
+also reported with this helper, but they are backend specific.
+
+\def\ShowValues#1{
+ \blank
+ \dontleavehmode
+ {\bf The return values of \type {node.values("#1"} are:}
+ \showvalues{#1}
+ \blank
+}
+
+\ShowValues{glue} \ShowValues{style} \ShowValues{math} \ShowValues{pagestate}
+
+\stopsection
+
+\startsection[title={Main text nodes}]
+
+\topicindex {nodes+text}
+
+These are the nodes that comprise actual typesetting commands. A few fields are
+present in all nodes regardless of their type, these are:
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{next} \NC node \NC the next node in a list, or nil \NC \NR
+\NC \type{id} \NC number \NC the node's type (\type {id}) number \NC \NR
+\NC \type{subtype} \NC number \NC the node \type {subtype} identifier \NC \NR
+\LL
+\stoptabulate
+
+The \type {subtype} is sometimes just a dummy entry because not all nodes
+actually use the \type {subtype}, but this way you can be sure that all nodes
+accept it as a valid field name, and that is often handy in node list traversal.
+In the following tables \type {next} and \type {id} are not explicitly mentioned.
+
+Besides these three fields, almost all nodes also have an \type {attr} field, and
+there is a also a field called \type {prev}. That last field is always present,
+but only initialized on explicit request: when the function \type {node.slide}
+is called, it will set up the \type {prev} fields to be a backwards pointer in
+the argument node list. By now most of \TEX's node processing makes sure that the
+\type {prev} nodes are valid but there can be exceptions, especially when the
+internal magic uses a leading \nod {temp} nodes to temporarily store a state.
+
+The \LUAMETATEX\ engine provides a lot of freedom and it is up to the user to
+make sure that the node lists remain sane. There are some safeguards but there
+can be cases where the engine just quits out of frustration. And, of course you
+can make the engine crash.
+
+\startsubsection[title={\nod {hlist} and \nod {vlist} nodes}]
+
+\topicindex {nodes+lists}
+\topicindex {lists}
+
+These lists share fields and subtypes although some subtypes can only occur in
+horizontal lists while others are unique for vertical lists. The possible
+fields are \showfields {hlist}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{list} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{width} \NC number \NC the width of the box \NC \NR
+\NC \type{height} \NC number \NC the height of the box \NC \NR
+\NC \type{depth} \NC number \NC the depth of the box \NC \NR
+\NC \type{direction} \NC number \NC the direction of this box, see~\in [dirnodes] \NC \NR
+\NC \type{shift} \NC number \NC a displacement perpendicular to the character
+ (hlist) or line (vlist) progression direction \NC \NR
+\NC \type{glue_order} \NC number \NC a number in the range $[0,4]$, indicating the
+ glue order \NC \NR
+\NC \type{glue_set} \NC number \NC the calculated glue ratio \NC \NR
+\NC \type{glue_sign} \NC number \NC 0 = \type {normal}, 1 = \type {stretching}, 2 =
+ \type {shrinking} \NC \NR
+\NC \type{list} \NC node \NC the first node of the body of this list \NC \NR
+\LL
+\stoptabulate
+
+The \type {orientation}, \type {woffset}, \type {hoffset}, \type {doffset},
+\type {xoffset} and \type {yoffset} fields are special. They can be used to make
+the backend rotate and shift boxes which can be handy in for instance vertical
+typesetting. Because they relate to (and depend on the) the backend they are not
+discussed here (yet).
+
+A warning: never assign a node list to the \type {list} field unless you are sure
+its internal link structure is correct, otherwise an error may result.
+
+Note: the field name \type {head} and \type {list} are both valid. Sometimes it
+makes more sense to refer to a list by \type {head}, sometimes \type {list} makes
+more sense.
+
+\stopsubsection
+
+\startsubsection[title={\nod {rule} nodes}]
+
+\topicindex {nodes+rules}
+\topicindex {rules}
+
+Contrary to traditional \TEX, \LUATEX\ has more \prm {rule} subtypes because we
+also use rules to store reuseable objects and images. User nodes are invisible
+and can be intercepted by a callback. The supported fields are \showfields {rule}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes {rule} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{width} \NC number \NC the width of the rule where the special value
+ $-1073741824$ is used for \quote {running} glue dimensions \NC \NR
+\NC \type{height} \NC number \NC the height of the rule (can be negative) \NC \NR
+\NC \type{depth} \NC number \NC the depth of the rule (can be negative) \NC \NR
+\NC \type{left} \NC number \NC shift at the left end (also subtracted from width) \NC \NR
+\NC \type{right} \NC number \NC (subtracted from width) \NC \NR
+\NC \type{dir} \NC string \NC the direction of this rule, see~\in[dirnodes] \NC \NR
+\NC \type{index} \NC number \NC an optional index that can be referred to \NC \NR
+\NC \type{transform} \NC number \NC an private variable (also used to specify outline width) \NC \NR
+\LL
+\stoptabulate
+
+The \type {left} and type {right} keys are somewhat special (and experimental).
+When rules are auto adapting to the surrounding box width you can enforce a shift
+to the right by setting \type {left}. The value is also subtracted from the width
+which can be a value set by the engine itself and is not entirely under user
+control. The \type {right} is also subtracted from the width. It all happens in
+the backend so these are not affecting the calculations in the frontend (actually
+the auto settings also happen in the backend). For a vertical rule \type {left}
+affects the height and \type {right} affects the depth. There is no matching
+interface at the \TEX\ end (although we can have more keywords for rules it would
+complicate matters and introduce a speed penalty.) However, you can just
+construct a rule node with \LUA\ and write it to the \TEX\ input. The \type
+{outline} subtype is just a convenient variant and the \type {transform} field
+specifies the width of the outline.
+
+The \type {xoffset} and \type {yoffset} fields are special. They can be used to
+shift rules. Because they relate to (and depend on the) the backend they are not
+discussed here (yet).
+
+\stopsubsection
+
+\startsubsection[title={\nod {ins} nodes}]
+
+\topicindex {nodes+insertions}
+\topicindex {insertions}
+
+This node relates to the \prm {insert} primitive and support the fields: \showfields{ins}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC the insertion class \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{cost} \NC number \NC the penalty associated with this insert \NC \NR
+\NC \type{height} \NC number \NC height of the insert \NC \NR
+\NC \type{depth} \NC number \NC depth of the insert \NC \NR
+\NC \type{list} \NC node \NC the first node of the body of this insert \NC \NR
+\LL
+\stoptabulate
+
+There is a set of extra fields that concern the associated glue: \type {width},
+\type {stretch}, \type {stretch_order}, \type {shrink} and \type {shrink_order}.
+These are all numbers.
+
+A warning: never assign a node list to the \type {head} field unless you are sure
+its internal link structure is correct, otherwise an error may result. You can use
+\type {list} instead (often in functions you want to use local variable with similar
+names and both names are equally sensible).
+
+\stopsubsection
+
+\startsubsection[title={\nod {mark} nodes}]
+
+\topicindex {nodes+marks}
+\topicindex {marks}
+
+This one relates to the \prm {mark} primitive and only has a few fields:
+\showfields {mark}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC unused \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{class} \NC number \NC the mark class \NC \NR
+\NC \type{mark} \NC table \NC a table representing a token list \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsection
+
+\startsubsection[title={\nod {adjust} nodes}]
+
+\topicindex {nodes+adjust}
+\topicindex {adjust}
+
+This node comes from \prm {vadjust} primitive and has fields: \showfields {adjust}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{adjust} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{list} \NC node \NC adjusted material \NC \NR
+\LL
+\stoptabulate
+
+A warning: never assign a node list to the \type {head} field unless you are sure
+its internal link structure is correct, otherwise an error may be the result.
+
+\stopsubsection
+
+\startsubsection[title={\nod {disc} nodes}]
+
+\topicindex {nodes+discretionaries}
+\topicindex {discretionaries}
+
+The \prm {discretionary} and \prm {-}, the \type {-} character but also the
+hyphenation mechanism produces these nodes. The available fields are: \showfields
+{disc}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{disc} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{pre} \NC node \NC pointer to the pre|-|break text \NC \NR
+\NC \type{post} \NC node \NC pointer to the post|-|break text \NC \NR
+\NC \type{replace} \NC node \NC pointer to the no|-|break text \NC \NR
+\NC \type{penalty} \NC number \NC the penalty associated with the break, normally
+ \prm {hyphenpenalty} or \prm {exhyphenpenalty} \NC \NR
+\LL
+\stoptabulate
+
+The subtype numbers~4 and~5 belong to the \quote {of-f-ice} explanation given
+elsewhere. These disc nodes are kind of special as at some point they also keep
+information about breakpoints and nested ligatures.
+
+The \type {pre}, \type {post} and \type {replace} fields at the \LUA\ end are in
+fact indirectly accessed and have a \type {prev} pointer that is not \type {nil}.
+This means that when you mess around with the head of these (three) lists, you
+also need to reassign them because that will restore the proper \type {prev}
+pointer, so:
+
+\starttyping
+pre = d.pre
+-- change the list starting with pre
+d.pre = pre
+\stoptyping
+
+Otherwise you can end up with an invalid internal perception of reality and
+\LUAMETATEX\ might even decide to crash on you. It also means that running forward
+over for instance \type {pre} is ok but backward you need to stop at \type {pre}.
+And you definitely must not mess with the node that \type {prev} points to, if
+only because it is not really a node but part of the disc data structure (so
+freeing it again might crash \LUAMETATEX).
+
+\stopsubsection
+
+\startsubsection[title={\nod {math} nodes}]
+
+\topicindex {nodes+math}
+\topicindex {math+nodes}
+
+Math nodes represent the boundaries of a math formula, normally wrapped into
+\type {$} signs. The following fields are available: \showfields {math}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{math} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{surround} \NC number \NC width of the \prm {mathsurround} kern \NC \NR
+\NC \type{width} \NC number \NC the horizontal or vertical displacement \NC \NR
+\NC \type{stretch} \NC number \NC extra (positive) displacement or stretch amount \NC \NR
+\NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR
+\NC \type{shrink} \NC number \NC extra (negative) displacement or shrink amount\NC \NR
+\NC \type{shrink_order} \NC number \NC factor applied to shrink amount \NC \NR
+\LL
+\stoptabulate
+
+The glue fields only kick in when the \type {surround} fields is zero.
+
+\stopsubsection
+
+\startsubsection[title={\nod {glue} nodes}]
+
+\topicindex {nodes+glue}
+\topicindex {glue}
+
+Skips are about the only type of data objects in traditional \TEX\ that are not a
+simple value. They are inserted when \TEX\ sees a space in the text flow but also
+by \prm {hskip} and \prm {vskip}. The structure that represents the glue
+components of a skip internally is called a \nod {glue_spec}. In \LUAMETATEX\ we
+don't use the spec itself but just its values. A glue node has the fields:
+\showfields {glue}.
+
+\starttabulate[|l|l|pA{flushleft,tolerant}|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{glue} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{leader} \NC node \NC pointer to a box or rule for leaders \NC \NR
+\NC \type{width} \NC number \NC the horizontal or vertical displacement \NC \NR
+\NC \type{stretch} \NC number \NC extra (positive) displacement or stretch amount \NC \NR
+\NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR
+\NC \type{shrink} \NC number \NC extra (negative) displacement or shrink amount\NC \NR
+\NC \type{shrink_order} \NC number \NC factor applied to shrink amount \NC \NR
+\LL
+\stoptabulate
+
+Note that we use the key \type {width} in both horizontal and vertical glue. This
+suits the \TEX\ internals well so we decided to stick to that naming.
+
+The effective width of some glue subtypes depends on the stretch or shrink needed
+to make the encapsulating box fit its dimensions. For instance, in a paragraph
+lines normally have glue representing spaces and these stretch or shrink to make
+the content fit in the available space. The \type {effective_glue} function that
+takes a glue node and a parent (hlist or vlist) returns the effective width of
+that glue item. When you pass \type {true} as third argument the value will be
+rounded.
+
+\stopsubsection
+
+\startsubsection[title={\nod {glue_spec} nodes}]
+
+\topicindex {nodes+glue}
+\topicindex {gluespec}
+
+Internally \LUAMETATEX\ (like its ancestors) also uses nodes to store data that
+is not seen in node lists. For instance the state of expression scanning (\type
+{\dimexpr} etc.) and conditionals (\type {\ifcase} etc.) is also kept in lists of
+nodes. A glue, which has five components, is stored in a node as well, so, where
+most registers store just a number, a skip register (of internal quantity) uses a
+pointer to a glue spec node. It has similar fields as glue nodes: \showfields
+{glue_spec}, which is not surprising because in the past (and other engines than
+\LUATEX) a glue node also has its values stored in a glue spec. This has some
+advantages because often the values are the same, so for instance spacing related
+skips were not resolved immediately but pointed to the current value of a space
+related internal register (like \type {\spaceskip}). But, in \LUATEX\ we do
+resolve these quantities immediately and we put the current values in the glue
+nodes.
+
+\starttabulate[|l|l|pA{flushleft,tolerant}|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{width} \NC number \NC the horizontal or vertical displacement \NC \NR
+\NC \type{stretch} \NC number \NC extra (positive) displacement or stretch amount \NC \NR
+\NC \type{stretch_order} \NC number \NC factor applied to stretch amount \NC \NR
+\NC \type{shrink} \NC number \NC extra (negative) displacement or shrink amount\NC \NR
+\NC \type{shrink_order} \NC number \NC factor applied to shrink amount \NC \NR
+\LL
+\stoptabulate
+
+You will only find these nodes in a few places, for instance when you query an
+internal quantity. In principle we could do without them as we have interfaces
+that use the five numbers instead. For compatibility reasons we keep glue spec
+nodes exposed but this might change in the future.
+
+\stopsubsection
+
+\startsubsection[title={\nod {kern} nodes}]
+
+\topicindex {nodes+kerns}
+\topicindex {kerns}
+
+The \prm {kern} command creates such nodes but for instance the font and math
+machinery can also add them. There are not that many fields: \showfields {kern}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{kern} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{kern} \NC number \NC fixed horizontal or vertical advance \NC \NR
+\NC \type{expansion_factor} \NC number \NC multiplier related to hz for font kerns \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsection
+
+\startsubsection[title={\nod {penalty} nodes}]
+
+\topicindex {nodes+penalty}
+\topicindex {penalty}
+
+The \prm {penalty} command is one that generates these nodes. It is one of the
+type of nodes often found in vertical lists. It has the fields: \showfields
+{penalty}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{penalty} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{penalty} \NC number \NC the penalty value \NC \NR
+\LL
+\stoptabulate
+
+The subtypes are just informative and \TEX\ itself doesn't use them. When you
+run into an \type {linebreakpenalty} you need to keep in mind that it's a
+accumulation of \type {club}, \type{widow} and other relevant penalties.
+
+\stopsubsection
+
+\startsubsection[title={\nod {glyph} nodes},reference=glyphnodes]
+
+\topicindex {nodes+glyph}
+\topicindex {glyphs}
+
+These are probably the mostly used nodes and although you can push them in the
+current list with for instance \prm {char} \TEX\ will normally do it for you when
+it considers some input to be text. Glyph nodes are relatively large and have many
+fields: \showfields {glyph}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC bit field \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{char} \NC number \NC the character index in the font \NC \NR
+\NC \type{font} \NC number \NC the font identifier \NC \NR
+\NC \type{lang} \NC number \NC the language identifier \NC \NR
+\NC \type{left} \NC number \NC the frozen \type {\lefthyphenmnin} value \NC \NR
+\NC \type{right} \NC number \NC the frozen \type {\righthyphenmnin} value \NC \NR
+\NC \type{uchyph} \NC boolean \NC the frozen \prm {uchyph} value \NC \NR
+\NC \type{components} \NC node \NC pointer to ligature components \NC \NR
+\NC \type{xoffset} \NC number \NC a virtual displacement in horizontal direction \NC \NR
+\NC \type{yoffset} \NC number \NC a virtual displacement in vertical direction \NC \NR
+\NC \type{width} \NC number \NC the (original) width of the character \NC \NR
+\NC \type{height} \NC number \NC the (original) height of the character\NC \NR
+\NC \type{depth} \NC number \NC the (original) depth of the character\NC \NR
+\NC \type{expansion_factor} \NC number \NC the to be applied expansion_factor \NC \NR
+\NC \type{data} \NC number \NC a general purpose field for users (we had room for it) \NC \NR
+\LL
+\stoptabulate
+
+The \type {width}, \type {height} and \type {depth} values are read|-|only. The
+\type {expansion_factor} is assigned in the par builder and used in the backend.
+
+A warning: never assign a node list to the components field unless you are sure
+its internal link structure is correct, otherwise an error may be result. Valid
+bits for the \type {subtype} field are:
+
+\starttabulate[|c|l|]
+\DB bit \BC meaning \NC \NR
+\TB
+\NC 0 \NC character \NC \NR
+\NC 1 \NC ligature \NC \NR
+\NC 2 \NC ghost \NC \NR
+\NC 3 \NC left \NC \NR
+\NC 4 \NC right \NC \NR
+\LL
+\stoptabulate
+
+The \type {expansion_factor} has been introduced as part of the separation
+between font- and backend. It is the result of extensive experiments with a more
+efficient implementation of expansion. Early versions of \LUATEX\ already
+replaced multiple instances of fonts in the backend by scaling but contrary to
+\PDFTEX\ in \LUATEX\ we now also got rid of font copies in the frontend and
+replaced them by expansion factors that travel with glyph nodes. Apart from a
+cleaner approach this is also a step towards a better separation between front-
+and backend.
+
+The \type {is_char} function checks if a node is a glyph node with a subtype still
+less than 256. This function can be used to determine if applying font logic to a
+glyph node makes sense. The value \type {nil} gets returned when the node is not
+a glyph, a character number is returned if the node is still tagged as character
+and \type {false} gets returned otherwise. When nil is returned, the id is also
+returned. The \type {is_glyph} variant doesn't check for a subtype being less
+than 256, so it returns either the character value or nil plus the id. These
+helpers are not always faster than separate calls but they sometimes permit
+making more readable tests. The \type {uses_font} helpers takes a node
+and font id and returns true when a glyph or disc node references that font.
+
+\stopsubsection
+
+\startsubsection[title={\nod {boundary} nodes}]
+
+\topicindex {nodes+boundary}
+\topicindex {boundary}
+
+This node relates to the \prm {noboundary}, \prm {boundary}, \prm
+{protrusionboundary} and \prm {wordboundary} primitives. These are small
+nodes: \showfields {boundary} are the only fields.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{boundary} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{value} \NC number \NC values 0--255 are reserved \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsection
+
+\startsubsection[title={\nod {local_par} nodes}]
+
+\topicindex {nodes+paragraphs}
+\topicindex {paragraphs}
+
+This node is inserted at the start of a paragraph. You should not mess
+too much with this one. Valid fields are: \showfields {local_par}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{pen_inter} \NC number \NC local interline penalty (from \lpr {localinterlinepenalty}) \NC \NR
+\NC \type{pen_broken} \NC number \NC local broken penalty (from \lpr {localbrokenpenalty}) \NC \NR
+\NC \type{dir} \NC string \NC the direction of this par. see~\in [dirnodes] \NC \NR
+\NC \type{box_left} \NC node \NC the \lpr {localleftbox} \NC \NR
+\NC \type{box_left_width} \NC number \NC width of the \lpr {localleftbox} \NC \NR
+\NC \type{box_right} \NC node \NC the \lpr {localrightbox} \NC \NR
+\NC \type{box_right_width} \NC number \NC width of the \lpr {localrightbox} \NC \NR
+\LL
+\stoptabulate
+
+A warning: never assign a node list to the \type {box_left} or \type {box_right}
+field unless you are sure its internal link structure is correct, otherwise an
+error may result.
+
+\stopsubsection
+
+\startsubsection[title={\nod {dir} nodes},reference=dirnodes]
+
+\topicindex {nodes+direction}
+\topicindex {directions}
+
+Direction nodes mark parts of the running text that need a change of direction
+and the \prm {textdir} command generates them. Again this is a small node, we
+just have \showfields {dir}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{dir} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{dir} \NC string \NC the direction (\type {0} = l2r, \type {1} = r2l) \NC \NR
+\NC \type{level} \NC number \NC nesting level of this direction \NC \NR
+\LL
+\stoptabulate
+
+There are only two directions: left|-|to|-|right (\type {0}) and
+right|-|to|-|left (\type {1}). This is different from \LUATEX\ that has four
+directions.
+
+\stopsubsection
+
+\startsubsection[title={\nod {marginkern} nodes}]
+
+\topicindex {nodes+paragraphs}
+\topicindex {paragraphs}
+\topicindex {protrusion}
+
+Margin kerns result from protrusion and have: \showfields {margin_kern}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{marginkern} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{width} \NC number \NC the advance of the kern \NC \NR
+\NC \type{glyph} \NC node \NC the glyph to be used \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsection
+
+\startsubsection[title={Whatsits}]
+
+A whatsit node is a real simple one and it only has a subtype. It is even less
+than a user node (which it actually could be) and uses hardly any memory. What
+you do with it it entirely up to you: it's is real minimalistic. You can assign a
+subtype and it has attributes. It is all up to the user how they are handled.
+
+\stopsubsection
+
+\startsubsection[title={Math noads}]
+
+\topicindex {nodes+math}
+\topicindex {math+nodes}
+
+\startsubsubsection[title=The concept]
+
+These are the so||called \quote {noad}s and the nodes that are specifically
+associated with math processing. When you enter a formula, \TEX\ creates a node
+list with regular nodes and noads. Then it hands over the list the math
+processing engine. The result of that is a nodelist without noads. Most of the
+noads contain subnodes so that the list of possible fields is actually quite
+small. Math formulas are both a linked list and a tree. For instance in $e =
+mc^2$ there is a linked list \type {e = m c} but the \type {c} has a superscript
+branch that itself can be a list with branches.
+
+First, there are the objects (the \TEX book calls them \quote {atoms}) that are
+associated with the simple math objects: ord, op, bin, rel, open, close, punct,
+inner, over, under, vcenter. These all have the same fields, and they are combined
+into a single node type with separate subtypes for differentiation: \showfields
+{noad}.
+
+Many object fields in math mode are either simple characters in a specific family
+or math lists or node lists: \type {math_char}, \type {math_text_char}, {sub_box}
+and \type {sub_mlist} and \type {delim}. These are endpoints and therefore the
+\type {next} and \type {prev} fields of these these subnodes are unused.
+
+Some of the more elaborate noads have an option field. The values in this bitset
+are common:
+
+\starttabulate[|l|r|]
+\DB meaning \BC bits \NC \NR
+\TB
+\NC set \NC \type{0x08} \NC \NR
+\NC internal \NC \type{0x00} + \type{0x08} \NC \NR
+\NC internal \NC \type{0x01} + \type{0x08} \NC \NR
+\NC axis \NC \type{0x02} + \type{0x08} \NC \NR
+\NC no axis \NC \type{0x04} + \type{0x08} \NC \NR
+\NC exact \NC \type{0x10} + \type{0x08} \NC \NR
+\NC left \NC \type{0x11} + \type{0x08} \NC \NR
+\NC middle \NC \type{0x12} + \type{0x08} \NC \NR
+\NC right \NC \type{0x14} + \type{0x08} \NC \NR
+\NC no subscript \NC \type{0x21} + \type{0x08} \NC \NR
+\NC no superscript \NC \type{0x22} + \type{0x08} \NC \NR
+\NC no script \NC \type{0x23} + \type{0x08} \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {math_char} and \nod {math_text_char} subnodes}]
+
+These are the most common ones, as they represent characters, and they both have
+the same fields: \showfields {math_char}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{char} \NC number \NC the character index \NC \NR
+\NC \type{fam} \NC number \NC the family number \NC \NR
+\LL
+\stoptabulate
+
+The \nod {math_char} is the simplest subnode field, it contains the character and
+family for a single glyph object. The family eventually resolves on a reference
+to a font. The \nod {math_text_char} is a special case that you will not normally
+encounter, it arises temporarily during math list conversion (its sole function
+is to suppress a following italic correction).
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {sub_box} and \nod {sub_mlist} subnodes}]
+
+These two subnode types are used for subsidiary list items. For \nod {sub_box},
+the \type {list} points to a \quote {normal} vbox or hbox. For \nod {sub_mlist},
+the \type {list} points to a math list that is yet to be converted. Their fields
+are: \showfields {sub_box}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{list} \NC node \NC list of nodes \NC \NR
+\LL
+\stoptabulate
+
+A warning: never assign a node list to the \type {list} field unless you are sure
+its internal link structure is correct, otherwise an error is triggered.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {delim} subnodes}]
+
+There is a fifth subnode type that is used exclusively for delimiter fields. As
+before, the \type {next} and \type {prev} fields are unused, but we do have:
+\showfields {delim}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{small_char} \NC number \NC character index of base character \NC \NR
+\NC \type{small_fam} \NC number \NC family number of base character \NC \NR
+\NC \type{large_char} \NC number \NC character index of next larger character \NC \NR
+\NC \type{large_fam} \NC number \NC family number of next larger character \NC \NR
+\LL
+\stoptabulate
+
+The fields \type {large_char} and \type {large_fam} can be zero, in that case the
+font that is set for the \type {small_fam} is expected to provide the large
+version as an extension to the \type {small_char}.
+
+\stopsubsubsection
+
+\startsubsubsection[title={simple \nod {noad} nodes}]
+
+In these noads, the \type {nucleus}, \type {sub} and \type {sup} fields can
+branch of. Its fields are: \showfields {noad}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{noad} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{nucleus} \NC kernel node \NC base \NC \NR
+\NC \type{sub} \NC kernel node \NC subscript \NC \NR
+\NC \type{sup} \NC kernel node \NC superscript \NC \NR
+\NC \type{options} \NC number \NC bitset of rendering options \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {accent} nodes}]
+
+Accent nodes deal with stuff on top or below a math constructs. They support:
+\showfields {accent}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{accent} \NC \NR
+\NC \type{nucleus} \NC kernel node \NC base \NC \NR
+\NC \type{sub} \NC kernel node \NC subscript \NC \NR
+\NC \type{sup} \NC kernel node \NC superscript \NC \NR
+\NC \type{accent} \NC kernel node \NC top accent \NC \NR
+\NC \type{bot_accent} \NC kernel node \NC bottom accent \NC \NR
+\NC \type{fraction} \NC number \NC larger step criterium (divided by 1000) \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {style} nodes}]
+
+These nodes are signals to switch to another math style. They are quite simple:
+\showfields {style}. Currently the subtype is actually used to store the style
+but don't rely on that for the future. Fields are: \showfields {style}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{style} \NC string \NC contains the style \NC \NR
+\LL
+\stoptabulate
+
+Valid styles are: \showvalues{style}.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {parameter} nodes}]
+
+These nodes are used to (locally) set math parameters: \showfields {parameter}.
+Fields are: \showfields {parameter}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{style} \NC string \NC contains the style \NC \NR
+\NC \type{name} \NC string \NC defines the parameter \NC \NR
+\NC \type{value} \NC number \NC holds the value, in case of a muglue multiple \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {choice} nodes}]
+
+Of its fields \showfields {choice} most are lists. Warning: never assign a node
+list unless you are sure its internal link structure is correct, otherwise an
+error can occur.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{display} \NC node \NC list of display size alternatives \NC \NR
+\NC \type{text} \NC node \NC list of text size alternatives \NC \NR
+\NC \type{script} \NC node \NC list of scriptsize alternatives \NC \NR
+\NC \type{scriptscript} \NC node \NC list of scriptscriptsize alternatives \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {radical} nodes}]
+
+Radical nodes are the most complex as they deal with scripts as well as
+constructed large symbols. Many fields: \showfields {radical}. Warning: never
+assign a node list to the \type {nucleus}, \type {sub}, \type {sup}, \type
+{left}, or \type {degree} field unless you are sure its internal link structure
+is correct, otherwise an error can be triggered.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{radical} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{nucleus} \NC kernel node \NC base \NC \NR
+\NC \type{sub} \NC kernel node \NC subscript \NC \NR
+\NC \type{sup} \NC kernel node \NC superscript \NC \NR
+\NC \type{left} \NC delimiter node \NC \NC \NR
+\NC \type{degree} \NC kernel node \NC only set by \lpr {Uroot} \NC \NR
+\NC \type{width} \NC number \NC required width \NC \NR
+\NC \type{options} \NC number \NC bitset of rendering options \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {fraction} nodes}]
+
+Fraction nodes are also used for delimited cases, hence the \type {left} and
+\type {right} fields among: \showfields {fraction}.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{width} \NC number \NC (optional) width of the fraction \NC \NR
+\NC \type{num} \NC kernel node \NC numerator \NC \NR
+\NC \type{denom} \NC kernel node \NC denominator \NC \NR
+\NC \type{left} \NC delimiter node \NC left side symbol \NC \NR
+\NC \type{right} \NC delimiter node \NC right side symbol \NC \NR
+\NC \type{middle} \NC delimiter node \NC middle symbol \NC \NR
+\NC \type{options} \NC number \NC bitset of rendering options \NC \NR
+\LL
+\stoptabulate
+
+Warning: never assign a node list to the \type {num}, or \type {denom} field
+unless you are sure its internal link structure is correct, otherwise an error
+can result.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {fence} nodes}]
+
+Fence nodes come in pairs but either one can be a dummy (this period driven empty
+fence). Fields are: \showfields {fence}. Some of these fields are used by the
+renderer and might get adapted in the process.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{subtype} \NC number \NC \showsubtypes{fence} \NC \NR
+\NC \type{attr} \NC node \NC list of attributes \NC \NR
+\NC \type{delim} \NC delimiter node \NC delimiter specification \NC \NR
+\NC \type{italic} \NC number \NC italic correction \NC \NR
+\NC \type{height} \NC number \NC required height \NC \NR
+\NC \type{depth} \NC number \NC required depth \NC \NR
+\NC \type{options} \NC number \NC bitset of rendering options \NC \NR
+\NC \type{class} \NC number \NC spacing related class \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsubsection
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={The \type {node} library}][library=node]
+
+\startsubsection[title={Introduction}]
+
+The \type {node} library provides methods that facilitate dealing with (lists of)
+nodes and their values. They allow you to create, alter, copy, delete, and insert
+node, the core objects within the typesetter. Nodes are represented in \LUA\ as
+userdata. The various parts within a node can be accessed using named fields.
+
+Each node has at least the three fields \type {next}, \type {id}, and \type
+{subtype}. The other available fields depend on the \type {id}.
+
+\startitemize[intro]
+
+\startitem
+ The \type {next} field returns the userdata object for the next node in a
+ linked list of nodes, or \type {nil}, if there is no next node.
+\stopitem
+
+\startitem
+ The \type {id} indicates \TEX's \quote{node type}. The field \type {id} has a
+ numeric value for efficiency reasons, but some of the library functions also
+ accept a string value instead of \type {id}.
+\stopitem
+
+\startitem
+ The \type {subtype} is another number. It often gives further information
+ about a node of a particular \type {id}.
+\stopitem
+
+\stopitemize
+
+% Support for \nod {unset} (alignment) nodes is partial: they can be queried and
+% modified from \LUA\ code, but not created.
+
+Nodes can be compared to each other, but: you are actually comparing indices into
+the node memory. This means that equality tests can only be trusted under very
+limited conditions. It will not work correctly in any situation where one of the
+two nodes has been freed and|/|or reallocated: in that case, there will be false
+positives. The general approach to a node related callback is as follows:
+
+\startitemize
+
+\startitem
+ Assume that the node list that you get is okay and properly double linked.
+ If for some reason the links are not right, you can apply \type {node.slide}
+ to the list.
+\stopitem
+
+\startitem
+ When you insert a node, make sure you use a previously removed one, a new one
+ or a copy. Don't simply inject the same node twice.
+\stopitem
+
+\startitem
+ When you remove a node, make sure that when this is permanent, you also free
+ the node or list. When you free a node its components are checked and when
+ they are nodes themselves they are also freed.
+\stopitem
+
+\startitem
+ Although you can fool the system, normally you will trigger an error when you
+ try to copy a nonexisting node, or free an already freed node. There is some
+ overhead involved in this checking but the current compromise is acceptable.
+\stopitem
+
+\startitem
+ When you're done, pass back (if needed) the result. It's your responsibility
+ to make sure that the list is properly linked (you can play safe and again
+ apply \type {node.slide}. In principle you can put nodes in a list that are
+ not acceptable in the following up actions. Some nodes get ignored, others
+ will trigger an error, and sometimes the engine will just crash.
+\stopitem
+
+\stopitemize
+
+So, from the above it will be clear then memory management of nodes has to be
+done explicitly by the user. Nodes are not \quote {seen} by the \LUA\ garbage
+collector, so you have to call the node freeing functions yourself when you are
+no longer in need of a node (list). Nodes form linked lists without reference
+counting, so you have to be careful that when control returns back to \LUATEX\
+itself, you have not deleted nodes that are still referenced from a \type {next}
+pointer elsewhere, and that you did not create nodes that are referenced more
+than once. Normally the setters and getters handle this for you.
+
+A good example are discretionary nodes that themselves have three sublists.
+Internally they use special pointers, but the user never sees them because when
+you query them or set fields, this property is hidden and taken care of. You just
+see a list. But, when you mess with these sub lists it is your responsibility
+that it only contains nodes that are permitted in a discretionary.
+
+There are statistics available with regards to the allocated node memory, which
+can be handy for tracing. Normally the amount of used nodes is not that large.
+Typesetting a page can involve thousands of them but most are freed when the page
+has been shipped out. Compared to other programs, node memory usage is not that
+excessive. So, if for some reason your application leaks nodes, if at the end of
+your run you lost as few hundred it's not a real problem. In fact, if you created
+boxes and made copies but not flushed them for good reason, your run will for
+sure end with used nodes and the statistics will mention that. The same is true
+for attributes and skips (glue spec nodes): keeping the current state involves
+using nodes.
+
+\stopsubsection
+
+\startsubsection[title={Housekeeping}]
+
+\startsubsubsection[title={\type {types}}]
+
+\libindex {types}
+
+This function returns an array that maps node id numbers to node type strings,
+providing an overview of the possible top|-|level \type {id} types.
+
+\startfunctioncall
+<table> t = node.types()
+\stopfunctioncall
+
+when we issue this command, we get a table. The currently visible types are
+\inlineluavalue { node.types() } where the numbers are the internal identifiers.
+Only those nodes are reported that make sense to users so there can be gaps in
+the range of numbers.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {id} and \type {type}}]
+
+\libindex{id}
+\libindex{type}
+
+This converts a single type name to its internal numeric representation.
+
+\startfunctioncall
+<number> id = node.id(<string> type)
+\stopfunctioncall
+
+The \type {node.id("glyph")} command returns the number \inlineluavalue { node.id
+("glyph") } and \type {node.id("hlist")} returns \inlineluavalue { node.id
+("hlist") } where the number don't relate to importance or some ordering; they
+just appear in the order that is handy for the engine. Commands like this are
+rather optimized so performance should be ok but you can of course always store
+the id in a \LUA\ number.
+
+The reverse operation is: \type {node.type} If the argument is a number, then the
+next function converts an internal numeric representation to an external string
+representation. Otherwise, it will return the string \type {node} if the object
+represents a node, and \type {nil} otherwise.
+
+\startfunctioncall
+<string> type = node.type(<any> n)
+\stopfunctioncall
+
+The \type {node.type(4)} command returns the string \inlineluavalue { node.type
+(4) } and \type {node.id(99)} returns \inlineluavalue { node.id (99) } because
+there is no node with that id.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {fields} and \type {has_field}}]
+
+\libindex {fields}
+\libindex {has_field}
+
+This function returns an indexed table with valid field names for a particular type of
+node.
+
+\startfunctioncall
+<table> t = node.fields(<number|string> id)
+\stopfunctioncall
+
+The function accepts a string or number, so \typ {node.fields ("glyph")} returns
+\inlineluavalue { node.fields ("glyph") } and \typ {node.fields (12)} gives
+\inlineluavalue { node.fields (12) }.
+
+The \type {has_field} function returns a boolean that is only true if \type {n}
+is actually a node, and it has the field.
+
+\startfunctioncall
+<boolean> t = node.has_field(<node> n, <string> field)
+\stopfunctioncall
+
+This function probably is not that useful but some nodes don't have a \type
+{subtype}, \type {attr} or \type {prev} field and this is a way to test for that.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {is_node}}]
+
+\topicindex {nodes+functions}
+
+\libindex {is_node}
+
+\startfunctioncall
+<boolean|integer> t = node.is_node(<any> item)
+\stopfunctioncall
+
+This function returns a number (the internal index of the node) if the argument
+is a userdata object of type \type {<node>} and false when no node is passed.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {new}}]
+
+\libindex{new}
+
+The \type {new} function creates a new node. All its fields are initialized to
+either zero or \type {nil} except for \type {id} and \type {subtype}. Instead of
+numbers you can also use strings (names). If you pass a second argument the
+subtype will be set too.
+
+\startfunctioncall
+<node> n = node.new(<number|string> id)
+<node> n = node.new(<number|string> id, <number|string> subtype)
+\stopfunctioncall
+
+As already has been mentioned, you are responsible for making sure that nodes
+created this way are used only once, and are freed when you don't pass them
+back somehow.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {free}, \type {flush_node} and \type {flush_list}}]
+
+\libindex{free}
+\libindex{flush_node}
+\libindex{flush_list}
+
+The next one frees node \type {n} from \TEX's memory. Be careful: no checks are
+done on whether this node is still pointed to from a register or some \type
+{next} field: it is up to you to make sure that the internal data structures
+remain correct. Fields that point to nodes or lists are flushed too. So, when
+you used their content for something else you need to set them to nil first.
+
+\startfunctioncall
+<node> next = node.free(<node> n)
+flush_node(<node> n)
+\stopfunctioncall
+
+The \type {free} function returns the next field of the freed node, while the
+\type {flush_node} alternative returns nothing.
+
+A list starting with node \type {n} can be flushed from \TEX's memory too. Be
+careful: no checks are done on whether any of these nodes is still pointed to
+from a register or some \type {next} field: it is up to you to make sure that the
+internal data structures remain correct.
+
+\startfunctioncall
+node.flush_list(<node> n)
+\stopfunctioncall
+
+When you free for instance a discretionary node, \type {flush_list} is applied to
+the \type {pre}, \type {post}, \type {replace} so you don't need to do that
+yourself. Assigning them \type {nil} won't free those lists!
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {copy} and \type {copy_list}}]
+
+\libindex{copy}
+\libindex{copy_list}
+
+This creates a deep copy of node \type {n}, including all nested lists as in the case
+of a hlist or vlist node. Only the \type {next} field is not copied.
+
+\startfunctioncall
+<node> m = node.copy(<node> n)
+\stopfunctioncall
+
+A deep copy of the node list that starts at \type {n} can be created too. If
+\type {m} is also given, the copy stops just before node \type {m}.
+
+\startfunctioncall
+<node> m = node.copy_list(<node> n)
+<node> m = node.copy_list(<node> n, <node> m)
+\stopfunctioncall
+
+Note that you cannot copy attribute lists this way. However, there is normally no
+need to copy attribute lists as when you do assignments to the \type {attr} field
+or make changes to specific attributes, the needed copying and freeing takes
+place automatically. When you change a value of an attribute {\em in} a list, it will
+affect all the nodes that share that list.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {write}}]
+
+\libindex {write}
+
+\startfunctioncall
+node.write(<node> n)
+\stopfunctioncall
+
+This function that will append a node list to \TEX's \quote {current list}. The
+node list is not deep|-|copied! There is no error checking either! You mignt need
+to enforce horizontal mode in order for this to work as expected.
+
+\stopsubsubsection
+
+\stopsubsection
+
+\startsubsection[title={Manipulating lists}]
+
+\startsubsubsection[title={\type {slide}}]
+
+\libindex {slide}
+
+This helper makes sure that the node lists is double linked and returns the found
+tail node.
+
+\startfunctioncall
+<node> tail = node.slide(<node> n)
+\stopfunctioncall
+
+After some callbacks automatic sliding takes place. This feature can be turned
+off with \type {node.fix_node_lists(false)} but you better make sure then that
+you don't mess up lists. In most cases \TEX\ itself only uses \type {next}
+pointers but your other callbacks might expect proper \type {prev} pointers too.
+Future versions of \LUATEX\ can add more checking but this will not influence
+usage.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {tail}}]
+
+\libindex {tail}
+
+\startfunctioncall
+<node> m = node.tail(<node> n)
+\stopfunctioncall
+
+Returns the last node of the node list that starts at \type {n}.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {length} and \type {count}}]
+
+\libindex {length}
+\libindex {count}
+
+\startfunctioncall
+<number> i = node.length(<node> n)
+<number> i = node.length(<node> n, <node> m)
+\stopfunctioncall
+
+Returns the number of nodes contained in the node list that starts at \type {n}.
+If \type {m} is also supplied it stops at \type {m} instead of at the end of the
+list. The node \type {m} is not counted.
+
+\startfunctioncall
+<number> i = node.count(<number> id, <node> n)
+<number> i = node.count(<number> id, <node> n, <node> m)
+\stopfunctioncall
+
+Returns the number of nodes contained in the node list that starts at \type {n}
+that have a matching \type {id} field. If \type {m} is also supplied, counting
+stops at \type {m} instead of at the end of the list. The node \type {m} is not
+counted. This function also accept string \type {id}'s.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {remove}}]
+
+\libindex {remove}
+
+\startfunctioncall
+<node> head, current, removed =
+ node.remove(<node> head, <node> current)
+<node> head, current =
+ node.remove(<node> head, <node> current, <boolean> true)
+\stopfunctioncall
+
+This function removes the node \type {current} from the list following \type
+{head}. It is your responsibility to make sure it is really part of that list.
+The return values are the new \type {head} and \type {current} nodes. The
+returned \type {current} is the node following the \type {current} in the calling
+argument, and is only passed back as a convenience (or \type {nil}, if there is
+no such node). The returned \type {head} is more important, because if the
+function is called with \type {current} equal to \type {head}, it will be
+changed. When the third argument is passed, the node is freed.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {insert_before}}]
+
+\libindex {insert_before}
+
+\startfunctioncall
+<node> head, new = node.insert_before(<node> head, <node> current, <node> new)
+\stopfunctioncall
+
+This function inserts the node \type {new} before \type {current} into the list
+following \type {head}. It is your responsibility to make sure that \type
+{current} is really part of that list. The return values are the (potentially
+mutated) \type {head} and the node \type {new}, set up to be part of the list
+(with correct \type {next} field). If \type {head} is initially \type {nil}, it
+will become \type {new}.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {insert_after}}]
+
+\libindex {insert_after}
+
+\startfunctioncall
+<node> head, new = node.insert_after(<node> head, <node> current, <node> new)
+\stopfunctioncall
+
+This function inserts the node \type {new} after \type {current} into the list
+following \type {head}. It is your responsibility to make sure that \type
+{current} is really part of that list. The return values are the \type {head} and
+the node \type {new}, set up to be part of the list (with correct \type {next}
+field). If \type {head} is initially \type {nil}, it will become \type {new}.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {last_node}}]
+
+\libindex {last_node}
+
+\startfunctioncall
+<node> n = node.last_node()
+\stopfunctioncall
+
+This function pops the last node from \TEX's \quote{current list}. It returns
+that node, or \type {nil} if the current list is empty.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {traverse}}]
+
+\libindex {traverse}
+
+\startfunctioncall
+<node> t, id, subtype = node.traverse(<node> n)
+\stopfunctioncall
+
+This is a \LUA\ iterator that loops over the node list that starts at \type {n}.
+Typically code looks like this:
+
+\starttyping
+for n in node.traverse(head) do
+ ...
+end
+\stoptyping
+
+is functionally equivalent to:
+
+\starttyping
+do
+ local n
+ local function f (head,var)
+ local t
+ if var == nil then
+ t = head
+ else
+ t = var.next
+ end
+ return t
+ end
+ while true do
+ n = f (head, n)
+ if n == nil then break end
+ ...
+ end
+end
+\stoptyping
+
+It should be clear from the definition of the function \type {f} that even though
+it is possible to add or remove nodes from the node list while traversing, you
+have to take great care to make sure all the \type {next} (and \type {prev})
+pointers remain valid.
+
+If the above is unclear to you, see the section \quote {For Statement} in the
+\LUA\ Reference Manual.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {traverse_id}}]
+
+\libindex {traverse_id}
+
+\startfunctioncall
+<node> t, subtype = node.traverse_id(<number> id, <node> n)
+\stopfunctioncall
+
+This is an iterator that loops over all the nodes in the list that starts at
+\type {n} that have a matching \type {id} field.
+
+See the previous section for details. The change is in the local function \type
+{f}, which now does an extra while loop checking against the upvalue \type {id}:
+
+\starttyping
+ local function f(head,var)
+ local t
+ if var == nil then
+ t = head
+ else
+ t = var.next
+ end
+ while not t.id == id do
+ t = t.next
+ end
+ return t
+ end
+\stoptyping
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {traverse_char} and \type {traverse_glyph}}]
+
+\libindex {traverse_char}
+\libindex {traverse_glyph}
+
+The \type{traverse_char} iterator loops over the \nod {glyph} nodes in a list.
+Only nodes with a subtype less than 256 are seen.
+
+\startfunctioncall
+<node> n, font, char = node.traverse_char(<node> n)
+\stopfunctioncall
+
+The \type{traverse_glyph} iterator loops over a list and returns the list and
+filters all glyphs:
+
+\startfunctioncall
+<node> n, font, char = node.traverse_glyph(<node> n)
+\stopfunctioncall
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {traverse_list}}]
+
+\libindex {traverse_list}
+
+This iterator loops over the \nod {hlist} and \nod {vlist} nodes in a list.
+
+\startfunctioncall
+<node> n, id, subtype, list = node.traverse_list(<node> n)
+\stopfunctioncall
+
+The four return values can save some time compared to fetching these fields but
+in practice you seldom need them all. So consider it a (side effect of
+experimental) convenience.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {find_node}}]
+
+\libindex {find_node}
+
+This helper returns the location of the first match at or after node \type {n}:
+
+\startfunctioncall
+<node> n = node.traverse_list(<node> n, <integer> subtype)
+<node> n, subtype = node.traverse_list(<node> n)
+\stopfunctioncall
+
+\stopsubsubsection
+
+
+\stopsubsection
+
+\startsubsection[title={Glue handling}][library=node]
+
+\startsubsubsection[title={\type {setglue}}]
+
+\libindex {setglue}
+
+You can set the properties of a glue in one go. If you pass no values, the glue
+will become a zero glue.
+
+\startfunctioncall
+node.setglue(<node> n)
+node.setglue(<node> n,width,stretch,shrink,stretch_order,shrink_order)
+\stopfunctioncall
+
+When you pass values, only arguments that are numbers are assigned so
+
+\starttyping
+node.setglue(n,655360,false,65536)
+\stoptyping
+
+will only adapt the width and shrink.
+
+When a list node is passed, you set the glue, order and sign instead.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {getglue}}]
+
+\libindex {getglue}
+
+The next call will return 5 values or nothing when no glue is passed.
+
+\startfunctioncall
+<integer> width, <integer> stretch, <integer> shrink, <integer> stretch_order,
+ <integer> shrink_order = node.getglue(<node> n)
+\stopfunctioncall
+
+When the second argument is false, only the width is returned (this is consistent
+with \type {tex.get}).
+
+When a list node is passed, you get back the glue that is set, the order of that
+glue and the sign.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {is_zero_glue}}]
+
+\libindex {is_zero_glue}
+
+This function returns \type {true} when the width, stretch and shrink properties
+are zero.
+
+\startfunctioncall
+<boolean> isglue = node.is_zero_glue(<node> n)
+\stopfunctioncall
+
+\stopsubsubsection
+
+\stopsubsection
+
+\startsubsection[title={Attribute handling}][library=node]
+
+\startsubsubsection[title={Attributes}]
+
+\topicindex {attributes}
+
+Assignments to attributes registers result in assigning lists with set attributes
+to nodes and the implementation is non|-|trivial because the value that is
+attached to a node is essentially a (sorted) sparse array of key|-|value pairs.
+It is generally easiest to deal with attribute lists and attributes by using the
+dedicated functions in the \type {node} library.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {attribute_list} nodes}]
+
+\topicindex {nodes+attributes}
+
+An \type {attribute_list} item is used as a head pointer for a list of attribute
+items. It has only one user|-|visible field:
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{next} \NC node \NC pointer to the first attribute \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsubsection
+
+\startsubsubsection[title={\nod {attr} nodes}]
+
+A normal node's attribute field will point to an item of type \nod
+{attribute_list}, and the \type {next} field in that item will point to the first
+defined \quote {attribute} item, whose \type {next} will point to the second
+\quote {attribute} item, etc.
+
+\starttabulate[|l|l|p|]
+\DB field \BC type \BC explanation \NC \NR
+\TB
+\NC \type{next} \NC node \NC pointer to the next attribute \NC \NR
+\NC \type{number} \NC number \NC the attribute type id \NC \NR
+\NC \type{value} \NC number \NC the attribute value \NC \NR
+\LL
+\stoptabulate
+
+As mentioned it's better to use the official helpers rather than edit these
+fields directly. For instance the \type {prev} field is used for other purposes
+and there is no double linked list.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {current_attr}}]
+
+\libindex{current_attr}
+
+This returns the currently active list of attributes, if there is one.
+
+\startfunctioncall
+<node> m = node.current_attr()
+\stopfunctioncall
+
+The intended usage of \type {current_attr} is as follows:
+
+\starttyping
+local x1 = node.new("glyph")
+x1.attr = node.current_attr()
+local x2 = node.new("glyph")
+x2.attr = node.current_attr()
+\stoptyping
+
+or:
+
+\starttyping
+local x1 = node.new("glyph")
+local x2 = node.new("glyph")
+local ca = node.current_attr()
+x1.attr = ca
+x2.attr = ca
+\stoptyping
+
+The attribute lists are ref counted and the assignment takes care of incrementing
+the refcount. You cannot expect the value \type {ca} to be valid any more when
+you assign attributes (using \type {tex.setattribute}) or when control has been
+passed back to \TEX.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {has_attribute}}]
+
+\libindex {has_attribute}
+
+\startfunctioncall
+<number> v = node.has_attribute(<node> n, <number> id)
+<number> v = node.has_attribute(<node> n, <number> id, <number> val)
+\stopfunctioncall
+
+Tests if a node has the attribute with number \type {id} set. If \type {val} is
+also supplied, also tests if the value matches \type {val}. It returns the value,
+or, if no match is found, \type {nil}.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {get_attribute}}]
+
+\libindex {get_attribute}
+
+\startfunctioncall
+<number> v = node.get_attribute(<node> n, <number> id)
+\stopfunctioncall
+
+Tests if a node has an attribute with number \type {id} set. It returns the
+value, or, if no match is found, \type {nil}. If no \type {id} is given then the
+zero attributes is assumed.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {find_attribute}}]
+
+\libindex {find_attribute}
+
+\startfunctioncall
+<number> v, <node> n = node.find_attribute(<node> n, <number> id)
+\stopfunctioncall
+
+Finds the first node that has attribute with number \type {id} set. It returns
+the value and the node if there is a match and otherwise nothing.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {set_attribute}}]
+
+\libindex {set_attribute}
+
+\startfunctioncall
+node.set_attribute(<node> n, <number> id, <number> val)
+\stopfunctioncall
+
+Sets the attribute with number \type {id} to the value \type {val}. Duplicate
+assignments are ignored.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {unset_attribute}}]
+
+\libindex {unset_attribute}
+
+\startfunctioncall
+<number> v =
+ node.unset_attribute(<node> n, <number> id)
+<number> v =
+ node.unset_attribute(<node> n, <number> id, <number> val)
+\stopfunctioncall
+
+Unsets the attribute with number \type {id}. If \type {val} is also supplied, it
+will only perform this operation if the value matches \type {val}. Missing
+attributes or attribute|-|value pairs are ignored.
+
+If the attribute was actually deleted, returns its old value. Otherwise, returns
+\type {nil}.
+
+\stopsubsubsection
+
+\stopsubsection
+
+\startsubsection[title={Glyph handling}][library=node]
+
+\startsubsubsection[title={\type {first_glyph}}]
+
+\libindex {first_glyph}
+
+\startfunctioncall
+<node> n = node.first_glyph(<node> n)
+<node> n = node.first_glyph(<node> n, <node> m)
+\stopfunctioncall
+
+Returns the first node in the list starting at \type {n} that is a glyph node
+with a subtype indicating it is a glyph, or \type {nil}. If \type {m} is given,
+processing stops at (but including) that node, otherwise processing stops at the
+end of the list.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {is_char} and \type {is_glyph}}]
+
+\libindex {is_char}
+\libindex {is_glyph}
+
+The subtype of a glyph node signals if the glyph is already turned into a character reference
+or not.
+
+\startfunctioncall
+<boolean> b = node.is_char(<node> n)
+<boolean> b = node.is_glyph(<node> n)
+\stopfunctioncall
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {has_glyph}}]
+
+\libindex {has_glyph}
+
+This function returns the first glyph or disc node in the given list:
+
+\startfunctioncall
+<node> n = node.has_glyph(<node> n)
+\stopfunctioncall
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {ligaturing}}]
+
+\libindex {ligaturing}
+
+\startfunctioncall
+<node> h, <node> t, <boolean> success = node.ligaturing(<node> n)
+<node> h, <node> t, <boolean> success = node.ligaturing(<node> n, <node> m)
+\stopfunctioncall
+
+Apply \TEX-style ligaturing to the specified nodelist. The tail node \type {m} is
+optional. The two returned nodes \type {h} and \type {t} are the new head and
+tail (both \type {n} and \type {m} can change into a new ligature).
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {kerning}}]
+
+\libindex {kerning}
+
+\startfunctioncall
+<node> h, <node> t, <boolean> success = node.kerning(<node> n)
+<node> h, <node> t, <boolean> success = node.kerning(<node> n, <node> m)
+\stopfunctioncall
+
+Apply \TEX|-|style kerning to the specified node list. The tail node \type {m} is
+optional. The two returned nodes \type {h} and \type {t} are the head and tail
+(either one of these can be an inserted kern node, because special kernings with
+word boundaries are possible).
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {unprotect_glyph[s]}}]
+
+\libindex {unprotect_glyphs}
+\libindex {unprotect_glyph}
+
+\startfunctioncall
+node.unprotect_glyph(<node> n)
+node.unprotect_glyphs(<node> n,[<node> n])
+\stopfunctioncall
+
+Subtracts 256 from all glyph node subtypes. This and the next function are
+helpers to convert from \type {characters} to \type {glyphs} during node
+processing. The second argument is optional and indicates the end of a range.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {protect_glyph[s]}}]
+
+\libindex {protect_glyphs}
+\libindex {protect_glyph}
+
+\startfunctioncall
+node.protect_glyph(<node> n)
+node.protect_glyphs(<node> n,[<node> n])
+\stopfunctioncall
+
+Adds 256 to all glyph node subtypes in the node list starting at \type {n},
+except that if the value is 1, it adds only 255. The special handling of 1 means
+that \type {characters} will become \type {glyphs} after subtraction of 256. A
+single character can be marked by the singular call. The second argument is
+optional and indicates the end of a range.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {protrusion_skippable}}]
+
+\libindex {protrusion_skippable}
+
+\startfunctioncall
+<boolean> skippable = node.protrusion_skippable(<node> n)
+\stopfunctioncall
+
+Returns \type {true} if, for the purpose of line boundary discovery when
+character protrusion is active, this node can be skipped.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {check_discretionary}, \type {check_discretionaries}}]
+
+\libindex{check_discretionary}
+\libindex{check_discretionaries}
+
+When you fool around with disc nodes you need to be aware of the fact that they
+have a special internal data structure. As long as you reassign the fields when
+you have extended the lists it's ok because then the tail pointers get updated,
+but when you add to list without reassigning you might end up in trouble when
+the linebreak routine kicks in. You can call this function to check the list for
+issues with disc nodes.
+
+\startfunctioncall
+node.check_discretionary(<node> n)
+node.check_discretionaries(<node> head)
+\stopfunctioncall
+
+The plural variant runs over all disc nodes in a list, the singular variant
+checks one node only (it also checks if the node is a disc node).
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {flatten_discretionaries}}]
+
+\libindex {flatten_discretionaries}
+
+This function will remove the discretionaries in the list and inject the replace
+field when set.
+
+\startfunctioncall
+<node> head, count = node.flatten_discretionaries(<node> n)
+\stopfunctioncall
+
+\stopsubsubsection
+
+\stopsubsection
+
+\startsubsection[title={Packaging}][library=node]
+
+\startsubsubsection[title={\type {hpack}}]
+
+\libindex {hpack}
+
+This function creates a new hlist by packaging the list that begins at node \type
+{n} into a horizontal box. With only a single argument, this box is created using
+the natural width of its components. In the three argument form, \type {info}
+must be either \type {additional} or \type {exactly}, and \type {w} is the
+additional (\type {\hbox spread}) or exact (\type {\hbox to}) width to be used.
+The second return value is the badness of the generated box.
+
+\startfunctioncall
+<node> h, <number> b =
+ node.hpack(<node> n)
+<node> h, <number> b =
+ node.hpack(<node> n, <number> w, <string> info)
+<node> h, <number> b =
+ node.hpack(<node> n, <number> w, <string> info, <string> dir)
+\stopfunctioncall
+
+Caveat: there can be unexpected side|-|effects to this function, like updating
+some of the \prm {marks} and \type {\inserts}. Also note that the content of
+\type {h} is the original node list \type {n}: if you call \type {node.free(h)}
+you will also free the node list itself, unless you explicitly set the \type
+{list} field to \type {nil} beforehand. And in a similar way, calling \type
+{node.free(n)} will invalidate \type {h} as well!
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {vpack}}]
+
+\libindex {vpack}
+
+This function creates a new vlist by packaging the list that begins at node \type
+{n} into a vertical box. With only a single argument, this box is created using
+the natural height of its components. In the three argument form, \type {info}
+must be either \type {additional} or \type {exactly}, and \type {w} is the
+additional (\type {\vbox spread}) or exact (\type {\vbox to}) height to be used.
+
+\startfunctioncall
+<node> h, <number> b =
+ node.vpack(<node> n)
+<node> h, <number> b =
+ node.vpack(<node> n, <number> w, <string> info)
+<node> h, <number> b =
+ node.vpack(<node> n, <number> w, <string> info, <string> dir)
+\stopfunctioncall
+
+The second return value is the badness of the generated box. See the description
+of \type {hpack} for a few memory allocation caveats.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {prepend_prevdepth}}]
+
+\libindex {prepend_prevdepth}
+
+This function is somewhat special in the sense that it is an experimental helper
+that adds the interlinespace to a line keeping the baselineskip and lineskip into
+account.
+
+\startfunctioncall
+<node> n, <number> delta =
+ node.prepend_prevdepth(<node> n,<number> prevdepth)
+\stopfunctioncall
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {dimensions}, \type {rangedimensions}, \type {naturalwidth}}]
+
+\libindex{dimensions}
+\libindex{rangedimensions}
+
+\startfunctioncall
+<number> w, <number> h, <number> d =
+ node.dimensions(<node> n)
+<number> w, <number> h, <number> d =
+ node.dimensions(<node> n, <node> t)
+\stopfunctioncall
+
+This function calculates the natural in|-|line dimensions of the node list starting
+at node \type {n} and terminating just before node \type {t} (or the end of the
+list, if there is no second argument). The return values are scaled points. An
+alternative format that starts with glue parameters as the first three arguments
+is also possible:
+
+\startfunctioncall
+<number> w, <number> h, <number> d =
+ node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,
+ <node> n)
+<number> w, <number> h, <number> d =
+ node.dimensions(<number> glue_set, <number> glue_sign, <number> glue_order,
+ <node> n, <node> t)
+\stopfunctioncall
+
+This calling method takes glue settings into account and is especially useful for
+finding the actual width of a sublist of nodes that are already boxed, for
+example in code like this, which prints the width of the space in between the
+\type {a} and \type {b} as it would be if \type {\box0} was used as-is:
+
+\starttyping
+\setbox0 = \hbox to 20pt {a b}
+
+\directlua{print (node.dimensions(
+ tex.box[0].glue_set,
+ tex.box[0].glue_sign,
+ tex.box[0].glue_order,
+ tex.box[0].head.next,
+ node.tail(tex.box[0].head)
+)) }
+\stoptyping
+
+You need to keep in mind that this is one of the few places in \TEX\ where floats
+are used, which means that you can get small differences in rounding when you
+compare the width reported by \type {hpack} with \type {dimensions}.
+
+The second alternative saves a few lookups and can be more convenient in some
+cases:
+
+\startfunctioncall
+<number> w, <number> h, <number> d =
+ node.rangedimensions(<node> parent, <node> first)
+<number> w, <number> h, <number> d =
+ node.rangedimensions(<node> parent, <node> first, <node> last)
+\stopfunctioncall
+
+A simple and somewhat more efficient variant is this:
+
+\startfunctioncall
+<number> w =
+ node.naturalwidth(<node> start, <node> stop)
+\stopfunctioncall
+
+\stopsubsubsection
+
+\stopsubsection
+
+\startsubsection[title={Math}][library=node]
+
+\startsubsubsection[title={\type {mlist_to_hlist}}]
+
+\libindex {mlist_to_hlist}
+
+\startfunctioncall
+<node> h =
+ node.mlist_to_hlist(<node> n, <string> display_type, <boolean> penalties)
+\stopfunctioncall
+
+This runs the internal mlist to hlist conversion, converting the math list in
+\type {n} into the horizontal list \type {h}. The interface is exactly the same
+as for the callback \cbk {mlist_to_hlist}.
+
+\stopsubsubsection
+
+\startsubsubsection[title={\type {end_of_math}}]
+
+\libindex {end_of_math}
+
+\startfunctioncall
+<node> t = node.end_of_math(<node> start)
+\stopfunctioncall
+
+Looks for and returns the next \type {math_node} following the \type {start}. If
+the given node is a math end node this helper returns that node, else it follows
+the list and returns the next math endnote. If no such node is found nil is
+returned.
+
+\stopsubsubsection
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Two access models}][library=node]
+
+\topicindex{nodes+direct}
+\topicindex{direct nodes}
+
+\libindex {todirect}
+\libindex {tonode}
+\libindex {tostring}
+
+Deep down in \TEX\ a node has a number which is a numeric entry in a memory
+table. In fact, this model, where \TEX\ manages memory is real fast and one of
+the reasons why plugging in callbacks that operate on nodes is quite fast too.
+Each node gets a number that is in fact an index in the memory table and that
+number often is reported when you print node related information. You go from
+userdata nodes and there numeric references and back with:
+
+\startfunctioncall
+<integer> d = node.todirect(<node> n))
+<node> n = node.tonode(<integer> d))
+\stopfunctioncall
+
+The userdata model is rather robust as it is a virtual interface with some
+additional checking while the more direct access which uses the node numbers
+directly. However, even with userdata you can get into troubles when you free
+nodes that are no longer allocated or mess up lists. if you apply \type
+{tostring} to a node you see its internal (direct) number and id.
+
+The first model provides key based access while the second always accesses fields
+via functions:
+
+\starttyping
+nodeobject.char
+getfield(nodenumber,"char")
+\stoptyping
+
+If you use the direct model, even if you know that you deal with numbers, you
+should not depend on that property but treat it as an abstraction just like
+traditional nodes. In fact, the fact that we use a simple basic datatype has the
+penalty that less checking can be done, but less checking is also the reason why
+it's somewhat faster. An important aspect is that one cannot mix both methods,
+but you can cast both models. So, multiplying a node number makes no sense.
+
+So our advice is: use the indexed (table) approach when possible and investigate
+the direct one when speed might be a real issue. For that reason \LUATEX\ also
+provide the \type {get*} and \type {set*} functions in the top level node
+namespace. There is a limited set of getters. When implementing this direct
+approach the regular index by key variant was also optimized, so direct access
+only makes sense when nodes are accessed millions of times (which happens in some
+font processing for instance).
+
+We're talking mostly of getters because setters are less important. Documents
+have not that many content related nodes and setting many thousands of properties
+is hardly a burden contrary to millions of consultations.
+
+Normally you will access nodes like this:
+
+\starttyping
+local next = current.next
+if next then
+ -- do something
+end
+\stoptyping
+
+Here \type {next} is not a real field, but a virtual one. Accessing it results in
+a metatable method being called. In practice it boils down to looking up the node
+type and based on the node type checking for the field name. In a worst case you
+have a node type that sits at the end of the lookup list and a field that is last
+in the lookup chain. However, in successive versions of \LUATEX\ these lookups
+have been optimized and the most frequently accessed nodes and fields have a
+higher priority.
+
+Because in practice the \type {next} accessor results in a function call, there
+is some overhead involved. The next code does the same and performs a tiny bit
+faster (but not that much because it is still a function call but one that knows
+what to look up).
+
+\starttyping
+local next = node.next(current)
+if next then
+ -- do something
+end
+\stoptyping
+
+In the direct namespace there are more helpers and most of them are accompanied
+by setters. The getters and setters are clever enough to see what node is meant.
+We don't deal with whatsit nodes: their fields are always accessed by name. It
+doesn't make sense to add getters for all fields, we just identifier the most
+likely candidates. In complex documents, many node and fields types never get
+seen, or seen only a few times, but for instance glyphs are candidates for such
+optimization. The \type {node.direct} interface has some more helpers. \footnote
+{We can define the helpers in the node namespace with \type {getfield} which is
+about as efficient, so at some point we might provide that as module.}
+
+The \type {setdisc} helper takes three (optional) arguments plus an optional
+fourth indicating the subtype. Its \type {getdisc} takes an optional boolean;
+when its value is \type {true} the tail nodes will also be returned. The \type
+{setfont} helper takes an optional second argument, it being the character. The
+directmode setter \type {setlink} takes a list of nodes and will link them,
+thereby ignoring \type {nil} entries. The first valid node is returned (beware:
+for good reason it assumes single nodes). For rarely used fields no helpers are
+provided and there are a few that probably are used seldom too but were added for
+consistency. You can of course always define additional accessors using \type
+{getfield} and \type {setfield} with little overhead. When the second argument of
+\type {setattributelist} is \type {true} the current attribute list is assumed.
+
+In \CONTEXT\ some of the not performance|-|critical userdata variants are
+emulated in \LUA\ and not in the engine, so we retain downward compatibility.
+
+\def\yes{$+$} \def\nop{$-$}
+
+\def\supported#1#2#3#4%
+ {\NC \type{#1}
+ \NC \ifx#2\yes\lix{node} {#1}\fi #2
+ \NC \ifx#3\yes\lix{node.direct}{#1}\fi #3 \NC
+ \NC #4 \NC
+ \NR}
+
+\starttabulate[|l|c|c|]
+\DB function \BC node \BC direct \NC emulated \NC \NR
+\TB
+\supported {check_discretionaries} \nop \yes \yes
+\supported {check_discretionary} \nop \yes \yes
+\supported {copy} \yes \yes \relax
+\supported {copy_list} \yes \yes \relax
+%supported {copy_only} \nop \yes \relax
+\supported {count} \nop \yes \yes
+\supported {current_attr} \yes \yes \relax
+\supported {dimensions} \nop \yes \yes
+\supported {effective_glue} \nop \yes \yes
+\supported {end_of_math} \nop \yes \yes
+\supported {find_attribute} \nop \yes \yes
+%supported {find_attribute_range} \nop \yes \relax
+%supported {find_node} \nop \yes \relax
+\supported {first_glyph} \nop \yes \yes
+\supported {flatten_discretionaries} \nop \yes \yes
+%supported {flush_components} \nop \yes \relax
+\supported {flush_list} \yes \yes \relax
+\supported {flush_node} \yes \yes \relax
+\supported {free} \yes \yes \relax
+\supported {get_attribute} \yes \yes \relax
+\supported {get_properties_table} \yes \yes \relax
+\supported {get_synctex_fields} \nop \yes \relax
+\supported {getattributelist} \nop \yes \relax
+\supported {getboth} \nop \yes \relax
+\supported {getbox} \nop \yes \relax
+\supported {getchar} \nop \yes \relax
+\supported {getcomponents} \nop \yes \relax
+\supported {getdata} \nop \yes \relax
+\supported {getdepth} \nop \yes \relax
+\supported {getdirection} \nop \yes \relax
+\supported {getdisc} \nop \yes \relax
+\supported {getexpansion} \nop \yes \relax
+\supported {getfam} \nop \yes \relax
+\supported {getfield} \yes \yes \relax
+\supported {getfont} \nop \yes \relax
+\supported {getglue} \nop \yes \yes
+%supported {getglyphdata} \nop \yes \relax % experiment
+\supported {getheight} \nop \yes \relax
+\supported {getid} \nop \yes \relax
+\supported {getkern} \nop \yes \relax
+\supported {getlang} \nop \yes \relax
+\supported {getleader} \nop \yes \relax
+\supported {getlist} \nop \yes \relax
+\supported {getnext} \nop \yes \relax
+\supported {getnormalizedline} \nop \yes \relax
+\supported {getnucleus} \nop \yes \relax
+\supported {getoffsets} \nop \yes \relax
+\supported {getorientation} \nop \yes \relax
+\supported {getpenalty} \nop \yes \relax
+\supported {getpost} \nop \yes \relax
+\supported {getpre} \nop \yes \relax
+\supported {getprev} \nop \yes \relax
+\supported {getproperty} \yes \yes \relax
+\supported {getreplace} \nop \yes \relax
+\supported {getshift} \nop \yes \relax
+\supported {getsub} \nop \yes \relax
+\supported {getsubtype} \nop \yes \relax
+\supported {getsup} \nop \yes \relax
+\supported {getwhd} \nop \yes \relax
+\supported {getwidth} \nop \yes \relax
+\supported {has_attribute} \yes \yes \relax
+\supported {has_dimensions} \nop \yes \relax
+\supported {has_field} \yes \yes \relax
+\supported {has_glyph} \nop \yes \yes
+\supported {hpack} \nop \yes \yes
+%supported {ignore_math_skip} \nop \yes \relax
+\supported {insert_after} \yes \yes \relax
+\supported {insert_before} \yes \yes \relax
+\supported {is_char} \nop \yes \relax
+\supported {is_direct} \nop \yes \relax
+\supported {is_glyph} \nop \yes \relax
+\supported {is_node} \yes \yes \relax
+\supported {is_valid} \nop \yes \relax
+\supported {is_zero_glue} \nop \yes \yes
+\supported {kerning} \nop \yes \yes
+\supported {last_node} \nop \yes \yes
+\supported {length} \nop \yes \yes
+\supported {ligaturing} \nop \yes \yes
+\supported {make_extensible} \nop \yes \yes
+\supported {mlist_to_hlist} \nop \yes \yes
+\supported {naturalwidth} \nop \yes \yes
+\supported {new} \yes \yes \relax
+\supported {prepend_prevdepth} \nop \yes \yes
+\supported {protect_glyphs} \nop \yes \yes
+\supported {protect_glyph} \nop \yes \yes
+\supported {protrusion_skippable} \nop \yes \yes
+\supported {rangedimensions} \nop \yes \yes
+\supported {remove} \yes \yes \relax
+\supported {set_attribute} \yes \yes \relax
+\supported {set_synctex_fields} \nop \yes \relax
+\supported {setattributelist} \nop \yes \relax
+\supported {setboth} \nop \yes \relax
+\supported {setbox} \nop \yes \relax
+\supported {setchar} \nop \yes \relax
+\supported {setcomponents} \nop \yes \relax
+\supported {setdata} \nop \yes \relax
+\supported {setdepth} \nop \yes \relax
+\supported {setdirection} \nop \yes \relax
+\supported {setdisc} \nop \yes \relax
+\supported {setexpansion} \nop \yes \relax
+\supported {setfam} \nop \yes \relax
+\supported {setfield} \yes \yes \relax
+\supported {setfont} \nop \yes \relax
+\supported {setglue} \yes \yes \relax
+%supported {setglyphdata} \nop \yes \relax % experiment
+\supported {setheight} \nop \yes \relax
+\supported {setkern} \nop \yes \relax
+\supported {setlang} \nop \yes \relax
+\supported {setleader} \nop \yes \relax
+\supported {setlink} \nop \yes \relax
+\supported {setlist} \nop \yes \relax
+\supported {setnext} \nop \yes \relax
+\supported {setnucleus} \nop \yes \relax
+\supported {setoffsets} \nop \yes \relax
+\supported {setorientation} \nop \yes \relax
+\supported {setpenalty} \nop \yes \relax
+\supported {setprev} \nop \yes \relax
+\supported {setproperty} \yes \yes \relax
+\supported {setshift} \nop \yes \relax
+\supported {setsplit} \nop \yes \relax
+\supported {setsub} \nop \yes \relax
+\supported {setsubtype} \nop \yes \relax
+\supported {setsup} \nop \yes \relax
+\supported {setwhd} \nop \yes \relax
+\supported {setwidth} \nop \yes \relax
+\supported {slide} \nop \yes \yes
+\supported {start_of_par} \nop \yes \relax
+\supported {subtype} \nop \nop \relax
+\supported {tail} \yes \yes \relax
+\supported {todirect} \nop \yes \relax
+\supported {tonode} \nop \yes \relax
+\supported {tostring} \yes \nop \relax
+\supported {traverse} \yes \yes \relax
+\supported {traverse_char} \yes \yes \relax
+\supported {traverse_glyph} \yes \yes \relax
+\supported {traverse_id} \yes \yes \relax
+\supported {traverse_list} \yes \yes \relax
+\supported {type} \yes \nop \relax
+\supported {unprotect_glyphs} \nop \yes \yes
+\supported {unprotect_glyph} \nop \yes \yes
+\supported {unset_attribute} \yes \yes \relax
+\supported {usedlist} \nop \yes \yes
+\supported {uses_font} \nop \yes \yes
+\supported {vpack} \nop \yes \yes
+\supported {write} \yes \yes \relax
+\LL
+\stoptabulate
+
+The \type {node.next} and \type {node.prev} functions will stay but for
+consistency there are variants called \type {getnext} and \type {getprev}. We had
+to use \type {get} because \type {node.id} and \type {node.subtype} are already
+taken for providing meta information about nodes. Note: The getters do only basic
+checking for valid keys. You should just stick to the keys mentioned in the
+sections that describe node properties.
+
+Some of the getters and setters handle multiple node types, given that the field
+is relevant. In that case, some field names are considered similar (like \type
+{kern} and \type {width}, or \type {data} and \type {value}. In retrospect we
+could have normalized field names better but we decided to stick to the original
+(internal) names as much as possible. After all, at the \LUA\ end one can easily
+create synonyms.
+
+Some nodes have indirect references. For instance a math character refers to a
+family instead of a font. In that case we provide a virtual font field as
+accessor. So, \type {getfont} and \type {.font} can be used on them. The same is
+true for the \type {width}, \type {height} and \type {depth} of glue nodes. These
+actually access the spec node properties, and here we can set as well as get the
+values.
+
+In some places \LUATEX\ can do a bit of extra checking for valid node lists and
+you can enable that with:
+
+\startfunctioncall
+node.fix_node_lists(<boolean> b)
+\stopfunctioncall
+
+You can set and query the \SYNCTEX\ fields, a file number aka tag and a line
+number, for a glue, kern, hlist, vlist, rule and math nodes as well as glyph
+nodes (although this last one is not used in native \SYNCTEX).
+
+\startfunctioncall
+node.set_synctex_fields(<integer> f, <integer> l)
+<integer> f, <integer> l =
+ node.get_synctex_fields(<node> n)
+\stopfunctioncall
+
+Of course you need to know what you're doing as no checking on sane values takes
+place. Also, the synctex interpreter used in editors is rather peculiar and has
+some assumptions (heuristics).
+
+\stopsection
+
+\startsection[title={Normalization}][library=node]
+
+As an experiment the lines resulting from paragraph construction can be normalized.
+There are several modes, that can be set and queried with:
+
+\startfunctioncall
+node.direct.setnormalize(<integer> n)
+<integer> n = node.direct.getnormalize()
+\stopfunctioncall
+
+The state of a line (a hlist) can be queried with:
+
+\startfunctioncall
+<integer> leftskip, <integer> rightskip,
+ <integer> lefthangskip, <integer> righthangskip,
+ <node> head, <node> tail,
+ <integer> parindent, <integer> parfillskip = node.direct.getnormalized()
+\stopfunctioncall
+
+The modes accumulate, so mode \type {4} includes \type {1} upto \type {3}:
+
+\starttabulate[|l|p|]
+\DB value \BC explanation \NC \NR
+\TB
+\NC \type{1} \NC left and right skips and directions \NC \NR
+\NC \type{2} \NC indentation and parfill skip \NC \NR
+\NC \type{3} \NC hanging indentation and par shapes \NC \NR
+\NC \type{4} \NC idem but before left and right skips \NC \NR
+\NC \type{5} \NC inject compensation for overflow \NC \NR
+\LL
+\stoptabulate
+
+This is experimental code and might take a while to become frozen.
+
+\stopsection
+
+\startsection[title={Properties}][library=node]
+
+\topicindex {nodes+properties}
+\topicindex {properties}
+
+\libindex{flush_properties_table}
+\libindex{get_properties_table}
+\libindex{set_properties_mode}
+
+Attributes are a convenient way to relate extra information to a node. You can
+assign them at the \TEX\ end as well as at the \LUA\ end and and consult them at
+the \LUA\ end. One big advantage is that they obey grouping. They are linked
+lists and normally checking for them is pretty efficient, even if you use a lot
+of them. A macro package has to provide some way to manage these attributes at the
+\TEX\ end because otherwise clashes in their usage can occur.
+
+Each node also can have a properties table and you can assign values to this
+table using the \type {setproperty} function and get properties using the \type
+{getproperty} function. Managing properties is way more demanding than managing
+attributes.
+
+Take the following example:
+
+\starttyping
+\directlua {
+ local n = node.new("glyph")
+
+ node.setproperty(n,"foo")
+ print(node.getproperty(n))
+
+ node.setproperty(n,"bar")
+ print(node.getproperty(n))
+
+ node.free(n)
+}
+\stoptyping
+
+This will print \type {foo} and \type {bar} which in itself is not that useful
+when multiple mechanisms want to use this feature. A variant is:
+
+\starttyping
+\directlua {
+ local n = node.new("glyph")
+
+ node.setproperty(n,{ one = "foo", two = "bar" })
+ print(node.getproperty(n).one)
+ print(node.getproperty(n).two)
+
+ node.free(n)
+}
+\stoptyping
+
+This time we store two properties with the node. It really makes sense to have a
+table as property because that way we can store more. But in order for that to
+work well you need to do it this way:
+
+\starttyping
+\directlua {
+ local n = node.new("glyph")
+
+ local t = node.getproperty(n)
+
+ if not t then
+ t = { }
+ node.setproperty(n,t)
+ end
+ t.one = "foo"
+ t.two = "bar"
+
+ print(node.getproperty(n).one)
+ print(node.getproperty(n).two)
+
+ node.free(n)
+}
+\stoptyping
+
+Here our own properties will not overwrite other users properties unless of
+course they use the same keys. So, eventually you will end up with something:
+
+\starttyping
+\directlua {
+ local n = node.new("glyph")
+
+ local t = node.getproperty(n)
+
+ if not t then
+ t = { }
+ node.setproperty(n,t)
+ end
+ t.myself = { one = "foo", two = "bar" }
+
+ print(node.getproperty(n).myself.one)
+ print(node.getproperty(n).myself.two)
+
+ node.free(n)
+}
+\stoptyping
+
+This assumes that only you use \type {myself} as subtable. The possibilities are
+endless but care is needed. For instance, the generic font handler that ships
+with \CONTEXT\ uses the \type {injections} subtable and you should not mess with
+that one!
+
+There are a few helper functions that you normally should not touch as user: \typ
+{flush_properties_table} will wipe the table (normally a bad idea), \typ
+{get_properties_table} and will give the table that stores properties (using
+direct entries) and you can best not mess too much with that one either because
+\LUATEX\ itself will make sure that entries related to nodes will get wiped when
+nodes get freed, so that the \LUA\ garbage collector can do its job. In fact, the
+main reason why we have this mechanism is that it saves the user (or macro
+package) some work. One can easily write a property mechanism in \LUA\ where
+after a shipout properties gets cleaned up but it's not entirely trivial to make
+sure that with each freed node also its properties get freed, due to the fact
+that there can be nodes left over for a next page. And having a callback bound to
+the node deallocator would add way to much overhead.
+
+When we copy a node list that has a table as property, there are several
+possibilities: we do the same as a new node, we copy the entry to the table in
+properties (a reference), we do a deep copy of a table in the properties, we
+create a new table and give it the original one as a metatable. After some
+experiments (that also included timing) with these scenarios we decided that a
+deep copy made no sense, nor did nilling. In the end both the shallow copy and
+the metatable variant were both ok, although the second one is slower. The most
+important aspect to keep in mind is that references to other nodes in properties
+no longer can be valid for that copy. We could use two tables (one unique and one
+shared) or metatables but that only complicates matters.
+
+When defining a new node, we could already allocate a table but it is rather easy
+to do that at the lua end e.g.\ using a metatable \type {__index} method. That
+way it is under macro package control. When deleting a node, we could keep the
+slot (e.g. setting it to false) but it could make memory consumption raise
+unneeded when we have temporary large node lists and after that only small lists.
+Both are not done.
+
+So in the end this is what happens now: when a node is copied, and it has a table
+as property, the new node will share that table. If the second argument of \typ
+{set_properties_mode} is \type {true} then a metatable approach is chosen: the
+copy gets its own table with the original table as metatable. If you use the
+generic font loader the mode is enabled that way.
+
+A few more xperiments were done. For instance: copy attributes to the properties
+so that we have fast access at the \LUA\ end. In the end the overhead is not
+compensated by speed and convenience, in fact, attributes are not that slow when
+it comes to accessing them. So this was rejected.
+
+Another experiment concerned a bitset in the node but again the gain compared to
+attributes was neglectable and given the small amount of available bits it also
+demands a pretty strong agreement over what bit represents what, and this is
+unlikely to succeed in the \TEX\ community. It doesn't pay off.
+
+Just in case one wonders why properties make sense: it is not so much speed that
+we gain, but more convenience: storing all kind of (temporary) data in attributes
+is no fun and this mechanism makes sure that properties are cleaned up when a
+node is freed. Also, the advantage of a more or less global properties table is
+that we stay at the \LUA\ end. An alternative is to store a reference in the node
+itself but that is complicated by the fact that the register has some limitations
+(no numeric keys) and we also don't want to mess with it too much.
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-pdf.tex b/doc/context/sources/general/manuals/luametatex/luametatex-pdf.tex
new file mode 100644
index 000000000..3e2150053
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-pdf.tex
@@ -0,0 +1,565 @@
+% language=uk
+
+% lua.newtable
+
+\environment luametatex-style
+
+\startcomponent luametatex-pdf
+
+\startchapter[reference=pdf,title={The \PDF\ related libraries}]
+
+\startsection[title={The \type {pdfe} library}][library=pdfe]
+
+\startsubsection[title={Introduction}]
+
+\topicindex{\PDF+objects}
+
+\topicindex{\PDF+analyze}
+\topicindex{\PDF+\type{pdfe}}
+
+The \type {pdfe} library replaces the \type {epdf} library and provides an
+interface to \PDF\ files. It uses the same code as is used for \PDF\ image
+inclusion. The \type {pplib} library by Paweł Jackowski replaces the \type
+{poppler} (derived from \type {xpdf}) library.
+
+A \PDF\ file is basically a tree of objects and one descends into the tree via
+dictionaries (key/value) and arrays (index/value). There are a few topmost
+dictionaries that start at root that are accessed more directly.
+
+Although everything in \PDF\ is basically an object we only wrap a few in so
+called userdata \LUA\ objects.
+
+\starttabulate[|l|l|]
+\DB type \BC mapping \NC \NR
+\TB
+\BC \PDF \BC \LUA \NC \NR
+\NC null \NC nil \NC \NR
+\NC boolean \NC boolean \NC \NR
+\NC integer \NC integer \NC \NR
+\NC float \NC number \NC \NR
+\NC name \NC string \NC \NR
+\NC string \NC string \NC \NR
+\NC array \NC array userdatum \NC \NR
+\NC dictionary \NC dictionary userdatum \NC \NR
+\NC stream \NC stream userdatum (with related dictionary) \NC \NR
+\NC reference \NC reference userdatum \NC \NR
+\LL
+\stoptabulate
+
+The regular getters return these \LUA\ data types but one can also get more
+detailed information.
+
+\stopsubsection
+
+\startsubsection[title={\type {open}, \type {openfile}, \type {new}, \type {getstatus}, \type {close}, \type {unencrypt}}]
+
+\libindex {open}
+\libindex {new}
+\libindex {new}
+\libindex {getstatus}
+\libindex {close}
+\libindex {unencrypt}
+
+A document is loaded from a file (by name or handle) or string:
+
+\starttyping
+<pdfe document> = pdfe.open(filename)
+<pdfe document> = pdfe.openfile(filehandle)
+<pdfe document> = pdfe.new(somestring,somelength)
+\stoptyping
+
+Such a document is closed with:
+
+\starttyping
+pdfe.close(<pdfe document>)
+\stoptyping
+
+You can check if a document opened well by:
+
+\starttyping
+pdfe.getstatus(<pdfe document>)
+\stoptyping
+
+The returned codes are:
+
+\starttabulate[|c|l|]
+\DB value \BC explanation \NC \NR
+\TB
+\NC \type {-2} \NC the document failed to open \NC \NR
+\NC \type {-1} \NC the document is (still) protected \NC \NR
+\NC \type {0} \NC the document is not encrypted \NC \NR
+\NC \type {2} \NC the document has been unencrypted \NC \NR
+\LL
+\stoptabulate
+
+An encrypted document can be unencrypted by the next command where instead of
+either password you can give \type {nil}:
+
+\starttyping
+pdfe.unencrypt(<pdfe document>,userpassword,ownerpassword)
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\type {getsize}, \type {getversion}, \type {getnofobjects}, \type {getnofpages}}]
+
+\libindex {getsize}
+\libindex {getversion}
+\libindex {getnofobjects}
+\libindex {getnofpages}
+
+A successfully opened document can provide some information:
+
+\starttyping
+bytes = getsize(<pdfe document>)
+major, minor = getversion(<pdfe document>)
+n = getnofobjects(<pdfe document>)
+n = getnofpages(<pdfe document>)
+bytes, waste = getnofpages(<pdfe document>)
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\type {get[catalog|trailer|info]}}]
+
+\libindex {getcatalog}
+\libindex {gettrailer}
+\libindex {getinfo}
+
+For accessing the document structure you start with the so called catalog, a
+dictionary:
+
+\starttyping
+<pdfe dictionary> = pdfe.getcatalog(<pdfe document>)
+\stoptyping
+
+The other two root dictionaries are accessed with:
+
+\starttyping
+<pdfe dictionary> = pdfe.gettrailer(<pdfe document>)
+<pdfe dictionary> = pdfe.getinfo(<pdfe document>)
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\type {getpage}, \type {getbox}}]
+
+\libindex {getpage}
+\libindex {getbox}
+
+A specific page can conveniently be reached with the next command, which
+returns a dictionary. The first argument is to be a page dictionary.
+
+\starttyping
+<pdfe dictionary> = pdfe.getpage(<pdfe document>,pagenumber)
+\stoptyping
+
+Another convenience command gives you the (bounding) box of a (normally page)
+which can be inherited from the document itself. An example of a valid box name
+is \type {MediaBox}.
+
+\starttyping
+pages = pdfe.getbox(<pdfe dictionary>,boxname)
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\type {get[string|integer|number|boolean|name]}}]
+
+\libindex {getstring}
+\libindex {getinteger}
+\libindex {getnumber}
+\libindex {getboolean}
+\libindex {getname}
+
+Common values in dictionaries and arrays are strings, integers, floats, booleans
+and names (which are also strings) and these are also normal \LUA\ objects:
+
+\starttyping
+s = getstring (<pdfe array|dictionary>,index|key)
+i = getinteger(<pdfe array|dictionary>,index|key)
+n = getnumber (<pdfe array|dictionary>,index|key)
+b = getboolean(<pdfe array|dictionary>,index|key)
+n = getname (<pdfe array|dictionary>,index|key)
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\type {get[from][dictionary|array|stream]}}]
+
+\libindex {getdictionary} \libindex {getfromdictionary}
+\libindex {getarray} \libindex {getfromarray}
+\libindex {getstream} \libindex {getfromstream}
+
+Normally you will use an index in an array and key in a dictionary but dictionaries
+also accept an index. The size of an array or dictionary is available with the
+usual \type {#} operator.
+
+\starttyping
+<pdfe dictionary> = getdictionary(<pdfe array|dictionary>,index|key)
+<pdfe array> = getarray (<pdfe array|dictionary>,index|key)
+<pdfe stream>,
+<pdfe dictionary> = getstream (<pdfe array|dictionary>,index|key)
+\stoptyping
+
+These commands return dictionaries, arrays and streams, which are dictionaries
+with a blob of data attached.
+
+Before we come to an alternative access mode, we mention that the objects provide
+access in a different way too, for instance this is valid:
+
+\starttyping
+print(pdfe.open("foo.pdf").Catalog.Type)
+\stoptyping
+
+At the topmost level there are \type {Catalog}, \type {Info}, \type {Trailer}
+and \type {Pages}, so this is also okay:
+
+\starttyping
+print(pdfe.open("foo.pdf").Pages[1])
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\type {[open|close|readfrom|whole|]stream}}]
+
+\libindex {openstream}
+\libindex {closestream}
+\libindex {readfromstream}
+\libindex {readfromwholestream}
+
+Streams are sort of special. When your index or key hits a stream you get back a
+stream object and dictionary object. The dictionary you can access in the usual
+way and for the stream there are the following methods:
+
+\starttyping
+okay = openstream(<pdfe stream>,[decode])
+ closestream(<pdfe stream>)
+str, n = readfromstream(<pdfe stream>)
+str, n = readwholestream(<pdfe stream>,[decode])
+\stoptyping
+
+You either read in chunks, or you ask for the whole. When reading in chunks, you
+need to open and close the stream yourself. The \type {n} value indicates the
+length read. The \type {decode} parameter controls if the stream data gets
+uncompressed.
+
+As with dictionaries, you can access fields in a stream dictionary in the usual
+\LUA\ way too. You get the content when you \quote {call} the stream. You can
+pass a boolean that indicates if the stream has to be decompressed.
+
+% pdfe.objectcodes = objectcodes
+% pdfe.stringcodes = stringcodes
+% pdfe.encryptioncodes = encryptioncodes
+
+\stopsubsection
+
+\startsubsection[title={\type {getfrom[dictionary|array]}}]
+
+\libindex {getfromdictionary}
+\libindex {getfromarray}
+
+In addition to the interface described before, there is also a bit lower level
+interface available.
+
+\starttyping
+key, type, value, detail = getfromdictionary(<pdfe dictionary>,index)
+type, value, detail = getfromarray(<pdfe array>,index)
+\stoptyping
+
+\starttabulate[|c|l|l|l|]
+\DB type \BC meaning \BC value \BC detail \NC \NR
+\NC \type {0} \NC none \NC nil \NC \NC \NR
+\NC \type {1} \NC null \NC nil \NC \NC \NR
+\NC \type {2} \NC boolean \NC boolean \NC \NC \NR
+\NC \type {3} \NC integer \NC integer \NC \NC \NR
+\NC \type {4} \NC number \NC float \NC \NC \NR
+\NC \type {5} \NC name \NC string \NC \NC \NR
+\NC \type {6} \NC string \NC string \NC hex \NC \NR
+\NC \type {7} \NC array \NC arrayobject \NC size \NC \NR
+\NC \type {8} \NC dictionary \NC dictionaryobject \NC size \NC \NR
+\NC \type {9} \NC stream \NC streamobject \NC dictionary size \NC \NR
+\NC \type {10} \NC reference \NC integer \NC \NC \NR
+\LL
+\stoptabulate
+
+A \type {hex} string is (in the \PDF\ file) surrounded by \type {<>} while plain
+strings are bounded by \type {<>}.
+
+\stopsubsection
+
+\startsubsection[title={\type {[dictionary|array]totable}}]
+
+\libindex {dictionarytotable}
+\libindex {arraytotable}
+
+All entries in a dictionary or table can be fetched with the following commands
+where the return values are a hashed or indexed table.
+
+\starttyping
+hash = dictionarytotable(<pdfe dictionary>)
+list = arraytotable(<pdfe array>)
+\stoptyping
+
+You can get a list of pages with:
+
+\starttyping
+{ { <pdfe dictionary>, size, objnum }, ... } = pagestotable(<pdfe document>)
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\type {getfromreference}}]
+
+\libindex {getfromreference}
+
+Because you can have unresolved references, a reference object can be resolved
+with:
+
+\starttyping
+type, <pdfe dictionary|array|stream>, detail = getfromreference(<pdfe reference>)
+\stoptyping
+
+So, as second value you get back a new \type {pdfe} userdata object that you can
+query.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={Memory streams}][library=pdfe]
+
+\topicindex{\PDF+memory streams}
+
+\libindex {new}
+
+The \type {pdfe.new} that takes three arguments:
+
+\starttabulate
+\DB value \BC explanation \NC \NR
+\TB
+\NC \type {stream} \NC this is a (in low level \LUA\ speak) light userdata
+ object, i.e.\ a pointer to a sequence of bytes \NC \NR
+\NC \type {length} \NC this is the length of the stream in bytes (the stream can
+ have embedded zeros) \NC \NR
+\NC \type {name} \NC optional, this is a unique identifier that is used for
+ hashing the stream, so that multiple doesn't use more
+ memory \NC \NR
+\LL
+\stoptabulate
+
+The third argument is optional. When it is not given the function will return an
+\type {pdfe} document object as with a regular file, otherwise it will return a
+filename that can be used elsewhere (e.g.\ in the image library) to reference the
+stream as pseudo file.
+
+Instead of a light userdata stream (which is actually fragile but handy when you
+come from a library) you can also pass a \LUA\ string, in which case the given
+length is (at most) the string length.
+
+The function returns an \type {pdfe} object and a string. The string can be used in
+the \type {img} library instead of a filename. You need to prevent garbage
+collection of the object when you use it as image (for instance by storing it
+somewhere).
+
+Both the memory stream and it's use in the image library is experimental and can
+change. In case you wonder where this can be used: when you use the swiglib
+library for \type {graphicmagick}, it can return such a userdata object. This
+permits conversion in memory and passing the result directly to the backend. This
+might save some runtime in one|-|pass workflows. This feature is currently not
+meant for production and we might come up with a better implementation.
+
+\stopsection
+
+\startsection[title={The \type {pdfscanner} library}][library=pdfscanner]
+
+This library is not available in \LUAMETATEX.
+
+\stopsection
+
+% \startsection[title={The \type {pdfscanner} library}][library=pdfscanner]
+%
+% \topicindex{\PDF+scanner}
+%
+% \libindex {scan}
+%
+% The \type {pdfscanner} library allows interpretation of \PDF\ content streams and
+% \type {/ToUnicode} (cmap) streams. You can get those streams from the \type
+% {pdfe} library, as explained in an earlier section. There is only a single
+% top|-|level function in this library:
+%
+% \startfunctioncall
+% pdfscanner.scan (<pdfe stream>, <table> operatortable, <table> info)
+% pdfscanner.scan (<pdfe array>, <table> operatortable, <table> info)
+% pdfscanner.scan (<string>, <table> operatortable, <table> info)
+% \stopfunctioncall
+%
+% The first argument should be a \LUA\ string or a stream or array onject coming
+% from the \type {pdfe} library. The second argument, \type {operatortable}, should
+% be a \LUA\ table where the keys are \PDF\ operator name strings and the values
+% are \LUA\ functions (defined by you) that are used to process those operators.
+% The functions are called whenever the scanner finds one of these \PDF\ operators
+% in the content stream(s). The functions are called with two arguments: the \type
+% {scanner} object itself, and the \type {info} table that was passed are the third
+% argument to \type {pdfscanner.scan}.
+%
+% Internally, \type {pdfscanner.scan} loops over the \PDF\ operators in the
+% stream(s), collecting operands on an internal stack until it finds a \PDF\
+% operator. If that \PDF\ operator's name exists in \type {operatortable}, then the
+% associated function is executed. After the function has run (or when there is no
+% function to execute) the internal operand stack is cleared in preparation for the
+% next operator, and processing continues.
+%
+% The \type {scanner} argument to the processing functions is needed because it
+% offers various methods to get the actual operands from the internal operand
+% stack.
+%
+% A simple example of processing a \PDF's document stream could look like this:
+%
+% \starttyping
+% local operatortable = { }
+%
+% operatortable.Do = function(scanner,info)
+% local resources = info.resources
+% if resources then
+% local val = scanner:pop()
+% local name = val[2]
+% local xobject = resources.XObject
+% print(info.space .. "Uses XObject " .. name)
+% local resources = xobject.Resources
+% if resources then
+% local newinfo = {
+% space = info.space .. " ",
+% resources = resources,
+% }
+% pdfscanner.scan(entry, operatortable, newinfo)
+% end
+% end
+% end
+%
+% local function Analyze(filename)
+% local doc = pdfe.open(filename)
+% if doc then
+% local pages = doc.Pages
+% for i=1,#pages do
+% local page = pages[i]
+% local info = {
+% space = " " ,
+% resources = page.Resources,
+% }
+% print("Page " .. i)
+% -- pdfscanner.scan(page.Contents,operatortable,info)
+% pdfscanner.scan(page.Contents(),operatortable,info)
+% end
+% end
+% end
+%
+% Analyze("foo.pdf")
+% \stoptyping
+%
+% This example iterates over all the actual content in the \PDF, and prints out the
+% found \type {XObject} names. While the code demonstrates quite some of the \type
+% {pdfe} functions, let's focus on the type \type {pdfscanner} specific code
+% instead.
+%
+% From the bottom up, the following line runs the scanner with the \PDF\ page's
+% top|-|level content given in the first argument.
+%
+% The third argument, \type {info}, contains two entries: \type {space} is used to
+% indent the printed output, and \type {resources} is needed so that embedded \type
+% {XForms} can find their own content.
+%
+% The second argument, \type {operatortable} defines a processing function for a
+% single \PDF\ operator, \type {Do}.
+%
+% The function \type {Do} prints the name of the current \type {XObject}, and then
+% starts a new scanner for that object's content stream, under the condition that
+% the \type {XObject} is in fact a \type {/Form}. That nested scanner is called
+% with new \type {info} argument with an updated \type {space} value so that the
+% indentation of the output nicely nests, and with a new \type {resources} field
+% to help the next iteration down to properly process any other, embedded \type
+% {XObject}s.
+%
+% Of course, this is not a very useful example in practice, but for the purpose of
+% demonstrating \type {pdfscanner}, it is just long enough. It makes use of only
+% one \type {scanner} method: \type {scanner:pop()}. That function pops the top
+% operand of the internal stack, and returns a \LUA\ table where the object at index
+% one is a string representing the type of the operand, and object two is its
+% value.
+%
+% The list of possible operand types and associated \LUA\ value types is:
+%
+% \starttabulate[|l|l|]
+% \DB types \BC type \NC \NR
+% \TB
+% \NC \type{integer} \NC <number> \NC \NR
+% \NC \type{real} \NC <number> \NC \NR
+% \NC \type{boolean} \NC <boolean> \NC \NR
+% \NC \type{name} \NC <string> \NC \NR
+% \NC \type{operator} \NC <string> \NC \NR
+% \NC \type{string} \NC <string> \NC \NR
+% \NC \type{array} \NC <table> \NC \NR
+% \NC \type{dict} \NC <table> \NC \NR
+% \LL
+% \stoptabulate
+%
+% In case of \type {integer} or \type {real}, the value is always a \LUA\ (floating
+% point) number. In case of \type {name}, the leading slash is always stripped.
+%
+% In case of \type {string}, please bear in mind that \PDF\ actually supports
+% different types of strings (with different encodings) in different parts of the
+% \PDF\ document, so you may need to reencode some of the results; \type {pdfscanner}
+% always outputs the byte stream without reencoding anything. \type {pdfscanner}
+% does not differentiate between literal strings and hexadecimal strings (the
+% hexadecimal values are decoded), and it treats the stream data for inline images
+% as a string that is the single operand for \type {EI}.
+%
+% In case of \type {array}, the table content is a list of \type {pop} return
+% values and in case of \type {dict}, the table keys are \PDF\ name strings and the
+% values are \type {pop} return values.
+%
+% \libindex{pop}
+% \libindex{popnumber}
+% \libindex{popname}
+% \libindex{popstring}
+% \libindex{poparray}
+% \libindex{popdictionary}
+% \libindex{popboolean}
+% \libindex{done}
+%
+% There are a few more methods defined that you can ask \type {scanner}:
+%
+% \starttabulate[|l|p|]
+% \DB method \BC explanation \NC \NR
+% \TB
+% \NC \type{pop} \NC see above \NC \NR
+% \NC \type{popnumber} \NC return only the value of a \type {real} or \type {integer} \NC \NR
+% \NC \type{popname} \NC return only the value of a \type {name} \NC \NR
+% \NC \type{popstring} \NC return only the value of a \type {string} \NC \NR
+% \NC \type{poparray} \NC return only the value of a \type {array} \NC \NR
+% \NC \type{popdictionary} \NC return only the value of a \type {dict} \NC \NR
+% \NC \type{popboolean} \NC return only the value of a \type {boolean} \NC \NR
+% \NC \type{done} \NC abort further processing of this \type {scan()} call \NC \NR
+% \LL
+% \stoptabulate
+%
+% The \type {pop*} are convenience functions, and come in handy when you know the
+% type of the operands beforehand (which you usually do, in \PDF). For example, the
+% \type {Do} function could have used \type {local name = scanner:popname()}
+% instead, because the single operand to the \type {Do} operator is always a \PDF\
+% name object.
+%
+% The \type {done} function allows you to abort processing of a stream once you
+% have learned everything you want to learn. This comes in handy while parsing
+% \type {/ToUnicode}, because there usually is trailing garbage that you are not
+% interested in. Without \type {done}, processing only ends at the end of the
+% stream, possibly wasting \CPU\ cycles.
+%
+% {\em We keep the older names \type {popNumber}, \type {popName}, \type
+% {popString}, \type {popArray}, \type {popDict} and \type {popBool} around.}
+%
+% \stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-preamble.tex b/doc/context/sources/general/manuals/luametatex/luametatex-preamble.tex
new file mode 100644
index 000000000..8f1400c9f
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-preamble.tex
@@ -0,0 +1,166 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-preamble
+
+\startchapter[reference=preamble,title={The internals}]
+
+\topicindex{nodes}
+\topicindex{boxes}
+\topicindex{\LUA}
+
+This is a reference manual, not a tutorial. This means that we discuss changes
+relative to traditional \TEX\ and also present new functionality. As a
+consequence we will refer to concepts that we assume to be known or that might be
+explained later. Because the \LUATEX\ and \LUAMETATEX\ engines open up \TEX\
+there's suddenly quite some more to explain, especially about the way a (to be)
+typeset stream moves through the machinery. However, discussing all that in
+detail makes not much sense, because deep knowledge is only relevant for those
+who write code not possible with regular \TEX\ and who are already familiar with
+these internals (or willing to spend time on figuring it out).
+
+So, the average user doesn't need to know much about what is in this manual. For
+instance fonts and languages are normally dealt with in the macro package that
+you use. Messing around with node lists is also often not really needed at the
+user level. If you do mess around, you'd better know what you're dealing with.
+Reading \quotation {The \TEX\ Book} by Donald Knuth is a good investment of time
+then also because it's good to know where it all started. A more summarizing
+overview is given by \quotation {\TEX\ by Topic} by Victor Eijkhout. You might
+want to peek in \quotation {The \ETEX\ manual} too.
+
+But \unknown\ if you're here because of \LUA, then all you need to know is that
+you can call it from within a run. If you want to learn the language, just read
+the well written \LUA\ book. The macro package that you use probably will provide
+a few wrapper mechanisms but the basic \lpr {directlua} command that does the job
+is:
+
+\starttyping
+\directlua{tex.print("Hi there")}
+\stoptyping
+
+You can put code between curly braces but if it's a lot you can also put it in a
+file and load that file with the usual \LUA\ commands. If you don't know what
+this means, you definitely need to have a look at the \LUA\ book first.
+
+If you still decide to read on, then it's good to know what nodes are, so we do a
+quick introduction here. If you input this text:
+
+\starttyping
+Hi There
+\stoptyping
+
+eventually we will get a linked lists of nodes, which in \ASCII\ art looks like:
+
+\starttyping
+H <=> i <=> [glue] <=> T <=> h <=> e <=> r <=> e
+\stoptyping
+
+When we have a paragraph, we actually get something:
+
+\starttyping
+[localpar] <=> H <=> i <=> [glue] <=> T <=> h <=> e <=> r <=> e <=> [glue]
+\stoptyping
+
+Each character becomes a so called glyph node, a record with properties like the
+current font, the character code and the current language. Spaces become glue
+nodes. There are many node types that we will discuss later. Each node points
+back to a previous node or next node, given that these exist. Sometimes
+multiple characters are represented by one glyphs, so one can also get:
+
+\starttyping
+[localpar] <=> H <=> i <=> [glue] <=> Th <=> e <=> r <=> e <=> [glue]
+\stoptyping
+
+And maybe some characters get positioned relative to each other, so we might
+see:
+
+\starttyping
+[localpar] <=> H <=> [kern] <=> i <=> [glue] <=> Th <=> e <=> r <=> e <=> [glue]
+\stoptyping
+
+It's also good to know beforehand that \TEX\ is basically centered around
+creating paragraphs and pages. The par builder takes a list and breaks it into
+lines. At some point horizontal blobs are wrapped into vertical ones. Lines are
+so called boxes and can be separated by glue, penalties and more. The page
+builder accumulates lines and when feasible triggers an output routine that will
+take the list so far. Constructing the actual page is not part of \TEX\ but done
+using primitives that permit manipulation of boxes. The result is handled back to
+\TEX\ and flushed to a (often \PDF) file.
+
+The \LUATEX\ engine provides hooks for \LUA\ code at nearly every reasonable
+point in the process: collecting content, hyphenating, applying font features,
+breaking into lines, etc. This means that you can overload \TEX's natural
+behaviour, which still is the benchmark. When we refer to \quote {callbacks} we
+means these hooks. The \TEX\ engine itself is pretty well optimized but when you
+kick in much \LUA\ code, you will notices that performance drops. Don't blame and
+bother the authors with performance issues. In \CONTEXT\ over 50\% of the time
+can be spent in \LUA, but so far we didn't get many complaints about efficiency.
+
+Where plain \TEX\ is basically a basic framework for writing a specific style,
+macro packages like \CONTEXT\ and \LATEX\ provide the user a whole lot of
+additional tools to make documents look good. They hide the dirty details of font
+management, language demands, turning structure into typeset results, wrapping
+pages, including images, and so on. You should be aware of the fact that when you
+hook in your own code to manipulate lists, this can interfere with the macro
+package that you use. Each successive step expects a certain result and if you
+mess around to much, the engine eventually might bark and quit. It can even
+crash, because testing everywhere for what users can do wrong is no real option.
+
+When you read about nodes in the following chapters it's good to keep in mind
+their commands that relate to then. Here are a few:
+
+\starttabulate[|l|l|p|]
+\DB command \BC node \BC explanation \NC \NR
+\TB
+\NC \prm {hbox} \NC \nod {hlist} \NC horizontal box \NC \NR
+\NC \prm {vbox} \NC \nod {vlist} \NC vertical box with the baseline at the bottom \NC \NR
+\NC \prm {vtop} \NC \nod {vlist} \NC vertical box with the baseline at the top \NC \NR
+\NC \prm {hskip} \NC \nod {glue} \NC horizontal skip with optional stretch and shrink \NC \NR
+\NC \prm {vskip} \NC \nod {glue} \NC vertical skip with optional stretch and shrink \NC \NR
+\NC \prm {kern} \NC \nod {kern} \NC horizontal or vertical fixed skip \NC \NR
+\NC \prm {discretionary} \NC \nod {disc} \NC hyphenation point (pre, post, replace) \NC \NR
+\NC \prm {char} \NC \nod {glyph} \NC a character \NC \NR
+\NC \prm {hrule} \NC \nod {rule} \NC a horizontal rule \NC \NR
+\NC \prm {vrule} \NC \nod {rule} \NC a vertical rule \NC \NR
+\NC \prm {textdirection} \NC \nod {dir} \NC a change in text direction \NC \NR
+\LL
+\stoptabulate
+
+Text (interspersed with macros) comes from an input medium. This can be a file,
+token list, macro body cq.\ arguments, \ some internal quantity (like a number),
+\LUA, etc. Macros get expanded. In the process \TEX\ can enter a group. Inside
+the group, changes to registers get saved on a stack, and restored after leaving
+the group. When conditionals are encountered, another kind of nesting happens,
+and again there is a stack involved. Tokens, expansion, stacks, input levels are
+all terms used in the next chapters. Don't worry, they loose their magic once you
+use \TEX\ a lot. You have access to most of the internals and when not, at least
+it is possible to query some state we're in or level we're at.
+
+When we talk about packing it can mean two things. When \TEX\ has consumed some
+tokens that represent text the next can happen. When the text is put into a so
+called \type {\hbox} it (normally) first gets hyphenated, next ligatures are
+build, and finally kerns are added. Each of that stages can be overloaded using
+\LUA\ code. When these three stages are finished, the dimension of the content is
+calculated and the box gets its width, height and depth. What happens with the
+box depends on what macros do with it.
+
+The other thing that can happen is that the text starts a new paragraph. In that
+case some (directional) information is put in front, indentation is prepended and
+some skip appended at the end. Again the three stages are applied but this time,
+afterwards, the long line is broken into lines and the result is either added to
+the content of a box or to the main vertical list (the running text so to say).
+This is called par building. At some point \TEX\ decides that enough is enough
+and it will trigger the page builder. So, building is another concept we will
+encounter. Another example of a builder is the one that turns an intermediate
+math list into something typeset.
+
+Wrapping something in a box is called packing. Adding something to a list is
+described in terms of contributing. The more complicated processes are wrapped
+into builders. For now this should be enough to enable you to understand the next
+chapters. The text is not as enlightening and entertaining as Don Knuths books,
+sorry.
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-primitives.tex b/doc/context/sources/general/manuals/luametatex/luametatex-primitives.tex
new file mode 100644
index 000000000..5c4724100
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-primitives.tex
@@ -0,0 +1,392 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-primitives
+
+\startchapter[reference=primitives,title={Primitives aka commands}]
+
+\startsection[title=Introduction]
+
+The starting point of \LUATEX\ is \PDFTEX, which itself contains regular \TEX\
+and \ETEX. Because directional support was needed we also took some code from
+\ALEPH\ (\OMEGA). In a later stage the backend specific commands were isolated in
+its own namespace which resulted in a cleaner code base where the backend code no
+longer was interwoven with the normal frontend primitives. We also promoted some
+generic constructs (like box resources and directions) to core functionality.
+
+Some of the \PDFTEX\ support primitives have been around from the start but when
+\LUA\ integration became better and when a token scanner library was added, not
+all of those made sense as primitives. In previous chapters we already mentioned
+what is gone from the core. Deep down some more has changes but not all is
+reflected at the primitive level. Because there is still a considerable amount of
+new primitives, a summary is given below.
+
+\stopsection
+
+\startsection[title=Languages]
+
+\starttabulate[|||p|]
+\NC \type {automatichyphenmode} \NC integer \NC \NC \NR
+\NC \type {automatichyphenpenalty} \NC integer \NC \NC \NR
+\NC \type {hyphenpenaltymode} \NC integer \NC \NC \NR
+\NC \type {compoundhyphenmode} \NC integer \NC \NC \NR
+\NC \type {exceptionpenalty} \NC integer \NC \NC \NR
+\NC \type {explicithyphenpenalty} \NC integer \NC \NC \NR
+\NC \type {hyphenationbounds} \NC integer \NC \NC \NR
+\NC \type {hjcode} \NC charactercode \NC \NC \NR
+\NC \type {hyphenationmin} \NC charactercode \NC \NC \NR
+\NC \type {postexhyphenchar} \NC charactercode \NC \NC \NR
+\NC \type {posthyphenchar} \NC charactercode \NC \NC \NR
+\NC \type {preexhyphenchar} \NC charactercode \NC \NC \NR
+\NC \type {prehyphenchar} \NC charactercode \NC \NC \NR
+\stoptabulate
+
+\stopsection
+
+\startsection[title=Fonts]
+
+\starttabulate[|||p|]
+\NC \type {tracingfonts} \NC integer \NC \NC \NR
+\NC \type {suppressfontnotfounderror} \NC integer \NC \NC \NR
+\NC \type {setfontid} \NC integer \NC \NC \NR
+\NC \type {fontid} \NC font \NC \NC \NR
+\NC \type {efcode} \NC font charactercode \NC \NC \NR
+\NC \type {lpcode} \NC font charactercode \NC \NC \NR
+\NC \type {rpcode} \NC font charactercode \NC \NC \NR
+\stoptabulate
+
+\stopsection
+
+\startsection[title=Math]
+
+\starttabulate[|||p|]
+\NC \type {matholdmode} \NC integer \NC \NC \NR
+\NC \type {mathstyle} \NC integer \NC \NC \NR
+\NC \type {matheqnogapstep} \NC integer \NC \NC \NR
+\NC \type {Uskewed} \NC \NC \NC \NR
+\NC \type {Uskewedwithdelims} \NC \NC \NC \NR
+\NC \type {Ustartdisplaymath} \NC \NC \NC \NR
+\NC \type {Ustartmath} \NC \NC \NC \NR
+\NC \type {Ustopdisplaymath} \NC \NC \NC \NR
+\NC \type {Ustopmath} \NC \NC \NC \NR
+\NC \type {crampeddisplaystyle} \NC \NC \NC \NR
+\NC \type {crampedtextstyle} \NC \NC \NC \NR
+\NC \type {crampedscriptstyle} \NC \NC \NC \NR
+\NC \type {crampedscriptscriptstyle} \NC \NC \NC \NR
+\NC \type {Umathchardef} \NC \NC \NC \NR
+\NC \type {Umathcharnumdef} \NC \NC \NC \NR
+\NC \type {mathdisplayskipmode} \NC integer \NC \NC \NR
+\NC \type {mathscriptsmode} \NC integer \NC \NC \NR
+\NC \type {mathnolimitsmode} \NC integer \NC \NC \NR
+\NC \type {mathitalicsmode} \NC integer \NC \NC \NR
+\NC \type {mathrulesmode} \NC integer \NC \NC \NR
+\NC \type {mathrulesfam} \NC integer \NC \NC \NR
+\NC \type {mathdelimitersmode} \NC integer \NC \NC \NR
+\NC \type {mathflattenmode} \NC integer \NC \NC \NR
+\NC \type {mathpenaltiesmode} \NC integer \NC \NC \NR
+\NC \type {mathrulethicknessmode} \NC integer \NC \NC \NR
+\NC \type {mathscriptboxmode} \NC integer \NC \NC \NR
+\NC \type {mathscriptcharmode} \NC integer \NC \NC \NR
+\NC \type {mathsurroundmode} \NC integer \NC \NC \NR
+\NC \type {nokerns} \NC integer \NC \NC \NR
+\NC \type {noligs} \NC integer \NC \NC \NR
+\NC \type {prebinoppenalty} \NC integer \NC \NC \NR
+\NC \type {predisplaygapfactor} \NC integer \NC \NC \NR
+\NC \type {prerelpenalty} \NC integer \NC \NC \NR
+\NC \type {Usuperscript} \NC command \NC \NC \NR
+\NC \type {Usubscript} \NC command \NC \NC \NR
+\NC \type {Unosuperscript} \NC command \NC \NC \NR
+\NC \type {Unosubscript} \NC command \NC \NC \NR
+\NC \type {Umathcode} \NC \NC \NC \NR
+\NC \type {Umathcodenum} \NC \NC \NC \NR
+\NC \type {Udelcode} \NC \NC \NC \NR
+\NC \type {Udelcodenum} \NC \NC \NC \NR
+\NC \type {Umathaxis} \NC family dimension \NC \NC \NR
+\NC \type {Umathbinbinspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathbinclosespacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathbininnerspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathbinopenspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathbinopspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathbinordspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathbinpunctspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathbinrelspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathclosebinspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathcloseclosespacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathcloseinnerspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathcloseopenspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathcloseopspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathcloseordspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathclosepunctspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathcloserelspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathconnectoroverlapmin} \NC family dimension \NC \NC \NR
+\NC \type {Umathfractiondelsize} \NC family dimension \NC \NC \NR
+\NC \type {Umathfractiondenomdown} \NC family dimension \NC \NC \NR
+\NC \type {Umathfractiondenomvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathfractionnumup} \NC family dimension \NC \NC \NR
+\NC \type {Umathfractionnumvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathfractionrule} \NC family dimension \NC \NC \NR
+\NC \type {Umathinnerbinspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathinnerclosespacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathinnerinnerspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathinneropenspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathinneropspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathinnerordspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathinnerpunctspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathinnerrelspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathlimitabovebgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathlimitabovekern} \NC family dimension \NC \NC \NR
+\NC \type {Umathlimitabovevgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathlimitbelowbgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathlimitbelowkern} \NC family dimension \NC \NC \NR
+\NC \type {Umathlimitbelowvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathnolimitsubfactor} \NC family dimension \NC \NC \NR
+\NC \type {Umathnolimitsupfactor} \NC family dimension \NC \NC \NR
+\NC \type {Umathopbinspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopclosespacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopenbinspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopenclosespacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopeninnerspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopenopenspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopenopspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopenordspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopenpunctspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopenrelspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathoperatorsize} \NC family dimension \NC \NC \NR
+\NC \type {Umathopinnerspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopopenspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopopspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathopordspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathoppunctspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathoprelspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathordbinspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathordclosespacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathordinnerspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathordopenspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathordopspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathordordspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathordpunctspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathordrelspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathoverbarkern} \NC family dimension \NC \NC \NR
+\NC \type {Umathoverbarrule} \NC family dimension \NC \NC \NR
+\NC \type {Umathoverbarvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathoverdelimiterbgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathoverdelimitervgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathpunctbinspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathpunctclosespacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathpunctinnerspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathpunctopenspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathpunctopspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathpunctordspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathpunctpunctspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathpunctrelspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathquad} \NC family dimension \NC \NC \NR
+\NC \type {Umathradicaldegreeafter} \NC family dimension \NC \NC \NR
+\NC \type {Umathradicaldegreebefore} \NC family dimension \NC \NC \NR
+\NC \type {Umathradicaldegreeraise} \NC family dimension \NC \NC \NR
+\NC \type {Umathradicalkern} \NC family dimension \NC \NC \NR
+\NC \type {Umathradicalrule} \NC family dimension \NC \NC \NR
+\NC \type {Umathradicalvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathrelbinspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathrelclosespacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathrelinnerspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathrelopenspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathrelopspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathrelordspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathrelpunctspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathrelrelspacing} \NC family dimension \NC \NC \NR
+\NC \type {Umathskewedfractionhgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathskewedfractionvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathspaceafterscript} \NC family dimension \NC \NC \NR
+\NC \type {Umathstackdenomdown} \NC family dimension \NC \NC \NR
+\NC \type {Umathstacknumup} \NC family dimension \NC \NC \NR
+\NC \type {Umathstackvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathsubshiftdown} \NC family dimension \NC \NC \NR
+\NC \type {Umathsubshiftdrop} \NC family dimension \NC \NC \NR
+\NC \type {Umathsubsupshiftdown} \NC family dimension \NC \NC \NR
+\NC \type {Umathsubsupvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathsubtopmax} \NC family dimension \NC \NC \NR
+\NC \type {Umathsupbottommin} \NC family dimension \NC \NC \NR
+\NC \type {Umathsupshiftdrop} \NC family dimension \NC \NC \NR
+\NC \type {Umathsupshiftup} \NC family dimension \NC \NC \NR
+\NC \type {Umathsupsubbottommax} \NC family dimension \NC \NC \NR
+\NC \type {Umathunderbarkern} \NC family dimension \NC \NC \NR
+\NC \type {Umathunderbarrule} \NC family dimension \NC \NC \NR
+\NC \type {Umathunderbarvgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathunderdelimiterbgap} \NC family dimension \NC \NC \NR
+\NC \type {Umathunderdelimitervgap} \NC family dimension \NC \NC \NR
+\NC \type {Udelimiter} \NC \NC \NC \NR
+\NC \type {Umathaccent} \NC \NC \NC \NR
+\NC \type {Umathchar} \NC \NC \NC \NR
+\NC \type {Umathcharnum} \NC \NC \NC \NR
+\NC \type {Ustack} \NC \NC \NC \NR
+\NC \type {Uradical} \NC \NC \NC \NR
+\NC \type {Uroot} \NC \NC \NC \NR
+\NC \type {Uunderdelimiter} \NC \NC \NC \NR
+\NC \type {Uoverdelimiter} \NC \NC \NC \NR
+\NC \type {Udelimiterunder} \NC \NC \NC \NR
+\NC \type {Udelimiterover} \NC \NC \NC \NR
+\NC \type {Uhextensible} \NC \NC \NC \NR
+\NC \type {Uchar} \NC \NC \NC \NR
+\NC \type {Umathcharclass} \NC \NC \NC \NR
+\NC \type {Umathcharfam} \NC \NC \NC \NR
+\NC \type {Umathcharslot} \NC \NC \NC \NR
+\NC \type {Uleft} \NC \NC \NC \NR
+\NC \type {Umiddle} \NC \NC \NC \NR
+\NC \type {Uright} \NC \NC \NC \NR
+\NC \type {Uvextensible} \NC \NC \NC \NR
+\stoptabulate
+
+\stopsection
+
+\startsection[title=Boxes and directions]
+
+\starttabulate[|||p|]
+\NC \type {pardirection} \NC direction \NC \NC \NR
+\NC \type {textdirection} \NC direction \NC \NC \NR
+\NC \type {mathdirection} \NC direction \NC \NC \NR
+\NC \type {linedirection} \NC direction \NC \NC \NR
+\NC \type {breakafterdirmode} \NC integer \NC \NC \NR
+\NC \type {shapemode} \NC integer \NC \NC \NR
+\NC \type {fixupboxesmode} \NC integer \NC \NC \NR
+\NC \type {boxdirection} \NC box direction \NC \NC \NR
+\NC \type {boxorientation} \NC box orientation \NC rotation over 90, 180, 270 degrees \NC \NR
+\NC \type {boxxoffset} \NC box xoffset \NC leaves dimensions untounched \NC \NR
+\NC \type {boxyoffset} \NC box yoffset \NC leaves dimensions untounched \NC \NR
+\NC \type {boxxmove} \NC box xmove \NC offsets that reflect on dimensions \NC \NR
+\NC \type {boxymove} \NC box ymove \NC offsets that reflect on dimensions \NC \NR
+\NC \type {boxtotal} \NC box ht+dp \NC height plus depth (and when assigned halfs) \NC \NR
+\NC \type {boxattr} \NC box attr value \NC (sets) a specific attribute to a value \NC \NR
+\stoptabulate
+
+\stopsection
+
+\startsection[title=Scanning]
+
+\starttabulate[|||p|]
+\NC \type {aftergrouped} \NC text \NC like aftergroup but for given list \NC \NR
+\NC \type {alignmark} \NC \NC equivalent to hash token \NC \NR
+\NC \type {aligntab} \NC \NC equivalent to tab token \NC \NR
+\NC \type {begincsname} \NC command \NC variant of \type {\csname} that ignores undefined commands \NC \NR
+\NC \type {catcodetable} \NC integer \NC switch to catcode table \NC \NR
+\NC \type {csstring} \NC command \NC the command without preceding escape character \NC \NR
+\NC \type {endlocalcontrol} \NC command \NC switches back to the main control loop \NC \NR
+\NC \type {etoksapp} \NC tokenregister text \NC append expanded text to given tokenregister \NC \NR
+\NC \type {etokspre} \NC tokenregister text \NC prepend expanded text to given tokenregister \NC \NR
+\NC \type {expanded} \NC text \NC expands the given text \NC \NR
+\NC \type {frozen} \NN prefix \NC \NC \NR
+\NC \type {futureexpand} \NN token command command \NC expands second ot third token depending on first match \NC \NR
+\NC \type {futureexpandis} \NN token command command \NC as \type {futureexpand} but also skips pars \NC \NR
+\NC \type {futureexpandisap} \NN token command command \NC same as idem but doesn't push back skipped spaces \NC \NR
+\NC \type {gtoksapp} \NC tokenregister text \NC globally append text to given tokenregister \NC \NR
+\NC \type {gtokspre} \NC tokenregister text \NC globally prepend text to given tokenregister \NC \NR
+\NC \type {ifabsdim} \NC dimension <=> dimension \NC test the absolute value of the given dimension \NC \NR
+\NC \type {ifabsnum} \NC integer <=> integer \NC test the absolute value of the given integer \NC \NR
+\NC \type {ifcondition} \NC command \NC assume the next token is a test (so skip as if) \NC \NR
+\NC \type {ifdimen} \NC possibly a dimension \NC acts like an \type {\ifcase} with 1 for valid and 2 for invalid \NC \NR
+\NC \type {ifincsname} \NC command \NC check if we're inside a csname expansion \NC \NR
+\NC \type {ifnumval} \NC \NC \NC \NR
+\NC \type {ifdimval} \NC \NC \NC \NR
+\NC \type {ifchknum} \NC \NC \NC \NR
+\NC \type {ifchkdim} \NC \NC \NC \NR
+\NC \type {ifcmpnum} \NC \NC \NC \NR
+\NC \type {ifcmpdim} \NC \NC \NC \NR
+\NC \type {ifusercmd} \NC command \NC \NC \NR
+\NC \type {ifprotected} \NC command \NC \NC \NR
+\NC \type {iffrozen} \NC command \NC \NC \NR
+\NC \type {iftok} \NC \NC \NC \NR
+\NC \type {ifcstok} \NC \NC \NC \NR
+\NC \type {internalcodesmode} \NC integer \NC \NC \NR
+%NC \type {ifprimitive} \NC command \NC check if the given command is a primitive \NC \NR
+\NC \type {immediateassigned} \NC command \NC (todo) expand the following assignment now\NC \NR
+\NC \type {immediateassignment} \NC command \NC (todo) expand the following assignment now\NC \NR
+\NC \type {initcatcodetable} \NC integer \NC initialize catcode table \NC \NR
+\NC \type {lastnamedcs} \NC command \NC last found command of \type {\ifcsname} construction \NC \NR
+\NC \type {nospaces} \NC integer \NC don't inject spaces \NC \NR
+%NC \type {primitive} \NC command \NC expands the next primitive equivalent \NC \NR
+\NC \type {orelse} \NC condition \NC \NC \NR
+\NC \type {pxdimen} \NC dimension \NC multiplier for the \type {px} unit \NC \NR
+\NC \type {savecatcodetable} \NC integer \NC save catcode table \NC \NR
+\NC \type {scantextokens} \NC text \NC \type {\scantokens} without file side effects \NC \NR
+\NC \type {suppressifcsnameerror} \NC integer \NC recover from issues in csname testing \NC \NR
+\NC \type {suppresslongerror} \NC integer \NC make \type {\long} a nop \NC \NR
+\NC \type {suppressmathparerror} \NC integer \NC accepts \type {\par} and empty lines in math \NC \NR
+\NC \type {suppressoutererror} \NC integer \NC make \type {\outer} a nop \NC \NR
+\NC \type {suppressprimitiveerror} \NC integer \NC don't report an invalid \type {\primitive} \NC \NR
+\NC \type {toksapp} \NC tokenregister text \NC append text to given tokenregister \NC \NR
+\NC \type {tokspre} \NC tokenregister text \NC prepend text to given tokenregister \NC \NR
+\NC \type {xtoksapp} \NC tokenregister text \NC globally append expanded text to given tokenregister \NC \NR
+\NC \type {xtokspre} \NC tokenregister text \NC globally prepend expanded text to given tokenregister \NC \NR
+\stoptabulate
+
+\stopsection
+
+\startsection[title=Typesetting]
+
+\starttabulate[|||p|]
+\NC \type {protrudechars} \NC integer \NC \NC \NR
+\NC \type {localbrokenpenalty} \NC integer \NC \NC \NR
+\NC \type {localinterlinepenalty} \NC integer \NC \NC \NR
+\NC \type {adjustspacing} \NC integer \NC \NC \NR
+\NC \type {boundary} \NC command \NC \NC \NR
+\NC \type {noboundary} \NC command \NC \NC \NR
+\NC \type {protrusionboundary} \NC command \NC \NC \NR
+\NC \type {wordboundary} \NC command \NC \NC \NR
+\NC \type {leftghost} \NC charactercode \NC \NC \NR
+\NC \type {rightghost} \NC charactercode \NC \NC \NR
+\NC \type {nohrule} \NC command \NC \NC \NR
+\NC \type {novrule} \NC command \NC \NC \NR
+\NC \type {insertht} \NC number \NC \NC \NR
+\NC \type {quitvmode} \NC command \NC \NC \NR
+\NC \type {leftmarginkern} \NC dimension \NC \NC \NR
+\NC \type {rightmarginkern} \NC dimension \NC \NC \NR
+\NC \type {localleftbox} \NC box \NC \NC \NR
+\NC \type {localrightbox} \NC box \NC \NC \NR
+\NC \type {gleaders} \NC command \NC \NC \NR
+\stoptabulate
+
+\stopsection
+
+\startsection[title=\LUA]
+
+\starttabulate[|||p|]
+\NC \type {luacopyinputnodes} \NC integer \NC \NC \NR
+\NC \type {luadef} \NC \NC \NC \NR
+\NC \type {luabytecodecall} \NC \NC \NC \NR
+\NC \type {luafunctioncall} \NC \NC \NC \NR
+\NC \type {latelua} \NC \NC \NC \NR
+\NC \type {lateluafunction} \NC \NC \NC \NR
+\NC \type {luabytecode} \NC \NC \NC \NR
+\NC \type {luaescapestring} \NC \NC \NC \NR
+\NC \type {luafunction} \NC \NC \NC \NR
+\stoptabulate
+
+\stopsection
+
+\startsection[title=Management]
+
+\starttabulate[|||p|]
+\NC \type {outputbox} \NC integer \NC \NC \NR
+\NC \type {clearmarks} \NC \NC \NC \NR
+\NC \type {attribute} \NC \NC \NC \NR
+\NC \type {glet} \NC \NC \NC \NR
+\NC \type {letcharcode} \NC \NC \NC \NR
+\NC \type {attributedef} \NC \NC \NC \NR
+\stoptabulate
+
+\stopsection
+
+\startsection[title=Miscellaneous]
+
+\starttabulate[|||p|]
+\NC \type {luatexversion} \NC \NC \NC \NR
+\NC \type {formatname} \NC \NC \NC \NR
+\NC \type {luatexbanner} \NC \NC \NC \NR
+\NC \type {luatexrevision} \NC \NC \NC \NR
+\stoptabulate
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-registers.tex b/doc/context/sources/general/manuals/luametatex/luametatex-registers.tex
new file mode 100644
index 000000000..f230a4500
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-registers.tex
@@ -0,0 +1,47 @@
+\environment luametatex-style
+
+\startcomponent luametatex-registers
+
+\startchapter[title=Topics]
+
+ \placeregister[topicindex]
+
+\stopchapter
+
+\startchapter[title=Primitives]
+
+ This register contains the primitives that are mentioned in the manual. There
+ are of course many more primitives. The \LUATEX\ primitives are typeset in
+ bold. The primitives from \PDFTEX\ are not supported that way but mentioned
+ anyway.
+
+ \placeregister[primitiveindex][indicator=no]
+
+\stopchapter
+
+\startchapter[title=Callbacks]
+
+ \placeregister[callbackindex]
+
+\stopchapter
+
+\startchapter[title=Nodes]
+
+ This register contains the nodes that are known to \LUATEX. The primary nodes
+ are in bold, whatsits that are determined by their subtype are normal. The
+ names prefixed by \type {pdf_} are backend specific.
+
+ \placeregister[nodeindex]
+
+\stopchapter
+
+\startchapter[title=Libraries]
+
+ This register contains the functions available in libraries. Not all functions
+ are documented, for instance because they can be experimental or obsolete.
+
+ \placeregister[libraryindex]
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-statistics.tex b/doc/context/sources/general/manuals/luametatex/luametatex-statistics.tex
new file mode 100644
index 000000000..d779a0899
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-statistics.tex
@@ -0,0 +1,17 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-statistics
+
+\startchapter[title={Statistics}]
+
+ \topicindex{fonts+used}
+
+ The following fonts are used in this document:
+
+ \showfontusage
+
+\stopchapter
+
+\stopcomponent
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-tex.tex b/doc/context/sources/general/manuals/luametatex/luametatex-tex.tex
new file mode 100644
index 000000000..4e5a4f584
--- /dev/null
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-tex.tex
@@ -0,0 +1,2477 @@
+% language=uk
+
+\environment luametatex-style
+
+\startcomponent luametatex-tex
+
+\startchapter[reference=tex,title={The \TEX\ related libraries}]
+
+\startsection[title={The \type {lua} library}][library=lua]
+
+\startsubsection[title={Version information}]
+
+\topicindex{libraries+\type{lua}}
+\topicindex{version}
+\topicindex{startupfile}
+
+\libindex{version}
+\libindex{startupfile}
+
+This library contains two read|-|only items:
+
+\starttyping
+<string> v = lua.version
+<string> s = lua.startupfile
+\stoptyping
+
+This returns the \LUA\ version identifier string. The value is currently
+\directlua {tex.print(lua.version)}.
+
+\stopsubsection
+
+\startsubsection[title={Table allocators}]
+
+\topicindex{tables}
+
+\libindex{newtable}
+\libindex{newindex}
+
+Sometimes performance (and memory usage) can benefit a little from
+it preallocating a table with \type {newtable}:
+
+\starttyping
+<table> t = lua.newtable(100,5000)
+\stoptyping
+
+This preallocates 100 hash entries and 5000 index entries. The \type
+{newindex} function create an indexed table with preset values:
+
+\starttyping
+<table> t = lua.newindex(2500,true)
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={Bytecode registers}]
+
+\topicindex{bytecodes}
+\topicindex{registers+bytecodes}
+
+\libindex{bytecode}
+\libindex{setbytecode}
+\libindex{getbytecode}
+
+\LUA\ registers can be used to store \LUA\ code chunks. The accepted values for
+assignments are functions and \type {nil}. Likewise, the retrieved value is
+either a function or \type {nil}.
+
+\starttyping
+lua.bytecode[<number> n] = <function> f
+lua.bytecode[<number> n]()
+\stoptyping
+
+The contents of the \type {lua.bytecode} array is stored inside the format file
+as actual \LUA\ bytecode, so it can also be used to preload \LUA\ code. The
+function must not contain any upvalues. The associated function calls are:
+
+\startfunctioncall
+<function> f = lua.getbytecode(<number> n)
+lua.setbytecode(<number> n, <function> f)
+\stopfunctioncall
+
+Note: Since a \LUA\ file loaded using \type {loadfile(filename)} is essentially
+an anonymous function, a complete file can be stored in a bytecode register like
+this:
+
+\startfunctioncall
+lua.bytecode[n] = loadfile(filename)
+\stopfunctioncall
+
+Now all definitions (functions, variables) contained in the file can be
+created by executing this bytecode register:
+
+\startfunctioncall
+lua.bytecode[n]()
+\stopfunctioncall
+
+Note that the path of the file is stored in the \LUA\ bytecode to be used in
+stack backtraces and therefore dumped into the format file if the above code is
+used in \INITEX. If it contains private information, i.e. the user name, this
+information is then contained in the format file as well. This should be kept in
+mind when preloading files into a bytecode register in \INITEX.
+
+\stopsubsection
+
+\startsubsection[title={Chunk name registers}]
+
+\libindex{name}
+\libindex{setluaname}
+\libindex{getluaname}
+
+There is an array of 65536 (0--65535) potential chunk names for use with the
+\prm {directlua} and \lpr {latelua} primitives.
+
+\startfunctioncall
+lua.name[<number> n] = <string> s
+<string> s = lua.name[<number> n]
+\stopfunctioncall
+
+If you want to unset a \LUA\ name, you can assign \type {nil} to it. The function
+accessors are:
+
+\startfunctioncall
+lua.setluaname(<string> s,<number> n])
+<string> s = lua.getluaname(<number> n)
+\stopfunctioncall
+
+\stopsubsection
+
+\startsubsection[title={Introspection}]
+
+\libindex{getstacktop}
+\libindex{getruntime}
+\libindex{getcurrenttime}
+\libindex{getpreciseticks}
+\libindex{getpreciseseconds}
+
+The \type {getstacktop} function return a number indicating how full the \LUA\
+stack is. This function only makes sense as breakpoint when checking some
+mechanism going haywire.
+
+There are four time related helpers. The \type {getruntime} function returns the
+time passed since startup. The \type {getcurrenttime} does what its name says.
+Just play with them to see how it pays off. The \type {getpreciseticks} returns a
+number that can be used later, after a similar call, to get a difference. The
+\type {getpreciseseconds} function gets such a tick (delta) as argument and
+returns the number of seconds. Ticks can differ per operating system, but one
+always creates a reference first and then deltas to this reference.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={The \type {status} library}][library=status]
+
+\topicindex{libraries+\type{status}}
+
+\libindex{list}
+\libindex{resetmessages}
+\libindex{setexitcode}
+
+This contains a number of run|-|time configuration items that you may find useful
+in message reporting, as well as an iterator function that gets all of the names
+and values as a table.
+
+\startfunctioncall
+<table> info = status.list()
+\stopfunctioncall
+
+The keys in the table are the known items, the value is the current value. Almost
+all of the values in \type {status} are fetched through a metatable at run|-|time
+whenever they are accessed, so you cannot use \type {pairs} on \type {status},
+but you {\it can\/} use \type {pairs} on \type {info}, of course. If you do not
+need the full list, you can also ask for a single item by using its name as an
+index into \type {status}. The current list is:
+
+\starttabulate[|l|p|]
+\DB key \BC explanation \NC \NR
+\TB
+\NC \type{banner} \NC terminal display banner \NC \NR
+\NC \type{best_page_break} \NC the current best break (a node) \NC \NR
+\NC \type{buf_size} \NC current allocated size of the line buffer \NC \NR
+\NC \type{callbacks} \NC total number of executed callbacks so far \NC \NR
+\NC \type{cs_count} \NC number of control sequences \NC \NR
+\NC \type{dest_names_size} \NC \PDF\ destination table size \NC \NR
+\NC \type{dvi_gone} \NC written \DVI\ bytes \NC \NR
+\NC \type{dvi_ptr} \NC not yet written \DVI\ bytes \NC \NR
+\NC \type{dyn_used} \NC token (multi|-|word) memory in use \NC \NR
+\NC \type{filename} \NC name of the current input file \NC \NR
+\NC \type{fix_mem_end} \NC maximum number of used tokens \NC \NR
+\NC \type{fix_mem_min} \NC minimum number of allocated words for tokens \NC \NR
+\NC \type{fix_mem_max} \NC maximum number of allocated words for tokens \NC \NR
+\NC \type{font_ptr} \NC number of active fonts \NC \NR
+\NC \type{hash_extra} \NC extra allowed hash \NC \NR
+\NC \type{hash_size} \NC size of hash \NC \NR
+\NC \type{indirect_callbacks} \NC number of those that were themselves a result of other callbacks (e.g. file readers) \NC \NR
+\NC \type{ini_version} \NC \type {true} if this is an \INITEX\ run \NC \NR
+\NC \type{init_pool_ptr} \NC \INITEX\ string pool index \NC \NR
+\NC \type{init_str_ptr} \NC number of \INITEX\ strings \NC \NR
+\NC \type{input_ptr} \NC the level of input we're at \NC \NR
+\NC \type{inputid} \NC numeric id of the current input \NC \NR
+\NC \type{largest_used_mark} \NC max referenced marks class \NC \NR
+\NC \type{lasterrorcontext} \NC last error context string (with newlines) \NC \NR
+\NC \type{lasterrorstring} \NC last \TEX\ error string \NC \NR
+\NC \type{lastluaerrorstring} \NC last \LUA\ error string \NC \NR
+\NC \type{lastwarningstring} \NC last warning tag, normally an indication of in what part\NC \NR
+\NC \type{lastwarningtag} \NC last warning string\NC \NR
+\NC \type{linenumber} \NC location in the current input file \NC \NR
+\NC \type{log_name} \NC name of the log file \NC \NR
+\NC \type{luabytecode_bytes} \NC number of bytes in \LUA\ bytecode registers \NC \NR
+\NC \type{luabytecodes} \NC number of active \LUA\ bytecode registers \NC \NR
+\NC \type{luastate_bytes} \NC number of bytes in use by \LUA\ interpreters \NC \NR
+\NC \type{luatex_engine} \NC the \LUATEX\ engine identifier \NC \NR
+\NC \type{luatex_hashchars} \NC length to which \LUA\ hashes strings ($2^n$) \NC \NR
+\NC \type{luatex_hashtype} \NC the hash method used (in \LUAJITTEX) \NC \NR
+\NC \type{luatex_version} \NC the \LUATEX\ version number \NC \NR
+\NC \type{luatex_revision} \NC the \LUATEX\ revision string \NC \NR
+\NC \type{max_buf_stack} \NC max used buffer position \NC \NR
+\NC \type{max_in_stack} \NC max used input stack entries \NC \NR
+\NC \type{max_nest_stack} \NC max used nesting stack entries \NC \NR
+\NC \type{max_param_stack} \NC max used parameter stack entries \NC \NR
+\NC \type{max_save_stack} \NC max used save stack entries \NC \NR
+\NC \type{max_strings} \NC maximum allowed strings \NC \NR
+\NC \type{nest_size} \NC nesting stack size \NC \NR
+\NC \type{node_mem_usage} \NC a string giving insight into currently used nodes \NC \NR
+\NC \type{obj_ptr} \NC max \PDF\ object pointer \NC \NR
+\NC \type{obj_tab_size} \NC \PDF\ object table size \NC \NR
+\NC \type{output_active} \NC \type {true} if the \prm {output} routine is active \NC \NR
+\NC \type{output_file_name} \NC name of the \PDF\ or \DVI\ file \NC \NR
+\NC \type{param_size} \NC parameter stack size \NC \NR
+\NC \type{pdf_dest_names_ptr} \NC max \PDF\ destination pointer \NC \NR
+\NC \type{pdf_gone} \NC written \PDF\ bytes \NC \NR
+\NC \type{pdf_mem_ptr} \NC max \PDF\ memory used \NC \NR
+\NC \type{pdf_mem_size} \NC \PDF\ memory size \NC \NR
+\NC \type{pdf_os_cntr} \NC max \PDF\ object stream pointer \NC \NR
+\NC \type{pdf_os_objidx} \NC \PDF\ object stream index \NC \NR
+\NC \type{pdf_ptr} \NC not yet written \PDF\ bytes \NC \NR
+\NC \type{pool_ptr} \NC string pool index \NC \NR
+\NC \type{pool_size} \NC current size allocated for string characters \NC \NR
+\NC \type{save_size} \NC save stack size \NC \NR
+\NC \type{shell_escape} \NC \type {0} means disabled, \type {1} means anything is permitted, and \type {2} is restricted \NC \NR
+\NC \type{safer_option} \NC \type {1} means safer is enforced \NC \NR
+\NC \type{kpse_used} \NC \type {1} means that kpse is used \NC \NR
+\NC \type{stack_size} \NC input stack size \NC \NR
+\NC \type{str_ptr} \NC number of strings \NC \NR
+\NC \type{total_pages} \NC number of written pages \NC \NR
+\NC \type{var_mem_max} \NC number of allocated words for nodes \NC \NR
+\NC \type{var_used} \NC variable (one|-|word) memory in use \NC \NR
+\NC \type{lc_collate} \NC the value of \type {LC_COLLATE} at startup time (becomes \type {C} at startup) \NC \NR
+\NC \type{lc_ctype} \NC the value of \type {LC_CTYPE} at startup time (becomes \type {C} at startup) \NC \NR
+%NC \type{lc_monetary} \NC the value of \type {LC_MONETARY} at startup time \NC \NR
+\NC \type{lc_numeric} \NC the value of \type {LC_NUMERIC} at startup time \NC \NR
+%NC \type{lc_time} \NC the value of \type {LC_TIME} at startup time (becomes \type {C} at startup) \NC \NR
+\LL
+\stoptabulate
+
+The error and warning messages can be wiped with the \type {resetmessages}
+function. A return value can be set with \type {setexitcode}.
+
+\stopsection
+
+\startsection[title={The \type {tex} library}][library=tex]
+
+\startsubsection[title={Introduction}]
+
+\topicindex{libraries+\type{tex}}
+
+The \type {tex} table contains a large list of virtual internal \TEX\
+parameters that are partially writable.
+
+The designation \quote {virtual} means that these items are not properly defined
+in \LUA, but are only front\-ends that are handled by a metatable that operates
+on the actual \TEX\ values. As a result, most of the \LUA\ table operators (like
+\type {pairs} and \type {#}) do not work on such items.
+
+At the moment, it is possible to access almost every parameter that you can use
+after \prm {the}, is a single tokens or is sort of special in \TEX. This excludes
+parameters that need extra arguments, like \type {\the\scriptfont}. The subset
+comprising simple integer and dimension registers are writable as well as
+readable (like \prm {tracingcommands} and \prm {parindent}).
+
+\stopsubsection
+
+\startsubsection[title={Internal parameter values, \type {set} and \type {get}}]
+
+\topicindex{parameters+internal}
+
+\libindex{set} \libindex{get}
+
+For all the parameters in this section, it is possible to access them directly
+using their names as index in the \type {tex} table, or by using one of the
+functions \type {tex.get} and \type {tex.set}.
+
+The exact parameters and return values differ depending on the actual parameter,
+and so does whether \type {tex.set} has any effect. For the parameters that {\em
+can} be set, it is possible to use \type {global} as the first argument to \type
+{tex.set}; this makes the assignment global instead of local.
+
+\startfunctioncall
+tex.set (["global",] <string> n, ...)
+... = tex.get (<string> n)
+\stopfunctioncall
+
+Glue is kind of special because there are five values involved. The return value
+is a \nod {glue_spec} node but when you pass \type {false} as last argument to
+\type {tex.get} you get the width of the glue and when you pass \type {true} you
+get all five values. Otherwise you get a node which is a copy of the internal
+value so you are responsible for its freeing at the \LUA\ end. When you set a
+glue quantity you can either pass a \nod {glue_spec} or upto five numbers. If
+you pass \type {true} to \type {get} you get 5 values returned for a glue and
+when you pass \type {false} you only get the width returned.
+
+\subsubsection{Integer parameters}
+
+The integer parameters accept and return \LUA\ numbers. These are read|-|write:
+
+\starttwocolumns
+\starttyping
+tex.adjdemerits
+tex.binoppenalty
+tex.brokenpenalty
+tex.catcodetable
+tex.clubpenalty
+tex.day
+tex.defaulthyphenchar
+tex.defaultskewchar
+tex.delimiterfactor
+tex.displaywidowpenalty
+tex.doublehyphendemerits
+tex.endlinechar
+tex.errorcontextlines
+tex.escapechar
+tex.exhyphenpenalty
+tex.fam
+tex.finalhyphendemerits
+tex.floatingpenalty
+tex.globaldefs
+tex.hangafter
+tex.hbadness
+tex.holdinginserts
+tex.hyphenpenalty
+tex.interlinepenalty
+tex.language
+tex.lastlinefit
+tex.lefthyphenmin
+tex.linepenalty
+tex.localbrokenpenalty
+tex.localinterlinepenalty
+tex.looseness
+tex.mag
+tex.maxdeadcycles
+tex.month
+tex.newlinechar
+tex.outputpenalty
+tex.pausing
+tex.postdisplaypenalty
+tex.predisplaydirection
+tex.predisplaypenalty
+tex.pretolerance
+tex.relpenalty
+tex.righthyphenmin
+tex.savinghyphcodes
+tex.savingvdiscards
+tex.showboxbreadth
+tex.showboxdepth
+tex.time
+tex.tolerance
+tex.tracingassigns
+tex.tracingcommands
+tex.tracinggroups
+tex.tracingifs
+tex.tracinglostchars
+tex.tracingmacros
+tex.tracingnesting
+tex.tracingonline
+tex.tracingoutput
+tex.tracingpages
+tex.tracingparagraphs
+tex.tracingrestores
+tex.tracingscantokens
+tex.tracingstats
+tex.uchyph
+tex.vbadness
+tex.widowpenalty
+tex.year
+\stoptyping
+\stoptwocolumns
+
+These are read|-|only:
+
+\startthreecolumns
+\starttyping
+tex.deadcycles
+tex.insertpenalties
+tex.parshape
+tex.prevgraf
+tex.spacefactor
+\stoptyping
+\stopthreecolumns
+
+\subsubsection{Dimension parameters}
+
+The dimension parameters accept \LUA\ numbers (signifying scaled points) or
+strings (with included dimension). The result is always a number in scaled
+points. These are read|-|write:
+
+\startthreecolumns
+\starttyping
+tex.boxmaxdepth
+tex.delimitershortfall
+tex.displayindent
+tex.displaywidth
+tex.emergencystretch
+tex.hangindent
+tex.hfuzz
+tex.hoffset
+tex.hsize
+tex.lineskiplimit
+tex.mathsurround
+tex.maxdepth
+tex.nulldelimiterspace
+tex.overfullrule
+tex.pagebottomoffset
+tex.pageheight
+tex.pageleftoffset
+tex.pagerightoffset
+tex.pagetopoffset
+tex.pagewidth
+tex.parindent
+tex.predisplaysize
+tex.scriptspace
+tex.splitmaxdepth
+tex.vfuzz
+tex.voffset
+tex.vsize
+tex.prevdepth
+tex.prevgraf
+tex.spacefactor
+\stoptyping
+\stopthreecolumns
+
+These are read|-|only:
+
+\startthreecolumns
+\starttyping
+tex.pagedepth
+tex.pagefilllstretch
+tex.pagefillstretch
+tex.pagefilstretch
+tex.pagegoal
+tex.pageshrink
+tex.pagestretch
+tex.pagetotal
+\stoptyping
+\stopthreecolumns
+
+Beware: as with all \LUA\ tables you can add values to them. So, the following is
+valid:
+
+\starttyping
+tex.foo = 123
+\stoptyping
+
+When you access a \TEX\ parameter a look up takes place. For read||only variables
+that means that you will get something back, but when you set them you create a
+new entry in the table thereby making the original invisible.
+
+There are a few special cases that we make an exception for: \type {prevdepth},
+\type {prevgraf} and \type {spacefactor}. These normally are accessed via the
+\type {tex.nest} table:
+
+\starttyping
+tex.nest[tex.nest.ptr].prevdepth = p
+tex.nest[tex.nest.ptr].spacefactor = s
+\stoptyping
+
+However, the following also works:
+
+\starttyping
+tex.prevdepth = p
+tex.spacefactor = s
+\stoptyping
+
+Keep in mind that when you mess with node lists directly at the \LUA\ end you
+might need to update the top of the nesting stack's \type {prevdepth} explicitly
+as there is no way \LUATEX\ can guess your intentions. By using the accessor in
+the \type {tex} tables, you get and set the values at the top of the nesting
+stack.
+
+\subsubsection{Direction parameters}
+
+The direction states can be queried and set with:
+
+\startthreecolumns
+\starttyping
+tex.gettextdir()
+tex.getlinedir()
+tex.getmathdir()
+tex.getpardir()
+tex.settextdir(<number>)
+tex.setlinedir(<number>)
+tex.setmathdir(<number>)
+tex.setpardir(<number>)
+\stoptyping
+\stopthreecolumns
+
+and also with:
+
+\startthreecolumns
+\starttyping
+tex.textdirection
+tex.linedirection
+tex.mathdirection
+tex.pardirection
+\stoptyping
+\stopthreecolumns
+
+
+\subsubsection{Glue parameters}
+
+The glue parameters accept and return a userdata object that represents a \nod {glue_spec} node.
+
+\startthreecolumns
+\starttyping
+tex.abovedisplayshortskip
+tex.abovedisplayskip
+tex.baselineskip
+tex.belowdisplayshortskip
+tex.belowdisplayskip
+tex.leftskip
+tex.lineskip
+tex.parfillskip
+tex.parskip
+tex.rightskip
+tex.spaceskip
+tex.splittopskip
+tex.tabskip
+tex.topskip
+tex.xspaceskip
+\stoptyping
+\stopthreecolumns
+
+\subsubsection{Muglue parameters}
+
+All muglue parameters are to be used read|-|only and return a \LUA\ string.
+
+\startthreecolumns
+\starttyping
+tex.medmuskip
+tex.thickmuskip
+tex.thinmuskip
+\stoptyping
+\stopthreecolumns
+
+\subsubsection{Tokenlist parameters}
+
+The tokenlist parameters accept and return \LUA\ strings. \LUA\ strings are
+converted to and from token lists using \prm {the} \prm {toks} style expansion:
+all category codes are either space (10) or other (12). It follows that assigning
+to some of these, like \quote {tex.output}, is actually useless, but it feels bad
+to make exceptions in view of a coming extension that will accept full|-|blown
+token strings.
+
+\startthreecolumns
+\starttyping
+tex.errhelp
+tex.everycr
+tex.everydisplay
+tex.everyeof
+tex.everyhbox
+tex.everyjob
+tex.everymath
+tex.everypar
+tex.everyvbox
+tex.output
+\stoptyping
+\stopthreecolumns
+
+\stopsubsection
+
+\startsubsection[title={Convert commands}]
+
+\topicindex{convert commands}
+
+All \quote {convert} commands are read|-|only and return a \LUA\ string. The
+supported commands at this moment are:
+
+\starttwocolumns
+\starttyping
+tex.eTeXVersion
+tex.eTeXrevision
+tex.formatname
+tex.jobname
+tex.luatexbanner
+tex.luatexrevision
+tex.fontname(number)
+tex.uniformdeviate(number)
+tex.number(number)
+tex.romannumeral(number)
+tex.fontidentifier(number)
+\stoptyping
+\stoptwocolumns
+
+If you are wondering why this list looks haphazard; these are all the cases of
+the \quote {convert} internal command that do not require an argument, as well as
+the ones that require only a simple numeric value. The special (\LUA|-|only) case
+of \type {tex.fontidentifier} returns the \type {csname} string that matches a
+font id number (if there is one).
+
+\stopsubsection
+
+\startsubsection[title={Last item commands}]
+
+\topicindex{last items}
+
+All \quote {last item} commands are read|-|only and return a number. The
+supported commands at this moment are:
+
+\startthreecolumns
+\starttyping
+tex.lastpenalty
+tex.lastkern
+tex.lastskip
+tex.lastnodetype
+tex.inputlineno
+tex.lastxpos
+tex.lastypos
+tex.randomseed
+tex.luatexversion
+tex.eTeXminorversion
+tex.eTeXversion
+tex.currentgrouplevel
+tex.currentgrouptype
+tex.currentiflevel
+tex.currentiftype
+tex.currentifbranch
+\stoptyping
+\stopthreecolumns
+
+\stopsubsection
+
+\startsubsection[title={Accessing registers: \type {set*}, \type {get*} and \type {is*}}]
+
+\topicindex{attributes}
+\topicindex{registers}
+
+\libindex{attribute} \libindex{setattribute} \libindex{getattribute} \libindex{isattribute}
+\libindex{count} \libindex{setcount} \libindex{getcount} \libindex{iscount}
+\libindex{dimen} \libindex{setdimen} \libindex{getdimen} \libindex{isdimen}
+\libindex{skip} \libindex{setskip} \libindex{getskip} \libindex{isskip}
+\libindex{muskip} \libindex{setmuskip} \libindex{getmuskip} \libindex{ismuskip}
+\libindex{glue} \libindex{setglue} \libindex{getglue} \libindex{isglue}
+\libindex{muglue} \libindex{setmuglue} \libindex{getmuglue} \libindex{ismuglue}
+\libindex{toks} \libindex{settoks} \libindex{gettoks} \libindex{istoks}
+\libindex{box} \libindex{setbox} \libindex{getbox} \libindex{isbox}
+
+\libindex{scantoks}
+
+\TEX's attributes (\lpr {attribute}), counters (\prm {count}), dimensions (\prm
+{dimen}), skips (\prm {skip}, \prm {muskip}) and token (\prm {toks}) registers
+can be accessed and written to using two times five virtual sub|-|tables of the
+\type {tex} table:
+
+\startthreecolumns
+\starttyping
+tex.attribute
+tex.count
+tex.dimen
+tex.skip
+tex.glue
+tex.muskip
+tex.muglue
+tex.toks
+\stoptyping
+\stopthreecolumns
+
+It is possible to use the names of relevant \lpr {attributedef}, \prm {countdef},
+\prm {dimendef}, \prm {skipdef}, or \prm {toksdef} control sequences as indices
+to these tables:
+
+\starttyping
+tex.count.scratchcounter = 0
+enormous = tex.dimen['maxdimen']
+\stoptyping
+
+In this case, \LUATEX\ looks up the value for you on the fly. You have to use a
+valid \prm {countdef} (or \lpr {attributedef}, or \prm {dimendef}, or \prm
+{skipdef}, or \prm {toksdef}), anything else will generate an error (the intent
+is to eventually also allow \type {<chardef tokens>} and even macros that expand
+into a number).
+
+\startitemize
+
+ \startitem
+ The count registers accept and return \LUA\ numbers.
+ \stopitem
+
+ \startitem
+ The dimension registers accept \LUA\ numbers (in scaled points) or
+ strings (with an included absolute dimension; \type {em} and \type {ex}
+ and \type {px} are forbidden). The result is always a number in scaled
+ points.
+ \stopitem
+
+ \startitem
+ The token registers accept and return \LUA\ strings. \LUA\ strings are
+ converted to and from token lists using \prm {the} \prm {toks} style
+ expansion: all category codes are either space (10) or other (12).
+ \stopitem
+
+ \startitem
+ The skip registers accept and return \nod {glue_spec} userdata node
+ objects (see the description of the node interface elsewhere in this
+ manual).
+ \stopitem
+
+ \startitem
+ The glue registers are just skip registers but instead of userdata
+ are verbose.
+ \stopitem
+
+ \startitem
+ Like the counts, the attribute registers accept and return \LUA\ numbers.
+ \stopitem
+
+\stopitemize
+
+As an alternative to array addressing, there are also accessor functions defined
+for all cases, for example, here is the set of possibilities for \prm {skip}
+registers:
+
+\startfunctioncall
+tex.setskip (["global",] <number> n, <node> s)
+tex.setskip (["global",] <string> s, <node> s)
+<node> s = tex.getskip (<number> n)
+<node> s = tex.getskip (<string> s)
+\stopfunctioncall
+
+We have similar setters for \type {count}, \type {dimen}, \type {muskip}, and
+\type {toks}. Counters and dimen are represented by numbers, skips and muskips by
+nodes, and toks by strings.
+
+Again the glue variants are not using the \nod {glue-spec} userdata nodes. The
+\type {setglue} function accepts upto 5 arguments: width, stretch, shrink,
+stretch order and shrink order and the \type {getglue} function reports them,
+unless the second argument is \type {false} in which care only the width is
+returned.
+
+Here is an example usign a threesome:
+
+\startfunctioncall
+local d = tex.getdimen("foo")
+if tex.isdimen("bar") then
+ tex.setdimen("bar",d)
+end
+\stopfunctioncall
+
+There are six extra skip (glue) related helpers:
+
+\startfunctioncall
+tex.setglue (["global"], <number> n,
+ width, stretch, shrink, stretch_order, shrink_order)
+tex.setglue (["global"], <string> s,
+ width, stretch, shrink, stretch_order, shrink_order)
+width, stretch, shrink, stretch_order, shrink_order =
+ tex.getglue (<number> n)
+width, stretch, shrink, stretch_order, shrink_order =
+ tex.getglue (<string> s)
+\stopfunctioncall
+
+The other two are \type {tex.setmuglue} and \type {tex.getmuglue}.
+
+There are such helpers for \type {dimen}, \type {count}, \type {skip}, \type
+{muskip}, \type {box} and \type {attribute} registers but the glue ones
+are special because they have to deal with more properties.
+
+As with the general \type {get} and \type {set} function discussed before, for
+the skip registers \type {getskip} returns a node and \type {getglue} returns
+numbers, while \type {setskip} accepts a node and \type {setglue} expects upto 5
+numbers. Again, when you pass \type {false} as second argument to \type {getglue}
+you only get the width returned. The same is true for the \type {mu} variants
+\type {getmuskip}, \type {setmuskip}, \type {getmuskip} and\type {setmuskip}.
+
+For tokens registers we have an alternative where a catcode table is specified:
+
+\startfunctioncall
+tex.scantoks(0,3,"$e=mc^2$")
+tex.scantoks("global",0,3,"$\int\limits^1_2$")
+\stopfunctioncall
+
+In the function-based interface, it is possible to define values globally by
+using the string \type {global} as the first function argument.
+
+\stopsubsection
+
+\startsubsection[title={Character code registers: \type {[get|set]*code[s]}}]
+
+\topicindex{characters+codes}
+
+\libindex{lccode} \libindex{setlccode} \libindex{getlccode}
+\libindex{uccode} \libindex{setuccode} \libindex{getuccode}
+\libindex{sfcode} \libindex{setsfcode} \libindex{getsfcode}
+\libindex{catcode} \libindex{setcatcode} \libindex{getcatcode}
+\libindex{mathcode} \libindex{setmathcode} \libindex{getmathcode}
+\libindex{delcode} \libindex{setdelcode} \libindex{getdelcode}
+
+\libindex{setdelcodes} \libindex{getdelcodes}
+\libindex{setmathcodes} \libindex{getmathcodes}
+
+\TEX's character code tables (\prm {lccode}, \prm {uccode}, \prm {sfcode}, \prm
+{catcode}, \prm {mathcode}, \prm {delcode}) can be accessed and written to using
+six virtual subtables of the \type {tex} table
+
+\startthreecolumns
+\starttyping
+tex.lccode
+tex.uccode
+tex.sfcode
+tex.catcode
+tex.mathcode
+tex.delcode
+\stoptyping
+\stopthreecolumns
+
+The function call interfaces are roughly as above, but there are a few twists.
+\type {sfcode}s are the simple ones:
+
+\startfunctioncall
+tex.setsfcode (["global",] <number> n, <number> s)
+<number> s = tex.getsfcode (<number> n)
+\stopfunctioncall
+
+The function call interface for \type {lccode} and \type {uccode} additionally
+allows you to set the associated sibling at the same time:
+
+\startfunctioncall
+tex.setlccode (["global"], <number> n, <number> lc)
+tex.setlccode (["global"], <number> n, <number> lc, <number> uc)
+<number> lc = tex.getlccode (<number> n)
+tex.setuccode (["global"], <number> n, <number> uc)
+tex.setuccode (["global"], <number> n, <number> uc, <number> lc)
+<number> uc = tex.getuccode (<number> n)
+\stopfunctioncall
+
+The function call interface for \type {catcode} also allows you to specify a
+category table to use on assignment or on query (default in both cases is the
+current one):
+
+\startfunctioncall
+tex.setcatcode (["global"], <number> n, <number> c)
+tex.setcatcode (["global"], <number> cattable, <number> n, <number> c)
+<number> lc = tex.getcatcode (<number> n)
+<number> lc = tex.getcatcode (<number> cattable, <number> n)
+\stopfunctioncall
+
+The interfaces for \type {delcode} and \type {mathcode} use small array tables to
+set and retrieve values:
+
+\startfunctioncall
+tex.setmathcode (["global"], <number> n, <table> mval )
+<table> mval = tex.getmathcode (<number> n)
+tex.setdelcode (["global"], <number> n, <table> dval )
+<table> dval = tex.getdelcode (<number> n)
+\stopfunctioncall
+
+Where the table for \type {mathcode} is an array of 3 numbers, like this:
+
+\starttyping
+{
+ <number> class,
+ <number> family,
+ <number> character
+}
+\stoptyping
+
+And the table for \type {delcode} is an array with 4 numbers, like this:
+
+\starttyping
+{
+ <number> small_fam,
+ <number> small_char,
+ <number> large_fam,
+ <number> large_char
+}
+\stoptyping
+
+You can also avoid the table:
+
+\startfunctioncall
+tex.setmathcode (["global"], <number> n, <number> class,
+ <number> family, <number> character)
+class, family, char =
+ tex.getmathcodes (<number> n)
+tex.setdelcode (["global"], <number> n, <number> smallfam,
+ <number> smallchar, <number> largefam, <number> largechar)
+smallfam, smallchar, largefam, largechar =
+ tex.getdelcodes (<number> n)
+\stopfunctioncall
+
+Normally, the third and fourth values in a delimiter code assignment will be zero
+according to \lpr {Udelcode} usage, but the returned table can have values there
+(if the delimiter code was set using \prm {delcode}, for example). Unset \type
+{delcode}'s can be recognized because \type {dval[1]} is $-1$.
+
+\stopsubsection
+
+\startsubsection[title={Box registers: \type {[get|set]box}}]
+
+\topicindex{registers}
+\topicindex{boxes}
+
+\libindex{box}
+\libindex{setbox} \libindex{getbox}
+
+It is possible to set and query actual boxes, coming for instance from \prm
+{hbox}, \prm {vbox} or \prm {vtop}, using the node interface as defined in the
+\type {node} library:
+
+\starttyping
+tex.box
+\stoptyping
+
+for array access, or
+
+\starttyping
+tex.setbox(["global",] <number> n, <node> s)
+tex.setbox(["global",] <string> cs, <node> s)
+<node> n = tex.getbox(<number> n)
+<node> n = tex.getbox(<string> cs)
+\stoptyping
+
+for function|-|based access. In the function-based interface, it is possible to
+define values globally by using the string \type {global} as the first function
+argument.
+
+Be warned that an assignment like
+
+\starttyping
+tex.box[0] = tex.box[2]
+\stoptyping
+
+does not copy the node list, it just duplicates a node pointer. If \type {\box2}
+will be cleared by \TEX\ commands later on, the contents of \type {\box0} becomes
+invalid as well. To prevent this from happening, always use \type
+{node.copy_list} unless you are assigning to a temporary variable:
+
+\starttyping
+tex.box[0] = node.copy_list(tex.box[2])
+\stoptyping
+
+\stopsubsection
+
+\startsubsection[title={\type {triggerbuildpage}}]
+
+\topicindex{pages}
+
+\libindex{triggerbuildpage}
+
+You should not expect to much from the \type {triggerbuildpage} helpers because
+often \TEX\ doesn't do much if it thinks nothing has to be done, but it might be
+useful for some applications. It just does as it says it calls the internal
+function that build a page, given that there is something to build.
+
+\stopsubsection
+
+\startsubsection[title={\type {splitbox}}]
+
+\topicindex{boxes+split}
+
+\libindex{splitbox}
+
+You can split a box:
+
+\starttyping
+local vlist = tex.splitbox(n,height,mode)
+\stoptyping
+
+The remainder is kept in the original box and a packaged vlist is returned. This
+operation is comparable to the \prm {vsplit} operation. The mode can be \type
+{additional} or \type {exactly} and concerns the split off box.
+
+\stopsubsection
+
+\startsubsection[title={Accessing math parameters: \type {[get|set]math}}]
+
+\topicindex{math+parameters}
+\topicindex{parameters+math}
+
+\libindex{setmath}
+\libindex{getmath}
+
+It is possible to set and query the internal math parameters using:
+
+\startfunctioncall
+tex.setmath(["global",] <string> n, <string> t, <number> n)
+<number> n = tex.getmath(<string> n, <string> t)
+\stopfunctioncall
+
+As before an optional first parameter \type {global} indicates a global
+assignment.
+
+The first string is the parameter name minus the leading \quote {Umath}, and the
+second string is the style name minus the trailing \quote {style}. Just to be
+complete, the values for the math parameter name are:
+
+\starttyping
+quad axis operatorsize
+overbarkern overbarrule overbarvgap
+underbarkern underbarrule underbarvgap
+radicalkern radicalrule radicalvgap
+radicaldegreebefore radicaldegreeafter radicaldegreeraise
+stackvgap stacknumup stackdenomdown
+fractionrule fractionnumvgap fractionnumup
+fractiondenomvgap fractiondenomdown fractiondelsize
+limitabovevgap limitabovebgap limitabovekern
+limitbelowvgap limitbelowbgap limitbelowkern
+underdelimitervgap underdelimiterbgap
+overdelimitervgap overdelimiterbgap
+subshiftdrop supshiftdrop subshiftdown
+subsupshiftdown subtopmax supshiftup
+supbottommin supsubbottommax subsupvgap
+spaceafterscript connectoroverlapmin
+ordordspacing ordopspacing ordbinspacing ordrelspacing
+ordopenspacing ordclosespacing ordpunctspacing ordinnerspacing
+opordspacing opopspacing opbinspacing oprelspacing
+opopenspacing opclosespacing oppunctspacing opinnerspacing
+binordspacing binopspacing binbinspacing binrelspacing
+binopenspacing binclosespacing binpunctspacing bininnerspacing
+relordspacing relopspacing relbinspacing relrelspacing
+relopenspacing relclosespacing relpunctspacing relinnerspacing
+openordspacing openopspacing openbinspacing openrelspacing
+openopenspacing openclosespacing openpunctspacing openinnerspacing
+closeordspacing closeopspacing closebinspacing closerelspacing
+closeopenspacing closeclosespacing closepunctspacing closeinnerspacing
+punctordspacing punctopspacing punctbinspacing punctrelspacing
+punctopenspacing punctclosespacing punctpunctspacing punctinnerspacing
+innerordspacing inneropspacing innerbinspacing innerrelspacing
+inneropenspacing innerclosespacing innerpunctspacing innerinnerspacing
+\stoptyping
+
+The values for the style parameter are:
+
+\starttyping
+display crampeddisplay
+text crampedtext
+script crampedscript
+scriptscript crampedscriptscript
+\stoptyping
+
+The value is either a number (representing a dimension or number) or a glue spec
+node representing a muskip for \type {ordordspacing} and similar spacing
+parameters.
+
+\stopsubsection
+
+\startsubsection[title={Special list heads: \type {[get|set]list}}]
+
+\topicindex{lists}
+
+\libindex{lists}
+\libindex{setlist}
+\libindex{getlist}
+
+The virtual table \type {tex.lists} contains the set of internal registers that
+keep track of building page lists.
+
+\starttabulate[|l|p|]
+\DB field \BC explanation \NC \NR
+\TB
+\NC \type{page_ins_head} \NC circular list of pending insertions \NC \NR
+\NC \type{contrib_head} \NC the recent contributions \NC \NR
+\NC \type{page_head} \NC the current page content \NC \NR
+%NC \type{temp_head} \NC \NC \NR
+\NC \type{hold_head} \NC used for held-over items for next page \NC \NR
+\NC \type{adjust_head} \NC head of the current \prm {vadjust} list \NC \NR
+\NC \type{pre_adjust_head} \NC head of the current \type {\vadjust pre} list \NC \NR
+%NC \type{align_head} \NC \NC \NR
+\NC \type{page_discards_head} \NC head of the discarded items of a page break \NC \NR
+\NC \type{split_discards_head} \NC head of the discarded items in a vsplit \NC \NR
+\LL
+\stoptabulate
+
+The getter and setter functions are \type {getlist} and \type {setlist}. You have
+to be careful with what you set as \TEX\ can have expectations with regards to
+how a list is constructed or in what state it is.
+
+\stopsubsection
+
+\startsubsection[title={Semantic nest levels: \type {getnest} and \type {ptr}}]
+
+\topicindex{nesting}
+
+\libindex{nest}
+\libindex{ptr}
+%libindex{setnest} % only a message
+\libindex{getnest}
+
+The virtual table \type {nest} contains the currently active semantic nesting
+state. It has two main parts: a zero-based array of userdata for the semantic
+nest itself, and the numerical value \type {ptr}, which gives the highest
+available index. Neither the array items in \type {nest[]} nor \type {ptr} can be
+assigned to (as this would confuse the typesetting engine beyond repair), but you
+can assign to the individual values inside the array items, e.g.\ \type
+{tex.nest[tex.nest.ptr].prevdepth}.
+
+\type {tex.nest[tex.nest.ptr]} is the current nest state, \type {nest[0]} the
+outermost (main vertical list) level. The getter function is \type {getnest}. You
+can pass a number (which gives you a list), nothing or \type {top}, which returns
+the topmost list, or the string \type {ptr} which gives you the index of the
+topmost list.
+
+The known fields are:
+
+\starttabulate[|l|l|l|p|]
+\DB key \BC type \BC modes \BC explanation \NC \NR
+\TB
+\NC \type{mode} \NC number \NC all \NC the meaning of these numbers depends on the engine
+ and sometimes even the version; you can use \typ
+ {tex.getmodevalues()} to get the mapping: positive
+ values signal vertical, horizontal and math mode,
+ while negative values indicate inner and inline
+ variants \NC \NR
+\NC \type{modeline} \NC number \NC all \NC source input line where this mode was entered in,
+ negative inside the output routine \NC \NR
+\NC \type{head} \NC node \NC all \NC the head of the current list \NC \NR
+\NC \type{tail} \NC node \NC all \NC the tail of the current list \NC \NR
+\NC \type{prevgraf} \NC number \NC vmode \NC number of lines in the previous paragraph \NC \NR
+\NC \type{prevdepth} \NC number \NC vmode \NC depth of the previous paragraph \NC \NR
+\NC \type{spacefactor} \NC number \NC hmode \NC the current space factor \NC \NR
+\NC \type{dirs} \NC node \NC hmode \NC used for temporary storage by the line break algorithm\NC \NR
+\NC \type{noad} \NC node \NC mmode \NC used for temporary storage of a pending fraction numerator,
+ for \prm {over} etc. \NC \NR
+\NC \type{delimptr} \NC node \NC mmode \NC used for temporary storage of the previous math delimiter,
+ for \prm {middle} \NC \NR
+\NC \type{mathdir} \NC boolean \NC mmode \NC true when during math processing the \lpr {mathdir} is not
+ the same as the surrounding \lpr {textdir} \NC \NR
+\NC \type{mathstyle} \NC number \NC mmode \NC the current \lpr {mathstyle} \NC \NR
+\LL
+\stoptabulate
+
+\stopsubsection
+
+\startsubsection[reference=sec:luaprint,title={Print functions}]
+
+\topicindex{printing}
+
+The \type {tex} table also contains the three print functions that are the major
+interface from \LUA\ scripting to \TEX. The arguments to these three functions
+are all stored in an in|-|memory virtual file that is fed to the \TEX\ scanner as
+the result of the expansion of \prm {directlua}.
+
+The total amount of returnable text from a \prm {directlua} command is only
+limited by available system \RAM. However, each separate printed string has to
+fit completely in \TEX's input buffer. The result of using these functions from
+inside callbacks is undefined at the moment.
+
+\subsubsection{\type {print}}
+
+\libindex{print}
+
+\startfunctioncall
+tex.print(<string> s, ...)
+tex.print(<number> n, <string> s, ...)
+tex.print(<table> t)
+tex.print(<number> n, <table> t)
+\stopfunctioncall
+
+Each string argument is treated by \TEX\ as a separate input line. If there is a
+table argument instead of a list of strings, this has to be a consecutive array
+of strings to print (the first non-string value will stop the printing process).
+
+The optional parameter can be used to print the strings using the catcode regime
+defined by \lpr {catcodetable}~\type {n}. If \type {n} is $-1$, the currently
+active catcode regime is used. If \type {n} is $-2$, the resulting catcodes are
+the result of \prm {the} \prm {toks}: all category codes are 12 (other) except for
+the space character, that has category code 10 (space). Otherwise, if \type {n}
+is not a valid catcode table, then it is ignored, and the currently active
+catcode regime is used instead.
+
+The very last string of the very last \type {tex.print} command in a \prm
+{directlua} will not have the \prm {endlinechar} appended, all others do.
+
+\subsubsection{\type {sprint}}
+
+\libindex{sprint}
+
+\startfunctioncall
+tex.sprint(<string> s, ...)
+tex.sprint(<number> n, <string> s, ...)
+tex.sprint(<table> t)
+tex.sprint(<number> n, <table> t)
+\stopfunctioncall
+
+Each string argument is treated by \TEX\ as a special kind of input line that
+makes it suitable for use as a partial line input mechanism:
+
+\startitemize[packed]
+\startitem
+ \TEX\ does not switch to the \quote {new line} state, so that leading spaces
+ are not ignored.
+\stopitem
+\startitem
+ No \prm {endlinechar} is inserted.
+\stopitem
+\startitem
+ Trailing spaces are not removed. Note that this does not prevent \TEX\ itself
+ from eating spaces as result of interpreting the line. For example, in
+
+ \starttyping
+ before\directlua{tex.sprint("\\relax")tex.sprint(" inbetween")}after
+ \stoptyping
+
+ the space before \type {in between} will be gobbled as a result of the \quote
+ {normal} scanning of \prm {relax}.
+\stopitem
+\stopitemize
+
+If there is a table argument instead of a list of strings, this has to be a
+consecutive array of strings to print (the first non-string value will stop the
+printing process).
+
+The optional argument sets the catcode regime, as with \type {tex.print}. This
+influences the string arguments (or numbers turned into strings).
+
+Although this needs to be used with care, you can also pass token or node
+userdata objects. These get injected into the stream. Tokens had best be valid
+tokens, while nodes need to be around when they get injected. Therefore it is
+important to realize the following:
+
+\startitemize
+\startitem
+ When you inject a token, you need to pass a valid token userdata object. This
+ object will be collected by \LUA\ when it no longer is referenced. When it gets
+ printed to \TEX\ the token itself gets copied so there is no interference with the
+ \LUA\ garbage collection. You manage the object yourself. Because tokens are
+ actually just numbers, there is no real extra overhead at the \TEX\ end.
+\stopitem
+\startitem
+ When you inject a node, you need to pass a valid node userdata object. The
+ node related to the object will not be collected by \LUA\ when it no longer
+ is referenced. It lives on at the \TEX\ end in its own memory space. When it
+ gets printed to \TEX\ the node reference is used assuming that node stays
+ around. There is no \LUA\ garbage collection involved. Again, you manage the
+ object yourself. The node itself is freed when \TEX\ is done with it.
+\stopitem
+\stopitemize
+
+If you consider the last remark you might realize that we have a problem when a
+printed mix of strings, tokens and nodes is reused. Inside \TEX\ the sequence
+becomes a linked list of input buffers. So, \type {"123"} or \type {"\foo{123}"}
+gets read and parsed on the fly, while \typ {<token userdata>} already is
+tokenized and effectively is a token list now. A \typ {<node userdata>} is also
+tokenized into a token list but it has a reference to a real node. Normally this
+goes fine. But now assume that you store the whole lot in a macro: in that case
+the tokenized node can be flushed many times. But, after the first such flush the
+node is used and its memory freed. You can prevent this by using copies which is
+controlled by setting \lpr {luacopyinputnodes} to a non|-|zero value. This is one
+of these fuzzy areas you have to live with if you really mess with these low
+level issues.
+
+\subsubsection{\type {tprint}}
+
+\libindex{tprint}
+
+\startfunctioncall
+tex.tprint({<number> n, <string> s, ...}, {...})
+\stopfunctioncall
+
+This function is basically a shortcut for repeated calls to \type
+{tex.sprint(<number> n, <string> s, ...)}, once for each of the supplied argument
+tables.
+
+\subsubsection{\type {cprint}}
+
+\libindex{cprint}
+
+This function takes a number indicating the to be used catcode, plus either a
+table of strings or an argument list of strings that will be pushed into the
+input stream.
+
+\startfunctioncall
+tex.cprint( 1," 1: $&{\\foo}") tex.print("\\par") -- a lot of \bgroup s
+tex.cprint( 2," 2: $&{\\foo}") tex.print("\\par") -- matching \egroup s
+tex.cprint( 9," 9: $&{\\foo}") tex.print("\\par") -- all get ignored
+tex.cprint(10,"10: $&{\\foo}") tex.print("\\par") -- all become spaces
+tex.cprint(11,"11: $&{\\foo}") tex.print("\\par") -- letters
+tex.cprint(12,"12: $&{\\foo}") tex.print("\\par") -- other characters
+tex.cprint(14,"12: $&{\\foo}") tex.print("\\par") -- comment triggers
+\stopfunctioncall
+
+% \subsubsection{\type {write}, \type {twrite}, \type {nwrite}}
+\subsubsection{\type {write}}
+
+\libindex{write}
+% \libindex{twrite}
+% \libindex{nwrite}
+
+\startfunctioncall
+tex.write(<string> s, ...)
+tex.write(<table> t)
+\stopfunctioncall
+
+Each string argument is treated by \TEX\ as a special kind of input line that
+makes it suitable for use as a quick way to dump information:
+
+\startitemize
+\item All catcodes on that line are either \quote{space} (for '~') or \quote
+ {character} (for all others).
+\item There is no \prm {endlinechar} appended.
+\stopitemize
+
+If there is a table argument instead of a list of strings, this has to be a
+consecutive array of strings to print (the first non-string value will stop the
+printing process).
+
+% The functions \type {twrite} and \type {nwrite} can be used to write a token or
+% node back to \TEX\, possibly intermixed with regular strings that will be
+% tokenized. You have to make sure that you pass the right data because sometimes
+% \TEX\ has expectations that need to be met.
+
+\stopsubsection
+
+\startsubsection[title={Helper functions}]
+
+\subsubsection{\type {round}}
+
+\topicindex {helpers}
+
+\libindex{round}
+
+\startfunctioncall
+<number> n = tex.round(<number> o)
+\stopfunctioncall
+
+Rounds \LUA\ number \type {o}, and returns a number that is in the range of a
+valid \TEX\ register value. If the number starts out of range, it generates a
+\quote {number too big} error as well.
+
+\subsubsection{\type {scale}}
+
+\libindex{scale}
+
+\startfunctioncall
+<number> n = tex.scale(<number> o, <number> delta)
+<table> n = tex.scale(table o, <number> delta)
+\stopfunctioncall
+
+Multiplies the \LUA\ numbers \type {o} and \nod {delta}, and returns a rounded
+number that is in the range of a valid \TEX\ register value. In the table
+version, it creates a copy of the table with all numeric top||level values scaled
+in that manner. If the multiplied number(s) are of range, it generates
+\quote{number too big} error(s) as well.
+
+Note: the precision of the output of this function will depend on your computer's
+architecture and operating system, so use with care! An interface to \LUATEX's
+internal, 100\% portable scale function will be added at a later date.
+
+\subsubsection{\type {number} and \type {romannumeral}}
+
+\libindex{number}
+\libindex{romannumeral}
+
+These are the companions to the primitives \prm {number} and \prm
+{romannumeral}. They can be used like:
+
+\startfunctioncall
+tex.print(tex.romannumeral(123))
+\stopfunctioncall
+
+\subsubsection{\type {fontidentifier} and \type {fontname}}
+
+\libindex{fontidentifier}
+\libindex{fontname}
+
+The first one returns the name only, the second one reports the size too.
+
+\startfunctioncall
+tex.print(tex.fontname(tex.fontname))
+tex.print(tex.fontname(tex.fontidentidier))
+\stopfunctioncall
+
+\subsubsection{\type {sp}}
+
+\libindex{sp}
+
+\startfunctioncall
+<number> n = tex.sp(<number> o)
+<number> n = tex.sp(<string> s)
+\stopfunctioncall
+
+Converts the number \type {o} or a string \type {s} that represents an explicit
+dimension into an integer number of scaled points.
+
+For parsing the string, the same scanning and conversion rules are used that
+\LUATEX\ would use if it was scanning a dimension specifier in its \TEX|-|like
+input language (this includes generating errors for bad values), expect for the
+following:
+
+\startitemize[n]
+\startitem
+ only explicit values are allowed, control sequences are not handled
+\stopitem
+\startitem
+ infinite dimension units (\type {fil...}) are forbidden
+\stopitem
+\startitem
+ \type {mu} units do not generate an error (but may not be useful either)
+\stopitem
+\stopitemize
+
+\subsubsection{\type {tex.getlinenumber} and \type {tex.setlinenumber}}
+
+\libindex{getlinenumber}
+\libindex{setlinenumber}
+
+You can mess with the current line number:
+
+\startfunctioncall
+local n = tex.getlinenumber()
+tex.setlinenumber(n+10)
+\stopfunctioncall
+
+which can be shortcut to:
+
+\startfunctioncall
+tex.setlinenumber(10,true)
+\stopfunctioncall
+
+This might be handy when you have a callback that read numbers from a file and
+combines them in one line (in which case an error message probably has to refer
+to the original line). Interference with \TEX's internal handling of numbers is
+of course possible.
+
+\subsubsection{\type {error}, \type {show_context} and \type {gethelptext}}
+
+\topicindex{errors}
+
+\libindex{error}
+\libindex{show_context}
+\libindex{gethelptext}
+
+\startfunctioncall
+tex.error(<string> s)
+tex.error(<string> s, <table> help)
+<string> s = tex.gethelptext()
+\stopfunctioncall
+
+This creates an error somewhat like the combination of \prm {errhelp} and \prm
+{errmessage} would. During this error, deletions are disabled.
+
+The array part of the \type {help} table has to contain strings, one for each
+line of error help.
+
+In case of an error the \type {show_context} function will show the current
+context where we're at (in the expansion).
+
+\subsubsection{\type {getfamilyoffont}}
+
+\libindex {getfamilyoffont}
+
+When you pass a proper family identifier the next helper will return the font
+currently associated with it.
+
+\startfunctioncall
+<integer> id = font.getfamilyoffont(<integer> fam)
+\stopfunctioncall
+
+\subsubsection{\type {[set|get]interaction}}
+
+\libindex{setinteraction}
+\libindex{getinteraction}
+
+The engine can in one of four modes:
+
+\starttabulate[|lT|l|pl|]
+\DB value \NC mode \BC meaning \NC \NR
+\TB
+\NC 0 \NC batch \NC omits all stops and omits terminal output \NC \NR
+\NC 1 \NC nonstop \NC omits all stops \NC \NR
+\NC 2 \NC scroll \NC omits error stops \NC \NR
+\NC 3 \NC errorstop \NC stops at every opportunity to interact \NC \NR
+\LL
+\stoptabulate
+
+The mode can be queried and set with:
+
+\startfunctioncall
+<integer> i = tex.getinteraction()
+tex.setinteraction(<integer> i)
+\stopfunctioncall
+
+\subsubsection{\type {runtoks} and \type {quittoks}}
+
+Because of the fact that \TEX\ is in a complex dance of expanding, dealing with
+fonts, typesetting paragraphs, messing around with boxes, building pages, and so
+on, you cannot easily run a nested \TEX\ run (read nested main loop). However,
+there is an option to force a local run with \type {runtoks}. The content of the
+given token list register gets expanded locally after which we return to where we
+triggered this expansion, at the \LUA\ end. Instead a function can get passed
+that does some work. You have to make sure that at the end \TEX\ is in a sane
+state and this is not always trivial. A more complex mechanism would complicate
+\TEX\ itself (and probably also harm performance) so this simple local expansion
+loop has to do.
+
+\startfunctioncall
+tex.runtoks(<token register>)
+tex.runtoks(<lua function>)
+tex.runtoks(<macro name>)
+tex.runtoks(<register name>)
+\stopfunctioncall
+
+When the \prm {tracingnesting} parameter is set to a value larger than~2 some
+information is reported about the state of the local loop. The return value indicates
+an error:
+
+\starttabulate[|lT|pl|]
+\DB value \NC meaning \NC \NR
+\TB
+\NC 0 \NC no error \NC \NR
+\NC 1 \NC bad register number \NC \NR
+\NC 2 \NC unknown macro or register name \NC \NR
+\NC 3 \NC macro is unsuitable for runtoks (has arguments) \NC \NR
+\LL
+\stoptabulate
+
+This function has two optional arguments in case a token register is passed:
+
+\startfunctioncall
+tex.runtoks(<token register>,force,grouped)
+\stopfunctioncall
+
+Inside for instance an \type {\edef} the \type {runtoks} function behaves (at
+least tries to) like it were an \type {\the}. This prevents unwanted side
+effects: normally in such an definition tokens remain tokens and (for instance)
+characters don't become nodes. With the second argument you can force the local
+main loop, no matter what. The third argument adds a level of grouping.
+
+You can quit the local loop with \type {\endlocalcontrol} or from the \LUA\ end
+with \type {tex.quittoks}. In that case you end one level up! Of course in the
+end that can mean that you arrive at the main level in which case an extra end
+will trigger a redundancy warning (not an abort!).
+
+\subsubsection{\type {forcehmode}}
+
+\libindex{forcehmode}
+
+An example of a (possible error triggering) complication is that \TEX\ expects to
+be in some state, say horizontal mode, and you have to make sure it is when you
+start feeding back something from \LUA\ into \TEX. Normally a user will not run
+into issues but when you start writing tokens or nodes or have a nested run there
+can be situations that you need to run \type {forcehmode}. There is no recipe for
+this and intercepting possible cases would weaken \LUATEX's flexibility.
+
+\subsubsection{\type {hashtokens}}
+
+\libindex{hashtokens}
+
+\topicindex{hash}
+
+\startfunctioncall
+for i,v in pairs (tex.hashtokens()) do ... end
+\stopfunctioncall
+
+Returns a list of names. This can be useful for debugging, but note that this
+also reports control sequences that may be unreachable at this moment due to
+local redefinitions: it is strictly a dump of the hash table. You can use \type
+{token.create} to inspect properties, for instance when the \type {command} key
+in a created table equals \type {123}, you have the \type {cmdname} value \type
+{undefined_cs}.
+
+\subsubsection{\type {definefont}}
+
+\topicindex{fonts+defining}
+
+\libindex{definefont}
+
+\startfunctioncall
+tex.definefont(<string> csname, <number> fontid)
+tex.definefont(<boolean> global, <string> csname, <number> fontid)
+\stopfunctioncall
+
+Associates \type {csname} with the internal font number \type {fontid}. The
+definition is global if (and only if) \type {global} is specified and true (the
+setting of \type {globaldefs} is not taken into account).
+
+\stopsubsection
+
+\startsubsection[reference=luaprimitives,title={Functions for dealing with primitives}]
+
+\subsubsection{\type {enableprimitives}}
+
+\libindex{enableprimitives}
+
+\topicindex{initialization}
+\topicindex{primitives}
+
+\startfunctioncall
+tex.enableprimitives(<string> prefix, <table> primitive names)
+\stopfunctioncall
+
+This function accepts a prefix string and an array of primitive names. For each
+combination of \quote {prefix} and \quote {name}, the \type
+{tex.enableprimitives} first verifies that \quote {name} is an actual primitive
+(it must be returned by one of the \type {tex.extraprimitives} calls explained
+below, or part of \TEX82, or \prm {directlua}). If it is not, \type
+{tex.enableprimitives} does nothing and skips to the next pair.
+
+But if it is, then it will construct a csname variable by concatenating the
+\quote {prefix} and \quote {name}, unless the \quote {prefix} is already the
+actual prefix of \quote {name}. In the latter case, it will discard the \quote
+{prefix}, and just use \quote {name}.
+
+Then it will check for the existence of the constructed csname. If the csname is
+currently undefined (note: that is not the same as \prm {relax}), it will
+globally define the csname to have the meaning: run code belonging to the
+primitive \quote {name}. If for some reason the csname is already defined, it
+does nothing and tries the next pair.
+
+An example:
+
+\starttyping
+tex.enableprimitives('LuaTeX', {'formatname'})
+\stoptyping
+
+will define \type {\LuaTeXformatname} with the same intrinsic meaning as the
+documented primitive \lpr {formatname}, provided that the control sequences \type
+{\LuaTeXformatname} is currently undefined.
+
+When \LUATEX\ is run with \type {--ini} only the \TEX82 primitives and \prm
+{directlua} are available, so no extra primitives {\bf at all}.
+
+If you want to have all the new functionality available using their default
+names, as it is now, you will have to add
+
+\starttyping
+\ifx\directlua\undefined \else
+ \directlua {tex.enableprimitives('',tex.extraprimitives ())}
+\fi
+\stoptyping
+
+near the beginning of your format generation file. Or you can choose different
+prefixes for different subsets, as you see fit.
+
+Calling some form of \type {tex.enableprimitives} is highly important though,
+because if you do not, you will end up with a \TEX82-lookalike that can run \LUA\
+code but not do much else. The defined csnames are (of course) saved in the
+format and will be available at runtime.
+
+\subsubsection{\type {extraprimitives}}
+
+\libindex{extraprimitives}
+
+\startfunctioncall
+<table> t = tex.extraprimitives(<string> s, ...)
+\stopfunctioncall
+
+This function returns a list of the primitives that originate from the engine(s)
+given by the requested string value(s). The possible values and their (current)
+return values are given in the following table. In addition the somewhat special
+primitives \quote{\tex{ }}, \quote{\tex {/}} and \quote{\type {-}} are defined.
+
+\startluacode
+function document.showprimitives(tag)
+ for k, v in table.sortedpairs(tex.extraprimitives(tag)) do
+ if v ~= ' ' and v ~= "/" and v ~= "-" then
+ context.type(v)
+ context.space()
+ end
+ end
+end
+\stopluacode
+
+\starttabulate[|l|pl|]
+\DB name \BC values \NC \NR
+\TB
+\NC tex \NC \ctxlua{document.showprimitives('tex') } \NC \NR
+\NC core \NC \ctxlua{document.showprimitives('core') } \NC \NR
+\NC etex \NC \ctxlua{document.showprimitives('etex') } \NC \NR
+\NC luatex \NC \ctxlua{document.showprimitives('luatex') } \NC \NR
+\LL
+\stoptabulate
+
+Note that \type {luatex} does not contain \type {directlua}, as that is
+considered to be a core primitive, along with all the \TEX82 primitives, so it is
+part of the list that is returned from \type {'core'}.
+
+Running \type {tex.extraprimitives} will give you the complete list of
+primitives \type {-ini} startup. It is exactly equivalent to \type
+{tex.extraprimitives("etex","luatex")}.
+
+\subsubsection{\type {primitives}}
+
+\libindex{primitives}
+
+\startfunctioncall
+<table> t = tex.primitives()
+\stopfunctioncall
+
+This function returns a list of all primitives that \LUATEX\ knows about.
+
+\stopsubsection
+
+\startsubsection[title={Core functionality interfaces}]
+
+\subsubsection{\type {badness}}
+
+\libindex{badness}
+
+\startfunctioncall
+<number> b = tex.badness(<number> t, <number> s)
+\stopfunctioncall
+
+This helper function is useful during linebreak calculations. \type {t} and \type
+{s} are scaled values; the function returns the badness for when total \type {t}
+is supposed to be made from amounts that sum to \type {s}. The returned number is
+a reasonable approximation of \mathematics {100(t/s)^3};
+
+\subsubsection{\type {tex.resetparagraph}}
+
+\topicindex {paragraphs+reset}
+
+\libindex{resetparagraph}
+
+This function resets the parameters that \TEX\ normally resets when a new paragraph
+is seen.
+
+\subsubsection{\type {linebreak}}
+
+\topicindex {linebreaks}
+
+\libindex{linebreak}
+
+\startfunctioncall
+local <node> nodelist, <table> info =
+ tex.linebreak(<node> listhead, <table> parameters)
+\stopfunctioncall
+
+The understood parameters are as follows:
+
+\starttabulate[|l|l|p|]
+\DB name \BC type \BC explanation \NC \NR
+\TB
+\NC \type{pardir} \NC string \NC \NC \NR
+\NC \type{pretolerance} \NC number \NC \NC \NR
+\NC \type{tracingparagraphs} \NC number \NC \NC \NR
+\NC \type{tolerance} \NC number \NC \NC \NR
+\NC \type{looseness} \NC number \NC \NC \NR
+\NC \type{hyphenpenalty} \NC number \NC \NC \NR
+\NC \type{exhyphenpenalty} \NC number \NC \NC \NR
+\NC \type{pdfadjustspacing} \NC number \NC \NC \NR
+\NC \type{adjdemerits} \NC number \NC \NC \NR
+\NC \type{pdfprotrudechars} \NC number \NC \NC \NR
+\NC \type{linepenalty} \NC number \NC \NC \NR
+\NC \type{lastlinefit} \NC number \NC \NC \NR
+\NC \type{doublehyphendemerits} \NC number \NC \NC \NR
+\NC \type{finalhyphendemerits} \NC number \NC \NC \NR
+\NC \type{hangafter} \NC number \NC \NC \NR
+\NC \type{interlinepenalty} \NC number or table \NC if a table, then it is an array like \prm {interlinepenalties} \NC \NR
+\NC \type{clubpenalty} \NC number or table \NC if a table, then it is an array like \prm {clubpenalties} \NC \NR
+\NC \type{widowpenalty} \NC number or table \NC if a table, then it is an array like \prm {widowpenalties} \NC \NR
+\NC \type{brokenpenalty} \NC number \NC \NC \NR
+\NC \type{emergencystretch} \NC number \NC in scaled points \NC \NR
+\NC \type{hangindent} \NC number \NC in scaled points \NC \NR
+\NC \type{hsize} \NC number \NC in scaled points \NC \NR
+\NC \type{leftskip} \NC glue_spec node \NC \NC \NR
+\NC \type{rightskip} \NC glue_spec node \NC \NC \NR
+\NC \type{parshape} \NC table \NC \NC \NR
+\LL
+\stoptabulate
+
+Note that there is no interface for \prm {displaywidowpenalties}, you have to
+pass the right choice for \type {widowpenalties} yourself.
+
+It is your own job to make sure that \type {listhead} is a proper paragraph list:
+this function does not add any nodes to it. To be exact, if you want to replace
+the core line breaking, you may have to do the following (when you are not
+actually working in the \cbk {pre_linebreak_filter} or \cbk {linebreak_filter}
+callbacks, or when the original list starting at listhead was generated in
+horizontal mode):
+
+\startitemize
+\startitem
+ add an \quote {indent box} and perhaps a \nod {local_par} node at the start
+ (only if you need them)
+\stopitem
+\startitem
+ replace any found final glue by an infinite penalty (or add such a penalty,
+ if the last node is not a glue)
+\stopitem
+\startitem
+ add a glue node for the \prm {parfillskip} after that penalty node
+\stopitem
+\startitem
+ make sure all the \type {prev} pointers are OK
+\stopitem
+\stopitemize
+
+The result is a node list, it still needs to be vpacked if you want to assign it
+to a \prm {vbox}. The returned \type {info} table contains four values that are
+all numbers:
+
+\starttabulate[|l|p|]
+\DB name \BC explanation \NC \NR
+\TB
+\NC prevdepth \NC depth of the last line in the broken paragraph \NC \NR
+\NC prevgraf \NC number of lines in the broken paragraph \NC \NR
+\NC looseness \NC the actual looseness value in the broken paragraph \NC \NR
+\NC demerits \NC the total demerits of the chosen solution \NC \NR
+\LL
+\stoptabulate
+
+Note there are a few things you cannot interface using this function: You cannot
+influence font expansion other than via \type {pdfadjustspacing}, because the
+settings for that take place elsewhere. The same is true for hbadness and hfuzz
+etc. All these are in the \type {hpack} routine, and that fetches its own
+variables via globals.
+
+\subsubsection{\type {shipout}}
+
+\topicindex {shipout}
+
+\libindex{shipout}
+
+\startfunctioncall
+tex.shipout(<number> n)
+\stopfunctioncall
+
+Ships out box number \type {n} to the output file, and clears the box register.
+
+\subsubsection{\type {getpagestate}}
+
+\topicindex {pages}
+
+\libindex{getpagestate}
+
+This helper reports the current page state: \type {empty}, \type {box_there} or
+\type {inserts_only} as integer value.
+
+\subsubsection{\type {getlocallevel}}
+
+\topicindex {nesting}
+
+\libindex{getlocallevel}
+
+This integer reports the current level of the local loop. It's only useful for
+debugging and the (relative state) numbers can change with the implementation.
+
+\stopsubsection
+
+\startsubsection[title={Randomizers}]
+
+\libindex{lua_math_random}
+\libindex{lua_math_randomseed}
+\libindex{init_rand}
+\libindex{normal_rand}
+\libindex{uniform_rand}
+\libindex{uniformdeviate}
+
+For practical reasons \LUATEX\ has its own random number generator. The original
+\LUA\ random function is available as \typ {tex.lua_math_random}. You can
+initialize with a new seed with \type {init_rand} (\typ {lua_math_randomseed} is
+equivalent to this one.
+
+There are three generators: \type {normal_rand} (no argument is used), \type
+{uniform_rand} (takes a number that will get rounded before being used) and \type
+{uniformdeviate} which behaves like the primitive and expects a scaled integer, so
+
+\startfunctioncall
+tex.print(tex.uniformdeviate(65536)/65536)
+\stopfunctioncall
+
+will give a random number between zero and one.
+
+\stopsubsection
+
+\startsubsection[reference=synctex,title={Functions related to synctex}]
+
+\topicindex {synctex}
+
+\libindex{set_synctex_mode} \libindex{get_synctex_mode}
+\libindex{set_synctex_no_files}
+\libindex{set_synctex_tag} \libindex{get_synctex_tag} \libindex{force_synctex_tag}
+\libindex{set_synctex_line} \libindex{get_synctex_line} \libindex{force_synctex_line}
+
+The next helpers only make sense when you implement your own synctex logic. Keep in
+mind that the library used in editors assumes a certain logic and is geared for
+plain and \LATEX, so after a decade users expect a certain behaviour.
+
+\starttabulate[|l|p|]
+\DB name \BC explanation \NC \NR
+\TB
+\NC \type{set_synctex_mode} \NC \type {0} is the default and used normal synctex
+ logic, \type {1} uses the values set by the next
+ helpers while \type {2} also sets these for glyph
+ nodes; \type{3} sets glyphs and glue and \type {4}
+ sets only glyphs \NC \NR
+\NC \type{set_synctex_tag} \NC set the current tag (file) value (obeys save stack) \NC \NR
+\NC \type{set_synctex_line} \NC set the current line value (obeys save stack) \NC \NR
+\NC \type{set_synctex_no_files} \NC disable synctex file logging \NC \NR
+\NC \type{get_synctex_mode} \NC returns the current mode (for values see above) \NC \NR
+\NC \type{get_synctex_tag} \NC get the currently set value of tag (file) \NC \NR
+\NC \type{get_synctex_line} \NC get the currently set value of line \NC \NR
+\NC \type{force_synctex_tag} \NC overload the tag (file) value (\type {0} resets) \NC \NR
+\NC \type{force_synctex_line} \NC overload the line value (\type {0} resets) \NC \NR
+\LL
+\stoptabulate
+
+The last one is somewhat special. Due to the way files are registered in \SYNCTEX\ we need
+to explicitly disable that feature if we provide our own alternative if we want to avoid
+that overhead. Passing a value of 1 disables registering.
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={The \type {texconfig} table},reference=texconfig][library=texconfig]
+
+\topicindex{libraries+\type{texconfig}}
+
+\topicindex {configuration}
+
+This is a table that is created empty. A startup \LUA\ script could fill this
+table with a number of settings that are read out by the executable after loading
+and executing the startup file.
+
+\starttabulate[|l|l|l|]
+\DB key \BC type \BC default \NC \NR
+\TB
+\NC \type{max_strings} \NC number \NC 100000 \NC \NR
+\NC \type{strings_free} \NC number \NC 100 \NC \NR
+\NC \type{nest_size} \NC number \NC 50 \NC \NR
+\NC \type{max_in_open} \NC number \NC 100 \NC \NR
+\NC \type{param_size} \NC number \NC 60 \NC \NR
+\NC \type{save_size} \NC number \NC 5000 \NC \NR
+\NC \type{stack_size} \NC number \NC 500 \NC \NR
+\NC \type{expand_depth} \NC number \NC 1000 \NC \NR
+\NC \type{function_size} \NC number \NC 0 \NC \NR
+\NC \type{error_line} \NC number \NC 79 \NC \NR
+\NC \type{half_error_line} \NC number \NC 50 \NC \NR
+\NC \type{hash_extra} \NC number \NC 0 \NC \NR
+\NC \type{formatname} \NC string \NC \NC \NR
+\NC \type{jobname} \NC string \NC \NC \NR
+\LL
+\stoptabulate
+
+If no format name or jobname is given on the command line, the related keys will
+be tested first instead of simply quitting.
+
+\stopsection
+
+\startsection[title={The \type {texio} library}][library=texio]
+
+\topicindex{libraries+\type{texio}}
+\topicindex{\IO}
+
+This library takes care of the low|-|level I/O interface: writing to the log file
+and|/|or console.
+
+\startsubsection[title={\type {write}}]
+
+\libindex{write}
+
+\startfunctioncall
+texio.write(<string> target, <string> s, ...)
+texio.write(<string> s, ...)
+\stopfunctioncall
+
+Without the \type {target} argument, writes all given strings to the same
+location(s) \TEX\ writes messages to at this moment. If \prm {batchmode} is in
+effect, it writes only to the log, otherwise it writes to the log and the
+terminal. The optional \type {target} can be one of three possibilities: \type
+{term}, \type {log} or \type {term and log}.
+
+Note: If several strings are given, and if the first of these strings is or might
+be one of the targets above, the \type {target} must be specified explicitly to
+prevent \LUA\ from interpreting the first string as the target.
+
+\stopsubsection
+
+\startsubsection[title={\type {write_nl}}]
+
+\libindex{write_nl}
+
+\startfunctioncall
+texio.write_nl(<string> target, <string> s, ...)
+texio.write_nl(<string> s, ...)
+\stopfunctioncall
+
+This function behaves like \type {texio.write}, but make sure that the given
+strings will appear at the beginning of a new line. You can pass a single empty
+string if you only want to move to the next line.
+
+\stopsubsection
+
+\startsubsection[title={\type {setescape}}]
+
+\libindex{setescape}
+
+You can disable \type {^^} escaping of control characters by passing a value of
+zero.
+
+\stopsubsection
+
+\startsubsection[title={\type {closeinput}}]
+
+\libindex{closeinput}
+
+This function that should be used with care. It acts as \prm {endinput} but at
+the \LUA\ end. You can use it to (sort of) force a jump back to \TEX. Normally a
+\LUA\ will just collect prints and at the end bump an input level and flush these
+prints. This function can help you stay at the current level but you need to know
+what you're doing (or more precise: what \TEX\ is doing with input).
+
+\stopsubsection
+
+\stopsection
+
+\startsection[title={The \type {token} library}][library=token]
+
+\startsubsection[title={The scanner}]
+
+\topicindex{libraries+\type{token}}
+\topicindex{tokens}
+
+\libindex{scan_keyword}
+\libindex{scan_keywordcs}
+\libindex{scan_int}
+\libindex{scan_real}
+\libindex{scan_float}
+\libindex{scan_dimen}
+\libindex{scan_glue}
+\libindex{scan_toks}
+\libindex{scan_code}
+\libindex{scan_string}
+\libindex{scan_argument}
+\libindex{scan_word}
+\libindex{scan_csname}
+\libindex{scan_list}
+
+The token library provides means to intercept the input and deal with it at the
+\LUA\ level. The library provides a basic scanner infrastructure that can be used
+to write macros that accept a wide range of arguments. This interface is on
+purpose kept general and as performance is quite ok. One can build additional
+parsers without too much overhead. It's up to macro package writers to see how
+they can benefit from this as the main principle behind \LUATEX\ is to provide a
+minimal set of tools and no solutions. The scanner functions are probably the
+most intriguing.
+
+\starttabulate[|l|l|p|]
+\DB function \BC argument \BC result \NC \NR
+\TB
+\NC \type{scan_keyword} \NC string \NC returns true if the given keyword is gobbled; as with
+ the regular \TEX\ keyword scanner this is case insensitive
+ (and \ASCII\ based) \NC \NR
+\NC \type{scan_keywordcs} \NC string \NC returns true if the given keyword is gobbled; this variant
+ is case sensitive and also suitable for \UTF8 \NC \NR
+\NC \type{scan_int} \NC \NC returns an integer \NC \NR
+\NC \type{scan_real} \NC \NC returns a number from e.g.\ \type {1}, \type {1.1}, \type {.1} with optional collapsed signs \NC \NR
+\NC \type{scan_float} \NC \NC returns a number from e.g.\ \type {1}, \type {1.1}, \type {.1}, \type {1.1E10}, , \type {.1e-10} with optional collapsed signs \NC \NR
+\NC \type{scan_dimen} \NC infinity, mu-units \NC returns a number representing a dimension and or two numbers being the filler and order \NC \NR
+\NC \type{scan_glue} \NC mu-units \NC returns a glue spec node \NC \NR
+\NC \type{scan_toks} \NC definer, expand \NC returns a table of tokens tokens \NC \NR
+\NC \type{scan_code} \NC bitset \NC returns a character if its category is in the given bitset (representing catcodes) \NC \NR
+\NC \type{scan_string} \NC \NC returns a string given between \type {{}}, as \type {\macro} or as sequence of characters with catcode 11 or 12 \NC \NR
+\NC \type{scan_argument} \NC \NC this one is simular to \type {scanstring} but also accepts a \type {\cs}
+ (which then get expanded) \NC \NR
+\NC \type{scan_word} \NC \NC returns a sequence of characters with catcode 11 or 12 as string \NC \NR
+\NC \type{scan_csname} \NC \NC returns \type {foo} after scanning \type {\foo} \NC \NR
+\NC \type{scan_list} \NC \NC picks up a box specification and returns a \type {[h|v]list} node \NC \NR
+\LL
+\stoptabulate
+
+The scanners can be considered stable apart from the one scanning for a token.
+The \type {scan_code} function takes an optional number, the \type {keyword}
+function a normal \LUA\ string. The \type {infinity} boolean signals that we also
+permit \type {fill} as dimension and the \type {mu-units} flags the scanner that
+we expect math units. When scanning tokens we can indicate that we are defining a
+macro, in which case the result will also provide information about what
+arguments are expected and in the result this is separated from the meaning by a
+separator token. The \type {expand} flag determines if the list will be expanded.
+
+The string scanner scans for something between curly braces and expands on the
+way, or when it sees a control sequence it will return its meaning. Otherwise it
+will scan characters with catcode \type {letter} or \type {other}. So, given the
+following definition:
+
+\startbuffer
+\def\bar{bar}
+\def\foo{foo-\bar}
+\stopbuffer
+
+\typebuffer \getbuffer
+
+we get:
+
+\starttabulate[|l|Tl|l|]
+\DB name \BC result \NC \NR
+\TB
+\NC \type {\directlua{token.scan_string()}{foo}} \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} {foo} \NC full expansion \NC \NR
+\NC \type {\directlua{token.scan_string()}foo} \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} foo \NC letters and others \NC \NR
+\NC \type {\directlua{token.scan_string()}\foo} \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")}\foo \NC meaning \NC \NR
+\LL
+\stoptabulate
+
+The \type {\foo} case only gives the meaning, but one can pass an already
+expanded definition (\prm {edef}'d). In the case of the braced variant one can of
+course use the \prm {detokenize} and \prm {unexpanded} primitives since there we
+do expand.
+
+The \type {scan_word} scanner can be used to implement for instance a number scanner:
+
+\starttyping
+function token.scan_number(base)
+ return tonumber(token.scan_word(),base)
+end
+\stoptyping
+
+This scanner accepts any valid \LUA\ number so it is a way to pick up floats
+in the input.
+
+You can use the \LUA\ interface as follows:
+
+\starttyping
+\directlua {
+ function mymacro(n)
+ ...
+ end
+}
+
+\def\mymacro#1{%
+ \directlua {
+ mymacro(\number\dimexpr#1)
+ }%
+}
+
+\mymacro{12pt}
+\mymacro{\dimen0}
+\stoptyping
+
+You can also do this:
+
+\starttyping
+\directlua {
+ function mymacro()
+ local d = token.scan_dimen()
+ ...
+ end
+}
+
+\def\mymacro{%
+ \directlua {
+ mymacro()
+ }%
+}
+
+\mymacro 12pt
+\mymacro \dimen0
+\stoptyping
+
+It is quite clear from looking at the code what the first method needs as
+argument(s). For the second method you need to look at the \LUA\ code to see what
+gets picked up. Instead of passing from \TEX\ to \LUA\ we let \LUA\ fetch from
+the input stream.
+
+In the first case the input is tokenized and then turned into a string, then it
+is passed to \LUA\ where it gets interpreted. In the second case only a function
+call gets interpreted but then the input is picked up by explicitly calling the
+scanner functions. These return proper \LUA\ variables so no further conversion
+has to be done. This is more efficient but in practice (given what \TEX\ has to
+do) this effect should not be overestimated. For numbers and dimensions it saves
+a bit but for passing strings conversion to and from tokens has to be done anyway
+(although we can probably speed up the process in later versions if needed).
+
+\stopsubsection
+
+\startsubsection[title= {Picking up one token}]
+
+\libindex {get_next}
+\libindex {scan_token}
+\libindex {expand}
+
+The scanners look for a sequence. When you want to pick up one token from the
+input you use \type {get_next}. This creates a token with the (low level)
+properties as discussed next. This token is just the next one. If you want to
+enforce expansion first you can use \type {scan_token}. Internally tokens are
+characterized by a number that packs a lot of information. In order to access
+the bits of information a token is wrapped in a userdata object.
+
+The \type {expand} function will trigger expansion of the next token in the
+input. This can be quite unpredictable but when you call it you probably know
+enough about \TEX\ not to be too worried about that. It basically is a call to
+the internal expand related function.
+
+\stopsubsection
+
+\startsubsection[title={Creating tokens}]
+
+\libindex{create}
+\libindex{new}
+
+\libindex{is_defined}
+\libindex{is_token}
+\libindex{biggest_char}
+
+\libindex{commands}
+\libindex{command_id}
+
+\libindex{get_command}
+\libindex{get_cmdname}
+\libindex{get_csname}
+\libindex{get_id}
+\libindex{get_active}
+\libindex{get_expandable}
+\libindex{get_protected}
+\libindex{get_mode}
+\libindex{get_index}
+\libindex{get_tok}
+
+\libindex{get_next}
+
+The creator function can be used as follows:
+
+\starttyping
+local t = token.create("relax")
+\stoptyping
+
+This gives back a token object that has the properties of the \prm {relax}
+primitive. The possible properties of tokens are:
+
+\starttabulate[|l|p|]
+\DB name \BC explanation \NC \NR
+\TB
+\NC \type {command} \NC a number representing the internal command number \NC \NR
+\NC \type {cmdname} \NC the type of the command (for instance the catcode in case of a
+ character or the classifier that determines the internal
+ treatment \NC \NR
+\NC \type {csname} \NC the associated control sequence (if applicable) \NC \NR
+\NC \type {id} \NC the unique id of the token \NC \NR
+\NC \type {tok} \NC the full token number as stored in \TEX \NC \NR
+\NC \type {active} \NC a boolean indicating the active state of the token \NC \NR
+\NC \type {expandable} \NC a boolean indicating if the token (macro) is expandable \NC \NR
+\NC \type {protected} \NC a boolean indicating if the token (macro) is protected \NC \NR
+\NC \type {mode} \NC a number either representing a character or another entity \NC \NR
+\NC \type {index} \NC a number running from 0x0000 upto 0xFFFF indicating a \TEX\ register index \NC \NR
+\LL
+\stoptabulate
+
+Alternatively you can use a getter \type {get_<fieldname>} to access a property
+of a token.
+
+The numbers that represent a catcode are the same as in \TEX\ itself, so using
+this information assumes that you know a bit about \TEX's internals. The other
+numbers and names are used consistently but are not frozen. So, when you use them
+for comparing you can best query a known primitive or character first to see the
+values.
+
+You can ask for a list of commands:
+
+\starttyping
+local t = token.commands()
+\stoptyping
+
+The id of a token class can be queried as follows:
+
+\starttyping
+local id = token.command_id("math_shift")
+\stoptyping
+
+If you really know what you're doing you can create character tokens by not
+passing a string but a number:
+
+\starttyping
+local letter_x = token.create(string.byte("x"))
+local other_x = token.create(string.byte("x"),12)
+\stoptyping
+
+Passing weird numbers can give side effects so don't expect too much help with
+that. As said, you need to know what you're doing. The best way to explore the
+way these internals work is to just look at how primitives or macros or \prm
+{chardef}'d commands are tokenized. Just create a known one and inspect its
+fields. A variant that ignores the current catcode table is:
+
+\starttyping
+local whatever = token.new(123,12)
+\stoptyping
+
+You can test if a control sequence is defined with \type {is_defined}, which
+accepts a string and returns a boolean:
+
+\starttyping
+local okay = token.is_defined("foo")
+\stoptyping
+
+The largest character possible is returned by \type {biggest_char}, just in case you
+need to know that boundary condition.
+
+\stopsubsection
+
+\startsubsection[title={Macros}]
+
+\topicindex {macros}
+
+\libindex{set_macro}
+\libindex{get_macro}
+\libindex{get_meaning}
+\libindex{set_char}
+\libindex{set_lua}
+\libindex{get_functions_table}
+
+The \type {set_macro} function can get upto 4 arguments:
+
+\starttyping
+set_macro("csname","content")
+set_macro("csname","content","global")
+set_macro("csname")
+\stoptyping
+
+You can pass a catcodetable identifier as first argument:
+
+\starttyping
+set_macro(catcodetable,"csname","content")
+set_macro(catcodetable,"csname","content","global")
+set_macro(catcodetable,"csname")
+\stoptyping
+
+The results are like:
+
+\starttyping
+ \def\csname{content}
+\gdef\csname{content}
+ \def\csname{}
+\stoptyping
+
+The \type {get_macro} function can be used to get the content of a macro while
+the \type {get_meaning} function gives the meaning including the argument
+specification (as usual in \TEX\ separated by \type {->}).
+
+The \type {set_char} function can be used to do a \prm {chardef} at the
+\LUA\ end, where invalid assignments are silently ignored:
+
+\starttyping
+set_char("csname",number)
+set_char("csname",number,"global")
+\stoptyping
+
+A special one is the following:
+
+\starttyping
+set_lua("mycode",id)
+set_lua("mycode",id,"global","protected")
+\stoptyping
+
+This creates a token that refers to a \LUA\ function with an entry in the table
+that you can access with \type {lua.get_functions_table}. It is the companion
+to \lpr {luadef}. When the first (and only) argument is true the size will preset
+to the value of \type {texconfig.function_size}.
+
+\stopsubsection
+
+\startsubsection[title={Pushing back}]
+
+\libindex{get_next}
+\libindex{put_next}
+
+There is a (for now) experimental putter:
+
+\starttyping
+local t1 = token.get_next()
+local t2 = token.get_next()
+local t3 = token.get_next()
+local t4 = token.get_next()
+-- watch out, we flush in sequence
+token.put_next { t1, t2 }
+-- but this one gets pushed in front
+token.put_next ( t3, t4 )
+\stoptyping
+
+When we scan \type {wxyz!} we get \type {yzwx!} back. The argument is either a table
+with tokens or a list of tokens. The \type {token.expand} function will trigger
+expansion but what happens really depends on what you're doing where.
+
+\stopsubsection
+
+\startsubsection[title={Nota bene}]
+
+When scanning for the next token you need to keep in mind that we're not scanning
+like \TEX\ does: expanding, changing modes and doing things as it goes. When we
+scan with \LUA\ we just pick up tokens. Say that we have:
+
+\starttyping
+\bar
+\stoptyping
+
+but \type {\bar} is undefined. Normally \TEX\ will then issue an error message.
+However, when we have:
+
+\starttyping
+\def\foo{\bar}
+\stoptyping
+
+We get no error, unless we expand \type {\foo} while \type {\bar} is still
+undefined. What happens is that as soon as \TEX\ sees an undefined macro it will
+create a hash entry and when later it gets defined that entry will be reused. So,
+\type {\bar} really exists but can be in an undefined state.
+
+\startbuffer[demo]
+bar : \directlua{tex.print(token.scan_csname())}\bar
+foo : \directlua{tex.print(token.scan_csname())}\foo
+myfirstbar : \directlua{tex.print(token.scan_csname())}\myfirstbar
+\stopbuffer
+
+\startlines
+\getbuffer[demo]
+\stoplines
+
+This was entered as:
+
+\typebuffer[demo]
+
+The reason that you see \type {bar} reported and not \type {myfirstbar} is that
+\type {\bar} was already used in a previous paragraph.
+
+If we now say:
+
+\startbuffer
+\def\foo{}
+\stopbuffer
+
+\typebuffer \getbuffer
+
+we get:
+
+\startlines
+\getbuffer[demo]
+\stoplines
+
+And if we say
+
+\startbuffer
+\def\foo{\bar}
+\stopbuffer
+
+\typebuffer \getbuffer
+
+we get:
+
+\startlines
+\getbuffer[demo]
+\stoplines
+
+When scanning from \LUA\ we are not in a mode that defines (undefined) macros at
+all. There we just get the real primitive undefined macro token.
+
+\startbuffer
+\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\myfirstbar
+\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\mysecondbar
+\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\mythirdbar
+\stopbuffer
+
+\startlines
+\getbuffer
+\stoplines
+
+This was generated with:
+
+\typebuffer
+
+So, we do get a unique token because after all we need some kind of \LUA\ object
+that can be used and garbage collected, but it is basically the same one,
+representing an undefined control sequence.
+
+\stopsubsection
+
+\stopsection
+
+\stopchapter
+
+\stopcomponent