2015-12-17 15:59:00

8 files changed, 320 insertions, 239 deletions

diff --git a/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex b/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex
index 6dfaecaba..19d99b74f 100644
--- a/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex-enhancements.tex
@@ -768,6 +768,37 @@ You can now open upto 127 files with \type {\openout}. When no file is open
 writes will go to the console and log. As a consequence a system command is
 no longer possible but one can use \type {os.execute} to do the same.
 
+\section{\type{\nospaces}}
+
+This new primitive can be used to overrule the usual \type {\spaceskip}
+related heuristics when a space character is seen in a text flow. The
+value~\type{1} triggers no injection while \type{2} results in injection of
+a zero skip. Below we see the results for four characters separated by a
+space.
+
+\startlinecorrection
+\startcombination[3*2]
+    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 10mm}}
+    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 10mm}}
+    {\ruledhbox to 5cm{\vtop{\hsize 10mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 10mm}}
+    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=0\relax x x x x \par}\hss}} {\type {0 / hsize 1mm}}
+    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=1\relax x x x x \par}\hss}} {\type {1 / hsize 1mm}}
+    {\ruledhbox to 5cm{\vtop{\hsize  1mm\nospaces=2\relax x x x x \par}\hss}} {\type {2 / hsize 1mm}}
+\stopcombination
+\stoplinecorrection
+
+\section{\type{\letcharcode}}
+
+This primitive is still experimental but can be used to assign a meaning to an active
+character, as in:
+
+\starttyping
+\def\foo{bar} \letcharcode123\foo
+\stoptyping
+
+This can be a bit nicer that using the uppercase tricks (using the property of
+\type {\uppercase} that it treats active characters special).
+
 \stopchapter
 
 \stopcomponent
diff --git a/doc/context/sources/general/manuals/luatex/luatex-introduction.tex b/doc/context/sources/general/manuals/luatex/luatex-introduction.tex
index 23b921129..5fcc96546 100644
--- a/doc/context/sources/general/manuals/luatex/luatex-introduction.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex-introduction.tex
@@ -7,7 +7,11 @@
 
 This book will eventually become the reference manual of \LUATEX. At the moment,
 it simply reports the behavior of the executable matching the snapshot or beta
-release date in the title page.
+release date in the title page. We don't claim it is complete and we assume that
+the reader knows about \TEX\ as described in \quotation {The \TEX\ Book}, the
+\quotation {\ETEX\ manual}, the \quotation {\PDFTEX\ manual}, etc. Additional
+reference material is published in journals of user groups and \CONTEXT\ related
+documentation.
 
 Features may come and go. The current version of \LUATEX\ can be used for
 production (in fact it is used in production by the authors) but users cannot
@@ -18,11 +22,21 @@ list. We're still not at version 1 but when we reach that state the interface
 will be stable. Of course we then can decide to move towards version 2 with
 different properties.
 
+This means that occasionally you can encounter functionality not described here.
+Just consider that experimental then. They are either a playground or are being
+tested in real situations first before being finalized. And we can equally well
+kick them out again. When they stay we will descibe them in the manual or more
+extensively in articles.
+
 Don't expect \LUATEX\ to behave the same as \PDFTEX ! Although the core
 functionality of that 8 bit engine is present, \LUATEX\ can behave different due
-to not only its 32 bit character: there is native \UTF\ input, support for wide
-fonts, and the math machinery is tuned for \OPENTYPE\ math. Also, the log output
-can differ (and will likely differ more as we move forward).
+to its wide (32 bit) characters, many registers and large memory support. There
+is native \UTF\ input, support for large (more that 8 bit) fonts, and the math
+machinery is tuned for \OPENTYPE\ math. Ther ei ssupport for directional
+typesetting too. The log output can differ from other engines and will likely
+differ more as we move forward. When you run plain \TEX\ for sure \LUATEX\ runs
+slower than \PDFTEX\ but when you run for instance \CONTEXT\ it might be faster. But
+in any case: 32 bit all||over combined with more features has a price.
 
 \LUATEX\ consists of a number of interrelated but (still) distinguishable parts.
 The organization of the source code is adapted so that it cna glue all these
@@ -67,7 +81,8 @@ code in \TEX\ engines (especially code that is not needed any longer).
 
 Neither \ALEPH's I/O translation processes, nor tcx files, nor \ENCTEX\ can be
 used, these encoding|-|related functions are superseded by a \LUA|-|based
-solution (reader callbacks).
+solution (reader callbacks). Most of the \PDFTEX\ backend is available but via a
+bit different interface.
 
 The yearly \TEXLIVE\ version is the stable version, any version between them is
 considered beta. Keep in mind that new (or changed) features also need to be
diff --git a/doc/context/sources/general/manuals/luatex/luatex-languages.tex b/doc/context/sources/general/manuals/luatex/luatex-languages.tex
index 83c251473..773cbdf81 100644
--- a/doc/context/sources/general/manuals/luatex/luatex-languages.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex-languages.tex
@@ -127,6 +127,32 @@ process the box using the current paragraph language unless there was a
 \type {\setlanguage} issued inside the box. In \LUATEX, all language variables are
 already frozen.
 
+In traditional \TEX\ the process of hyphenation is driven by so called lccodes.
+In \LUATEX\ we made this dependency less strong. There are several strategies
+possible. When you do nothing, the currently used lccodes are used, when loading
+patterns, setting exceptions or hyphenating a list.
+
+When you set \type {\savinghyphcodes} to a value larger than zero the current set of
+lccodes will be saved with the language. In that case changing a lccode afterwards
+has no effect. However, you can adapt the set with:
+
+\starttyping
+\hjcode`a=`a
+\stoptyping
+
+This change is global which makes sense if you keep in mind that the moment that
+hyphenation happens is (normally) when the paragraph or a horizontal box is
+constructed. When \type {\savinghyphcodes} was zero when the language got
+initialized you start out with nothing, otherwise you already have a set.
+
+Carrying all this information with each glyph would give too much overhead and
+also make the definition more complex. A solution with hj codesets was considered
+but rejected because in practice the current approach is sufficient and it would
+not be compatible anyway.
+
+Beware: the values are always saved in the format, independent of the setting
+of \type {\savinghyphcodes} at the mnoment the format is dumped.
+
 \section{The main control loop}
 
 In \LUATEX's main loop, almost all input characters that are to be typeset are
diff --git a/doc/context/sources/general/manuals/luatex/luatex-libraries.tex b/doc/context/sources/general/manuals/luatex/luatex-libraries.tex
index 339f25eaf..d7f4bf63d 100644
--- a/doc/context/sources/general/manuals/luatex/luatex-libraries.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex-libraries.tex
@@ -392,36 +392,36 @@ inside this function or infinite recursion may occur. If you return \type {nil},
 
 This callback does not replace any internal code.
 
-\subsubsection{\type {token_filter}}
-
-This callback allows you to replace the way \LUATEX\ fetches lexical tokens.
-
-\startfunctioncall
-function()
-    return <table> token
-end
-\stopfunctioncall
-
-The calling convention for this callback is a bit more complicated than for most
-other callbacks. The function should either return a \LUA\ table representing a
-valid to|-|be|-|processed token or tokenlist, or something else like \type {nil}
-or an empty table.
-
-If your \LUA\ function does not return a table representing a valid token, it
-will be immediately called again, until it eventually does return a useful token
-or tokenlist (or until you reset the callback value to nil). See the description
-of \type {token} for some handy functions to be used in conjunction with this
-callback.
-
-If your function returns a single usable token, then that token will be processed
-by \LUATEX\ immediately. If the function returns a token list (a table consisting
-of a list of consecutive token tables), then that list will be pushed to the
-input stack at a completely new token list level, with its token type set to
-\quote {inserted}. In either case, the returned token(s) will not be fed back
-into the callback function.
-
-Setting this callback to \type {false} has no effect (because otherwise nothing
-would happen, forever).
+% \subsubsection{\type {token_filter}}
+%
+% This callback allows you to replace the way \LUATEX\ fetches lexical tokens.
+%
+% \startfunctioncall
+% function()
+%     return <table> token
+% end
+% \stopfunctioncall
+%
+% The calling convention for this callback is a bit more complicated than for most
+% other callbacks. The function should either return a \LUA\ table representing a
+% valid to|-|be|-|processed token or tokenlist, or something else like \type {nil}
+% or an empty table.
+%
+% If your \LUA\ function does not return a table representing a valid token, it
+% will be immediately called again, until it eventually does return a useful token
+% or tokenlist (or until you reset the callback value to nil). See the description
+% of \type {token} for some handy functions to be used in conjunction with this
+% callback.
+%
+% If your function returns a single usable token, then that token will be processed
+% by \LUATEX\ immediately. If the function returns a token list (a table consisting
+% of a list of consecutive token tables), then that list will be pushed to the
+% input stack at a completely new token list level, with its token type set to
+% \quote {inserted}. In either case, the returned token(s) will not be fed back
+% into the callback function.
+%
+% Setting this callback to \type {false} has no effect (because otherwise nothing
+% would happen, forever).
 
 \subsection{Node list processing callbacks}
 
@@ -606,25 +606,6 @@ end
 
 This callback does not replace any internal code.
 
-\subsubsection{\type {hpack_quality} and \type {vpack_quality}}
-
-These two callbacks can be used to intercept the overfull messages. They get a few
-arguments:
-
-\startfunctioncall
-function(<string> incident, <number> detail, <node> head, <number> first,
-         <number> last)
-    return <node> whatever
-end
-\stopfunctioncall
-
-The incident is one of \type {overfull}, \type {underfull}, \type {loose} or
-\type {tight}. The detail is either the amount of overflow (in case of \type
-{overerful}, or the badness. The head is the list that is constructed (when
-protrusion or expansion is enabled, this is an intermediate list). For a hpack
-you can return a node (for instance an overfull rule indicator). That node will
-be appended to the list (just like \TEX's own rule would).
-
 \subsubsection{\type {hpack_quality}}
 
 This callback can be used to intercept the overfull messages that can result from
@@ -661,6 +642,13 @@ The incident is one of \type {overfull}, \type {underfull}, \type {loose} or
 \type {tight}. The detail is either the amount of overflow in case of \type
 {overfull}, or the badness otherwise. The head is the list that is constructed.
 
+\subsubsection{\type {process_rule}}
+
+This is an experimental callback. It can be used with rules of subtype~4
+(user). The callback gets three arguments: the node, the width and the
+height. The callback can use \type {pdf.print} to write code to the \PDF\
+file but beware of not messing up the final result. No checking is done.
+
 \subsubsection{\type {pre_output_filter}}
 
 This callback is called when \TEX\ is ready to start boxing the box 255 for \type
@@ -3189,9 +3177,10 @@ ones. The syntax of the string is explained in~\in {section}
 lang.clear_hyphenation(<language> l)
 \stopfunctioncall
 
-Clears the exception dictionary for this language.
+Clears the exception dictionary (string) for this language.
 
 \startfunctioncall
+<string> n = lang.clean(<language> l, <string> o)
 <string> n = lang.clean(<string> o)
 \stopfunctioncall
 
@@ -3261,6 +3250,16 @@ nodes with the node subtype having the value \type {1}. Glyph modes with
 different subtypes are not processed. See \in {section~} [charsandglyphs] for
 more details.
 
+The following two commands can be used to set or query hj codes:
+
+\startfunctioncall
+lang.sethjcode(<language> l, <number> char, <number> usedchar)
+<number> usedchar = lang.gethjcode(<language> l, <number> char)
+\stopfunctioncall
+
+When you set a hjcode the current sets get initialized unless the set was already
+initialized due to \type {\savinghyphcodes} being larger than zero.
+
 \section{The \type {lua} library}
 
 This library contains one read|-|only  item:
@@ -4378,12 +4377,27 @@ The preferred interface is now \type {pdf.setxformattributes} and \type
 These two functions set the level of compression. The minimum valu sis~0,
 the maximum is~9.
 
+\subsection{\type {pdf.setdecimaldigits} and \type {pdf.getdecimaldigits}}
+
+These two functions set the accuracy of floats written to the \PDF file.
+
 \subsection{\type {pdf.lastobj}, \type {pdf.lastlink}, \type {pdf.lastannot},
 and \type {pdf.retval}}
 
 These status variables are similar to the ones traditionally used at the \TEX\
 end.
 
+\subsection{\type {pdf.setorigin}}
+
+This one is used to set the horizonal and/or vertical offset (a traditional
+backend property).
+
+\starttyping
+pdf.setorigin() -- sets both to 0pt
+pdf.setorigin(tex.sp("1in")) -- sets both to 1in
+pdf.setorigin(tex.sp("1in"),tex.sp("1in"))
+\stoptyping
+
 \subsection{\type {pdf.h}, \type {pdf.v}}
 
 These are the \type {h} and \type {v} values that define the current location on
@@ -4810,10 +4824,11 @@ The current list is:
 \NC callbacks          \NC total number of executed callbacks so far \NC \NR
 \NC indirect_callbacks \NC number of those that were themselves
                            a result of other callbacks (e.g. file readers) \NC \NR
-\NC luatex_svn         \NC the luatex repository id  \NC \NR
 \NC luatex_version     \NC the luatex version number \NC \NR
 \NC luatex_revision    \NC the luatex revision string \NC \NR
 \NC ini_version        \NC \type {true} if this is an \INITEX\ run \NC \NR
+\NC shell_escape       \NC \type {0} means disabled, \type {1} is restricted and
+                           \type {2} means anything is permitted \NC \NR
 \stoptabulate
 
 The error and warning messages can be wiped with the \type {resetmessages}
@@ -4920,24 +4935,6 @@ tex.month
 tex.newlinechar
 tex.outputpenalty
 tex.pausing
-tex.pdfadjustspacing
-tex.pdfcompresslevel
-tex.pdfdecimaldigits
-tex.pdfgamma
-tex.pdfgentounicode
-tex.pdfimageapplygamma
-tex.pdfimagegamma
-tex.pdfimagehicolor
-tex.pdfimageresolution
-tex.pdfinclusionerrorlevel
-tex.pdfminorversion
-tex.pdfobjcompresslevel
-tex.pdfoutput
-tex.pdfpagebox
-tex.pdfpkresolution
-tex.pdfprotrudechars
-tex.pdftracingfonts
-tex.pdfuniqueresname
 tex.postdisplaypenalty
 tex.predisplaydirection
 tex.predisplaypenalty
@@ -4971,6 +4968,25 @@ tex.year
 \stoptyping
 \stoptwocolumns
 
+% tex.pdfadjustspacing
+% tex.pdfcompresslevel
+% tex.pdfdecimaldigits
+% tex.pdfgamma
+% tex.pdfgentounicode
+% tex.pdfimageapplygamma
+% tex.pdfimagegamma
+% tex.pdfimagehicolor
+% tex.pdfimageresolution
+% tex.pdfinclusionerrorlevel
+% tex.pdfminorversion
+% tex.pdfobjcompresslevel
+% tex.pdfoutput
+% tex.pdfpagebox
+% tex.pdfpkresolution
+% tex.pdfprotrudechars
+% tex.pdftracingfonts
+% tex.pdfuniqueresname
+
 Read|-|only:
 
 \startthreecolumns
@@ -6135,131 +6151,136 @@ This function behaves like \type {texio.write}, but make sure that the given
 strings will appear at the beginning of a new line. You can pass a single empty
 string if you only want to move to the next line.
 
-\section[luatokens]{The \type {oldtoken} library (obsolete)}
-
-{\em Nota Bene: This library will disappear soon. It is replaced by the \type
-{token} library, that used to be called \type {newroken}.}
-
-The \type {token} table contains interface functions to \TEX's handling of
-tokens. These functions are most useful when combined with the \type
-{token_filter} callback, but they could be used standalone as well.
-
-A token is represented in \LUA\ as a small table. For the moment, this table
-consists of three numeric entries:
-
-\starttabulate[|l|l|p|]
-\NC \bf index \NC \bf meaning         \NC \bf description \NC \NR
-\NC 1         \NC command code        \NC this is a value between~$0$ and~$130$ (approximately)\NC \NR
-\NC 2         \NC command modifier    \NC this is a value between~$0$ and~$2^{21}$ \NC \NR
-\NC 3         \NC control sequence id \NC for commands that are not the result of control
-                                          sequences, like letters and characters, it is zero,
-                                          otherwise, it is a number pointing into the \quote
-                                          {equivalence table} \NC \NR
-\stoptabulate
-
-\subsection{\type {oldtoken.get_next}}
+\subsubsection{\type {texio.setescape}}
 
-\startfunctioncall
-token t = oldtoken.get_next()
-\stopfunctioncall
-
-This fetches the next input token from the current input source, without
-expansion.
-
-\subsection{\type {oldtoken.is_expandable}}
-
-\startfunctioncall
-<boolean> b = oldtoken.is_expandable(<token> t)
-\stopfunctioncall
-
-This tests if the token \type {t} could be expanded.
-
-\subsection{\type {oldtoken.expand}}
-
-\startfunctioncall
-oldtoken.expand(<token> t)
-\stopfunctioncall
-
-If a token is expandable, this will expand one level of it, so that the first
-token of the expansion will now be the next token to be read by \type
-{oldtoken.get_next()}.
-
-\subsection{\type {oldtoken.is_activechar}}
-
-\startfunctioncall
-<boolean> b = oldtoken.is_activechar(<token> t)
-\stopfunctioncall
-
-This is a special test that is sometimes handy. Discovering whether some control
-sequence is the result of an active character turned out to be very hard
-otherwise.
-
-\subsection{\type {oldtoken.create}}
-
-\startfunctioncall
-token t = oldtoken.create(<string> csname)
-token t = oldtoken.create(<number> charcode)
-token t = oldtoken.create(<number> charcode, <number> catcode)
-\stopfunctioncall
-
-This is the token factory. If you feed it a string, then it is the name of a
-control sequence (without leading backslash), and it will be looked up in the
-equivalence table.
-
-If you feed it number, then this is assumed to be an input character, and an
-optional second number gives its category code. This means it is possible to
-overrule a character's category code, with a few exceptions: the category codes~0
-(escape), 9~(ignored), 13~(active), 14~(comment), and 15 (invalid) cannot occur
-inside a token. The values~0, 9, 14 and~15 are therefore illegal as input to
-\type {oldtoken.create()}, and active characters will be resolved immediately.
-
-Note: unknown string sequences and never defined active characters will result in
-a token representing an \quote {undefined control sequence} with a near|-|random
-name. It is {\em not} possible to define brand new control sequences using
-\type {oldtoken.create}!
-
-\subsection{\type {oldtoken.command_name}}
-
-\startfunctioncall
-<string> commandname = oldtoken.command_name(<token> t)
-\stopfunctioncall
-
-This returns the name associated with the \quote {command} value of the token in
-\LUATEX. There is not always a direct connection between these names and
-primitives. For instance, all \type {\ifxxx} tests are grouped under \type
-{if_test}, and the \quote {command modifier} defines which test is to be run.
-
-\subsection{\type {oldtoken.command_id}}
-
-\startfunctioncall
-<number> i = oldtoken.command_id(<string> commandname)
-\stopfunctioncall
-
-This returns a number that is the inverse operation of the previous command, to
-be used as the first item in a token table.
-
-\subsection{\type {oldtoken.csname_name}}
-
-\startfunctioncall
-<string> csname = oldtoken.csname_name(<token> t)
-\stopfunctioncall
-
-This returns the name associated with the \quote {equivalence table} value of the
-token in \LUATEX. It returns the string value of the command used to create the
-current token, or an empty string if there is no associated control sequence.
-
-Keep in mind that there are potentially two control sequences that return the
-same csname string: single character control sequences and active characters have
-the same \quote {name}.
-
-\subsection{\type {oldtoken.csname_id}}
-
-\startfunctioncall
-<number> i = oldtoken.csname_id(<string> csname)
-\stopfunctioncall
+You can disable \type {^^} escaping of control characters by passing a value of
+zero.
 
-This returns a number that is the inverse operation of the previous command, to
-be used as the third item in a token table.
+% \section[luatokens]{The \type {oldtoken} library (obsolete)}
+%
+% {\em Nota Bene: This library will disappear soon. It is replaced by the \type
+% {token} library, that used to be called \type {newroken}.}
+%
+% The \type {token} table contains interface functions to \TEX's handling of
+% tokens. These functions are most useful when combined with the \type
+% {token_filter} callback, but they could be used standalone as well.
+%
+% A token is represented in \LUA\ as a small table. For the moment, this table
+% consists of three numeric entries:
+%
+% \starttabulate[|l|l|p|]
+% \NC \bf index \NC \bf meaning         \NC \bf description \NC \NR
+% \NC 1         \NC command code        \NC this is a value between~$0$ and~$130$ (approximately)\NC \NR
+% \NC 2         \NC command modifier    \NC this is a value between~$0$ and~$2^{21}$ \NC \NR
+% \NC 3         \NC control sequence id \NC for commands that are not the result of control
+%                                           sequences, like letters and characters, it is zero,
+%                                           otherwise, it is a number pointing into the \quote
+%                                           {equivalence table} \NC \NR
+% \stoptabulate
+%
+% \subsection{\type {oldtoken.get_next}}
+%
+% \startfunctioncall
+% token t = oldtoken.get_next()
+% \stopfunctioncall
+%
+% This fetches the next input token from the current input source, without
+% expansion.
+%
+% \subsection{\type {oldtoken.is_expandable}}
+%
+% \startfunctioncall
+% <boolean> b = oldtoken.is_expandable(<token> t)
+% \stopfunctioncall
+%
+% This tests if the token \type {t} could be expanded.
+%
+% \subsection{\type {oldtoken.expand}}
+%
+% \startfunctioncall
+% oldtoken.expand(<token> t)
+% \stopfunctioncall
+%
+% If a token is expandable, this will expand one level of it, so that the first
+% token of the expansion will now be the next token to be read by \type
+% {oldtoken.get_next()}.
+%
+% \subsection{\type {oldtoken.is_activechar}}
+%
+% \startfunctioncall
+% <boolean> b = oldtoken.is_activechar(<token> t)
+% \stopfunctioncall
+%
+% This is a special test that is sometimes handy. Discovering whether some control
+% sequence is the result of an active character turned out to be very hard
+% otherwise.
+%
+% \subsection{\type {oldtoken.create}}
+%
+% \startfunctioncall
+% token t = oldtoken.create(<string> csname)
+% token t = oldtoken.create(<number> charcode)
+% token t = oldtoken.create(<number> charcode, <number> catcode)
+% \stopfunctioncall
+%
+% This is the token factory. If you feed it a string, then it is the name of a
+% control sequence (without leading backslash), and it will be looked up in the
+% equivalence table.
+%
+% If you feed it number, then this is assumed to be an input character, and an
+% optional second number gives its category code. This means it is possible to
+% overrule a character's category code, with a few exceptions: the category codes~0
+% (escape), 9~(ignored), 13~(active), 14~(comment), and 15 (invalid) cannot occur
+% inside a token. The values~0, 9, 14 and~15 are therefore illegal as input to
+% \type {oldtoken.create()}, and active characters will be resolved immediately.
+%
+% Note: unknown string sequences and never defined active characters will result in
+% a token representing an \quote {undefined control sequence} with a near|-|random
+% name. It is {\em not} possible to define brand new control sequences using
+% \type {oldtoken.create}!
+%
+% \subsection{\type {oldtoken.command_name}}
+%
+% \startfunctioncall
+% <string> commandname = oldtoken.command_name(<token> t)
+% \stopfunctioncall
+%
+% This returns the name associated with the \quote {command} value of the token in
+% \LUATEX. There is not always a direct connection between these names and
+% primitives. For instance, all \type {\ifxxx} tests are grouped under \type
+% {if_test}, and the \quote {command modifier} defines which test is to be run.
+%
+% \subsection{\type {oldtoken.command_id}}
+%
+% \startfunctioncall
+% <number> i = oldtoken.command_id(<string> commandname)
+% \stopfunctioncall
+%
+% This returns a number that is the inverse operation of the previous command, to
+% be used as the first item in a token table.
+%
+% \subsection{\type {oldtoken.csname_name}}
+%
+% \startfunctioncall
+% <string> csname = oldtoken.csname_name(<token> t)
+% \stopfunctioncall
+%
+% This returns the name associated with the \quote {equivalence table} value of the
+% token in \LUATEX. It returns the string value of the command used to create the
+% current token, or an empty string if there is no associated control sequence.
+%
+% Keep in mind that there are potentially two control sequences that return the
+% same csname string: single character control sequences and active characters have
+% the same \quote {name}.
+%
+% \subsection{\type {oldtoken.csname_id}}
+%
+% \startfunctioncall
+% <number> i = oldtoken.csname_id(<string> csname)
+% \stopfunctioncall
+%
+% This returns a number that is the inverse operation of the previous command, to
+% be used as the third item in a token table.
 
 \subsection{The \type {token} libray}
 
diff --git a/doc/context/sources/general/manuals/luatex/luatex-math.tex b/doc/context/sources/general/manuals/luatex/luatex-math.tex
index 6c83b7786..0fcf0e52f 100644
--- a/doc/context/sources/general/manuals/luatex/luatex-math.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex-math.tex
@@ -958,6 +958,11 @@ formula.
 
 % if needed we can put the value in stylenodes but maybe more should go there
 
+\subsection {Tracing}
+
+Because there are quite some math related parameters and values, it is possible
+to limit tracing. Only when \type {tracingassigns} and|/|or \type
+{tracingrestores} are set to~2 or more they will be traced.
 
 \subsection {Math options}
 
diff --git a/doc/context/sources/general/manuals/luatex/luatex-modifications.tex b/doc/context/sources/general/manuals/luatex/luatex-modifications.tex
index 5b35f3ec1..0fbd16dc7 100644
--- a/doc/context/sources/general/manuals/luatex/luatex-modifications.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex-modifications.tex
@@ -96,11 +96,6 @@ functionality, but with a few small adaptions.
 \stopitem
 
 \startitem
-    The \type {\savinghyphcodes} command is a no|-|op. \in {Chapter} [languages]
-    explains why.
-\stopitem
-
-\startitem
     When kpathsea is used to find files, \LUATEX\ uses the \type {ofm} file
     format to search for font metrics. In turn, this means that \LUATEX\ looks at
     the \type {OFMFONTS} configuration variable (like \OMEGA\ and \ALEPH) instead
@@ -250,16 +245,6 @@ which in turn triggered renaming primitives.
 \stopitem
 
 \startitem
-    Candidates for removal are \type {\pdfcolorstackinit} and \type
-    {\pdfcolorstack}.
-\stopitem
-
-\startitem
-    Candidates for replacement are \type {\pdfoutput} (\type {\outputmode}) and
-    \type {\pdfmatrix} (something with a normal syntax).
-\stopitem
-
-\startitem
     The introspective primitives \type {\pdflastximagecolordepth} and \type
     {\pdfximagebbox} have been removed. One can use external applications to
     determine these properties or use the built|-|in \type {img} library.
@@ -370,8 +355,13 @@ we say next applies to both these programs.
 \stopitem
 
 \startitem
-    The \type {^^} notation can come in five and six item repetitions also, to
-    insert characters that do not fit in the BMP.
+    The \type {^^} notation has been extended: after \type {^^^^} four hexadecimal
+    characters are expected and after \type {^^^^^^} six hexadecimal characters
+    have to be given. The original \TEX\ interpretation is still valid for the
+    \type {^^} case but the four and six variants do no backtracking, i.e.\ when
+    they are not followed by the right number of hexadecimal digits they issue an
+    error message. Because \type{^^^} is a normal \TEX\ case, we don't support the
+    odd number of \type {^^^^^} either.
 \stopitem
 
 \startitem
@@ -553,7 +543,6 @@ The configuration related registers have become:
 \edef\pdfpkresolution             {\pdfvariable pkresolution}
 \edef\pdfinclusioncopyfonts       {\pdfvariable inclusioncopyfonts}
 \edef\pdfinclusionerrorlevel      {\pdfvariable inclusionerrorlevel}
-\edef\pdfreplacefont              {\pdfvariable replacefont}
 \edef\pdfgentounicode             {\pdfvariable gentounicode}
 \edef\pdfpagebox                  {\pdfvariable pagebox}
 \edef\pdfminorversion             {\pdfvariable minorversion}
@@ -607,7 +596,6 @@ the defaults; these are:
 \pdfpkresolution         72
 \pdfinclusioncopyfonts    0
 \pdfinclusionerrorlevel   0
-\pdfreplacefont           0
 \pdfgentounicode          0
 \pdfpagebox               0
 \pdfminorversion          4
diff --git a/doc/context/sources/general/manuals/luatex/luatex-nodes.tex b/doc/context/sources/general/manuals/luatex/luatex-nodes.tex
index d59efbf19..f9d240689 100644
--- a/doc/context/sources/general/manuals/luatex/luatex-nodes.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex-nodes.tex
@@ -1105,32 +1105,18 @@ The \type {type} can have one of five distinct values:
 
 \section{Two access models}
 
-After doing lots of tests with \LUATEX\ and \LUAJITTEX\, with and without just in
-time compilation enabled, and with and without using ffi, we came to the
-conclusion that userdata prevents a speedup. We also found that the checking of
-metatables as well as assignment comes with overhead that can't be neglected.
-This is normally not really a problem but when processing fonts for more complex
-scripts it could have quite some overhead.
-
-Because the userdata approach has some benefits, this remains the recommended way
-to access nodes. We did several experiments with faster access using this model,
-but eventually settled for the \quote {direct} approach. For code that is proven
-to be okay, one can use this access model that operates on nodes more directly.
-
-Deep down in \TEX\ a node has a number which is an entry in a memory table. In
-fact, this model, where \TEX\ manages memory is real fast and one of the reasons
-why plugging in callbacks that operate on nodes is quite fast. No matter what
-future memory model \LUATEX\ has, an internal reference will always be a simple
-data type (like a number or light userdata in \LUA\ speak). So, if you use the
-direct model, even if you know that you currently deal with numbers, you should
+Deep down in \TEX\ a node has a number which is an numeric entry in a memory
+table. In fact, this model, where \TEX\ manages memory is real fast and one of
+the reasons why plugging in callbacks that operate on nodes is quite fast. So, if
+you use the direct model, even if you know that you deal with numbers, you should
 not depend on that property but treat it an abstraction just like traditional
 nodes. In fact, the fact that we use a simple basic datatype has the penalty that
 less checking can be done, but less checking is also the reason why it's somewhat
 faster. An important aspect is that one cannot mix both methods, but you can cast
 both models.
 
-So our advice is: use the indexed approach when possible and investigate the
-direct one when speed might be an issue. For that reason we also provide the
+So our advice is: use the indexed (table) approach when possible and investigate
+the direct one when speed might be an issue. For that reason we also provide the
 \type {get*} and \type {set*} functions in the top level node namespace. There is
 a limited set of getters. When implementing this direct approach the regular
 index by key variant was also optimized, so direct access only makes sense when
@@ -1262,14 +1248,15 @@ summarized this:
 \NC \type {is_char}              \NC \yes \NC \yes  \NC \NR
 \NC \type {is_direct}            \NC \nop \NC \yes  \NC \NR
 \NC \type {is_node}              \NC \yes \NC \yes  \NC \NR
-\NC \type {kerning}              \NC \yes \NC \nop  \NC \NR
+\NC \type {kerning}              \NC \yes \NC \yes  \NC \NR
 \NC \type {last_node}            \NC \yes \NC \yes  \NC \NR
 \NC \type {length}               \NC \yes \NC \yes  \NC \NR
-\NC \type {ligaturing}           \NC \yes \NC \nop  \NC \NR
+\NC \type {ligaturing}           \NC \yes \NC \yes  \NC \NR
 \NC \type {mlist_to_hlist}       \NC \yes \NC \nop  \NC \NR
 \NC \type {new}                  \NC \yes \NC \yes  \NC \NR
 \NC \type {next}                 \NC \yes \NC \nop  \NC \NR
 \NC \type {prev}                 \NC \yes \NC \nop  \NC \NR
+\NC \type {protect_glyph}        \NC \yes \NC \yes  \NC \NR
 \NC \type {protect_glyphs}       \NC \yes \NC \yes  \NC \NR
 \NC \type {protrusion_skippable} \NC \yes \NC \yes  \NC \NR
 \NC \type {remove}               \NC \yes \NC \yes  \NC \NR
@@ -1290,6 +1277,7 @@ summarized this:
 \NC \type {tonode}               \NC \yes \NC \yes  \NC \NR
 \NC \type {tostring}             \NC \yes \NC \yes  \NC \NR
 \NC \type {traverse_id}          \NC \yes \NC \yes  \NC \NR
+\NC \type {traverse_char}        \NC \yes \NC \yes  \NC \NR
 \NC \type {traverse}             \NC \yes \NC \yes  \NC \NR
 \NC \type {types}                \NC \yes \NC \nop  \NC \NR
 \NC \type {type}                 \NC \yes \NC \nop  \NC \NR
diff --git a/doc/context/sources/general/manuals/luatex/luatex.tex b/doc/context/sources/general/manuals/luatex/luatex.tex
index bcb543c54..a04592760 100644
--- a/doc/context/sources/general/manuals/luatex/luatex.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex.tex
@@ -1,13 +1,20 @@
+% macros=mkvi
+
 % \tex vs \type vs \syntax vs. \luatex
 % \em \it \/
 
+% \enabledirectives[nodes.basepass*]
+
+% \unprotect
+% \protect
+
 \environment luatex-style
 \environment luatex-logos
 
 \dontcomplain
 
 \startdocument
-  [beta=0.85.0]
+  [beta=0.87.0]
 
 \component luatex-titlepage