2022-11-14 22:56:00

5 files changed, 176 insertions, 73 deletions

diff --git a/doc/context/documents/general/manuals/luametatex.pdf b/doc/context/documents/general/manuals/luametatex.pdf
index e540e7f6d..432cf05e4 100644
--- a/doc/context/documents/general/manuals/luametatex.pdf
+++ b/doc/context/documents/general/manuals/luametatex.pdf
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-fonts.tex b/doc/context/sources/general/manuals/luametatex/luametatex-fonts.tex
index 1bbdebb09..64eb06bc9 100644
--- a/doc/context/sources/general/manuals/luametatex/luametatex-fonts.tex
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-fonts.tex
@@ -107,6 +107,11 @@ in combination with \LUA\ code that does this. In \CONTEXT\ we have base mode
 that uses the engine, and node mode that uses \LUA. A monospaced font normally
 has no ligatures and kerns and is normally not processed at all.
 
+We can group the parameters. All characters have the following base set. It must
+be noted here that \OPENTYPE\ doesn't have a italic property and that the height
+and depth are also not part of the design: one can choose to derive them from the
+bounding box.
+
 \starttabulate[|l|l|pl|]
 \DB key                     \BC type    \BC description \NC\NR
 \TB
@@ -114,24 +119,87 @@ has no ligatures and kerns and is normally not processed at all.
 \NC \type {height}          \NC number  \NC height in sp (default 0) \NC\NR
 \NC \type {depth}           \NC number  \NC depth in sp (default 0) \NC\NR
 \NC \type {italic}          \NC number  \NC italic correction in sp (default 0) \NC\NR
-\NC \type {topaccent}       \NC number  \NC top accent alignment place in sp (default zero) \NC\NR
-\NC \type {bottomaccent}    \NC number  \NC bottom accent alignment place, in sp (default zero) \NC\NR
+\stoptabulate
+
+Then there are three parameters that are more optional and relate to advanced
+optical paragraph optimization:
+
+\starttabulate[|l|l|pl|]
+\DB key                     \BC type    \BC description \NC\NR
+\TB
 \NC \type {leftprotruding}  \NC number  \NC left protruding factor (\prm {lpcode}) \NC\NR
 \NC \type {rightprotruding} \NC number  \NC right protruding factor (\prm {rpcode}) \NC\NR
 \NC \type {expansion}       \NC number  \NC expansion factor (\prm {efcode}) \NC\NR
-\NC \type {next}            \NC number  \NC \quote {next larger} character index \NC\NR
-\NC \type {extensible}      \NC table   \NC constituent parts of an extensible (traditional) recipe \NC\NR
-\NC \type {vparts}          \NC table   \NC constituent parts of a vertical (\OPENTYPE) recipe \NC \NR
-\NC \type {hparts}          \NC table   \NC constituent parts of a horizontal (\OPENTYPE)recipe \NC \NR
-\NC \type {kerns}           \NC table   \NC kerning information \NC\NR
+\stoptabulate
+
+From \TEX\ we inherit the following tables. Ligatures are only used in so call
+base mode, when the engine does the font magic. Kerns are used in text and
+optionally in math: More details follow below.
+
+\starttabulate[|l|l|pl|]
+\DB key                     \BC type    \BC description \NC\NR
+\TB
 \NC \type {ligatures}       \NC table   \NC ligaturing information \NC\NR
-\NC \type {mathkern}        \NC table   \NC math cut-in specifications \NC\NR
-\NC \type {smaller}         \NC number  \NC the next smaller math size character \NC\NR
+\NC \type {kerns}           \NC table   \NC kerning information \NC\NR
+\stoptabulate
+
+The next two fields control the engine and are a variant on \TEX's \TFM\ tag
+property. In a future we might provide a bit more (local) control although
+currently we see no need. Originally the tag and next field were combined into a
+packed integer but in current \LUAMETATEX\ we have a 32 bit tag and the next
+field moved to the math blob as it only is used as variant selector.
+
+\starttabulate[|l|l|pl|]
+\DB key                     \BC type    \BC description \NC\NR
+\TB
+\NC \type {tag}             \NC number  \NC a bitset, currently not really exposed \NC\NR
+%NC \type {reserved}        \NC number  \NC note for myself \NC \NR
+\stoptabulate
+
+In a math font characters have many more fields.
+
+\starttabulate[|l|l|pl|]
+\DB key                      \BC type    \BC description \NC\NR
+\TB
+\NC \type {smaller}          \NC number  \NC the next smaller math size character \NC\NR
+\NC \type {mirror}           \NC number  \NC a right to left alternative \NC\NR
+\NC \type {flataccent}       \NC number  \NC an accent alternative with less height (\OPENTYPE) \NC\NR
+\NC \type {next}             \NC number  \NC \quote {next larger} character index \NC\NR
+\HL
+\NC \type {topleft}          \NC number  \NC alternative script kern \NC\NR
+\NC \type {topright}         \NC number  \NC alternative script kern \NC\NR
+\NC \type {bottomleft}       \NC number  \NC alternative script kern \NC\NR
+\NC \type {bottomright}      \NC number  \NC alternative script kern \NC\NR
+\HL
+\NC \type {topmargin}        \NC number  \NC alternative accent calculation margin \NC\NR
+\NC \type {bottomargin}      \NC number  \NC alternative accent calculation margin \NC\NR
+\NC \type {leftmargin}       \NC number  \NC alternative accent calculation margin \NC\NR
+\NC \type {rightmargin}      \NC number  \NC alternative accent calculation margin \NC\NR
+\HL
+\NC \type {topovershoot}     \NC number  \NC accent width tolerance \NC\NR
+\NC \type {bottomovershoot}  \NC number  \NC accent width tolerance \NC\NR
+\HL
+\NC \type {topanchor}        \NC number  \NC horizontal top accent alignment position \NC\NR
+\NC \type {bottomanchor}     \NC number  \NC horizontal bottom accent alignment position \NC\NR
+\HL
+\NC \type {innerlocation}    \NC string  \NC \type {left} or \type {right}  \NC\NR
+\NC \type {innerxoffset}     \NC number  \NC radical degree horizontal position \NC\NR
+\NC \type {inneryoffset}     \NC number  \NC radical degree vertical position \NC\NR
+\HL
+\NC \type {parts}            \NC table   \NC constituent parts of an extensible \NC \NR
+\NC \type {partsitalic}      \NC number  \NC the italic correction applied with the extensible \NC \NR
+\NC \type {partsorientation} \NC number  \NC \type {horizontal} or \type {vertical} \NC\NR
+\HL
+\NC \type {mathkerns}        \NC table   \NC math cut-in specifications \NC\NR
+\HL
+\NC \type {extensible}       \NC table   \NC stretch a fixed width accent to fit\NC\NR
 \LL
 \stoptabulate
 
-For example, here is the character \quote {f} (decimal 102) in the font \type
-{cmr10 at 10pt}. The numbers that represent dimensions are in scaled points.
+Now some more details follow. For example, here is the character \quote {f}
+(decimal 102) in the font \type {cmr10 at 10pt}. The numbers that represent
+dimensions are in scaled points. Of course you will use Latin Modern \OPENTYPE\
+instead but the principles are the same:
 
 \starttyping
 [102] = {
@@ -159,17 +227,23 @@ virtual character whose ligatures and kerns are used to handle word boundary
 processing. \type {rightboundary} is similar but not actually used for anything
 (yet).
 
-The values of \type {topaccent}, \type {bottomaccent} and \type {mathkern} are
+The values of \type {leftprotrusion} and \type {rightprotrusion} are used only
+when \prm {protrudechars} is non-zero. Whether or not \type {expansion} is used
+depends on the font's global expansion settings, as well as on the value of \prm
+{adjustspacing}.
+
+The values of \type {topanchor}, \type {bottomanchor} and \type {mathkern} are
 used only for math accent and superscript placement, see \at {page} [math] in
-this manual for details. The values of \type {leftprotrusion} and \type
-{rightprotrusion} are used only when \prm {protrudechars} is non-zero. Whether or
-not \type {expansion} is used depends on the font's global expansion settings, as
-well as on the value of \prm {adjustspacing}.
+this manual for details. The italic corrections are a story in themselves and
+discussed in detail in other manuals. The additional parameters that deal with
+kerns, margins, overshoots, inner anchoring, etc. are engine specific and not
+part of \OPENTYPE. More information can be found in the \CONTEXT\ distribution;
+they relate the upgraded math engine project by Mikael and Hans.
 
 A math character can have a \type {next} field that points to a next larger
 shape. However, the presence of \type {extensible} will overrule \type {next}, if
 that is also present. The \type {extensible} field in turn can be overruled by
-\type {vparts}, the \OPENTYPE\ version. The \type {extensible} table is very
+\type {parts}, the \OPENTYPE\ version. The \type {extensible} table is very
 simple:
 
 \starttabulate[|l|l|p|]
@@ -182,8 +256,8 @@ simple:
 \LL
 \stoptabulate
 
-The \type {hparts} and \type {vparts} are arrays of components. Each of
-those components is itself a hash of up to five keys:
+The \type {parts} entru is an arrays of components. Each of those components is
+itself a hash of up to five keys:
 
 \starttabulate[|l|l|p|]
 \DB key             \BC type   \BC explanation \NC \NR
@@ -198,15 +272,15 @@ those components is itself a hash of up to five keys:
 \LL
 \stoptabulate
 
-The \type {kerns} table is a hash indexed by character index (and \quote
-{character index} is defined as either a non|-|negative integer or the string
-value \type {rightboundary}), with the values of the kerning to be applied, in
-scaled points.
+The traditional (text and math) \type {kerns} table is a hash indexed by
+character index (and \quote {character index} is defined as either a
+non|-|negative integer or the string value \type {rightboundary}), with the
+values of the kerning to be applied, in scaled points.
 
-The \type {ligatures} table is a hash indexed by character index (and \quote
-{character index} is defined as either a non|-|negative integer or the string
-value \type {rightboundary}), with the values being yet another small hash, with
-two fields:
+The traditional (text) \type {ligatures} table is a hash indexed by character
+index (and \quote {character index} is defined as either a non|-|negative integer
+or the string value \type {rightboundary}), with the values being yet another
+small hash, with two fields:
 
 \starttabulate[|l|l|p|]
 \DB key         \BC type   \BC description \NC \NR
@@ -243,46 +317,53 @@ The default value is~0, and can be left out. That signifies a \quote {normal}
 ligature where the ligature replaces both original glyphs. In this table the~\type {|}
 indicates the final insertion point.
 
-The \type {mathcontrol} bitset is mostly there for experimental purposes. Because
-there is inconsistency in the \OPENTYPE\ math fonts with respect to for instance
-glyph dimensions, it is possible to force the traditional code path. We just mention
-the possible flags:
-
-\startluacode
-    context.starttabulate { "|||" }
-    context.DB() context("value") context.BC() context("effect") context.NC() context.NR()
-    context.TB()
-    for k, v in table.sortedhash(tex.getmathcontrolvalues()) do
-        context.NC() context("0x%04X",k) context.NC() context(v) context.NC() context.NR()
-    end
-    context.LL()
-    context.stoptabulate()
-\stopluacode
+% The \type {mathcontrol} bitset is mostly there for experimental purposes. Because
+% there is inconsistency in the \OPENTYPE\ math fonts with respect to for instance
+% glyph dimensions, it is possible to force the traditional code path. We just mention
+% the possible flags:
+%
+% \startluacode
+%     context.starttabulate { "|||" }
+%     context.DB() context("value") context.BC() context("effect") context.NC() context.NR()
+%     context.TB()
+%     for k, v in table.sortedhash(tex.getmathcontrolvalues()) do
+%         context.NC() context("0x%04X",k) context.NC() context(v) context.NC() context.NR()
+%     end
+%     context.LL()
+%     context.stoptabulate()
+% \stopluacode
 
 Compact math is an experimental feature. The smaller field in a character
 definition of a text character can point to a script character that itself can
 point to a scriptscript one. When set the \type {textscale}, \type {scriptscale}
 and \type {scriptscriptscale} is applied to those.
 
-The \type {textcontrol} field is used to control some aspects of text processing.
-More options might be added in the future.
-
-\startluacode
-    context.starttabulate { "|||" }
-    context.DB() context("value") context.BC() context("effect") context.NC() context.NR()
-    context.TB()
-    for k, v in table.sortedhash(tex.gettextcontrolvalues()) do
-        context.NC() context("0x%04X",k) context.NC() context(v) context.NC() context.NR()
-    end
-    context.LL()
-    context.stoptabulate()
-\stopluacode
-
-In \CONTEXT\ these are interfaced via pseudo features. The math control flags of
-a font can be overloaded by \prm {mathcontrolmode} on the spot and the set
-controls of a font can be queried by \prm {fontmathcontrol}. The text control
-flags in a font always win over the ones set by other parameters, like \prm
-{hyphenationmode}. They can be queried with \prm {fonttextcontrol}.
+Bidirectional math is also experimental and driven by (in \CONTEXT\ speak) tweaks
+which means that it has to be set up explicitly as it uses a combination of
+fonts. In \CONTEXT\ is also uses specific features of the font subsystems that
+hook into the backend where we have a more advanced virtual font subsystem than
+in \LUATEX. Because this is macro package dependent it will not be discussed
+here.
+
+% The \type {textcontrol} field is used to control some aspects of text processing.
+% More options might be added in the future.
+
+% \startluacode
+%     context.starttabulate { "|||" }
+%     context.DB() context("value") context.BC() context("effect") context.NC() context.NR()
+%     context.TB()
+%     for k, v in table.sortedhash(tex.gettextcontrolvalues()) do
+%         context.NC() context("0x%04X",k) context.NC() context(v) context.NC() context.NR()
+%     end
+%     context.LL()
+%     context.stoptabulate()
+% \stopluacode
+%
+% In \CONTEXT\ these are interfaced via pseudo features. The math control flags of
+% a font can be overloaded by \prm {mathcontrolmode} on the spot and the set
+% controls of a font can be queried by \prm {fontmathcontrol}. The text control
+% flags in a font always win over the ones set by other parameters, like \prm
+% {hyphenationmode}. They can be queried with \prm {fonttextcontrol}.
 
 \stopsection
 
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex-math.tex b/doc/context/sources/general/manuals/luametatex/luametatex-math.tex
index 88d3eafc3..66a62ee72 100644
--- a/doc/context/sources/general/manuals/luametatex/luametatex-math.tex
+++ b/doc/context/sources/general/manuals/luametatex/luametatex-math.tex
@@ -2192,7 +2192,9 @@ influence the spacing. The numbers are the same as for character classes.
 
 \topicindex {math+codes}
 
-You can extract the components of a math character. Say that we have defined:
+You should not really depend on the number that comes from \prm {Umathcode} because
+the engine can (at some point) use a different amount of families and classes. Given this,
+you can extract the components of a math character. Say that we have defined:
 
 \starttyping
 \Umathcode 1 2 3 4
@@ -2201,17 +2203,21 @@ You can extract the components of a math character. Say that we have defined:
 then
 
 \starttyping
-[\Umathcharclass1] [\Umathcharfam1] [\Umathcharslot1]
+[\Umathcharclass\Umathcode1] [\Umathcharfam\Umathcode1] [\Umathcharslot\Umathcode1]
 \stoptyping
 
-will return:
+which will return:
 
 \starttyping
 [2] [3] [4]
 \stoptyping
 
-These commands are provided as convenience. Before they come available you could
-do the following:
+You can of course store the code in for instance a register and use that as
+argument. The three commands also accept a specification (and maybe more in the
+future).
+
+These commands are provided as convenience. Before they became available you
+could do the following:
 
 \starttyping
 \def\Umathcharclass{\numexpr
diff --git a/doc/context/sources/general/manuals/luametatex/luametatex.tex b/doc/context/sources/general/manuals/luametatex/luametatex.tex
index aead9b304..96f700355 100644
--- a/doc/context/sources/general/manuals/luametatex/luametatex.tex
+++ b/doc/context/sources/general/manuals/luametatex/luametatex.tex
@@ -110,6 +110,8 @@
 \ifdefined\linebuffering \linebuffering \fi
 
 \enableexperiments[fonts.compact]
+%disableexperiments[fonts.accurate]
+
 % \enabledirectives[fonts.injections.method=advance] % tricky ... not all xoffsets are advance robust
 
 % This is only for testing, use the command line (or banner line) instead.
diff --git a/doc/context/sources/general/manuals/luatex/luatex-math.tex b/doc/context/sources/general/manuals/luatex/luatex-math.tex
index 4ab65fa18..4b86c5864 100644
--- a/doc/context/sources/general/manuals/luatex/luatex-math.tex
+++ b/doc/context/sources/general/manuals/luatex/luatex-math.tex
@@ -130,6 +130,20 @@ sections:
 \LL
 \stoptabulate
 
+Instead of the pseudo class variable (7) you can use a family number as signal
+for using the current family. This permits classifying characters with a class
+and still let the family adapt. The trigger family is set with \lpr
+{variablefam}. So:
+
+\starttyping
+\variablefam"24
+\Umathchardef\foo "3 "24 123
+\foo \fam9
+\stoptyping
+
+Results in a curly left brace taken from family 9 with class \quote {relation}
+and spacing around it will be accordingly.
+
 \stopsection
 
 \startsection[title={Math styles}]
@@ -774,13 +788,13 @@ right end. The question is: how often is this implemented, and if so, do the
 kerns assume correction too. Anyway, with this parameter one can control it.
 
 \starttabulate[|l|ck1|ck1|ck1|ck1|ck1|ck1|]
-    \NC
-        \NC \mathnolimitsmode0    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode1    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode2    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode3    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode4    $\displaystyle\int\nolimits^0_1$
-        \NC \mathnolimitsmode8000 $\displaystyle\int\nolimits^0_1$
+    \NC % probably not ok, we need a raw int here
+        \NC \mathnolimitsmode0    $\displaystyle\mathop{\normalint}\nolimits^0_1$
+        \NC \mathnolimitsmode1    $\displaystyle\mathop{\normalint}\nolimits^0_1$
+        \NC \mathnolimitsmode2    $\displaystyle\mathop{\normalint}\nolimits^0_1$
+        \NC \mathnolimitsmode3    $\displaystyle\mathop{\normalint}\nolimits^0_1$
+        \NC \mathnolimitsmode4    $\displaystyle\mathop{\normalint}\nolimits^0_1$
+        \NC \mathnolimitsmode8000 $\displaystyle\mathop{\normalint}\nolimits^0_1$
     \NC \NR
     \TB
     \BC mode