path: root/doc/context/sources/general/manuals/primitives/primitives.tex

% language=us runpath=texruns:manuals/primitives

% The usual time stamp. This written while listening intermized to Fish (just ran
% into), Lazulli (some yt videos too, looking forward to a next live act) and
% because this all is boring checking out Sarah Coopers channel for new DT syncs
% every few hours.

% When you feel unhappy about the lack of detail in this manual, just keep in mind
% that you cannot really demand anything from volunteers: just hope for more (or
% pay for it). Friendly comments and corrections are of course always welcome. As
% we like what we're doing here, it all might eventually evolve to perfection, stay
% tuned.
%
% Hans Hagen | j.hagen @ xs4all . nl | ntg-context @ ntg . nl

\usemodule[article-basic]
\usemodule[abbreviations-logos]
\usemodule[scite]

\definecolor[maincolor][darkblue]
\definecolor[primcolor][darkblue]
\definecolor[nonecolor][darkgray]

\setuptyping
  [option=tex]

\setuptype
  [option=tex]

\setuphead
  [subject]
  [color=maincolor]

\definehead
  [newprimitive]
  [section]
  [color=maincolor]

\definehead
  [oldprimitive]
  [section]
  [color=nonecolor]

\setuplist
  [newprimitive]
  [textcolor=maincolor]

\setuplist
  [oldprimitive]
  [textcolor=nonecolor]

% We use the next one because we want to check what has been done. In a document
% like this using \type {\foo} makes more sense.

\protected\def\prm#1{\doifmode{*bodypart}{\index{\tex{#1}}}\tex{#1}}

% This is why we need to tag bodymatter.

\starttext

\startbodymatter

\pushoverloadmode

\startMPpage
    fill Page
        withcolor "darkgray" ;
    draw textext("\sstf {\white new} primitives")
        xysized (.9bbwidth(Page),bbheight(Page)-2cm)
        shifted center Page
        withcolor "maincolor" ;
    draw textext.ulft("\sstf in luametatex")
        xysized (.9bbwidth(Page)/3,(bbheight(Page)-2cm)/6)
        shifted center lrcorner Page
        shifted (-.1bbwidth(Page),.05bbwidth(Page))
        withcolor "white" ;
    setbounds currentpicture to Page ;
\stopMPpage

\startsubject[title={Introduction}]

Here I will discuss some of the new primitives in \LUATEX\ and \LUAMETATEX, the
later being a successor that permits the \CONTEXT\ folks to experiment with new
features. The order is arbitrary. When you compare \LUATEX\ with \PDFTEX, there
are actually quite some differences. Some primitives that \PDFTEX\ introduced
have been dropped in \LUATEX\ because they can be done better in \LUA. Others
have been promoted to core primitives that no longer have a \type {pdf} prefix.
Then there are lots of new primitives, some introduce new concepts, some are a
side effect of for instance new math font technologies, and then there are those
that are handy extensions to the macro language. The \LUAMETATEX\ engine drops
quite some primitives, like those related to \PDFTEX\ specific font or backend
features. It also adds some new primitives, mostly concerning the macro language.

We also discuss the primitives that fit into the macro programming scope that are
present in traditional \TEX\ and \ETEX\ but there are for sure better of
descriptions out there already. Primitives that relate to typesetting, like those
controlling math, fonts, boxes, attributes, directions, catcodes, \LUA\
(functions) etc are not discussed here.

There are for instance primitives to create aliases to low level registers like
counters and dimensions, as well as other (semi|-|numeric) quantities like
characters, but normally these are wrapped into high level macros so that
definitions can't clash too much. Numbers, dimensions etc can be advanced,
multiplied and divided and there is a simple expression mechanism to deal with
them. These are not discussed here.

\startcolumns
    \placelist
      [newprimitive,oldprimitive]
      [alternative=c]
\stopcolumns

In this document the section titles that discuss the \color [nonecolor] {original
\TEX\ and \ETEX\ primitives} have a different color those explaining the \color
[primcolor] {\LUATEX\ and \LUAMETATEX\ primitives}.

Primitives that extend typesetting related functionality, provide control over
subsystems (like math), allocate additional datatypes and resources, deal with
fonts and languages, manipulate boxes and glyphs, etc.\ are not discussed here.
In this document we concentrate on the programming aspects.

% \startalign[verytolerant,stretch]
% To be described here: \typ {enforced}, \typ {immutable}, \typ {instance},
% \typ {aliased}, \typ {mutable}, \typ {overloaded}, \typ {overloadmode}, \typ
% {tolerant}, \typ {permanent}, \typ {ifflags}, \typ {ifinsert}, \typ
% {ifmathparameter}, \typ {ifmathstyle}, \typ {ifparameter}, \typ
% {ifparameters}, \typ {ignorearguments}, \typ {ignorepars}, \typ
% {parametercount}, \typ {lastchkdim}, \typ {lastchknum}.
%
% Of a different order: \typ {adjustspacing}, \typ {adjustspacingshrink}, \typ
% {adjustspacingstep}, \typ {adjustspacingstretch}, \typ {attribute}, \typ
% {attributedef}, \typ {automaticdiscretionary}, \typ {automatichyphenpenalty},
% \typ {automigrationmode}, \typ {boundary}, \typ {boxattribute}, \typ
% {boxdirection}, \typ {boxorientation}, \typ {boxtotal}, \typ {boxxmove}, \typ
% {boxxoffset}, \typ {boxymove}, \typ {boxyoffset}, \typ {catcodetable}, \typ
% {clearmarks}, \typ {crampeddisplaystyle}, \typ {crampedscriptscriptstyle}, \typ
% {crampedscriptstyle}, \typ {crampedtextstyle}, \typ {directlua}, \typ {efcode},
% \typ {everybeforepar}, \typ {everytab}, \typ {exceptionpenalty}, \typ
% {explicitdiscretionary}, \typ {explicithyphenpenalty}, \typ {firstvalidlanguage},
% \typ {fontid}, \typ {fontmathcontrol}, \typ {fontspecifiedsize}, \typ
% {fonttextcontrol}, \typ {formatname}, \typ {gleaders}, \typ {gluespecdef}, \typ
% {glyphdatafield}, \typ {glyphoptions}, \typ {glyphscale}, \typ
% {glyphscriptfield}, \typ {glyphscriptscale}, \typ {glyphscriptscriptscale}, \typ
% {glyphstatefield}, \typ {glyphtextscale}, \typ {glyphxoffset}, \typ
% {glyphxscale}, \typ {glyphyoffset}, \typ {glyphyscale}, \typ {hccode}, \typ
% {hjcode}, \typ {hpack}, \typ {hyphenationmin}, \typ {hyphenationmode}, \typ
% {initcatcodetable}, \typ {insertbox},\typ {insertcopy}, \typ {insertdepth}, \typ
% {insertdistance}, \typ {insertheight}, \typ {insertheights}, \typ {insertlimit},
% \typ {insertmode}, \typ {insertmultiplier}, \typ {insertprogress}, \typ
% {insertunbox}, \typ {insertuncopy}, \typ {insertwidth}, \typ {lastnodesubtype},
% \typ {leftmarginkern}, \typ {letcharcode}, \typ {linedirection}, \typ {linepar},
% \typ {localbrokenpenalty}, \typ {localinterlinepenalty}, \typ {lpcode}, \typ
% {luabytecode}, \typ {luabytecodecall}, \typ {luacopyinputnodes}, \typ {luadef},
% \typ {luaescapestring}, \typ {luafunction}, \typ {luafunctioncall}, \typ
% {luatexbanner}, \typ {luatexrevision}, \typ {luatexversion}, \typ
% {mathcontrolmode}, \typ {mathdelimitersmode}, \typ {mathdirection}, \typ
% {mathdisplayskipmode}, \typ {matheqnogapstep}, \typ {mathflattenmode}, \typ
% {mathfontcontrol}, \typ {mathitalicsmode}, \typ {mathnolimitsmode}, \typ
% {mathpenaltiesmode}, \typ {mathrulesfam}, \typ {mathrulesmode}, \typ
% {mathrulethicknessmode}, \typ {mathscale}, \typ {mathscriptboxmode}, \typ
% {mathscriptcharmode}, \typ {mathscriptsmode}, \typ {mathstyle}, \typ
% {mathsurroundmode}, \typ {mathsurroundskip}, \typ {mugluespecdef}, \typ
% {noaligned}, \typ {noboundary}, \typ {nohrule}, \typ {normalizelinemode}, \typ
% {nospaces}, \typ {novrule}, \typ {numericscale}, \typ {numexpression}, \typ
% {outputbox}, \typ {parattribute}, \typ {pardirection}, \typ {postexhyphenchar},
% \typ {posthyphenchar}, \typ {prebinoppenalty}, \typ {predisplaygapfactor}, \typ
% {preexhyphenchar}, \typ {prehyphenchar}, \typ {prerelpenalty}, \typ
% {protrudechars}, \typ {protrusionboundary}, \typ {pxdimen}, \typ {quitvmode},
% \typ {retokenize}, \typ {rightmarginkern}, \typ {rpcode}, \typ
% {savecatcodetable}, \typ {scantextokens}, \typ {semiexpanded}, \typ {setfontid},
% \typ {snapshotpar}, \typ {supmarkmode}, \typ {textdirection}, \typ
% {thewithoutunit}, \typ {tpack}, \typ {tracingexpressions}, \typ {tracingfonts},
% \typ {tracinghyphenation}, \typ {tracingmath}, \typ {undent}, \typ {vpack}, \typ
% {wordboundary}, \typ {wrapuppar}.
% \stopalign
%
% {\em Some new primitives in this list might be forgotten or already became
% obsolete. Let me know if you run into one.}

\stopsubject

\page

% When writing this manual I also decided to merge some of the condition related
% code so that it dealt a bit more natural with the newer features. A usual side
% effects if writing manuals.

\startoldprimitive[title={\prm {meaning}}]

We start with a primitive that will be used in the following sections. The
reported meaning can look a bit different than the one reported by other engines
which is a side effect of additional properties and more extensive argument
parsing.

\startbuffer
\tolerant\permanent\protected\gdef\foo[#1]#*[#2]{(#1)(#2)} \meaning\foo
\stopbuffer

\typebuffer \getbuffer

\stopoldprimitive

\startnewprimitive[title={\prm {meaningless}}]

This one reports a bit less than \prm {meaning}.

\startbuffer
\tolerant\permanent\protected\gdef\foo[#1]#*[#2]{(#1)(#2)} \meaningless\foo
\stopbuffer

\typebuffer \getbuffer

\stopnewprimitive

\startnewprimitive[title={\prm {meaningfull}}]

This one reports a bit more than \prm {meaning}.

\startbuffer
\tolerant\permanent\protected\gdef\foo[#1]#*[#2]{(#1)(#2)} \meaningfull\foo
\stopbuffer

\typebuffer \getbuffer

\stopnewprimitive

\startnewprimitive[title={\prm {meaningasis}}]

Although it is not really round trip with the original due to information
being lost this primitive tries to return an equivalent definition.

\startbuffer
\tolerant\permanent\protected\gdef\foo[#1]#*[#2]{(#1)(#2)} \meaningasis\foo
\stopbuffer

\typebuffer \getbuffer

\stopnewprimitive

% \startoldprimitive[title={\prm {par}}]
% \stopoldprimitive
%
% \startoldprimitive[title={\prm {linepar}}]
% \stopoldprimitive

\startoldprimitive[title={\prm {afterassignment}}]

The token following \prm {afterassignment}, a traditional \TEX\ primitive, is
saved and gets injected (and then expanded) after a following assignment took
place.

\startbuffer
\afterassignment !\def\MyMacro {}\quad
\afterassignment !\let\MyMacro ?\quad
\afterassignment !\scratchcounter 123\quad
\afterassignment !%
\afterassignment ?\advance\scratchcounter by 1
\stopbuffer

\typebuffer

The \prm {afterassignment}s are not accumulated, the last one wins:

{\getbuffer}

\stopoldprimitive

\startnewprimitive[title={\prm {afterassigned}}]

The \prm {afterassignment} primitive stores a token to be injected (and thereby
expanded) after an assignment has happened. Unlike \prm {aftergroup}, multiple
calls are not accumulated, and changing that would be too incompatible. This is
why we have \prm {afterassigned}, which can be used to inject a bunch of
tokens. But in order to be consistent this one is also not accumulative.

\startbuffer
\afterassigned{done}%
\afterassigned{{\bf done}}%
\scratchcounter=123
\stopbuffer

\typebuffer

results in: \inlinebuffer\ being typeset.

\stopnewprimitive

\startoldprimitive[title={\prm {aftergroup}}]

The traditional \TEX\ \prm {aftergroup} primitive stores the next token and
expands that after the group has been closed.

\startbuffer
before{ ! \aftergroup a\aftergroup f\aftergroup t\aftergroup e\aftergroup r}
\stopbuffer

Multiple \prm {aftergroup}s are combined:

\typebuffer

\getbuffer

\stopoldprimitive

\startnewprimitive[title={\prm {aftergrouped}}]

The in itself powerful \prm {aftergroup} primitives works quite well, even
if you need to do more than one thing: you can either use it multiple times, or
you can define a macro that does multiple things and apply that after the group.
However, you can avoid that by using this primitive which takes a list of tokens.

\startbuffer
regular
\bgroup
\aftergrouped{regular}%
\bf bold
\egroup
\stopbuffer

\typebuffer

Because it happens after the group, we're no longer typesetting in bold.

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {atendofgroup}}]

The token provided will be injected just before the group ends. Because
these tokens are collected, you need to be aware of possible interference
between them. However, normally this is managed by the macro package.

\startbuffer
\bgroup
\atendofgroup\unskip
\atendofgroup )%
(but it works okay
\egroup
\stopbuffer

\typebuffer

Of course these effects can also be achieved by combining (extra) grouping with
\prm {aftergroup} calls, so this is more a convenience primitives than a real
necessity: {\inlinebuffer}, as proven here.

\stopnewprimitive

\startnewprimitive[title={\prm {atendofgrouped}}]

This is the multi token variant of \prm {atendofgroup}. Of course the next
example is somewhat naive when it comes to spacing and so, but it shows the
purpose.

\startbuffer
\bgroup
\atendofgrouped{\bf QED}%
\atendofgrouped{ (indeed)}%
This sometimes looks nicer.
\egroup
\stopbuffer

\typebuffer

Multiple invocations are accumulated: {\inlinebuffer}.

\stopnewprimitive

\startnewprimitive[title={\prm {toksapp}}]

One way to append something to a token list is the following:

\starttyping
\scratchtoks\expandafter{\the\scratchtoks more stuff}
\stoptyping

This works all right, but it involves a copy of what is already in \type
{\scratchtoks}. This is seldom a real issue unless we have large token lists and
many appends. This is why \LUATEX\ introduced:

\starttyping
\toksapp\scratchtoks{more stuff}
\toksapp\scratchtoksone\scratchtokstwo
\stoptyping

At some point, when working on \LUAMETATEX, I realized that primitives like this
one and the next appenders and prependers to be discussed were always on the
radar of Taco and me. Some were even implemented in what we called \type {eetex}:
extended \ETEX, and we even found back the prototypes, dating from pre|-|\PDFTEX\
times.

\stopnewprimitive

\startnewprimitive[title={\prm {etoksapp}}]

A variant of \prm {toksapp} is the following: it expands the to be appended
content.

\starttyping
\def\temp{more stuff}
\etoksapp\scratchtoks{some \temp}
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {tokspre}}]

Where appending something is easy because of the possible \prm {expandafter}
trickery a prepend would involve more work, either using temporary token
registers and|/|or using a mixture of the (no)expansion added by \ETEX, but all
are kind of inefficient and cumbersome.

\starttyping
\tokspre\scratchtoks{less stuff}
\tokspre\scratchtoksone\scratchtokstwo
\stoptyping

This prepends the token list that is provided.

\stopnewprimitive

\startnewprimitive[title={\prm {etokspre}}]

A variant of \prm {tokspre} is the following: it expands the to be prepended
content.

\starttyping
\def\temp{less stuff}
\etokspre\scratchtoks{a bit \temp}
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {gtoksapp}}]

This is the global variant of \prm {toksapp}.

\stopnewprimitive

\startnewprimitive[title={\prm {xtoksapp}}]

This is the global variant of \prm {etoksapp}.

\stopnewprimitive

\startnewprimitive[title={\prm {gtokspre}}]

This is the global variant of \prm {tokspre}.

\stopnewprimitive

\startnewprimitive[title={\prm {xtokspre}}]

This is the global variant of \prm {etokspre}.

\stopnewprimitive

\startoldprimitive[title={\prm {csname}}]

This original \TEX\ primitive starts the construction of a control sequence
reference. It does a lookup and when no sequence with than name is found, it will
create a hash entry and defaults its meaning to \prm {relax}.

\starttyping
\csname letters and other characters\endcsname
\stoptyping

\stopoldprimitive

\startoldprimitive[title={\prm {endcsname}}]

This primitive is used in combination with \prm {csname}, \prm {ifcsname} and
\prm {begincsname} where its end the scanning for the to be constructed control
sequence token.

\stopoldprimitive

\startnewprimitive[title={\prm {begincsname}}]

The next code creates a control sequence token from the given serialized tokens:

\starttyping
\csname mymacro\endcsname
\stoptyping

When \type {\mymacro} is not defined a control sequence will be created with the
meaning \prm {relax}. A side effect is that a test for its existence might fail
because it now exists. The next sequence will {\em not} create an controil
sequence:

\starttyping
\begincsname mymacro\endcsname
\stoptyping

This actually is kind of equivalent to:

\starttyping
\ifcsname mymacro\endcsname
    \csname mymacro\endcsname
\fi
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {lastnamedcs}}]

The example code in the previous section has some redundancy, in the sense that
there to be looked up control sequence name \type {mymacro} is assembled twice.
This is no big deal in a traditional eight bit \TEX\ but in a \UNICODE\ engine
multi|-|byte sequences demand some more processing (although it is unlikely that
control sequences have many multi|-|byte \UTF8\ characters).

\starttyping
\ifcsname mymacro\endcsname
    \csname mymacro\endcsname
\fi
\stoptyping

Instead we can say:

\starttyping
\ifcsname mymacro\endcsname
    \lastnamedcs
\fi
\stoptyping

Although there can be some performance benefits another advantage is that it uses
less tokens and parsing. It might even look nicer.

\stopnewprimitive

\startnewprimitive[title={\prm {futureexpand}}]

This primitive can be used as an alternative to a \prm {futurelet} approach,
which is where the name comes from. \footnote {In the engine primitives
that have similar behavior are grouped in commands that are then dealt with
together, code wise.}

\startbuffer
\def\variantone<#1>{(#1)}
\def\varianttwo#1{[#1]}
\futureexpand<\variantone\varianttwo<one>
\futureexpand<\variantone\varianttwo{two}
\stopbuffer

\typebuffer

So, the next token determines which of the two variants is taken:

{\getbuffer}

Because we look ahead there is some magic involved: spaces are ignored but when
we have no match they are pushed back into the input. The next variant
demonstrates this:

\startbuffer
\def\variantone<#1>{(#1)}
\def\varianttwo{}
\def\temp{\futureexpand<\variantone\varianttwo}
[\temp <one>]
[\temp {two}]
[\expandafter\temp\space <one>]
[\expandafter\temp\space {two}]
\stopbuffer

\typebuffer

This gives us:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {futureexpandis}}]

We assume that the previous section is read. This variant will not push back
spaces, which permits a consistent approach i.e.\ the user can assume that macro
always gobbles the spaces.

\startbuffer
\def\variantone<#1>{(#1)}
\def\varianttwo{}
\def\temp{\futureexpandis<\variantone\varianttwo}
[\temp <one>]
[\temp {two}]
[\expandafter\temp\space <one>]
[\expandafter\temp\space {two}]
\stopbuffer

\typebuffer

So, here no spaces are pushed back. This \type {is} in the name of this primitive
means \quote {ignore spaces}, but having that added to the name would have made
the primitive even more verbose (after all, we also don't have \type
{\expandeddef} but \prm {edef} and no \type {\globalexpandeddef} but \prm
{xdef}.

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {futureexpandisap}}]

This primitive is like the one in the previous section but also ignores par
tokens, so \type {isap} means \quote {ignore spaces and paragraphs}.

\stopnewprimitive

\startoldprimitive[title={\prm {expandafter}}]

This original \TEX\ primitive stores the next token, does a one level expansion
of what follows it, which actually can be an not expandable token, and
reinjects the stored token in the input. Like:

\starttyping
\expandafter\let\csname my weird macro name\endcsname{m w m n}
\stoptyping

Without \prm {expandafter} the \prm {csname} primitive would have been let to
the left brace (effectively then a begin group). Actually in this particular case
the control sequence with the weird name is injected and when it didn't yet exist
it will get the meaning \prm {relax} so we sort of have two assignments in a
row then.

\stopoldprimitive

\startnewprimitive[title={\prm {expandafterspaces}}]

This is a gobbler: the next token is reinjected after following spaces have been
read. Here is a simple example:

\startbuffer
[\expandafterspaces 1 2]
[\expandafterspaces 3
4]
[\expandafterspaces 5

6]
\stopbuffer

\typebuffer

We get this typeset: \inlinebuffer, because a newline normally is configured to be
a space (and leading spaces in a line are normally being ingored anyway).

\stopnewprimitive

\startnewprimitive[title={\prm {expandafterpars}}]

Here is another gobbler: the next token is reinjected after following spaces
and par tokens have been read. So:

\startbuffer
[\expandafterpars 1 2]
[\expandafterpars 3
4]
[\expandafterpars 5

6]
\stopbuffer

\typebuffer

gives us: \inlinebuffer, because empty lines are like \prm {par} and therefore
ignored.

\stopnewprimitive

\startnewprimitive[title={\prm {expandtoken}}]

This primitive creates a token with a specific combination of catcode and
character code. Because it assumes some knowledge of \TEX\ we can show it
using some \prm {expandafter} magic:

\startbuffer
\expandafter\let\expandafter\temp\expandtoken 11 `X \meaning\temp
\expandafter\let\expandafter\temp\expandtoken 12 `X \meaning\temp
\stopbuffer

\typebuffer

The meanings are:

\startlines \tttf \getbuffer \stoplines

Using other catcodes is possible but the results of injecting them into the input
directly (or here by injecting \type {\temp}) can be unexpected because of what
\TEX\ expects. You can get messages you normally won't get, for instance about
unexpected alignment interference, which is a side effect of \TEX\ using some
catcode|/|character combinations as signals and there is no reason to change
those internals. That said:

\startbuffer
\edef\tempA{\expandtoken  9 `X} \meaning\tempA
\edef\tempB{\expandtoken 10 `X} \meaning\tempB
\edef\tempC{\expandtoken 11 `X} \meaning\tempC
\edef\tempD{\expandtoken 12 `X} \meaning\tempD
\stopbuffer

\typebuffer

are all valid and from the meaning you cannot really deduce what's in there:

\startlines \tttf \getbuffer \stoplines

But you can be assured that:

\startbuffer
[AB: \ifx\tempA\tempB Y\else N\fi]
[AC: \ifx\tempA\tempC Y\else N\fi]
[AD: \ifx\tempA\tempD Y\else N\fi]
[BC: \ifx\tempB\tempC Y\else N\fi]
[BD: \ifx\tempB\tempD Y\else N\fi]
[CD: \ifx\tempC\tempD Y\else N\fi]
\stopbuffer

\typebuffer

makes clear that they're different: \inlinebuffer, and in case you wonder, the
characters with catcode 10 are spaces, while those with code 9 are ignored.

\stopnewprimitive

\startnewprimitive[title={\prm {expandcstoken}}]

The rationale behind this primitive is that when we \prm {let} a single token
like a character it is hard to compare that with something similar, stored in a
macro. This primitive pushes back a single token alias created by \prm {let}
into the input.

\startbuffer
\let\tempA + \meaning\tempA

\let\tempB X \meaning\tempB \crlf
\let\tempC $ \meaning\tempC \par

\edef\temp         {\tempA} \doifelse{\temp}{+}{Y}{N} \meaning\temp \crlf
\edef\temp         {\tempB} \doifelse{\temp}{X}{Y}{N} \meaning\temp \crlf
\edef\temp         {\tempC} \doifelse{\temp}{X}{Y}{N} \meaning\temp \par

\edef\temp{\expandcstoken\tempA} \doifelse{\temp}{+}{Y}{N} \meaning\temp \crlf
\edef\temp{\expandcstoken\tempB} \doifelse{\temp}{X}{Y}{N} \meaning\temp \crlf
\edef\temp{\expandcstoken\tempC} \doifelse{\temp}{$}{Y}{N} \meaning\temp \par

\doifelse{\expandcstoken\tempA}{+}{Y}{N}
\doifelse{\expandcstoken\tempB}{X}{Y}{N}
\doifelse{\expandcstoken\tempC}{$}{Y}{N} \par
\stopbuffer

\typebuffer

The meaning of the \prm {let} macros shows that we have a shortcut to a
character with (in this case) catcode letter, other (here \quote {other
character} gets abbreviated to \quote {character}), math shift etc.

\start \tttf \getbuffer \stop

Here we use the \CONTEXT\ macro \type {\doifelse} which can be implemented in
different ways, but the only property relevant to the user is that the expanded
content of the two arguments is compared.

\stopnewprimitive

\startnewprimitive[title={\prm {expand}}]

Normally a protected macro will not be expanded inside for instance an \prm
{edef} but there is a way out: \footnote {This primitive is dedicated to Hans vd
Meer who was playing with the unprotected version of \type {\doifelse} and
wondered about the reason for it not being expandable in the first place.}

\startbuffer
\edef\temp       {\doifelse{a}{b}{c}{d}} \meaning\temp \crlf
\edef\temp{\expand\doifelse{a}{b}{c}{d}} \meaning\temp \par
\stopbuffer

\typebuffer

In the second case, the \type {\doifelse} command {\em is} expanded, but keep in mind
that this only makes sense when the body of such a macro is expandable. This is the
case in \CONTEXT\ \LMTX, but not in \MKIV.

{\getbuffer}

\stopnewprimitive

\startoldprimitive[title={\prm {ignorespaces}}]

This traditional \TEX\ primitive signals the scanner to ignore the following
spaces, if any. We mention it because we show a companion in the next section.

\stopoldprimitive

\startnewprimitive[title={\prm {ignorepars}}]

This is a variant of \prm {ignorespaces}: following spaces {\em and} \type
{\par} equivalent tokens are ignored, so for instance:

\startbuffer
one + \ignorepars

two = \ignorepars \par
three
\stopbuffer

\typebuffer

renders as: \inlinebuffer. Traditionally \TEX\ has been sensitive to \prm {par}
tokens in some of its building blocks. This has to do with the fact that it could
indicate a runaway argument which in the times of slower machines and terminals
was best to catch early. In \LUAMETATEX\ we no longer have long macros and the
mechanisms that are sensitive can be told to accept \prm {par} tokens (and
\CONTEXT\ set them such that this is the case).

\stopnewprimitive

\startnewprimitive[title={\prm {ignorearguments}}]

This primitive will quit argument scanning and start expansion of the body of a
macro. The number of grabbed arguments can be tested as follows:

\startbuffer
\def\MyMacro[#1][#2][#3]%
 {\ifarguments zero\or one\or two\or three \else hm\fi}

\MyMacro          \ignorearguments \quad
\MyMacro       [1]\ignorearguments \quad
\MyMacro    [1][2]\ignorearguments \quad
\MyMacro [1][2][3]\ignorearguments \par
\stopbuffer

\typebuffer

{\getbuffer}

{\em Todo: explain optional delimiters.}

\stopnewprimitive

\startnewprimitive[title={\prm {lastarguments}}]

\startbuffer
\def\MyMacro    #1{\the\lastarguments (#1) }          \MyMacro{1}       \crlf
\def\MyMacro  #1#2{\the\lastarguments (#1) (#2)}      \MyMacro{1}{2}    \crlf
\def\MyMacro#1#2#3{\the\lastarguments (#1) (#2) (#3)} \MyMacro{1}{2}{3} \par

\def\MyMacro    #1{(#1)           \the\lastarguments} \MyMacro{1}       \crlf
\def\MyMacro  #1#2{(#1) (#2)      \the\lastarguments} \MyMacro{1}{2}    \crlf
\def\MyMacro#1#2#3{(#1) (#2) (#3) \the\lastarguments} \MyMacro{1}{2}{3} \par
\stopbuffer

\typebuffer

The value of \prm {lastarguments} can only be trusted in the expansion until
another macro is seen and expanded. For instance in these examples, as soon as a
character (like the left parenthesis) is seen, horizontal mode is entered and
\prm {everypar} is expanded which in turn can involve macros. You can see that
in the second block (that is: unless we changed \prm {everypar} in the
meantime).

{\getbuffer}

\stopnewprimitive

\startoldprimitive[title={\prm {scantokens}}]

Just forget about this \ETEX\ primnitive, just take the one in the next section.

\stopoldprimitive

\startnewprimitive[title={\prm {scantextokens}}]

This primitive scans the input as if it comes from a file. In the next examples
the \prm {detokenize} primitive turns tokenized code into verbatim code that is
similar to what is read from a file.

\startbuffer
\edef\whatever{\detokenize{This is {\bf bold} and this is not.}}
\detokenize   {This is {\bf bold} and this is not.}\crlf
\scantextokens{This is {\bf bold} and this is not.}\crlf
\scantextokens{\whatever}\crlf
\scantextokens\expandafter{\whatever}\par
\stopbuffer

\typebuffer

This primitive does not have the end|-|of|-|file side effects of its precursor
\prm {scantokens}.

{\getbuffer}

\stopnewprimitive

\startoldprimitive[title={\prm {number}}]

This \TEX\ primitive serializes the next token into a number, assuming that it
is indeed a number, like

\starttyping
\number`A
\number65
\number\scratchcounter
\stoptyping

For counters and such the \prm {the} primitive does the same, but when you're
not sure if what follows is a verbose number or (for instance) a counter the
\prm {number} primitive is a safer bet, because \type {\the 65} will not work.

\stopoldprimitive

\startnewprimitive[title={\prm {tointeger}}]

\startbuffer
\scratchcounter = 1234 \tointeger\scratchcounter
\stopbuffer

The following code gives this: {\nospacing\inlinebuffer} and is equivalent to
\prm {number}.

\typebuffer

\stopnewprimitive

\startnewprimitive[title={\prm {tohexadecimal}}]

\startbuffer
\scratchcounter = 1234 \tohexadecimal\scratchcounter
\stopbuffer

The following code gives this: {\nospacing\inlinebuffer} with uppercase letters.

\typebuffer

\stopnewprimitive

\startnewprimitive[title={\prm {todimension}}]

\startbuffer
\scratchdimen = 1234pt \todimension\scratchdimen
\stopbuffer

The following code gives this: {\nospacing\inlinebuffer} and like its numeric
counterparts accepts anything that resembles a number this one goes beyond
(user, internal or pseudo) registers values too.

\typebuffer

\stopnewprimitive

\startnewprimitive[title={\prm {toscaled}}]

\startbuffer
\scratchdimen = 1234pt \toscaled\scratchdimen
\stopbuffer

The following code gives this: {\nospacing\inlinebuffer} is similar to \prm
{todimension} but omits the \type {pt} so that we don't need to revert to some
nasty stripping code.

\typebuffer

\stopnewprimitive

\startnewprimitive[title={\prm {tosparsedimension}}]

\startbuffer
\scratchdimen = 1234pt \tosparsedimension\scratchdimen
\stopbuffer

The following code gives this: {\nospacing\inlinebuffer} where \quote {sparse}
indicates that redundant trailing zeros are not shown.

\typebuffer

\stopnewprimitive

\startnewprimitive[title={\prm {tosparsescaled}}]

\startbuffer
\scratchdimen = 1234pt \tosparsescaled\scratchdimen
\stopbuffer

The following code gives this: {\nospacing\inlinebuffer} where \quote {sparse}
means that redundant trailing zeros are omitted.

\typebuffer

\stopnewprimitive

\startoldprimitive[title={\prm {numericscale}}]

This primitive can best be explained by a few examples:

\startbuffer
\the\numericscale 1323
\the\numericscale 1323.0
\the\numericscale 1.323
\the\numericscale 13.23
\stopbuffer

\typebuffer

In several places \TEX\ uses a scale but due to the lack of floats it then uses
1000 as 1.0 replacement. This primitive can be used for \quote {real} scales and
the period signals this:

\startlines \getbuffer \stoplines

When there is a period (indicating the fraction) the result is an integer (count)
that has the multiplier 1000 applied.

\stopnewprimitive

\startoldprimitive[title={\prm {string}}]

We mention this original primitive because of the one in the next section. It
expands the next token or control sequence as if it was just entered, so normally
a control sequence becomes a backslash followed by characters and a space.

\stopoldprimitive

\startnewprimitive[title={\prm {csstring}}]

This primitive returns the name of the control sequence given without the leading
escape character (normally a backslash). Of course you could strip that character
with a simple helper but this is more natural.

\startbuffer
\csstring\mymacro
\stopbuffer

\typebuffer

We get the name, not the meaning: {\tt \inlinebuffer}.

\stopnewprimitive

\startoldprimitive[title={\prm {unexpanded}}]

This is an \ETEX\ enhancement. The content will not be expanded in a context
where expansion is happening, like in an \prm {edef}. In \CONTEXT\ you need to
use \type {\normalunexpanded} because we already had a macro with that name.

\startbuffer
\def \A{!}                       \meaning\A
\def \B{?}                       \meaning\B
\edef\C{\A\B}                    \meaning\C
\edef\C{\normalunexpanded{\A}\B} \meaning\C
\stopbuffer

\typebuffer

\startlines \tttf \getbuffer \stoplines

\stopoldprimitive

\startoldprimitive[title={\prm {detokenize}}]

This \ETEX\ primitive turns the content of the provides list will become
characters, kind of verbatim.

\startbuffer
\expandafter\let\expandafter\temp\detokenize{1} \meaning\temp
\expandafter\let\expandafter\temp\detokenize{A} \meaning\temp
\stopbuffer

\typebuffer

\startlines \tttf \getbuffer \stoplines

\stopoldprimitive

\startnewprimitive[title={\prm {tokenized}}]

Just as \prm {expanded} has a counterpart \prm {unexpanded}, it makes sense to give
\prm {detokenize} a companion:

\startbuffer
\edef\foo{\detokenize{\inframed{foo}}}
\edef\oof{\detokenize{\inframed{oof}}}

\meaning\foo \crlf \dontleavehmode\foo

\edef\foo{\tokenized{\foo\foo}}

\meaning\foo \crlf \dontleavehmode\foo

\dontleavehmode\tokenized{\foo\oof}
\stopbuffer

\typebuffer {\tttf \getbuffer}

This primitive is similar to:

\starttyping
\def\tokenized#1{\scantextokens\expandafter{\normalexpanded{#1}}}
\stoptyping

and should be more efficient, not that it matters much as we don't use it that
much (if at all).

\stopnewprimitive

\startnewprimitive[title={\prm {expanded}}]

This primitive complements the two expansion related primitives mentioned in the
previous two sections. This time the content will be expanded and then pushed
back into the input. Protected macros will not be expanded, so you can use this
primitive to expand the arguments in a call. In \CONTEXT\ you need to use \type
{\normalexpanded} because we already had a macro with that name. We give some
examples:

\startbuffer
\def\A{!}
          \def\B#1{\string#1}                          \B{\A}  \crlf
          \def\B#1{\string#1} \normalexpanded{\noexpand\B{\A}} \crlf
\protected\def\B#1{\string#1}                          \B{\A}  \par
\stopbuffer

\typebuffer {\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {numexpression}}]

The normal \prm {numexpr} primitive understands the \type {+}, \type {-}, \type
{*} and \type {/} operators but in \LUAMETATEX\ we also can use \type {:} for a
non rounded integer division (think of \LUA's \type {//}). if you want more than
that, you can use the new expression primitive where you can use the following
operators.

\starttabulate[||cT|cT|]
\BC add       \NC +                    \NC        \NC \NR
\BC subtract  \NC -                    \NC        \NC \NR
\BC multiply  \NC *                    \NC        \NC \NR
\BC divide    \NC / :                  \NC        \NC \NR
\BC mod       \NC \letterpercent       \NC mod    \NC \NR
\BC band      \NC &                    \NC band   \NC \NR
\BC bxor      \NC ^                    \NC bxor   \NC \NR
\BC bor       \NC \letterbar \space v  \NC bor    \NC \NR
\BC and       \NC &&                   \NC and    \NC \NR
\BC or        \NC \letterbar\letterbar \NC or     \NC \NR
\BC setbit    \NC <undecided>          \NC bset   \NC \NR
\BC resetbit  \NC <undecided>          \NC breset \NC \NR
\BC left      \NC <<                   \NC        \NC \NR
\BC right     \NC >>                   \NC        \NC \NR
\BC less      \NC <                    \NC        \NC \NR
\BC lessequal \NC <=                   \NC        \NC \NR
\BC equal     \NC = ==                 \NC        \NC \NR
\BC moreequal \NC >=                   \NC        \NC \NR
\BC more      \NC >                    \NC        \NC \NR
\BC unequal   \NC <> != \lettertilde = \NC        \NC \NR
\BC not       \NC ! \lettertilde       \NC not    \NC \NR
\stoptabulate

An example of the verbose bitwise operators is:

\starttyping
\scratchcounter = \numexpression
    "00000 bor "00001 bor "00020 bor "00400 bor "08000 bor "F0000
\relax
\stoptyping

In the table you might have notices that some operators have equivalents. This
makes the scanner a bit less sensitive for catcode regimes.

When \prm {tracingexpressions} is set to one or higher the intermediate \quote
{reverse polish notation} stack that is used for the calculation is shown, for
instance:

\starttyping
4:8: {numexpression rpn: 2 5 > 4 5 > and}
\stoptyping

When you want the output on your console, you need to say:

\starttyping
\tracingexpressions 1
\tracingonline      1
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {dimexpression}}]

This command is like \prm {numexpression} but results in a dimension instead of
an integer. Where \prm {dimexpr} doesn't like \typ {2 * 10pt} this expression
primitive is quite happy with it.

\stopnewprimitive

\startoldprimitive[title={\prm {if}}]

This traditional \TEX\ conditional checks if two character codes are the same. In
order to understand unexpanded results it is good to know that internally \TEX\
groups primitives in a way that serves the implementation. Each primitive has a
command code and a character code, but only for real characters the name
character code makes sense. This condition only really tests for character codes
when we have a character, in all other cases, the result is true.

\startbuffer
\def\A{A}\def\B{B} \chardef\C=`C \chardef\D=`D \def\AA{AA}

[\if AA   YES \else NOP \fi] [\if AB   YES \else NOP \fi]
[\if \A\B YES \else NOP \fi] [\if \A\A YES \else NOP \fi]
[\if \C\D YES \else NOP \fi] [\if \C\C YES \else NOP \fi]
[\if \count\dimen YES \else NOP \fi] [\if \AA\A YES \else NOP \fi]

\stopbuffer

\typebuffer

The last example demonstrates that the tokens get expanded, which is why
we get the extra \type {A}:

{\getbuffer}

\stopoldprimitive

\startoldprimitive[title={\prm {ifcat}}]

Another traditional \TEX\ primitive: what happens with what gets read in depends
on the catcode of a character, think of characters marked to start math mode, or
alphabetic characters (letters) versus other characters (like punctuation).

\startbuffer
\def\A{A}\def\B{,} \chardef\C=`C \chardef\D=`, \def\AA{AA}

[\ifcat $!   YES \else NOP \fi] [\ifcat ()   YES \else NOP \fi]
[\ifcat AA   YES \else NOP \fi] [\ifcat AB   YES \else NOP \fi]
[\ifcat \A\B YES \else NOP \fi] [\ifcat \A\A YES \else NOP \fi]
[\ifcat \C\D YES \else NOP \fi] [\ifcat \C\C YES \else NOP \fi]
[\ifcat \count\dimen YES \else NOP \fi] [\ifcat \AA\A YES \else NOP \fi]
\stopbuffer

\typebuffer

Close reading is needed here:

{\getbuffer}

This traditional \TEX\ condition as a well as the one in the previous section are
hardly used in \CONTEXT, if only because they expand what follows and we seldom
need to compare characters.

\stopoldprimitive

\startoldprimitive[title={\prm {ifnum}}]

This is a frequently used conditional: it compares two numbers where a number is
anything that can be seen as such.

\startbuffer
\scratchcounter=65 \chardef\A=65

\ifnum65=`A              YES \else NOP\fi
\ifnum\scratchcounter=65 YES \else NOP\fi
\ifnum\scratchcounter=\A YES \else NOP\fi
\stopbuffer

\typebuffer

Unless a number is an unexpandable token it ends with a space or \prm {relax},
so when you end up in the true branch, you'd better check if \TEX\ could
determine where the number ends.

{\getbuffer}

% When comparing integers, definitions (for instance characters) that can be seen
% as such, or any converter that produces a number (like the \type {`} or \prm
% {number} the usual \type {=}, \type {<} or \type {>} can be used. However, in
% \LUAMETATEX\ you can negate such a comparison by \type {!}: \type {!=}, \type
% {!<} or \type {!>}. Successive \type {!} toggle the negation state.

On top of these \ASCII\ combinations, the engine also accepts some \UNICODE\
characters. This brings the full repertoire to:

\starttabulate[|l|cT|cT|l|]
\FL
\BC character      \BC               \BC    \BC operation         \NC \NR
\ML
\NC \type {0x003C} \NC $\Uchar"003C$ \NC    \NC less              \NC \NR
\NC \type {0x003D} \NC $\Uchar"003D$ \NC    \NC equal             \NC \NR
\NC \type {0x003E} \NC $\Uchar"003E$ \NC    \NC more              \NC \NR
\NC \type {0x2208} \NC $\Uchar"2208$ \NC    \NC element of        \NC \NR
\NC \type {0x2209} \NC $\Uchar"2209$ \NC    \NC not element of    \NC \NR
\NC \type {0x2260} \NC $\Uchar"2260$ \NC != \NC not equal         \NC \NR
\NC \type {0x2264} \NC $\Uchar"2264$ \NC !> \NC less equal        \NC \NR
\NC \type {0x2265} \NC $\Uchar"2265$ \NC !< \NC greater equal     \NC \NR
\NC \type {0x2270} \NC $\Uchar"2270$ \NC    \NC not less equal    \NC \NR
\NC \type {0x2271} \NC $\Uchar"2271$ \NC    \NC not greater equal \NC \NR
\LL
\stoptabulate

This also applied to \prm {ifdim} although in the case of element we discard the
fractional part (read: divide the numeric representation by 65536).

\stopoldprimitive

\startoldprimitive[title={\prm {ifdim}}]

Dimensions can be compared with this traditional \TEX\ primitive.

\startbuffer
\scratchdimen=1pt \scratchcounter=65536

\ifdim\scratchdimen=\scratchcounter sp YES \else NOP\fi
\ifdim\scratchdimen=1               pt YES \else NOP\fi
\stopbuffer

\typebuffer

The units are mandate:

{\getbuffer}

\stopoldprimitive

\startoldprimitive[title={\prm {ifodd}}]

One reason for this condition to be around is that in a double sided layout we
need test for being on an odd or even page. It scans for a number the same was
as other primitives,

\startbuffer
\ifodd65 YES \else NO\fi &
\ifodd`B YES \else NO\fi .
\stopbuffer

\typebuffer

So: {\inlinebuffer}

\stopoldprimitive

\startoldprimitive[title={\prm {ifvmode}}]

This traditional conditional checks we are in (internal) vertical mode.

\stopoldprimitive

\startoldprimitive[title={\prm {ifhmode}}]

This traditional conditional checks we are in (restricted) horizontal mode.

\stopoldprimitive

\startoldprimitive[title={\prm {ifmmode}}]

This traditional conditional checks we are in (inline or display) math mode mode.

\stopoldprimitive

\startoldprimitive[title={\prm {ifinner}}]

This traditional one can be confusing. It is true when we are in restricted
horizontal mode (a box), internal vertical mode (a box), or inline math mode.

\startbuffer
test \ifhmode \ifinner INNER\fi HMODE\fi\crlf
\hbox{test \ifhmode \ifinner INNER \fi HMODE\fi} \par

\ifvmode \ifinner INNER\fi VMODE \fi\crlf
\vbox{\ifvmode \ifinner INNER \fi VMODE\fi} \crlf
\vbox{\ifinner INNER \ifvmode VMODE \fi \fi} \par
\stopbuffer

\typebuffer

Watch the last line: because we typeset \type {INNER} we enter horizontal mode:

{\getbuffer}

\stopoldprimitive

\startoldprimitive[title={\prm {ifvoid}}]

This traditional conditional checks if a given box register or internal box
variable has any content.

\stopoldprimitive

\startoldprimitive[title={\prm {ifhbox}}]

This traditional conditional checks if a given box register or internal box
variable represents a horizontal box,

\stopoldprimitive

\startoldprimitive[title={\prm {ifvbox}}]

This traditional conditional checks if a given box register or internal box
variable represents a vertical box,

\stopoldprimitive

\startoldprimitive[title={\prm {ifx}}]

We use this traditional \TEX\ conditional a lot in \CONTEXT. Contrary to \prm {if}
the two tokens that are compared are not expanded. This makes it possible to compare
the meaning of two macros. Depending on the need, these macros can have their content
expanded or not. A different number of parameters results in false.

Control sequences are identical when they have the same command code and
character code. Because a \prm {let} macro is just a reference, both let macros
are the same and equal to \prm {relax}:

\starttyping
\let\one\relax \let\two\relax
\stoptyping

The same is true for other definitions that result in the same (primitive) or
meaning encoded in the character field (think of \prm {chardef}s and so).

\stopoldprimitive

\startoldprimitive[title={\prm {ifeof}}]

This traditional conditional checks if current pointer into the the file bound to
the given index is past the end of file. The read and write channels are not
really used in \CONTEXT: in \MKII\ we only had one file for all multi|-|pass
data, and in \MKIV\ all file related stuff is dealt with in \LUATEX.

\stopoldprimitive

\startoldprimitive[title={\prm {iftrue}}]

Here we have a traditional \TEX\ conditional that is always true (therefore the
same is true for any macro that is \prm {let} to this primitive).

\stopoldprimitive

\startoldprimitive[title={\prm {iffalse}}]

Here we have a traditional \TEX\ conditional that is always false (therefore the
same is true for any macro that is \prm {let} to this primitive).

\stopoldprimitive

\startoldprimitive[title={\prm {ifcase}}]

This numeric \TEX\ conditional takes a counter (literal, register, shortcut to a
character, internal quantity) and goes to the branch that matches.

\startbuffer
\ifcase 3 zero\or one\or two\or three\or four\else five or more\fi
\stopbuffer

\typebuffer

Indeed: \inlinebuffer\ equals three. In later sections we will see some
\LUAMETATEX\ primitives that behave like an \prm {ifcase}.

\stopoldprimitive

\startoldprimitive[title={\prm {ifdefined}}]

In traditional \TEX\ checking for a macro to exist was a bit tricky and therefore
\ETEX\ introduced a convenient conditional. We can do this:

\starttyping
\ifx\MyMacro\undefined ... \else ... \fi
\stoptyping

but that assumes that \type {\undefined} is indeed undefined. Another test often
seen was this:

\starttyping
\expandafter\ifx\csname MyMacro\endcsname\relax ... \else ... \fi
\stoptyping

Instead of comparing with \type {\undefined} we need to check with \prm {relax}
because the control sequence is defined when not yet present and defaults to
\prm {relax}. This is not pretty.

\stopoldprimitive

\startoldprimitive[title={\prm {ifcsname}}]

This is an \ETEX\ conditional that complements the one on the previous section:

\starttyping
\expandafter\ifx\csname MyMacro\endcsname\relax ... \else ... \fi
            \ifcsname   MyMacro\endcsname       ... \else ... \fi
\stoptyping

Here the first one has the side effect of defining the macro and defaulting it to
\prm {relax}, while the second one doesn't do that. Juts think of checking a
few million different names: the first one will deplete the hash table and
probably string space too.

In \LUAMETATEX\ the construction stops when there is no letter or other character
seen (\TEX\ expands on the go so expandable macros are dealt with). Instead of an
error message, the match is simply false and all tokens till the \prm
{endcsname} are gobbled.

\stopoldprimitive

\startoldprimitive[title={\prm {iffontchar}}]

This is an \ETEX\ conditional. It takes a font identifier and a character number.
In modern fonts simply checking could not be enough because complex font features
can swap in other ones and their index can be anything. Also, a font mechanism
can provide fallback fonts and characters, so don't rely on this one too much. It
just reports true when the font passed to the frontend has a slot filled.

\stopoldprimitive

\startnewprimitive[title={\prm {ifincsname}}]

This conditional is sort of obsolete and can be used to check if we're inside a
\prm {csname} or \prm {ifcsname} construction. It's not used in \CONTEXT.

\stopnewprimitive

\startnewprimitive[title={\prm {ifabsnum}}]

This test will negate negative numbers before comparison, as in:

\startbuffer
\def\TestA#1{\ifnum   #1<100 too small\orelse\ifnum   #1>200 too large\else okay\fi}
\def\TestB#1{\ifabsnum#1<100 too small\orelse\ifabsnum#1>200 too large\else okay\fi}

\TestA {10}\quad\TestA {150}\quad\TestA {210}\crlf
\TestB {10}\quad\TestB {150}\quad\TestB {210}\crlf
\TestB{-10}\quad\TestB{-150}\quad\TestB{-210}\par
\stopbuffer

\typebuffer

Here we get the same result each time:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifabsdim}}]

This test will negate negative dimensions before comparison, as in:

\startbuffer
\def\TestA#1{\ifdim   #1<2pt too small\orelse\ifdim   #1>4pt too large\else okay\fi}
\def\TestB#1{\ifabsdim#1<2pt too small\orelse\ifabsdim#1>4pt too large\else okay\fi}

\TestA {1pt}\quad\TestA {3pt}\quad\TestA {5pt}\crlf
\TestB {1pt}\quad\TestB {3pt}\quad\TestB {5pt}\crlf
\TestB{-1pt}\quad\TestB{-3pt}\quad\TestB{-5pt}\par
\stopbuffer

\typebuffer

So we get this:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifzerodim}}]

This tests for a dimen (dimension) being zero so we have:

\starttyping
\ifdim<dimension>=0pt
\ifzerodim<dimension>
\ifcase<dimension register>
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {ifzeronum}}]

This tests for a number (integer) being zero so we have these variants now:

\starttyping
\ifnum<integer or equivalent>=0pt
\ifzeronum<integer or equivalent>
\ifcase<integer or equivalent>
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {ifchknum}}]

In \CONTEXT\ there are quite some cases where a variable can have a number or a
keyword indicating a symbolic name of a number or maybe even some special
treatment. Checking if a valid number is given is possible to some extend, but a
native checker makes much sense too. So here is one:

\startbuffer
\ifchknum oeps\or okay\else error\fi\quad
\ifchknum 12  \or okay\else error\fi\quad
\ifchknum 12pt\or okay\else error\fi
\stopbuffer

\typebuffer

The result is as expected:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifchkdim}}]

A variant on the checker in the previous section is a dimension checker:

\startbuffer
\ifchkdim oeps\or okay\else error\fi\quad
\ifchkdim 12  \or okay\else error\fi\quad
\ifchkdim 12pt\or okay\else error\fi
\stopbuffer

\typebuffer

We get:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifcmpnum}}]

This conditional compares two numbers and the resulting \prm {ifcase} reflects
their relation:

\startbuffer
[1 2 : \ifcmpnum 1 2 less\or equal\or more\fi]\quad
[1 1 : \ifcmpnum 1 1 less\or equal\or more\fi]\quad
[2 1 : \ifcmpnum 2 1 less\or equal\or more\fi]
\stopbuffer

\typebuffer

This gives:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifcmpdim}}]

This conditional compares two dimensions and the resulting \prm {ifcase}
reflects their relation:

\startbuffer
[1pt 2pt : \ifcmpdim 1pt 2pt less\or equal\or more\fi]\quad
[1pt 1pt : \ifcmpdim 1pt 1pt less\or equal\or more\fi]\quad
[2pt 1pt : \ifcmpdim 2pt 1pt less\or equal\or more\fi]
\stopbuffer

\typebuffer

This gives:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifnumval}}]

This conditional is a variant on \prm {ifchknum}. This time we get
some more detail about the value:

\startbuffer
[-12  : \ifnumval  -12\or negative\or zero\or positive\else error\fi]\quad
[0    : \ifnumval    0\or negative\or zero\or positive\else error\fi]\quad
[12   : \ifnumval   12\or negative\or zero\or positive\else error\fi]\quad
[oeps : \ifnumval oeps\or negative\or zero\or positive\else error\fi]
\stopbuffer

\typebuffer

This gives:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifdimval}}]

This conditional is a variant on \prm {ifchkdim} and provides some more
detailed information about the value:

\startbuffer
[-12pt : \ifdimval-12pt\or negative\or zero\or positive\else error\fi]\quad
[0pt   : \ifdimval  0pt\or negative\or zero\or positive\else error\fi]\quad
[12pt  : \ifdimval 12pt\or negative\or zero\or positive\else error\fi]\quad
[oeps  : \ifdimval oeps\or negative\or zero\or positive\else error\fi]
\stopbuffer

\typebuffer

This gives:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {iftok}}]

When you want to compare two arguments, the usual way to do this is the
following:

\starttyping
\edef\tempA{#1}
\edef\tempb{#2}
\ifx\tempA\tempB
    the same
\else
    different
\fi
\stoptyping

This works quite well but the fact that we need to define two macros can be
considered a bit of a nuisance. It also makes macros that use this method to be
not so called \quote {fully expandable}. The next one avoids both issues:

\starttyping
\iftok{#1}{#2}
    the same
\else
    different
\fi
\stoptyping

Instead of direct list you can also pass registers, so given:

\startbuffer[a]
\scratchtoks{a}%
\toks0{a}%
\stopbuffer

\typebuffer[a]

This:

\startbuffer[b]
\iftok 0 \scratchtoks          Y\else N\fi\space
\iftok{a}\scratchtoks          Y\else N\fi\space
\iftok\scratchtoks\scratchtoks Y\else N\fi
\stopbuffer

\typebuffer[b]

{\getbuffer[a]gives: \inlinebuffer[b].}

\stopnewprimitive

\startnewprimitive[title={\prm {ifcstok}}]

A variant on the primitive mentioned in the previous section is one that
operates on lists and macros:

\startbuffer[a]
\def\a{a} \def\b{b} \def\c{a}
\stopbuffer

\typebuffer[a]

\startbuffer[b]
\ifcstok\a\b   Y\else N\fi\space
\ifcstok\a\c   Y\else N\fi\space
\ifcstok{\a}\c Y\else N\fi\space
\ifcstok{a}\c  Y\else N\fi
\stopbuffer

This:

\typebuffer[b]

{\getbuffer[a]will give us: \inlinebuffer[b].}

\stopnewprimitive

\startnewprimitive[title={\prm {ifcondition}}]

The conditionals in \TEX\ are hard coded as primitives and although it might
look like \type {\newif} creates one, it actually just defined three macros.

\startbuffer
\newif\ifMyTest
\meaning\MyTesttrue  \crlf
\meaning\MyTestfalse \crlf
\meaning\ifMyTest    \crlf \MyTesttrue
\meaning\ifMyTest    \par
\stopbuffer

\typebuffer {\tttf \getbuffer}

This means that when you say:

\starttyping
\ifMytest ... \else ... \fi
\stoptyping

You actually have one of:

\starttyping
\iftrue  ... \else ... \fi
\iffalse ... \else ... \fi
\stoptyping

and because these are proper conditions nesting them like:

\starttyping
\ifnum\scratchcounter > 0 \ifMyTest A\else B\fi \fi
\stoptyping

will work out well too. This is not true for macros, so for instance:

\starttyping
\scratchcounter = 1
\unexpanded\def\ifMyTest{\iftrue}
\ifnum\scratchcounter > 0 \ifMyTest A\else B\fi \fi
\stoptyping

will make a run fail with an error (or simply loop forever, depending on your
code). This is where \prm {ifcondition} enters the picture:

\starttyping
\def\MyTest{\iftrue} \scratchcounter0
\ifnum\scratchcounter > 0
    \ifcondition\MyTest A\else B\fi
\else
    x
\fi
\stoptyping

This primitive is seen as a proper condition when \TEX\ is in \quotation {fast
skipping unused branches} mode but when it is expanding a branch, it checks if
the next expanded token is a proper tests and if so, it deals with that test,
otherwise it fails. The main condition here is that the \type {\MyTest} macro
expands to a proper true or false test, so, a definition like:

\starttyping
\def\MyTest{\ifnum\scratchcounter<10 }
\stoptyping

is also okay. Now, is that neat or not?

\stopnewprimitive

\startnewprimitive[title={\prm {iffrozen}}]

This conditional checks if a control sequence is frozen:

\starttyping
is \iffrozen\MyMacro \else not \fi frozen
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {ifprotected}}]

This conditional checks if a control sequence is protected:

\starttyping
is \ifprotected\MyMacro \else not \fi protected
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {ifusercmd}}]

This conditional checks if a control sequence is not one of the primitives:

\starttyping
is \ifusercmd\MyMacro \else not \fi a primitive
\stoptyping

It is not always possible to determine this but it should work okay for regular
macros, register allocations and character definitions.

\stopnewprimitive

\startnewprimitive[title={\prm {ifrelax}}]

This is a convenient shortcut for \typ {\ifx\relax} and the motivation for adding
this one is (as with some others) to get less tracing.

\stopnewprimitive

\startnewprimitive[title={\prm {ifempty}}]

This conditional checks if a control sequence is empty:

\starttyping
is \ifempty\MyMacro \else not \fi empty
\stoptyping

It is basically a shortcut of:

\starttyping
is \ifx\MyMacro\empty \else not \fi empty
\stoptyping

with:

\starttyping
\def\empty{}
\stoptyping

Of course this is not empty at all:

\starttyping
\def\notempty#1{}
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {ifboolean}}]

This tests a number (register or equivalent) and any nonzero value represents
\type {true}, which is nicer than using an \type {\unless \ifcase}.

\stopnewprimitive

\startnewprimitive[title={\prm {ifmathparameter}}]

This is an \prm {ifcase} where the value depends on if the given math parameter
is zero, (\type {0}), set (\type {1}), or unset (\type {2}).

\starttyping
\ifmathparameter\Umathpunctclosespacing\displaystyle
    zero    \or
    nonzero \or
    unset   \fi
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {ifmathstyle}}]

This is a variant of \prm {ifcase} were the number is one of the seven possible
styles: display, text, cramped text, script, cramped script, script script,
cramped script script.

\starttyping
\ifmathstyle
  display
\or
  text
\or
  cramped text
\else
  normally smaller than text
\fi
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {ifarguments}}]

This is a variant of \prm {ifcase} were the selector is the number of arguments
picked up. For example:

\startbuffer
\def\MyMacro#1#2#3{\ifarguments\0\or1\or2\or3\else ?\fi} \MyMacro{A}{B}{C}
\def\MyMacro#1#0#3{\ifarguments\0\or1\or2\or3\else ?\fi} \MyMacro{A}{B}{C}
\def\MyMacro#1#-#2{\ifarguments\0\or1\or2\or3\else ?\fi} \MyMacro{A}{B}{C}\par
\stopbuffer

\typebuffer

Watch the non counted, ignored, argument in the last case. Normally this test will
be used in combination with \prm {ignorearguments}.

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifhastok}}]

This conditional looks for occurrences in token lists where each argument has to
be a proper list.

\startbuffer
\def\scratchtoks{x}

\ifhastoks{yz}         {xyz} Y\else N\fi\quad
\ifhastoks\scratchtoks {xyz} Y\else N\fi
\stopbuffer

\typebuffer

We get:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifhastoks}}]

This test compares two token lists. When a macro is passed it's meaning
gets used.

\startbuffer
\def\x  {x}
\def\xyz{xyz}

(\ifhastoks  {x}  {xyz}Y\else N\fi)\quad
(\ifhastoks {\x}  {xyz}Y\else N\fi)\quad
(\ifhastoks  \x   {xyz}Y\else N\fi)\quad
(\ifhastoks  {y}  {xyz}Y\else N\fi)\quad
(\ifhastoks {yz}  {xyz}Y\else N\fi)\quad
(\ifhastoks {yz} {\xyz}Y\else N\fi)
\stopbuffer

\typebuffer {\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifhasxtoks}}]

This primitive is like the one in the previous section but this time the
given lists are expanded.

\startbuffer
\def\x  {x}
\def\xyz{\x yz}

(\ifhasxtoks  {x}  {xyz}Y\else N\fi)\quad
(\ifhasxtoks {\x}  {xyz}Y\else N\fi)\quad
(\ifhastoks   \x   {xyz}Y\else N\fi)\quad
(\ifhasxtoks  {y}  {xyz}Y\else N\fi)\quad
(\ifhasxtoks {yz}  {xyz}Y\else N\fi)\quad
(\ifhasxtoks {yz} {\xyz}Y\else N\fi)
\stopbuffer

\typebuffer {\getbuffer}

This primitive has some special properties.

\startbuffer
\edef\+{\expandtoken 9 `+}

\ifhasxtoks {xy}   {xyz}Y\else N\fi\quad
\ifhasxtoks {x\+y} {xyz}Y\else N\fi
\stopbuffer

\typebuffer

Here the first argument has a token that has category code \quote {ignore} which
means that such a character will be skipped when seen. So the result is:

{\getbuffer}

This permits checks like these:

\startbuffer
\edef\,{\expandtoken 9 `,}

\ifhasxtoks{\,x\,} {,x,y,z,}Y\else N\fi\quad
\ifhasxtoks{\,y\,} {,x,y,z,}Y\else N\fi\quad
\ifhasxtoks{\,z\,} {,x,y,z,}Y\else N\fi\quad
\ifhasxtoks{\,x\,}  {,xy,z,}Y\else N\fi
\stopbuffer

\typebuffer

I admit that it needs a bit of a twisted mind to come up with this, but it works
ok:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {ifhaschar}}]

This one is a simplified variant of the above:

\startbuffer
\ifhaschar !{this ! works} yes \else no \fi
\stopbuffer

\typebuffer

and indeed we get: \inlinebuffer ! Of course the spaces in this this example
code are normally not present in such a test.

\stopnewprimitive

\startnewprimitive[title={\prm {ifnumexpression}}]

Here is an example of a conditional using expressions:

\startbuffer
\ifnumexpression (\scratchcounterone > 5) and (\scratchcountertwo > 5) \relax
    do-something
\fi
\stopbuffer

This matches when the result is non zero, and you can mix calculations and tests
as with normal expressions.

\stopnewprimitive

\startnewprimitive[title={\prm {ifdimexpression}}]

The companion of the previous primitive is:

\startbuffer
\ifdimexpression 10pt > 10bp \relax
    do-something
\fi
\stopbuffer

This matches when the result is non zero, and you can mix calculations and tests
as with normal expressions. Contrary to the number variant units can be used and
precision kicks in.

\stopnewprimitive

\startoldprimitive[title={\prm {else}}]

This traditional primitive is part of the condition testing mechanism. When a
condition matches, \TEX\ will continue till it sees an \prm {else} or \prm
{or} or \prm {orelse} (to be discussed later). It will then do a fast skipping
pass till it sees an \prm {fi}.

\stopoldprimitive

\startoldprimitive[title={\prm {or}}]

This traditional primitive is part of the condition testing mechanism and relates
to an \prm {ifcase} test (or a similar test to be introduced in later
sections). Depending on the value, \TEX\ will do a fast scanning till the right
\prm {or} is seen, then it will continue expanding till it sees a \prm {or}
or \prm {else} or \prm {orelse} (to be discussed later). It will then do a
fast skipping pass till it sees an \prm {fi}.

\stopoldprimitive

\startoldprimitive[title={\prm {fi}}]

This traditional primitive is part of the condition testing mechanism and ends a
test. So, we have:

\starttyping
\ifsomething ... \else ... \fi
\ifsomething ... \or ... \or ... \else ... \fi
\ifsomething ... \orelse \ifsometing  ... \else ... \fi
\ifsomething ... \or ... \orelse \ifsometing  ... \else ... \fi
\stoptyping

The \prm {orelse} is new in \LUAMETATEX\ and a continuation like we find in
other programming languages (see later section).

\stopoldprimitive

\startoldprimitive[title={\prm {unless}}]

This \ETEX\ prefix will negate the test (when applicable).

\starttyping
       \ifx\one\two YES\else NO\fi
\unless\ifx\one\two NO\else YES\fi
\stoptyping

This primitive is hardly used in \CONTEXT\ and we probably could get rid of these
few cases.

\stopoldprimitive

\startnewprimitive[title={\prm {orelse}}]

This primitive provides a convenient way to flatten your conditional tests. So
instead of

\starttyping
\ifnum\scratchcounter<-10
    too small
\else\ifnum\scratchcounter>10
    too large
\else
    just right
\fi\fi
\stoptyping

You can say this:

\starttyping
\ifnum\scratchcounter<-10
    too small
\orelse\ifnum\scratchcounter>10
    too large
\else
    just right
\fi
\stoptyping

You can mix tests and even the case variants will work in most cases \footnote {I
just play safe because there are corner cases that might not work yet.}

\starttyping
\ifcase\scratchcounter          zero
\or                             one
\or                             two
\orelse\ifnum\scratchcounter<10 less than ten
\else                           ten or more
\fi
\stoptyping

Performance wise there are no real benefits although in principle there is a bit
less housekeeping involved than with nested checks. However you might like this:

\starttyping
\ifnum\scratchcounter<-10
    \expandafter\toosmall
\orelse\ifnum\scratchcounter>10
    \expandafter\toolarge
\else
    \expandafter\justright
\fi
\stoptyping

over:

\starttyping
\ifnum\scratchcounter<-10
    \expandafter\toosmall
\else\ifnum\scratchcounter>10
    \expandafter\expandafter\expandafter\toolarge
\else
    \expandafter\expandafter\expandafter\justright
\fi\fi
\stoptyping

or the more \CONTEXT\ specific:

\starttyping
\ifnum\scratchcounter<-10
    \expandafter\toosmall
\else\ifnum\scratchcounter>10
    \doubleexpandafter\toolarge
\else
    \doubleexpandafter\justright
\fi\fi
\stoptyping

But then, some \TEX ies like complex and obscure code and throwing away working
old code that took ages to perfect and get working and also showed that one
masters \TEX\ might hurt.

\stopnewprimitive

\startnewprimitive[title={\prm {orunless}}]

This is the negated variant of \prm {orelse} (prefixing that one with \tex
{unless} doesn't work well.

\stopnewprimitive

\startoldprimitive[title={\prm {futurelet}}]

The original \TEX\ primitive \prm {futurelet} can be used to create an alias to a next token,
push it back into the input and then expand a given token.

\startbuffer
\let\MySpecialToken[
\def\DoWhatever{\ifx\NextToken\MySpecialToken YES\else NOP\fi : }
\futurelet\NextToken\DoWhatever [A]\crlf
\futurelet\NextToken\DoWhatever (A)\par
\stopbuffer

\typebuffer

This is typically the kind of primitive that most users will never use because it
expects a sane follow up handler (here \type {\DoWhatever}) and therefore is
related to user interfacing.

{\getbuffer}

\stopoldprimitive

\startnewprimitive[title={\prm {futuredef}}]

We elaborate on the example of using \prm {futurelet} in the previous section.
Compare that one with the next:

\startbuffer
\def\MySpecialToken{[}
\def\DoWhatever{\ifx\NextToken\MySpecialToken YES\else NOP\fi : }
\futurelet\NextToken\DoWhatever [A]\crlf
\futurelet\NextToken\DoWhatever (A)\par
\stopbuffer

\typebuffer

This time we get:

{\getbuffer}

It is for that reason that we now also have \prm {futuredef}:

\startbuffer
\def\MySpecialToken{[}
\def\DoWhatever{\ifx\NextToken\MySpecialToken YES\else NOP\fi : }
\futuredef\NextToken\DoWhatever [A]\crlf
\futuredef\NextToken\DoWhatever (A)\par
\stopbuffer

\typebuffer

So we're back to what we want:

{\getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {futurecsname}}]

In order to make the repertoire of \type {def}, \type {let} and \type {futurelet}
primitives complete we also have:

\starttyping
\futurecsname MyMacro:1\endcsname\MyAction
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {letcharcode}}]

Assigning a meaning to an active character can sometimes be a bit cumbersome;
think of using some documented uppercase magic that one tends to forget as it's
used only a few times and then never looked at again. So we have this:

\startbuffer
{\letcharcode 65 1 \catcode 65 13 A : \meaning A}\crlf
{\letcharcode 65 2 \catcode 65 13 A : \meaning A}\par
\stopbuffer

\typebuffer

here we define \type {A} as an active charcter with meaning \type {1} in the
first line and \type {2} in the second.

{\tttf \getbuffer}

Normally one will assign a control sequence:

\startbuffer
{\letcharcode 66 \bf \catcode 66 13 {B   bold}: \meaning B}\crlf
{\letcharcode 73 \it \catcode 73 13 {I italic}: \meaning I}\par
\stopbuffer

\typebuffer

Of course \type {\bf} and \type {\it} are \CONTEXT\ specific commands:

{\tttf \getbuffer}

\stopnewprimitive

\startoldprimitive[title={\prm {global}}]

This is one of the original prefixes that can be used when we define a macro of
change some register.

\starttyping
\bgroup
       \def\MyMacroA{a}
\global\def\MyMacroB{a}
      \gdef\MyMacroC{a}
\egroup
\stoptyping

The macro defined in the first line is forgotten when the groups is left. The
second and third definition are both global and these definitions are retained.

\stopoldprimitive

\startoldprimitive[title={\prm {long}}]

This original prefix gave the macro being defined the property that it could not
have \prm {par} (or the often equivalent empty lines) in its arguments. It was
mostly a protection against a forgotten right curly brace, resulting in a so called
run|-|away argument. That mattered on a paper terminal or slow system where such a
situation should be catched early. In \LUATEX\ it was already optional, and in
\LUAMETATEX\ we dropped this feature completely (so that we could introduce others).

\stopoldprimitive

\startoldprimitive[title={\prm {outer}}]

An outer macro is one that can only be used at the outer level. This property is
no longer supported. Like \prm {long}, the \prm {outer} prefix is now an
no|-|op (and we don't expect this to have unfortunate side effects).

\stopoldprimitive

\startoldprimitive[title={\prm {protected}}]

A protected macro is one that doesn't get expanded unless it is time to do so.
For instance, inside an \prm {edef} it just stays what it is. It often makes
sense to pass macros as|-|is to (multi|-|pass) file (for tables of contents).

In \CONTEXT\ we use either \prm {protected} or \prm {unexpanded} because the
later was the command we used to achieve the same results before \ETEX\
introduced this protection primitive. Originally the \prm {protected} macro was
also defined but it has been dropped.

\stopoldprimitive

\startnewprimitive[title={\prm {expand}}]

Beware, this is not a prefix but a directive to ignore the protected characters of
the following macro.

\startbuffer
\protected \def \testa{\the\scratchcounter}
           \edef\testb{\testa}
           \edef\testc{\expand\testa}
\stopbuffer

\typebuffer

The meaning of the three macros is:

\startlines \getbuffer \tttf
\meaningfull\testa
\meaningfull\testb
\meaningfull\testc
\stoplines

\stopnewprimitive

\startnewprimitive[title={\prm {untraced}}]

Related to the meaning providers is the \prm {untraced} prefix. It marks a macro
as to be reported by name only. It makes the macro look like a primitive.

\starttyping
         \def\foo{}
\untraced\def\oof{}

\scratchtoks{\foo\foo\oof\oof}

\tracingall \the\scratchtoks \tracingnone
\stoptyping

This will show up in the log as follows:

\starttyping
1:4: {\the}
1:5: \foo ->
1:5: \foo ->
1:5: \oof
1:5: \oof
\stoptyping

This is again a trick to avoid too much clutter in a log. Often it doesn't matter
to users what the meaning of a macro is (if they trace at all). \footnote {An
earlier variant could also hide the expansion completely but that was just
confusing.}

\startoldprimitive[title={\prm {immediate}}]

This one has no effect unless you intercept it at the \LUA\ end and act upon it.
In original \TEX\ immediate is used in combination with read from and write to
file operations. So, this is an old primitive with a new meaning.

\stopoldprimitive

\startnewprimitive[title={\prm {frozen}}]

You can define a macro as being frozen:

\starttyping
\frozen\def\MyMacro{...}
\stoptyping

When you redefine this macro you get an error:

\starttyping
! You can't redefine a frozen macro.
\stoptyping

This is a prefix like \prm {global} and it can be combined with other prefixes.
\footnote {The \prm {outer} and \prm {long} prefixes are no|-|ops in
\LUAMETATEX\ and \LUATEX\ can be configured to ignore them.}

\stopnewprimitive

\startnewprimitive[title={\prm {letfrozen}}]

You can explicitly freeze an unfrozen macro:

\starttyping
\def\MyMacro{...}
\letfrozen\MyMacro
\stoptyping

A redefinition will now give:

\starttyping
! You can't redefine a frozen macro.
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {unletfrozen}}]

A frozen macro cannot be redefined: you get an error. But as nothing in \TEX\ is set
in stone, you can do this:

\starttyping
\frozen\def\MyMacro{...}
\unletfrozen\MyMacro
\stoptyping

and \type {\MyMacro} is no longer protected from overloading. It is still
undecided to what extend \CONTEXT\ will use this feature.

\stopnewprimitive

\startnewprimitive[title={\prm {letprotected}}]

Say that you have these definitions:

\startbuffer
             \def  \MyMacroA{alpha}
\protected   \def  \MyMacroB{beta}
             \edef \MyMacroC{\MyMacroA\MyMacroB}
\letprotected      \MyMacroA
             \edef \MyMacroD{\MyMacroA\MyMacroB}
\meaning           \MyMacroC\crlf
\meaning           \MyMacroD\par
\stopbuffer

\typebuffer

The typeset meaning in this example is:

{\tttf \getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {unletprotected}}]

The complementary operation of \prm {letprotected} can be used to unprotect
a macro, so that it gets expandable.

\startbuffer
               \def  \MyMacroA{alpha}
\protected     \def  \MyMacroB{beta}
               \edef \MyMacroC{\MyMacroA\MyMacroB}
\unletprotected      \MyMacroB
               \edef \MyMacroD{\MyMacroA\MyMacroB}
\meaning             \MyMacroC\crlf
\meaning             \MyMacroD\par
\stopbuffer

\typebuffer

Compare this with the example in the previous section:

{\tttf \getbuffer}

\stopnewprimitive

% \startnewprimitive[title={\prm {letdatacode}}]
%   {\em Todo.}
% \stopnewprimitive

\startnewprimitive[title={\prm {beginlocalcontrol}}]

Once \TEX\ is initialized it will enter the main loop. In there certain commands
trigger a function that itself can trigger further scanning and functions. In
\LUAMETATEX\ we can have local main loops and we can either enter it from the
\LUA\ end (which we don't discuss here) or at the \TEX\ end using this primitive.

\startbuffer
\scratchcounter100

\edef\whatever{
    a
    \beginlocalcontrol
        \advance\scratchcounter 10
        b
    \endlocalcontrol
    \beginlocalcontrol
        c
    \endlocalcontrol
    d
    \advance\scratchcounter 10
}

\the\scratchcounter
\whatever
\the\scratchcounter
\stopbuffer

\typebuffer

A bit of close reading probably gives an impression of what happens here:

{\getbuffer}

The local loop can actually result in material being injected in the current node
list. However, where normally assignments are not taking place in an \prm
{edef}, here they are applied just fine. Basically we have a local \TEX\ job, be
it that it shares all variables with the parent loop.

\stopnewprimitive

\startnewprimitive[title={\prm {endlocalcontrol}}]

     See previous section.

\stopnewprimitive

\startnewprimitive[title={\prm {localcontrolled}}]

The previously described local control feature comes with two extra helpers. The
\prm {localcontrolled} primitive takes a token list and wraps this into a local
control sidetrack. For example:

\startbuffer
\edef\testa{\scratchcounter123 \the\scratchcounter}
\edef\testb{\localcontrolled{\scratchcounter123}\the\scratchcounter}
\stopbuffer

\typebuffer

The two meanings are:

\start \getbuffer
\starttabulate[|T|T|]
\NC \string\testa \NC \meaningfull\testa \NC \NR
\NC \string\testb \NC \meaningfull\testb \NC \NR
\stoptabulate
\stop

The assignment is applied immediately in the expanded definition.

\stopnewprimitive

\startnewprimitive[title={\prm {localcontrol}}]

This primitive takes a single token:

\startbuffer
\edef\testa{\scratchcounter123 \the\scratchcounter}
\edef\testc{\testa \the\scratchcounter}
\edef\testd{\localcontrol\testa \the\scratchcounter}
\stopbuffer

\typebuffer

The three meanings are:

\start \getbuffer
\starttabulate[|T|T|]
\NC \string\testa \NC \meaning\testa \NC \NR
\NC \string\testc \NC \meaning\testc \NC \NR
\NC \string\testd \NC \meaning\testd \NC \NR
\stoptabulate
\stop

The \prm {localcontrol} makes that the following token gets expanded so we don't
see the yet to be expanded assignment show up in the macro body.

\stopnewprimitive

\startnewprimitive[title={\prm {alignmark}}]

When you have the \type {#} not set up as macro parameter character cq.\ align
mark, you can use this primitive instead. The same rules apply with respect to
multiple such tokens in (nested) macros and alignments.

\stopnewprimitive

\startnewprimitive[title={\prm {aligntab}}]

When you have the \type {&} not set up as align tab, you can use this primitive
instead. The same rules apply with respect to multiple such tokens in (nested)
macros and alignments.

\stopnewprimitive

\startnewprimitive[title={\prm {defcsname}}]

We now get a series of log clutter avoidance primitives. It's fine if you argue
that they are not really needed, just don't use them.

\starttyping
\expandafter\def\csname MyMacro:1\endcsname{...}
             \defcsname MyMacro:1\endcsname{...}
\stoptyping

The fact that \TEX\ has three (expanded and global) companions can be seen as a
signal that less verbosity makes sense. It's just that macro packages use plenty
of \prm {csname}'s.

\stopnewprimitive

\startnewprimitive[title={\prm {edefcsname}}]

This is the companion of \prm {edef}:

\starttyping
\expandafter\edef\csname MyMacro:1\endcsname{...}
             \edefcsname MyMacro:1\endcsname{...}
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {gdefcsname}}]

As with standard \TEX\ we also define global ones:

\starttyping
\expandafter\gdef\csname MyMacro:1\endcsname{...}
             \gdefcsname MyMacro:1\endcsname{...}
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {xdefcsname}}]

This is the companion of \prm {xdef}:

\starttyping
\expandafter\xdef\csname MyMacro:1\endcsname{...}
             \xdefcsname MyMacro:1\endcsname{...}
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {glet}}]

This is the global companion of \prm {let}. The fact that it is not an original
primitive is probably due to the expectation for it not it not being used (as)
often (as in \CONTEXT).

\stopnewprimitive

\startnewprimitive[title={\prm {letcsname}}]

It is easy to see that we save two tokens when we use this primitive. As with the
\type {..defcs..} variants it also saves a push back of the composed macro name.

\starttyping
\expandafter\let\csname MyMacro:1\endcsname\relax
             \letcsname MyMacro:1\endcsname\relax
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {gletcsname}}]

Naturally \LUAMETATEX\ also provides a global variant:

\starttyping
\expandafter\global\expandafter\let\csname MyMacro:1\endcsname\relax
\expandafter                  \glet\csname MyMacro:1\endcsname\relax
                               \gletcsname MyMacro:1\endcsname\relax
\stoptyping

So, here we save even more.

\stopnewprimitive

\startnewprimitive[title={\prm {cdef}}]

This primitive is like \prm {edef} but in some usage scenarios is slightly
more efficient because (delayed) expansion is ignored which in turn saves
building a temporary token list.

\startbuffer
\edef\FooA{this is foo} \meaningfull\FooA\crlf
\cdef\FooB{this is foo} \meaningfull\FooB\par
\stopbuffer

\typebuffer {\tttf \getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {cdefcsname}}]

This primitive is like \prm {edefcsame} but in some usage scenarios is slightly
more efficient because (delayed) expansion is ignored which in turn saves
building a temporary token list.

\startbuffer
\edefcsname FooA\endcsname{this is foo} \meaningasis\FooA\crlf
\cdefcsname FooB\endcsname{this is foo} \meaningasis\FooB\par
\stopbuffer

\typebuffer {\tttf \getbuffer}

\stopnewprimitive

\startnewprimitive[title={\prm {lettonothing}}]

This one let's a control sequence to nothing. Assuming that \tex {empty}
is indeed empty, these two lines are equivalent.

\starttyping
\let         \foo\empty
\lettonothing\oof
\stoptyping

\stopnewprimitive

\startnewprimitive[title={\prm {glettonothing}}]

This is the global companion of \prm {lettonothing}.

\stopnewprimitive

\startnewprimitive[title={\prm {norelax}}]

The rationale for this command can be shown by a few examples:

\startbuffer
\dimen0 1pt \dimen2 1pt \dimen4 2pt
\edef\testa{\ifdim\dimen0=\dimen2\norelax N\else Y\fi}
\edef\testb{\ifdim\dimen0=\dimen2\relax   N\else Y\fi}
\edef\testc{\ifdim\dimen0=\dimen4\norelax N\else Y\fi}
\edef\testd{\ifdim\dimen0=\dimen4\relax   N\else Y\fi}
\edef\teste{\norelax}
\stopbuffer

\typebuffer

The five meanings are:

\start \getbuffer \starttabulate[|T|T|]
\NC \string\testa \NC \meaning\testa \NC \NR
\NC \string\testb \NC \meaning\testb \NC \NR
\NC \string\testc \NC \meaning\testc \NC \NR
\NC \string\testd \NC \meaning\testd \NC \NR
\NC \string\teste \NC \meaning\teste \NC \NR
\stoptabulate \stop

So, the \prm {norelax} acts like \prm {relax} but is not pushed back as
usual (in some cases).

\stopnewprimitive

\startnewprimitive[title={\prm {swapcsvalues}}]

Because we mention some \type {def} and \type {let} primitives here, it makes
sense to also mention a primitive that will swap two values (meanings). This one
has to be used with care. Of course that what gets swapped has to be of the same
type (or at least similar enough not to cause issues). Registers for instance
store their values in the token, but as soon as we are dealing with token lists
we also need to keep an eye on reference counting. So, to some extend this is
an experimental feature.

\stopnewprimitive

\startnewprimitive[title={\prm {integerdef}}]

You can alias to a count (integer) register with \prm {countdef}:

\starttyping
\countdef\MyCount134
\stoptyping

Afterwards the next two are equivalent:

\starttyping
\MyCount   = 99
\count1234 = 99
\stoptyping

where \type {\MyCount} can be a bit more efficient because no index needs to be
scanned. However, in terms of storage the value (here 99) is always in the register
so \type {\MyCount} has to get there. This indirectness has the benefit that directly
setting the value is reflected in the indirect accessor.

\starttyping
\integerdef\MyCount = 99
\stoptyping

This primitive also defines a numeric equivalent but this time the number is stored
with the equivalent. This means that:

\starttyping
\let\MyCopyOfCount = \MyCount
\stoptyping

will store the {\em current} value of \type {\MyCount} in \type {\MyCopyOfCount} and
changing either of them is not reflected in the other.

The usual \prm {advance}, \prm {multiply} and \prm {divide} can be used with these
integers and they behave like any number. But compared to registers they are actually
more a constant.

\stopnewprimitive

\startnewprimitive[title={\prm {dimensiondef}}]

A variant of \prm {integerdef} is:

\starttyping
\dimensiondef\MyDimen = 1234pt
\stoptyping

The properties are comparable to the ones described in the section \prm
{integerdef}.

\stopnewprimitive

\startnewprimitive[title={\prm {gluespecdef}}]

A variant of \prm {integerdef} and \prm {dimensiondef} is:

\starttyping
\gluespecdef\MyGlue = 3pt plus 2pt minus 1pt
\stoptyping

The properties are comparable to the ones described in the previous sections.

\stopnewprimitive

\startnewprimitive[title={\prm {mugluespecdef}}]

A variant of \prm {gluespecdef} that expects \type {mu} units is:

\starttyping
\mugluespecdef\MyGlue = 3mu plus 2mu minus 1mu
\stoptyping

The properties are comparable to the ones described in the previous sections.

\stopnewprimitive

\startnewprimitive[title={\prm {advanceby}}]

This is slightly more efficient variant of \prm {advance} that doesn't look for
\type {by} and therefore, if one is missing, doesn't need to push back the last
seen token. Using \prm {advance} with \type {by} is nearly as efficient but takes
more tokens.

\stopnewprimitive

\startnewprimitive[title={\prm {multiplyby}}]

This is slightly more efficient variant of \prm {multiply} that doesn't look for
\type {by}. See previous section.

\stopnewprimitive

\startnewprimitive[title={\prm {divideby}}]

This is slightly more efficient variant of \prm {divide} that doesn't look for
\type {by}. See previous section.

\stopnewprimitive

\startnewprimitive[title={\prm {localcontrolledloop}}]

As with more of the primitives discussed here, there is a manual in the \quote
{lowlevel} subset that goes into more detail. So, here a simple example has to
do:

\startbuffer
\localcontrolledloop 1 100 1 {%
    \ifnum\currentloopiterator>6\relax
        \quitloop
    \else
        [\number\currentloopnesting:\number\currentloopiterator]
        \localcontrolledloop 1 8 1 {%
            (\number\currentloopnesting:\number\currentloopiterator)
        }\par
    \fi
}
\stopbuffer

\typebuffer

Here we see the main loop primitive being used nested. The code shows how we can
\prm {quitloop} and have access to the \prm {currentloopiterator} as well as the
nesting depth \prm {currentloopnesting}.

\startpacked \getbuffer \stoppacked

Be aware of the fact that \prm {quitloop} will end the loop at the {\em next}
iteration so any content after it will show up. Normally this one will be issued
in a condition and we want to end that properly. Also keep in mind that because
we use local control (a nested \TEX\ expansion loop) anything you feed back can
be injected out of order.

The three numbers can be separated by an equal sign which is a trick to avoid
look ahead issues that can result from multiple serialized numbers without spaces
that indicate the end of sequence of digits.

\stopnewprimitive

\startnewprimitive[title={\prm {expandedloop}}]

This variant of the previously introduced \prm {localcontrolledloop} doesn't
enter a local branch but immediately does its work. This means that it can be
used inside an expansion context like \prm {edef}.

\startbuffer
\edef\whatever
  {\expandedloop 1 10 1
     {\scratchcounter=\the\currentloopiterator\relax}}

\meaningasis\whatever
\stopbuffer

\typebuffer

\start \veryraggedright \tt\tfx \getbuffer \stop \blank

The next section shows a companion primitive.

\stopnewprimitive

\startnewprimitive[title={\prm {unexpandedloop}}]

As follow up on \prm {expandedloop} we now show its counterpart:

\startbuffer
\edef\whatever
  {\unexpandedloop 1 10 1
     {\scratchcounter=\the\currentloopiterator\relax}}

\meaningasis\whatever
\stopbuffer

\typebuffer

\start \veryraggedright \tt\tfx \getbuffer \stop \blank

The difference between the (un)expanded loops and a local controlled
one is shown here. Watch the out of order injection of \type {A}'s.

\startbuffer
\edef\TestA{\localcontrolledloop 1 5 1 {A}} % out of order
\edef\TestB{\expandedloop        1 5 1 {B}}
\edef\TestC{\unexpandedloop      1 5 1 {C\relax}}
\stopbuffer

\typebuffer \getbuffer

We show the effective definition as well as the outcome of using them

\startbuffer
\meaningasis\TestA
\meaningasis\TestB
\meaningasis\TestC

A: \TestA
B: \TestB
C: \TestC
\stopbuffer

\typebuffer \startlines \tttf \getbuffer \stoplines

Watch how because it is empty \type {\TestA} has become a constant macro because
that's what deep down empty boils down to.

\stopnewprimitive

\startsubject[title=Obsolete]

The \LUAMETATEX\ engine has more than its \LUATEX\ ancestor but it also has less.
Because in the end the local control mechanism performed quite okay I decided to
drop the \prm {immediateassignment} and \prm {immediateassigned} variants. They
sort of used the same trick so there isn't much to gain and it was less generic
(read: error prone).

% \startnewprimitive[title={\prm {immediateassignment}}]
%
% Assignments are not expandable which means that you cannot define fully
% expandable macros that have assignments. But, there is a way out of this:
%
% \startbuffer
% \scratchcounter = 10
% \edef\whatever{%
%     (\the\scratchcounter)
%     \immediateassignment\scratchcounter\numexpr\scratchcounter+10\relax
%     \immediateassignment\advance\scratchcounter -5
%     (\the\scratchcounter)
% }
% \meaning\whatever
% \stopbuffer
%
% \typebuffer
%
% Don't expect miracles: you can't mix|-|in content or unexpandable tokens as they
% will either show up or quit the scanning.
%
% {\getbuffer}
%
% \stopnewprimitive
%
% \startnewprimitive[title={\prm {immediateassigned}}]
%
% This is the multi|-|token variant of the primitive mentioned in the previous
% section.
%
% \startbuffer
% \scratchcounter = 10
% \edef\whatever{%
%     (\the\scratchcounter)
%     \immediateassigned{
%         \scratchcounter\numexpr\scratchcounter+10\relax
%         \advance\scratchcounter -5
%     }%
%     (\the\scratchcounter)
% }
% \meaning\whatever
% \stopbuffer
%
% \typebuffer
%
% The results are the same as in the previous section:
%
% {\getbuffer}
%
% \stopnewprimitive

\stopsubject

\page

% % It doesn't make sense to typeset this, also because it makes me feel old.
%
% \startsubject[title=A few notes on extensions] % ,placeholder=todo]
%
% This is a companion to the regular \LUAMETATEX\ reference manual, which is mostly
% a concise summary of the program and its features. They don't replace each other,
% and none of them claims completeness. There might be more manuals that discuss
% specific kind of extensions in the future. First some comments on extensions.
%
% The starting point of all \TEX\ engines is \TEX. The first follow up was \TEX\
% with support for 8 bit and languages. After that it took some time, but then two
% projects started that extended \TEX: \ETEX\ and \OMEGA. In the end the first
% brought some extensions to the macro machinery, more registers, a simple right to
% left typesetting feature, some more tracing, etc. The second was more ambitious
% and has input translation mechanisms, larger fonts, and multi directional
% typesetting. By the time \ETEX\ became stable, the \PDFTEX\ engine had showed up
% and at some point it integrated \ETEX. But \PDFTEX\ itself also extended the
% several components that make up \TEX. The \NTS\ project was started as follow up
% on \ETEX\ but although an engine written in \JAVA\ was the result it never was
% used for extensions; this project was fully funded by the german language user
% group.
%
% The \CONTEXT\ macro package was an early adopter of the \ETEX\ and \PDFTEX\
% extensions. Probably the most significant effect was that we got more registers
% (some other features were already kind of present in macro form). In between
% these engines we played with \type {eetex} (extended \ETEX) because we had some
% wishes of our own. We also explored extensions to the \DVI\ format but in the end
% \PDF\ won that race. An example of a new mechanism that we introduced in
% \CONTEXT\ was position tracking: marking positions that can be saved when the
% output is created and used in a second run. This started as a \DVI\ postprocessor
% in \PERL\ written by me, later turned into a \CLANGUAGE\ program by Taco, and
% eventually integrated in \PDFTEX\ by Thanh (\PDFTEX\ was a phd project). At some
% point \XETEX\ was developed, funded and driven by an organization that did high
% end multi lingual typesetting; it was based on \ETEX\ and uses a \DVI\ to \PDF\
% backend processor. Both \PDFTEX\ and \XETEX\ are supported by \CONTEXT\ \MKII,
% and both engines are basically stable and frozen.
%
% At some point Hartmut and I started playing with \LUA\ in \PDFTEX\ but soon Taco,
% Hartmut and I decided to start a follow up project. All the work on \LUATEX\ (and
% later \LUAMETATEX) is done whenever there is time and without financial
% compensation, so we have a slow but steady development track. Early in the
% \LUATEX\ development there has been some funding for the initial transition (by
% Taco) from \PDFTEX\ to what became the early versions of \LUATEX . You can read
% more about the oriental \TEX\ project in other documents and articles in user
% group journals. There has been some funded development of a library subsystem
% (which for some reason never took off) as well as \LUAJIT\ integration (by
% Luigi). The initial \METAPOST\ library (also done by Taco) was funded by a couple
% of user groups. Then there are the (ongoing) font projects by GUST that got
% funded by user groups as these were much needed for the \UNICODE\ engines.
%
% After the jump start, most work was and is still done in the usual \TEX\ spirit,
% on a voluntary basis, by folks from the \CONTEXT\ community, and after a decades
% we reached the stable version 1.00. It's one of the engines in \TEXLIVE\ and
% Luigi makes sure it integrates well in there. We did continue and around 1.10 the
% more of less final version was reached and \LUAMETATEX\ took off. In \LUATEX\
% only bugs get fixed, occasionally some helpers can get added, and we might port
% some of \LUAMETATEX\ back to its parent, when it doesn't harm compatibility.
%
% Already early in development some primitives were added that enhance the macro
% language. More were added later. It's these extensions that are discussed in this
% document. There are several documents in the \CONTEXT\ distribution that discuss
% the (ongoing) development, right from the start, and these often contain
% examples. For instance some of the new primitives have been introduced there,
% complete with a rationale and examples of usage.
%
% Just for the record: the \CONTEXT\ group runs the build farm that is used to
% generate binaries for all sorts of platforms. We make sure that there are always
% versions that can be used for real production jobs. You can expect regular
% updates as long as there are developments (of course, eventually we're done).
%
% \stopsubject

\startsubject[title=Rationale] % ,placeholder=todo]

Some words about the why and how it came. One of the early adopters of \CONTEXT\
was Taco Hoekwater and we spent numerous trips to \TEX\ meetings all over the
globe. He was also the only one I knew who had read the \TEX\ sources. Because
\CONTEXT\ has always been on the edge of what is possible and at that time we
both used it for rather advanced rendering, we also ran into the limitations. I'm
not talking of \TEX\ features here. Naturally old school \TEX\ is not really
geared for dealing with images of all kind, colors in all kind of color spaces,
highly interactive documents, input methods like \XML, etc. The nice thing is
that it offers some escapes, like specials and writes and later execution of
programs that opened up lots of possibilities, so in practice there were no real
limitations to what one could do. But coming up with a consistent and extensible
(multi lingual) user interface was non trivial, because it had an impact in
memory usage and performance. A lot could be done given some programming, as
\CONTEXT\ \MKII\ proves, but it was not always pretty under the hood. The move to
\LUATEX\ and \MKIV\ transferred some action to \LUA, and because \LUATEX\
effectively was a \CONTEXT\ related project, we could easily keep them in sync.

Our traveling together, meeting several times per year, and eventually email and
intense \LUATEX\ developments (lots of Skype sessions) for a couple of years,
gave us enough opportunity to discuss all kind of nice features not present in
the engine. The previous century we discussed lots of them, rejected some, stayed
with others, and I admit that forgot about most of the arguments already. Some
that we did was already explored in \type {eetex}, some of those ended up in
\LUATEX, and eventually what we have in \LUAMETATEX\ can been seen as the result
of years of programming in \TEX, improving macros, getting more performance and
efficiency out of existing \CONTEXT\ code and inspiration that we got out of the
\CONTEXT\ community, a demanding lot, always willing to experiment with us.

Once I decided to work on \LUAMETATEX\ and bind its source to the \CONTEXT\
distribution so that we can be sure that it won't get messed up and might
interfere with the \CONTEXT\ expectations, some more primitives saw their way
into it. It is very easy to come up with all kind of bells and whistles but it is
equally easy to hurt performance of an engine and what might go unnoticed in
simple tests can really affect a macro package that depends on stability. So, what
I did was mostly looking at the \CONTEXT\ code and wondering how to make some of
the low level macros look more natural, also because I know that there are users
who look into these sources. We spend a lot of time making them look consistent
and nice and the nicer the better. Getting a better performance was seldom an
argument because much is already as fast as can be so there is not that much to
gain, but less clutter in tracing was an argument for some new primitives. Also,
the fact that we soon might need to fall back on our phones to use \TEX\ a
smaller memory footprint and less byte shuffling also was a consideration. The
\LUAMETATEX\ memory footprint is somewhat smaller than the \LUATEX\ footprint.
By binding \LUAMETATEX\ to \CONTEXT\ we can also guarantee that the combinations
works as expected.

I'm aware of the fact that \CONTEXT\ is in a somewhat unique position. First of
all it has always been kind of cutting edge so its users are willing to
experiment. There are users who immediately update and run tests, so bugs can and
will be fixed fast. Already for a long time the community has an convenient
infrastructure for updating and the build farm for generating binaries (also for
other engines) is running smoothly.

Then there is the \CONTEXT\ user interface that is quite consistent and permits
extensions with staying backward compatible. Sometimes users run into old manuals
or examples and then complain that \CONTEXT\ is not compatible but that then
involves obsolete technology: we no longer need font and input encodings and font
definitions are different for \OPENTYPE\ fonts. We always had an abstract backend
model, but nowadays \PDF\ is kind of dominant and drives a lot of expectations.
So, some of the \MKII\ commands are gone and \MKIV\ has some more. Also, as
\METAPOST\ evolved that department in \CONTEXT\ also evolved. Think of it like
cars: soon all are electric so one cannot expect a hole to poor in some fluid but
gets a (often incompatible) plug instead. And buttons became touch panels. There
is no need to use much force to steer or brake. Navigation is different, as are
many controls. And do we need to steer ourselves a decade from now?

So, just look at \TEX\ and \CONTEXT\ in the same way. A system from the nineties
in the previous century differs from one three decades later. Demands differ,
input differs, resources change, editing and processing moves on, and so on.
Manuals, although still being written are seldom read from cover to cover because
online searching replaced them. And who buys books about programming? So
\LUAMETATEX, while still being \TEX\ also moves on, as do the way we do our low
level coding. This makes sense because the original \TEX\ ecosystem was not made
with a huge and complex macro package in mind, that just happened. An author was
supposed to make a style for each document. An often used argument for using
another macro package over \CONTEXT\ was that the later evolved and other macro
packages would work the same forever and not change from the perspective of the
user. In retrospect those arguments were somewhat strange because the world,
computers, users etc.\ do change. Standards come and go, as do software politics
and preferences. In many aspects the \TEX\ community is not different from other
large software projects, operating system wars, library devotees, programming
language addicts, paradigm shifts. But, don't worry, if you don't like
\LUAMETATEX\ and its new primitives, just forget about them. The other engines
will be there forever and are a safe bet, although \LUATEX\ already stirred up
the pot I guess. But keep in mind that new features in the latest greatest
\CONTEXT\ version will more and more rely on \LUAMETATEX\ being used; after all
that is where it's made for. And this manual might help understand its users why,
where and how the low level code differs between \MKII, \MKIV\ and \LMTX.

Can we expect more new primitives than the ones introduced here? Given the amount
of time I spend on experimenting and considering what made sense and what not,
the answer probably is \quotation {no}, or at least \quotation {not that much}.
As in the past no user ever requested the kind of primitives that were added, I
don't expect users to come up with requests in the future either. Of course,
those more closely related to \CONTEXT\ development look at it from the other
end. Because it's there where the low level action really is, demands might still
evolve.

Hans Hagen \crlf Hasselt NL

\stopsubject

\popoverloadmode

\startluacode
    local match = string.match
    local find  = string.match

    function document.CheckCompleteness()
        local primitives = token.getprimitives()
        local luametatex = { }
        local indexed    = { }

        for i=1,#primitives do
            local p = primitives[i]
            if p[4] == 4 then
                local name = p[3]
                if find(name,"U") or find (name,"math") then
                    -- ignore
                    luametatex[name] = nil
                else
                    luametatex[name] = false
                end
            end
        end

        local function collect(index)
            if index then
                local data = index.entries
                for i=1,#data do
                    local name = match(data[i].list[1][1],"\\tex%s*{(.-)}") or ""
                    if luametatex[name] == false then
                        luametatex[name] = true
                    end
                    indexed[name] = true
                end
            end
        end

        collect(structures.registers.collected and structures.registers.collected.index)

        context("To be checked primitives:")

        context.blank()
        context.startcolumns { n = 2 }
        for k, v in table.sortedhash(luametatex) do
            if not v then
                context.dontleavehmode()
                context.type(k)
                context.crlf()
            end
        end
        context.stopcolumns()
        context.blank()

        context("Indexed primitives:")

        context.blank()
        context.startcolumns { n = 2 }
        for k, v in table.sortedhash(indexed) do
            context.dontleavehmode()
            if luametatex[k] == true then
                context("\\color[darkgreen]{\\tttf %s}",k)
            elseif luametatex[k] == false then
                context("\\color[darkred]{\\tttf %s}",k)
            else
                context("{\\tttf %s}",k)
            end
            context.crlf()
        end
        context.stopcolumns()
        context.blank()

    end
\stopluacode

% \startmode[atpragma]
%     \startluacode
%         context.page()
%         document.CheckCompleteness()
%     \stopluacode
% \stopmode

\stopbodymatter

\stoptext