path: root/doc/context/sources/general/manuals/musings/musings-unicode.tex

% language=us runpath=texruns:manuals/math

\def\unichar#1#2{#1 (U+#2: \char"#2)}

\def\APL{\ss apl}

% \useMPlibrary[dum]

\startcomponent musings-unicode

\environment musings-style

% \usemodule[mathfun]

\startchapter[title=Unicode]

\startsection[title=Introduction]

When working on a \TEX\ macro package for decades one can hardly avoid dealing
with math; after all \TEX\ is pretty much about math. When this wonderful
typesetting infrastructure was written it was all about quality and how to make
your documents look nice. And for sure, Don Knuths documents looks nice, also
because he pays a lot of attention to the \quotation {fine points of math
typesetting}.

The constraints of those time (like hardware, compilers, fonts, and for sure also
time) made \TEX\ into what it is: eight bit character sets, eight bit fonts,
eight bit hyphenation patterns, efficient memory usage and therefore carrying
around as little as possible. It all makes sense. But one needs to pay attention.
\footnote {And that is what Mikael Sundqvist and I have been doing a lot since we
started upgrading math in \CONTEXT\ in combination with enhancing the math engine
in \LUAMETATEX. The story here is a byproduct of our explorations and very much a
combined effort.}

Math typesetting is actually a sort of separated process in the engine:
unprocessed lists go in and after some juggling a list of assembled boxes,
glyphs, glues and penalties come out. I will not go into detail about that and
only mention that in \LUAMETATEX\ we extended all this to be a bit more flexible
and controllable, something that has been driven by the fact that we need to
support \UNICODE\ fonts. This is all part of a related effort to move from eight
bit \quote {everything} to \UNICODE\ \quote {everywhere}.

Now, one can say a lot about \UNICODE\ but the main advantage is that it tries to
cover \quote {all} characters ever encountered, including scripts (used in
languages) that are long gone, as well as these little pictures that people like
to see on the web: emojis. One can safely say that \UNICODE\ simplifies mixing
languages and scripts, and thereby makes \TEX\ macro packages less complex. On
the other hand, \UNICODE\ (or more precisely, related wide) fonts makes all kind
of features possible and thereby add a complication.

So, how about math? When Don Knuth gave us \TEX\ he also gave us fonts and there
are plenty symbols in these fonts. But, as mathematicians seem to love variations
on symbols soon more fonts arrived, most noticeably those from the \AMS\ that
also added some more alphabets: mathematicians also love to render the shapes of
letters differently. In order to access these glyphs names were invented that
also sometimes suggested that there was some order in the matter. And, for some
reason these names got aliases and soon we had a huge list of often obscure and
inconsistent macro names. It didn't take long for a little mess and confusion to
creep in.

It has been said that the verbose \TEX\ math \ASCII\ input format is also a way
for mathematicians to communicate, just because many use the same tool to render
the formulas. Of course that gets obscured when one starts to add additional
macros. It gets even more tricky once we start talking \quote {standard} as in
\quotation {\LATEX\ is the standard}. That has for instance resulted in browsers
interpreting \TEX\ like input without using \TEX\ (so how about expansion?). It
has also sort of put \TEX\ into the range of possible word processing systems,
which in turn leads to these \MSWORD\ versus Google docs versus \LATEX\ debates
that can get rather nasty and unrealistic when it comes to discussing usage and
quality. Interestingly, \MSWORD\ now has reasonable math, to some extent
modelled after \TEX. It has some verbose \TEX\ like (but constrained) input and
would do well for probably mostly people who occasionally have to inject some
math. There were also attempts by the people at \MICROSOFT\ to normalize the
input but we leave that aside now.

However, because we now do have all these symbols and because source code editors
make them accessible and show them there is a good chance that users will inject
them, if only by cut and paste, so we do have to deal with that. This
automatically puts us in the position that we need to deal with different
meanings for the same symbol, which in turn might demand different spacing,
penalties and such. In the end it is users that drive all this, not publishers;
they don't really care and out|-|source typesetting anyway. We're not aware of
any research and development being done and I suppose we would have noticed
because after all we're involved in developing \LUATEX. It is one of the engines
that does \OPENTYPE\ and \UNICODE\ math and no publisher or supplier ever took
serious interest in it. From our perspective what users do is visible, everything
else is hidden behind corporate curtains. And this is why nowadays we only need
to care about users (mainly authors).

Back to typesetting. For a long time all went well: one could typeset documents
that looked good. Okay, not all looked good because not everyone paid attention to
details, and the more the web evolved the more patching cut'n'paste of bad
examples made its way into documents, but let's not start talking quality here.
But then came \UNICODE\ and a while later people started talking about
accessibility, cutting and pasting and more. In the meantime there had been
developments like \MATHML\ and \OPENMATH\ that tried to structure and organize
formulas in a more symbolic way. \footnote {It probably went unnoticed that
\CONTEXT\ always supported rendering \MATHML, and as such had to deal with all
the weird aspects (read: way it was used). Although one is not supposed to
directly edit \MATHML\ we work with authors who are quite happy to do that simply
because they code the documents in \XML\ because there is a need for high quality
\PDF\ as well as \HTML\ output and a \CONTEXT\ based workflow can handle the
\XML\ well. We're talking of large volumes here (mostly for basically free
school math).}

In the meantime the \TEX\ community had lost the edge on fonts, and \OPENTYPE\
math was invented by \MICROSOFT\ and implemented in \MSWORD\ before a substantial
number of \TEX\ users understood what was happening. They had it coming. To a
large extend one can say the same about math in \UNICODE. Where a Greek capital
\quote {A} is seen as different from a Latin capital \quote {A}, even when they
often have the same shape, a math italic variable \quote {h} was made synonym to
\quote {Planck constant}, as if the letters used in math had no meaning at all.
We'll see that a wide hat is an extensible character of zero width combining hat
accent, which makes for curious handling of the initial character. There is more
granularity in some symbols, especially popular symbols like slashes and bars,
than in letters. It is as if the math community didn't care much about how the
letters (variables) were communicated and perceived but were picky about the
slope of slashes. It seems more of a visual world, which might actually be the
reason structured input never really took of. Maybe \TEX ies just love the mix of
characters, commands, spacing directives. Maybe they just love to reposition and
space these glyphs to suit all kind of curious non|-|standard math rendering.

All this makes it pretty hard to communicate meaning, and it is just one of the
examples where the \TEX\ community, for as far involved, failed to make a strong
case. Our personal opinion is that no one really cared because in the \TEX\
community it is all about rendering. The fact that we use math to communicate
only gained attention when accessibility became hot and by then it was too late.
Efforts like \OPENMATH\ started ambitious and in the end basically failed. Coding
in \XML\ using \MATHML\ isn't much better and one always had to adapt to the
latest fashion. Also, once plenty code shows up bugs become features. Browser
support came and went and came back. Simplified input using for instance
\ASCIIMATH\ started indeed simple but quickly became a (somewhat inconsistent)
mess. What we see here is the same as everything web (and computer languages): we
can do better, we start some project, then move on, and we end up with half|-|way
abandoned results. The development cycles are short, results have to be achieved
fast, there is no time (or interest) for iterating and refactoring. The word
\quote {standard} and mantra \quote {everyone should use this} are quite popular.

So where does that leave us with \TEX ? Well, with a mess. Decades of various
efforts have not brought us a coherent system of organizing symbols and
properties, made us end up with inconsistencies, made users revert to hacks,
didn't make math easily transferable and complicates rendering. Personally we
find it sort of strange that we spend time on for instance tagging and
accessibility before we get these math alphabets and shared math specific symbols
sorted out. If we cannot make good arguments for that (math being a script on its
own with semantics and such) we waste energy and are pulling a dead horse. What
puzzles us most is that one would expect mathematicians to be able to come up
with strong arguments for a structured approach. But maybe it was simply the fact
that \TEX\ math typesetting was pretty much driven by large commercial publishers
and those providing services for them: the first category doesn't invest in these
matters and even less today, and the second category makes money from sorting out
the mess, so why get rid of it. Who knows. For us, it means that any complain
about these matters deserves the same answer: the \TEX\ community created this
mess, so it has to live with it. And the bad thing is: bugs and work|-|arounds
eventually become features and then one is supposed to conform, even if deep down
one knows better. It doesn't help that the community is proud of what it can
render and has built itself a reputation that all is good.

So why this criticism? Why not just abandon \TEX ? The answer is simple: \TEX\ is
quite okay and cannot be blamed for where we are now. We need to think of
solutions and in that respect the \CONTEXT\ users are lucky! They have always
been told not to use this macro package for math because there are other
standards and because publishers want \LATEX\ (even if they just let the
manuscripts be recoded). That means that we don't really need to care much about
the past. Those who use \CONTEXT\ can benefit from the compatibility we have
anyway but also move forward to more structured and consistent math. It is in
this perspective that we will discuss some more details next so that eventually
we can draw some conclusions. The end goal is to have an additional layer of
grouping math symbols that permits consistent high quality rendering in a mixed
input environment.

\stopsection

\startsection[title=Molecules]

Before we go into details about some characters, we spend some word on the
rendering. The building blocks of a formula are atoms and internally the term
nucleus is used for what we have without scripts. The simple sequence \type {1 +
x} will result in a linked list of three atoms with three nuclei. In \type {x^2}
the \type {x} is the nucleus. Atoms can have scripts: prescripts, postscripts and
a prime. The majority of \UNICODE\ math characters become such atoms (nuclei and
scripts) and they get a class property that determines their spacing, but that is
not part of the \UNICODE\ specification. From the upcoming sections it will be
clear that when we classify we don't get that much help from \MATHML\ or even the
\TEX\ community either.

In addition to these atoms the \LUAMETATEX\ engine (which builds upon \TEX) has
what we can call molecules. There are several types: fractions, accents, fences,
radicals. This distinction is to some extent present in \UNICODE: plenty of
fraction related slashes, all kind of accents, vertical delimiters that can be
made from snippets and act as fences, and a radical symbol. In \MATHML\ we see
similar constructs but there in practice quite often operators need to be
interpreted in a way that can distinguish between atoms and molecules. That is
partly a side effect of applications that generate \MATHML. And as usual with
standards pushed upon the world without years of exploration the confusion became
part of the norm and will stay.

In the \TEX\ engine over and under delimiters are implemented on top of radicals
(using the same noad, the wrapper node for yet unprocessed math) but they have
different code paths. Basically we have vertically fenced material and just like
fractions have left and right fences as part of the concept (for binominals) the
radical has a sort of left fence too. You can also wonder why we need accent
noads while we support other delimiters with radicals. This organization mostly
relates to subtypes and classes (and likely some limitations of the past) that
have related spacing properties, but we can think if a generic structure noad and
meaningful subtypes. However, that is not what we get so let's be more precise:

{\bf Fractions:} these stack two atoms (or molecules) and separate them by a
visible or phantom rule, or in \LUAMETATEX\ by a delimiter. They can have a left
and right fence which originates in them also suitable for binominals. You may
wonder why we don't use regular fences here. One reason we can think of is that
when you fence something, you have an open and close class at the edges while
with a fenced fraction the whole still is fraction. In \LUAMETATEX\ we can tweak
classes at the edges but in regular \TEX\ there are fewer classes, so there
constructs become ordinary or inner.

{\bf Accents:} these put something on top of or below an atom (or molecule) and
are driven by characters. The accent related commands take an integer
(traditional) or three integers (extended) and it is this expected input that
drives it. However, they are treated like delimiters. In traditional \TEX\ a
delimiter is defined by two characters: the direct unscaled one, and when not
found a second one drives the lookup from wider variants and eventually an
extensible character. Accents just have the second one, which probably relates to
the fact that the text ones that would be the starting point make no sense. It is
this \quote {looking} for a single code point that makes that accents are not
merged with the more general radical command space. Another reason is that
accents deal a bit different with spacing and italic correction so even if we
could merge, it would be more confusing in the end.

{\bf Fences:} these come in pairs with optional middle ones. The reason for
pairing is that they need to get the same size. That means that before we
construct them the atom or molecule that they fence has to be analyzed. It also
makes the result a construct of its own, although in \LUAMETATEX\ we can unpack
that result so that it can be broken across lines. In practice that was never an
issue because in a running text unscaled fences are used (just atoms with open
and close classes assigned) but as soon as one goes to multi|-|line displays
formulas things become more hairy. The related commands expect delimiters (the
two part character definitions) but in the meantime are also happy with a single
one because in the end \OPENTYPE\ math has all in one font.

{\bf Radicals:} originally this only concerned roots but because they are
basically wrappers we also use them for content that gets a delimiter above,
below or both. In that sense the term radical can also be interpreted as \quote
{extreme}, more than a carrot looking symbol. The related commands take one or
more delimiters (or character) because we support left as well as right
delimiters connected by a rule, so in the end radicals evolved into a construct
with delimiters of all kind. So, the unique property of radicals is that the
fences assume a cooperation between one or more glyphs and a rule. In \CONTEXT\
we support actuarian hooks as radicals that are used for annuity expressions,
otherwise the \UNICODE\ symbols is useless and the \MATHML\ construct complex.

So, where accents take numbers as delimiter specification, fences, fractions and
radicals take specific math quantities or just letters. This makes that we will
not merge these into one scanner and handler even if they all use the same
(large) noad to store and carry around their properties. Also, it has some charm
to keep the original \TEX\ distinctions. After all, it's not like \UNICODE,
\MATHML\ or \OPENTYPE\ math fonts have brought some new insights: in the end they
all draw from \TEX\ and they way it's done there.

\stopsection

\startsection[title=Symbols]

There are plenty of symbols in \UNICODE. When we try to get an idea how we ended
up with that set we're surprised that not much seems to be known about it. There
are references to \ISO\ standards, usage by specific organizations (like those
dealing with patents), there are references to lists of publishers. In personal
communications with people involved it becomes clear that the criterion that some
symbols really has to be used somewhere doesn't apply to these math symbols.
There are bizarre specimens that we cannot locate anywhere. They are often
assigned the \quote {relation} property which for \TEX\ is a safe bet because
binary and relations get similar spacing, but binary makes an exception when it
sits at the front. The fact that relation spacing is used can even obscure the
fact that some characters have zero width properties; the results just look
somewhat bad and one can always blame the font or renderer and adding some thin
spacing is accepted behavior. So one can make the argument that because \TEX\ was
the main renderer of math, a safe bet was better than a confusing and
unproven|-|by|-|usage assignment to some category.

In \TEX\ some symbols have multiple names, even when they have the same class.
This indicates the wish for meaning at one end but shape at the other, and once a
name has been assigned it sticks. It would be interesting to know how
mathematicians see formulas: if one puts \type {\bar}s around a variable does one
see \quotation {bar x bar} or \quotation {the modulus of x}, and how is translation
to audio to be performed?

One important aspect of using any symbol in \TEX, or basically any typesetting
system that deals with math, is that the spacing depends on the meaning. Now, in
the perspective of \UNICODE\ meaning is somewhat diffuse. A Latin capital \quote
{A} related to \quote {a} is not the same as a Greek capital \quote {A} that
relates to \quote {\alpha}. So, from the shape one cannot beforehand deduce what is
meant, but when copying it the \UNICODE\ will expose the meaning. This is not the
case in math: although many symbols have one meaning only, there are also plenty
that can mean different things and the (\TEX) math community has not been able to
make a strong case for providing different slots. Maybe the reason was that there
already was a tradition of using commands that then relate a shape to a class
that then results in appropriate spacing. Maybe it is also assumed that an
article or book starts by explaining what a specific symbol means in that
particular context. But that doesn't help much for copying. It also doesn't help
with direct \UNICODE\ input. The way out for this last problem is that in
\CONTEXT\ we will add additional properties to characters that then can
communicate the class and thereby control the spacing. Although we initially did
that at the \LUA\ end we now use the lightweight dictionary feature of the
engine: a property, group, slot model. The main reason is that we foresee that at
some point we might have to add property based rendering to the engine, and this
opens up that possibility. Ever since we started with \LUATEX\ and \MKIV\ we have
used the character database (in \LUA\ format) to store most properties so that we
have all in one place.

For figuring out the properties we can look at how traditionally symbols got
multiple commands associated, how \MATHML\ looks at it, what \UNICODE\ reveals and
what we find in fonts. It is a bit of jungle out there so for sure we have to
make decisions ourselves. We next turn to that exploration.

\stopsection

\startsection[title=Slashes]

The definition on the \WIKIPEDIA\ page [1] of slashes is as follows:

\startquotation
    The slash is an oblique slanting line punctuation mark /. Once used to mark
    periods and commas, the slash is now used to represent exclusive or inclusive
    or, division and fractions, and as a date separator. It is called a solidus
    in \UNICODE, is also known as an oblique stroke, and has several other
    historical or technical names including oblique and virgule.
\stopquotation

The page then has a very detailed description on how slashes are used in text,
mathematics, computing, currency, dates, numbering, linguistic transcriptions,
line breaks, abbreviations, proofreading, fiction, libraries, addresses, poetry,
music, sports, and text messages. It is a pretty good and detailed page which also
gives a nice summary of usage in math.

In mathematics, we use the slash (a forward leaning bar) for fractions, division,
and quotient of set. Examples of fractions are $\vfrac {1} {2}$ but also
$\percent$ sits in this category.

\starttabulate[|T|l|l|]
\NC U+0002F \NC \switchtobodyfont[stixtwo]$\utfchar{"0002F}$ \NC this is the official solidus    \NC \NR % /
\NC U+02044 \NC \switchtobodyfont[stixtwo]$\utfchar{"02044}$ \NC the mathematical fraction slash \NC \NR % ⁄
\NC U+02215 \NC \switchtobodyfont[stixtwo]$\utfchar{"02215}$ \NC the mathematical division slash \NC \NR % ∕
\NC U+02571 \NC \switchtobodyfont[stixtwo]$\utfchar{"02571}$ \NC a diagonal box drawing line     \NC \NR % ╱
\NC U+029F8 \NC \switchtobodyfont[stixtwo]$\utfchar{"029F8}$ \NC the mathematical big solidus    \NC \NR % ⧸
\NC U+0FF0F \NC \switchtobodyfont[stixtwo]$\utfchar{"0FF0F}$ \NC a full width solidus            \NC \NR % ／
\NC U+1F67C \NC \switchtobodyfont[stixtwo]$\utfchar{"1F67C}$ \NC the very heavy solidus          \NC \NR % 🙼
\stoptabulate

The \STIX\ fonts have the first five, the rest is not there, so we can safely
assume that they are not used in math. That brings us to the question that, say
that the other ones are used, how does the user access them? In the editor they
often look pretty much the same. For \TEX ies the answer is easy: you use a
command. But as we already mentioned, there we enter a real fuzzy area: these
commands either describe a shape or they communicate a meaning, at least, in an
ideal world. Sometimes wrapping in a macro helps, like \typ {$\vfrac {1} {2}$}.

In the document that explains \UNICODE\ math there is a section \quotation
{Fraction Slash and Other Diagonals}. Even if we limit ourselves to the forward
leaning slashes it looks like we need to include
exotic symbols, as the empty set symbol with an left arrow on top: \type
{U+29B4} a circle with left pointing arrow on top, that doesn't show up in most
math fonts but \STIX\ has it {\switchtobodyfont[stixtwo]{$⦴$}}. We quote:

\startquotation
    \type {U+2044 ⁄} \typ {FRACTION SLASH} is typically used to build up simple
    skewed fractions in running text. It applies to immediately adjacent
    sequences of decimal digits, that is, to spans of characters with the General
    Category property value \type {Nd}. For example, \type {1⁄2} should be
    displayed as \type {½}. In ordinary plain text, any character other than a
    digit delimits the numerator or denominator. So \type {5 1⁄2} should be
    displayed as \type {5½} since a space follows the \type {5}. In general
    mathematical use, a more versatile method for layout of fractions is needed
    (see, for example, Section 2.1 of [UnicodeMath]), however parsers of
    mathematical texts should be prepared to handle \typ {FRACTION SLASH} when it
    is received from other sources. \type {U+27CB}
    \typ {MATHEMATICAL RISING DIAGONAL} and \type {U+27CD}
    \typ {MATHEMATICAL FALLING DIAGONAL} are
    mathematical symbols for specific uses, to be distinguished from the more
    widely used solidi and reverse solidi operators as well as from
    nonmathematical diagonals.
\stopquotation

In \TEX\ there is no parsing going on: we just get sequences of atoms and the
inter atom spacing applies. Curly braced arguments are used to communicate units
that needs to be treated a while. As side note: where for some scripts there are
special characters that tell where something (state) starts and ends this is not
available for math, which makes it impossible to mark a sequence of characters as
being something math. The whole repertoire of pre|-|composed fractions and super-
and subscripted \UNICODE\ symbols are not to be used in math.

Most documents that somehow relate to or (partially) originate in \TEX\ can
be rather fuzzy, so we can read here:

\startquotation
    \type {U+27CB} corresponds to the \LATEX\ entity \type {\diagup} and \type
    {U+27CD} to \type {\diagdown}. Their glyphs are invariably drawn with 45° and
    135° slopes, respectively, instead of the more upright slants typical for the
    solidi operators. The diagonals are also to be distinguished from the two box
    drawing characters \type {U+2571} and \type {U+2572}. While in some fonts
    those characters may be drawn with 45° and 135° slopes, respectively, they
    are not intended to be used as mathematical symbols. One usage recorded for
    \type {U+27CB} and \type {U+27CD} is in the notation for spaces of double
    cosets.
\stopquotation

So, it is the angles that math users should translate into meaning which I guess
is natural for them. From the above we cannot deduce if we should take them into
account in a macro package.

The \MATHML\ specification [3] keeps it abstract and talks about division without
mentioning the rendering. In content \MATHML\ we have:

\starttyping
divide = element divide { CommonAtt, DefEncAtt, empty}
\stoptyping

and the suggested rendering (from an example) is a slash.

In the chapter \quotation {Characters, Entities and Fonts} there is mentioning of:

\startquotation
    There is one more case where combining characters turn up naturally in
    mathematical markup. Some relations have associated negations, such as \type
    {U+226F} [\typ {NOT GREATER-THAN}] for the negation of U+003E [\typ
    {GREATER-THAN SIGN}]. The glyph for U+226F [NOT GREATER-THAN] is usually just
    that for U+003E [\typ {GREATER-THAN SIGN}] with a slash through it. Thus it
    could also be expressed by \type {U+003E}|-|\type {U+0338} making use of the
    combining slash \type {U+0338} [COMBINING LONG SOLIDUS OVERLAY]. That is true
    of 25 other characters in common enough mathematical use to merit their own
    \UNICODE\ code points. In the other direction there are 31 character entity
    names listed in [\typ {Entities}] which are to be expressed using \type
    {U+0338} [\typ {COMBINING LONG SOLIDUS OVERLAY}].
\stopquotation

A curious note is this:

\startquotation
    For special purposes, one may need a symbol which does not have a \UNICODE\
    representation. In these cases one may use the \type {mglyph} element for
    direct access to a glyph as an image, or (in some systems) from a font that
    uses a non|-|\UNICODE\ encoding. All \MATHML\ token elements accept
    characters in their content and also accept an \type {mglyph} there. Beware,
    however, that use of \type {mglyph} to access a font is deprecated and the
    mechanism may not work in all systems. The \type {mglyph} element should
    always supply a useful alternative representation in its alt attribute.
\stopquotation

At some point we experimented with very precise positioned \HTML\ from \TEX\
(read: \CONTEXT) and that worked very well: the rendering was exactly the same as
\PDF\ but then suddenly it was no longer possible to access glyphs from fonts. The
assumption had become that one should feed text into the font rendering machinery
and use \OPENTYPE\ features to access specific shapes, which of course is a
fragile approach (the libraries and logic keep evolving, and the most robust
access is simply by index, or by glyph name if present, assuming that one uses
the font that was meant to be used). So, how the \MATHML\ glyph element is
supposed to work out well is not clear. Anyway, as we want nicely typeset math we
don't care that much if features present in \LUAMETATEX\ and \CONTEXT\ are unique
and cannot be reproduced otherwise.

In \type {mathclass.txt} [4] which is \quotation {{\em not} formally part of the
\UNICODE\ Character Database at this time} we see a classification:

\starttabulate[|T|l|]
\NC U+0002F \NC binary \NC \NR
\NC U+02044 \NC binary \NC \NR
\NC U+02215 \NC binary \NC \NR
\NC U+02571 \NC not mentioned \NC \NR
\NC U+029F8 \NC n-ary or large operator, often takes limits \NC \NR
\NC U+0FF0F \NC not mentioned \NC \NR
\NC U+1F67C \NC not mentioned \NC \NR
\stoptabulate

So, in the end we can focus on the four that are mentioned, and we will do that
with the above in mind as well as what is common in the \TEX\ world. We will look
at usage, classification (groups) and classes.

% modern   % ok, both the same
% cambria  % different, no extensible /
% bonum    % ok, both the same
% pagella  % ok, both the same
% stixtwo  % only / extensible, 2044 useless
% lucida   % both extensible, 2044 looks bad and more slope

Unfortunately this sort of mess also results in a mess in fonts. For instance
when we checked out the difference between \type {U+002F} and \type {U+2044} we
found that in the fonts produced by the \TEX Gyre project both have proper
dimensions (and look the same), so they can be used stand alone, but also as
delimiters. In Cambria the dimensions are okay but only \type {U+2044} has
extensible characters. In \CONTEXT\ we have defined \type {\slash} to use that slot but
when you test Lucida and \STIX2 the results are disappointing: In Lucida the
width of \type {U+2044} makes it unusable (it looks bad anyway), and in \STIX2 it
is a bit wider so in the end it even becomes fuzzy what to recommend as fix:
quarter width, half width or full width. Defining \type {\slash} as any of them
gives at some point an issue so in the end we just patch the font in the goodie
file: we make them the same and make sure they have extensible characters. After all,
chances are slim that this will ever be fixed. In that respect a newer engine
doesn't change the problem: we need to handle it in the macro package, but at
least that can be done a bit more natural. \footnote {In principle, we can support
the goodies in the generic font handler, but we think it makes no sense because it
also relates to the way math is handled in general and supporting a wide range of
different applications can only cripple the code, let along that agreeing on
matters can be hard.}

% \ctxlua{table.tocontext(characters.data[0x002F],"[0x002F]")}
% \ctxlua{table.tocontext(characters.data[0x2044],"[0x2044]")}
% \ctxlua{table.tocontext(characters.data[0x2215],"[0x2215]")}
% \ctxlua{table.tocontext(characters.data[0x2571],"[0x2571]")}
% \ctxlua{table.tocontext(characters.data[0x29F8],"[0x29F8]")}

\stopsection

\startsection[title=Bars]

Again we start with the \WIKIPEDIA\ page, this time the one dedicated to bars
[5]. The page starts with mathematics so that suggests that the (initial) author
is familiar with usage in that field: if we cut and paste the itemized list we
even get \TEX\ math (sort of). Examples of usage are: absolute value,
cardinality, conditional probability, determinant, distance, divisibility,
function evaluation, length, norm, order, restriction, set|-|builder notation,
the Sheffer stroke in logic, subtraction, but also \quotation {A vertical bar can
be used to separate variables from fixed parameters in a function, or in the
notation for elliptic integrals}.

Among the objectives of our exploration are grouping symbols in sets that
represent related meanings and usage. Within these groups we can fine tune with
classes but that is more geared at rendering. Although currently users enter
specific usage of symbols with the same shape (or even \UNICODE) with commands we
can imagine them entering the \quote {real} characters and in that case we need
some automatic class assignment based on a group (or set of groups). The
\WIKIPEDIA\ page mentions that in physics \quotation {The vertical bar is used in
bra|–|ket notation in quantum physics}. It then goes on about usage in computing,
phonetics and literature. This ordering is different from the slashes, but okay.

The page then makes a distinction between solid and broken bars and there is some
interesting history behind that, which relates to typewriters, terminals and
printers in the perspective of distinction and indeed we noticed that on our
keyboard the broken bar is still used, even if the rendering is solid. The
page ends with the \UNICODE\ bars and entities. We mention most:

\starttabulate[|T|l|l|]
\NC U+007C \NC \switchtobodyfont[stixtwo]$\utfchar{"007C}$ \NC a single vertical line         \NC \NR % |
\NC U+00A6 \NC \switchtobodyfont[stixtwo]$\utfchar{"00A6}$ \NC a single broken line          \NC \NR % ¦
\NC U+2016 \NC \switchtobodyfont[stixtwo]$\utfchar{"2016}$ \NC a double vertical line (norms) \NC \NR % ‖
\NC U+2223 \NC \switchtobodyfont[stixtwo]$\utfchar{"2223}$ \NC divides                        \NC \NR % ∣
\NC U+2225 \NC \switchtobodyfont[stixtwo]$\utfchar{"2225}$ \NC parallel lines                 \NC \NR % ∥
\NC U+2502 \NC \switchtobodyfont[stixtwo]$\utfchar{"2502}$ \NC a vertical box drawing line    \NC \NR % │
\NC U+FF5C \NC \switchtobodyfont[stixtwo]$\utfchar{"FF5C}$ \NC a fullwidth vertical line      \NC \NR % ｜
\stoptabulate

Given the mentioned wide range of usage it will be clear bars that can be confusing
and are pretty overloaded. We're not aware of broken bars being used in math, so
we ignore these.

The \UNICODE\ math draft talks of \quote {vertical lines} and distinguishes two
series, delimiters:

\starttabulate[|T|l|l|]
\NC U+007C \NC \switchtobodyfont[stixtwo]$\utfchar{"007C}$ \NC single vertical lines \NC \NR
\NC U+2016 \NC \switchtobodyfont[stixtwo]$\utfchar{"2016}$ \NC double vertical lines \NC \NR
\NC U+2980 \NC \switchtobodyfont[stixtwo]$\utfchar{"2980}$ \NC triple vertical lines \NC \NR
\stoptabulate

and operators:

\starttabulate[|T|l|l|]
\NC U+2223 \NC \switchtobodyfont[stixtwo]$\utfchar{"2223}$ \NC divides (single line)           \NC \NR
\NC U+2225 \NC \switchtobodyfont[stixtwo]$\utfchar{"2225}$ \NC parallel (double lines)         \NC \NR
\NC U+2AF4 \NC \switchtobodyfont[stixtwo]$\utfchar{"2AF4}$ \NC binary relation (tripple lines) \NC \NR
\NC U+2AFC \NC \switchtobodyfont[stixtwo]$\utfchar{"2AFC}$ \NC s large triplle operator        \NC \NR
\stoptabulate

Watch the triples: these are not (yet) in the \WIKIPEDIA\ summary. Rightfully
there is a remark that the official \UNICODE\ descriptions use \typ {BAR} and
\typ {LINE} but \TEX ies can't complain about that, can they? After all, they
also use these terms mixed.

The delimiters sit at the edges but sometimes also in the middle. The operators
are between other elements and the document states that they also should grow.
And is it mentioned that spacing depends on usage. The large triple is an n-ary
operator but as usual with math symbols the user (reader) has to guess what that
actually means.

It is actually unfortunate that the fences have no left, middle and right
variant. Even if these render the same it would make life easier and consistency
with other fences is also worth something. One wonders how it would have looked
if accessibility demands had kicked in earlier. The \UNICODE\ \type
{mathclass.txt} [4] provides:

\starttabulate[|T|l|]
\NC U+007C \NC fence (unpaired delimiter) \NC \NR
\NC U+2016 \NC fence (unpaired delimiter) \NC \NR
\NC U+2980 \NC fence (unpaired delimiter) \NC \NR
\stoptabulate

We assume that the unpaired qualification is actually an indication that usage as
what in \TEX\ is called \quote {middle} is okay. The operators are classified as:

\starttabulate[|T|l|]
\NC U+2223 \NC relation    \NC \NR
\NC U+2225 \NC relation    \NC \NR
\NC U+2AF4 \NC binary      \NC \NR
\NC U+2AFC \NC large n-ary \NC \NR
\stoptabulate

% \ctxlua{table.tocontext(characters.data[0x007C],"[0x007C]")}
% \ctxlua{table.tocontext(characters.data[0x00A6],"[0x00A6]")}
% \ctxlua{table.tocontext(characters.data[0x2016],"[0x2016]")}
% \ctxlua{table.tocontext(characters.data[0x2980],"[0x2980]")}
% \ctxlua{table.tocontext(characters.data[0x2223],"[0x2223]")}
% \ctxlua{table.tocontext(characters.data[0x2225],"[0x2225]")}
% \ctxlua{table.tocontext(characters.data[0x2AF4],"[0x2AF4]")}
% \ctxlua{table.tocontext(characters.data[0x2AFC],"[0x2AFC]")}

The main problem with bars in \TEX\ is that there is no distinction between a
left and right bar which makes it impossible to use them directly as fences. On
can consider this to be an omission to \UNICODE\ math because shape rules over
meaning. So anyway, this is something that a macro package has to deal with. If
needed these can get a class on their own in which case we can define atom
spacing rules that deal with them ending up left or right. In \UNICODE\ there are
signals that deal with bidirectional text, so we see no reason why there shouldn't
be similar provisions for math.

\stopsection

\startsection[title=Hyphens and Dashes]

This section applies to text and math as both are riddled with horizontal lines:
easy to scratch in wood, chisel in stone or draw on paper symbols. We limit
ourselves to the straight ones, but similar observations can be made for curved
ones.

\WIKIPEDIA\ distinguishes hyphens, minus, and dashes so there are multiple pages
dedicated to this. The page about minus mentions that there are three usages
(somewhat rephrased):

\startitemize[packed]
    \startitem
        It is used as subtraction operator and therefore a binary operator
        that indicates the operation of subtraction.
    \stopitem
    \startitem
        It can be function whose value for any real or complex argument is the
        additive inverse of that argument.
    \stopitem
    \startitem
        It can serve as a prefix of a numeric constant. When it is placed
        immediately before an unsigned numeral, the combination names a negative
        number, the additive inverse of the positive number that the numeral
        would otherwise name.
    \stopitem
\stopitemize

The functional variant is how content \MATHML\ sees it: you apply a minus
operator to something, singular of multiple. We were surprised to see that there
is a distinctive rendering suggested, something we have argued for at several
occasions (mostly \TEX\ meetings):

\startquotation
    In many contexts, it does not matter whether the second or the third of these
    usages is intended: \type {−5} is the same number. When it is important to
    distinguish them, a raised minus sign \type {¯} is sometimes used for negative
    constants, as in elementary education, the programming language \APL, and some
    early graphing calculators.
\stopquotation

Unfortunately that distinction was not recognized by the \TEX\ community at large
which (we guess) is why we don't see it in \UNICODE, which on the other hand has
plenty dashes as we will see soon.

The page mentions usage in indicating blood types and music, which is a nice
detail. It also mentions usage in computing, including regular expressions and in
physics and chemistry indicating charge. It lists these codes for minus symbols:

\starttabulate[|Tl|l|]
\NC U+002D \NC hyphen minus            \NC \NR
\NC U+2212 \NC minus                   \NC \NR
\NC U+FE63 \NC small hyphen minus      \NC \NR
\NC U+FF0D \NC full width hyphen minus \NC \NR
\stoptabulate

The page also mentions the commercial minus \type {⁒} (see also [7]) and division
sign \type {÷} (see also [8]) and we think these should be supported in math mode
simply because they can be part of (even simple text style) formulas.

The fact that we use the hyphen as minus and expect it to render as a wider dash
like shape is something that related to math mode in \TEX\ speak. In text mode we
expect it to be seen as hyphenation related indicator. We won't go into details
about automated hyphenation and explicit hyphens in text mode but here are the
hyphens as mentioned on the hyphen specific \WIKIPEDIA\ page:

\starttabulate[|Tl|l|]
\NC U+002D \NC hyphen minus \NC \NR
\NC U+00AD \NC soft hyphen \NC \NR
\NC U+2010 \NC hyphen \NC \NR
\NC U+2011 \NC non breaking hyphen \NC \NR
\stoptabulate

You might wonder why we mention text variants here and one reason is that we
actually might need to provide a catch for the last two: maybe when a user copies
these from a document (when rendered at all) we need to treat them as the simple
hyphen minus and just remap them to the math minus when in math mode. Below, we
will discuss dashes, and although these are also meant for text, a reason for
exploring these can be found in the fact that \TEX\ users like to decorate the
content in unexpected ways and lines (or rules) fit into that. The \WIKIPEDIA\
pages go into some details about the hyphens being used in compounds and there
can be some confusion about whether to use endashes or hyphens for that. We're
pretty sure that typesetting wars have been fought over that. Usage as pre- and
suffixes definitely is worth noting (and we use them as such in this sentence).

We leave out all the other usages and see what there is to tell about related
symbols. The \WIKIPEDIA\ page about dashes is an extensive one. It starts out with
the distinction between \unichar {figure dash} {2012}, \unichar {endash} {2013},
\unichar {emdash} {2014} and \unichar {horizontal bar} {2015}. Of these a \TEX ie
will for sure recognize the endash and emdash. The hyphen is not a dash but if
you look at \TEX\ input that double or triple hyphens get ligatured into en- and
emdashes! The only certainty one has is that the endash is often half the width
of an emdash. Also, the width of the emdash is often the same as the font size.

One reason why a language subsystem of a \TEX\ macro package is complex is that
it has to deal with cultural aspects and the usage as well as spacing around all
these dashes can differ. When trying to support that a macro writer soon finds
out that one user of language~X can tell you the rules are done this way, and a
while later you get a mail from another user who claims that in language~X the
rules are done that way. Word processing and dominance of English probably adds
to the confusion. The same is true for quotes, but math doesn't need these, so we
skip them. Now wait, you will say: does math use these dashes? Users probably
will mix them in but more important is that the width of these dashes also has
associated skips: \type {\enspace} and \type {\emspace} or \type {\quad} and
these one definitely see users mix into math.

The figure dash has the same width as digits which makes them useful in tables. In
the fonts that come with \TEX\ it is the reverse: the digits have the same width
and that width matches the endash. There is no habit of using the figuredash, but
we might need to change that. After all, we now have the fonts! We do need to
deal with the figure dash because users might mix math and text in tables, and
although you can find plenty of badly typeset by \TEX\ tables, this is no excuse
for using a mix of minus and figure dash in inconsistent ways.

The \WIKIPEDIA\ page mentions the usage of the endash: as connector, as compound
hyphen, and as sentence interrupter. Now the one that needs some attention is the
second one. In Dutch, we can combine words in many ways and for educational
purposes adding a compound dash makes sense. However, because the weight of the
hyphen and endash in \TEX\ fonts is rather incompatible, in \CONTEXT\ we use(d)
fakes: two overlapping hyphens. Another complication is that one has to wrap that
in a discretionary node in order to make the hyphenator happy, but that is now
delegated to the engine that can be configured to see certain characters as valid
hyphenation points. Although we support discretionaries in math this doesn't
relate to dashes but to pluses and minuses and such. The engine supports explicit
discretionaries but can also automatically repeat symbols that are set up as
repeatable across lines. We're not sure if users actually use en- and emdashes in
math mode, but one can occasionally run into examples (on the web) where special
effects are achieved in curious ways. \footnote {The math stream doesn't go
through the font handler although embedded \type {\hbox}es get that treatment.
This means that two hyphens in a row are just two atoms and not get collapsed to
an endash.}

It is worth pointing out that \WIKIPEDIA\ discusses \quotation {Ranges of values}
and this is something we need to investigate in the perspective of math! Strictly
spoken that is a text thing, but \unknown\  Among the many observed and suggested
patterns we note that among \TEX ies using the endash as itemize symbols is
also popular.

Usage of the emdash is related to the use of parenthesis or colons, so it is more
a kind of punctuation. It can also be used as an interrupt and again it is a
candidate for an itemize symbol. There is of course a \TEX\ thing there: lack of
text symbols made for a rather mixed usage of math and text symbols in
itemizations. For instance a dotted one uses the well visible math dot instead of
the often hardly visible text dot that simply was not present in \TEX\ fonts, so
our eyes got accustomed to the bolder ones. It is one of the reasons why a \TEX\
macro package load a math font even when no math is used. Over the years in \TEX\
math and text symbols have been mixed in various ways, also a side effect if the
limited amount of characters in text fonts and the abundance of them in math
mode, even if most are only accessible by name. We need to deal with that
historic mix.

The page rightfully mentions that \TEX\ has no horizontal bar, also known as
\quote {quotation dash}, used for dialogues in some languages. We should make a
note then that it might be good to see if we have to reconfigure the
sub|-|sentence presets to match that expectation. The proposed hack {\red MPS:
where?} for a missing symbol is somewhat curious:

\starttyping
x \hbox{---}\kern-.5em--- x
\stoptyping

\startbuffer[dash-example]
\uleaders \hbox to 1.5em {---\hskip 0pt minus .5em---} \hskip.125em minus .125em \relax
\stopbuffer

Why not \type {\hbox {---\kern-.5em---}} or just \type {---\kern-.5em---} to get
the same effect? This also assumes that the font collapses these three hyphens
into a dash, then it backtracks the symbol width and does a second one.
\footnote {Here is some food for thought: for this kind of usage one can argue
that such a dash should have some stretch. In \LUAMETATEX\ and therefore
\CONTEXT\ we can do this: \typeinlinebuffer [dash-example] and get: \dorecurse
{30} {x \getbuffer [dash-example] x}. Boxed material can be stretched and be
taken into account when creating paragraphs. It is no big deal to wrap that in a
macro, say \type {\figuredashed}.} Anyway, where figure dashes are related to
minuses we can probably ignore this super minus resembling horizontal bar.
\footnote {We can actually issue a warning when it is used in math mode.}

The \WIKIPEDIA\ page ends with a summary of all kind of dashes, including
underscores, script specific symbols, accents (like macron), modifiers and curly
ones. Here we only mention the ones that can end up in some source when one cuts
and pastes. Doing that can result in missing characters (because not all fonts
provides them) or a change in meaning (for as far as the symbols relates to an
intention). We show some that fit into this discussion and also mention the
\UNICODE\ description:

\starttabulate[|T|lb{\ttx}|p|]
\NC U+002D \NC HYPHEN-MINUS                  \NC the usual hyphen but also used as minus \NC \NR
\NC U+005F \NC LOW LINE                      \NC aka underscore \NC \NR
\NC U+00AD \NC SOFT HYPHEN                   \NC valid hyphenation point (invisible) \NC \NR
\NC U+2010 \NC HYPHEN                        \NC the real hyphen but more work on a keyboard \NC \NR
\NC U+2011 \NC NON-BREAKING HYPHEN           \NC a hard hyphen, disables following hyphenation \NC \NR
\NC U+2012 \NC FIGURE DASH                   \NC see discussion above \NC \NR
\NC U+2013 \NC EN DASH                       \NC see discussion above \NC \NR
\NC U+2014 \NC EM DASH                       \NC see discussion above \NC \NR
\NC U+2015 \NC HORIZONTAL BAR                \NC see discussion above \NC \NR
\NC U+2043 \NC HYPHEN BULLET                 \NC used in itemized lists \NC \NR
\NC U+207B \NC SUPERSCRIPT MINUS             \NC combined with pre-superscripted characters \NC \NR
\NC U+208B \NC SUBSCRIPT MINUS               \NC combined with pre-subscripted characters \NC \NR
\NC U+2212 \NC MINUS SIGN                    \NC the math minus (rendering of hyphen) \NC \NR
\NC U+23AF \NC HORIZONTAL LINE EXTENSION     \NC build long connected horizontal lines \NC \NR
\NC U+23E4 \NC STRAIGHTNESS                  \NC represents line straightness in technical context \NC \NR
\NC U+2500 \NC BOX DRAWINGS LIGHT HORIZONTAL \NC part of the box-drawing repertoire \NC \NR
\NC U+2796 \NC HEAVY MINUS SIGN              \NC a visual variant with no meaning \NC \NR
\NC U+2E3A \NC TWO-EM DASH                   \NC a visual variant with no meaning \NC \NR
\NC U+2E3B \NC THREE-EM DASH                 \NC a visual variant with no meaning \NC \NR
\NC U+FE58 \NC SMALL EM DASH                 \NC a visual variant with no meaning \NC \NR
\NC U+FE63 \NC SMALL HYPHEN-MINUS            \NC a visual variant with no meaning \NC \NR
\NC U+FF0D \NC FULLWIDTH HYPHEN-MINUS        \NC a visual variant with no meaning \NC \NR
\stoptabulate

The \UNICODE\ math draft only mentions the hyphen: \footnote {When I copy this
snippet into the document source there are \typ {START OF TEXT} symbols at the
places where a hyphenation occurs, which is probably a side effect of a bad \type
{TOUNICODE} entry in the \PDF\ file, but it is kind of interesting in this
perspective as definitely a hyphen is rendered.}

\startquotation
    Minus sign. \type {U+2212} [or] \type{−} [known as] \typ {MINUS SIGN} is the
    preferred representation of the unary and binary minus sign rather than the
    \ASCII|-|derived \type {U+002D} [or] \type {-} [known as] \typ
    {HYPHEN-MINUS}, because minus sign is unambiguous and because it is rendered
    with a more desirable length, usually longer than a hyphen.
\stopquotation

and elsewhere we can read:

\startquotation
    The \ASCII\ hyphen minus \type {U+002D} [or] \type {-} is a weakly
    mathematical character that may be used for the subtraction operator, but
    \type {U+2212} [or] \type {−} [known as] \typ {MINUS SIGN} is preferred for
    this purpose and looks better.
\stopquotation

We are not aware of the concept of weak mathematical characters, so we will not
take that property too serious when we try to improve the rendering.

This is basically it. There is no mentioning of classes (after all, traditional
\TEX\ has no unary class) so it is assumed that the renderer does the right
thing: interpreting the sequence of characters and apply spacing accordingly.
There are users who like to see a unary minus being rendered differently, just as
the minus that a student is supposed to key in a calculator and while the
\WIKIPEDIA\ page mentions this explicitly, it is ignored here. Yes, having two
distinctive slots for this would have been great. Maybe it is not seen as
relevant enough by the community that would benefit most, but who knows what had
happened it the \WIKIPEDIA\ page had been there before!

The minus is mentioned in the somewhat curious section about how shapes should be
positioned relative to the baseline, where the position of the minus relates to
what in \TEX\ speak is the math axis. There is also some mentioning of non-mathematical use, like:

\startquotation
    The concept of mathematical use is deliberately kept broad; therefore the
    Math property is also given to characters that are used as operators, but are
    not part of standard mathematical notation, such as \type {U+2052} \typ
    {COMMERCIAL MINUS}.
\stopquotation

There should be no confusion with the \typ {SET MINUS} which renders as a
backslash, a \typ {(NEG\-ATED) MINUS TILDE} or \typ {(NEG\-ATED) SIMILAR MINUS
SIMILAR} that look more like relations. {\red MPS: overfull hbox, and do you
intend to hyphenate?}

The \MATHML\ document recognizes the minus as being unary or binary. In content
\MATHML\ it is easy: when applied to a single atom it is a unary. In presentation
\MATHML\ minus is an operator that sits at the front of a row (unary) or in the
middle (binary). Keep in mind that we are limited to \type {mn} for numbers,
\type {mi} for alphabetic symbols and \type {mo} for operators, not to be
confused with \TEX's math operators, because in \MATHML\ relations are also
operators. One can wonder about a minus in \type {mn} elements.

So to summarize: we definitely need to make sure that (whatever renders as)
hyphens is dealt with in math as minus. We can wonder what to do with
(especially) en- and emdashes and the other horizontal lines that actually might
show up as (what we call) middle delimiters in mathematical constructs: if it's
there, \TEX ies will use it! The lack of specific symbols for unary minus has to
be compensated at the macro package level.

% \ctxlua{table.tocontext(characters.data[0x002D],"[0x002D]")}
% \ctxlua{table.tocontext(characters.data[0x2010],"[0x2010]")}
% \ctxlua{table.tocontext(characters.data[0x2011],"[0x2011]")}
% \ctxlua{table.tocontext(characters.data[0x2212],"[0x2212]")}
% \ctxlua{table.tocontext(characters.data[0x2212],"[0x2213]")}
% \ctxlua{table.tocontext(characters.data[0x2212],"[0x2214]")}
% \ctxlua{table.tocontext(characters.data[0x2212],"[0x2215]")}
% \ctxlua{table.tocontext(characters.data[0xFE63],"[0xFE63]")}
% \ctxlua{table.tocontext(characters.data[0xFF0D],"[0xFF0D]")}

% U+2043 HYPHEN BULLET
% U+207B SUPERSCRIPT MINUS
% U+208B SUBSCRIPT MINUS

\stopsection

\startsection[title=Pieces]

In \UNICODE\ one can find all kind of constructors, for instance characters that
find their origin in those character sets that had lines and corners for drawing
on a terminal. It is therefore no surprise that there are also some constructors
that relate to math. An example demonstrates this:

\startbuffer[definition]
\def\makeweird#1#2#3#4%
  {\vcenter\bgroup
     \offinterlineskip
     \hbox{$\scriptscriptstyle\char"#1$}\par
     \hbox{$\scriptscriptstyle\char"#2$}\par
     \hbox{$\scriptscriptstyle\char"#3$}\par
     \hbox{$\scriptscriptstyle\char"#4$}%
   \egroup}

\def\lwA{\mathopen {\makeweird{23A7}{23A8}{23A8}{23A9}}}
\def\rwA{\mathclose{\makeweird{23AB}{23AC}{23AC}{23AD}}}
\def\lwB{\mathopen {\makeweird{23A7}{23AC}{23AC}{23A9}}}
\def\rwB{\mathclose{\makeweird{23AB}{23A8}{23A8}{23AD}}}
\def\lwC{\mathopen {\makeweird{23A7}{23AC}{23A8}{23A9}}}
\def\rwC{\mathclose{\makeweird{23AB}{23A8}{23AC}{23AD}}}
\stopbuffer

\startbuffer[demo]
$\lwA x + 4 + \lwB x^2 + 4^2 + \lwC x^3 + 4^3 \rwC \rwB \rwA$
\stopbuffer

\typebuffer[definition,demo]

This renders as:

\startlinecorrection
\getbuffer[definition]
\scale[width=\textwidth]{\getbuffer[demo]}
\stoplinecorrection

So, we have official \UNICODE\ characters for constructing large fences. In the
\UNICODE\ math documents there is some mentioning of this and interesting is that
there are suggested compositions expressed in 2, 3, 5 etc. stacked \quote {lines}
which makes one wonder how math is perceived (or supposed to be rendered). But
what is really weird is that there are plenty of arrows but no snippets defined that
can be used to create extended ones. Why vertical snippets and no horizontal
ones? This is clearly an omission and the \TEX\ community did take care of this
need. So, for horizontal arrows and alike one expects the font to handle it and
for fences not?

It is not only fences that have snippets, we also find them for integrals. But
for vertical arrows they are lacking: that is completely up to the font. Now, for
us that is fine, but again, for consistency they could have been there. It would
make it possible to filter bits and pieces from fonts using official slots
instead of private ones. So, to some extent we can best assume there is nothing
like that and ignore whatever pieces are in \UNICODE\ anyway (like the braces in
the example). One can even argue that because of this inconsistency a font
designed can as well only use private slots and not provide snippets at all.

So, how do we get out of this situation? Because no one cared getting it in
\UNICODE, we can do as we like. Of course, we can define arrow fillers as has
always been done in \TEX, but because in \LUAMETATEX\ we have a bit more in our
toolkit, and because we want to support stretch fractions (where the rule is
replaced by a horizontal delimiter) it was decided to define a tweak that deals
with this: when the basic arrows have no horizontal parts defined, we just
assemble them. For those arrows that have a hook or so at the other end, we use
the space as extender. \footnote {Actually we no longer do that because the
engine will center the arrow anyway when it's too short.} If we ever end up with
proper snippets un \UNICODE\ then we also need adapted fonts, and then we can get
rid of these hacks. That said: because all decent math fonts do have the three
pairs or fences (brace, parenthesis, bracket) the vertical snippets are rather
useless, unless one wants to construct assembled weird ones. This would be
different for horizontal assemblies, because there is more variety in them.

The official name for all related to characters that can stretch is \quote
{delimiter}. In traditional \TEX\ one can define a command that becomes a
character. In that case a family, class and slot is assigned. You can also
directly access a character in which case one will assign these properties
otherwise (no command is defined). The same is true for these delimiters.
However, in traditional \TEX\ the larger character usually comes from a so called
extension font and uses family~3). In \OPENTYPE\ fonts we have all in one font so
there the large family, class, and slot are not used.

An interesting side effect of the updated math machinery in \LUAMETATEX\ is that
we no longer really need delimiter specifications when we use \OPENTYPE\ fonts.
This is because in practice the only two classes that really matter are the open
and close ones. There are basically two kinds of delimiters: fences and
singulars. Fences need open and close and only bars have a dual character. So,
when we don't define it as delimiter, the engine can still use that character and
take its assigned class when used stand|-|alone, while in the case of fences
these themselves are of class open and close. And, for instance a left brace can
get class open because when used stand alone it is an unscaled left fence. In the
rare case that one really need a different class we are using commands: some
characters can be binary, ordinary or whatever so then commands relate a name to
a class|-|character combination. Actually, in \CONTEXT\ we will switch to using
dictionaries and field specific rendering instead, but that is a different story.
We can illustrate the arrows with an example:

\startbuffer
$ x +
    \left\downarrow a \uparrow \frac{1}{b} \downarrow c \right\uparrow
= y $
\stopbuffer

\typebuffer

The stand alone arrows are defines with class relation but when used as fences
their spacing is driven by the fences themselves.

\startlinecorrection
\scale[width=\textwidth]{\showmakeup[mathglue]\mathspacingmode1\showglyphs\getbuffer}
\stoplinecorrection

This means that in \CONTEXT\ \LMTX\ we no longer have delimiter code definitions.
Of course the engine has to be able to use math characters of any kind (by
commands, direct or as \UTF) as delimiters, but that was not that hard to
provide. It also simplifies the code we use for fencing as it can be less
selective.

Another interesting side effect of once again looking into these stretched
characters is that the fraction mechanism that already was extended with skewed
fractions, now supports any stretchable character as alternative for a fraction
rule.

\startbuffer
$
    p \leftarrowtext {a + b + c + d}{x + y} q
    \quad
    p \frac {a + b + c + d}{x + y} q
$
\stopbuffer

\typebuffer

Watch the difference in spacing: here the class of the used delimiter determines the
spacing around the (pseudo) fraction:

\startlinecorrection
\scale[width=\textwidth]{\showmakeup[mathglue]\mathspacingmode1\showglyphs\getbuffer}
\stoplinecorrection

Again this simplifies some code because normally one ends up with stacking stuff
using leaders in between.

\stopsection

\startsection[title=Accents]

When we talk about accents, we refer to tiny symbols that anchor themselves onto
base characters. We limit ourselves to the ones common in Latin scripts because
they are the ones used in math. Accents in \UNICODE\ are somewhat special. In
the past, when encoding vectors were limited, accents were entered as part of an
input sequence and then anchored by the renderer. Nowadays often pre|-|composed
characters are used. A very cheap way of anchoring is to have accents that just
overlay, and in practice centering an accent over a base character works sort of
okay. As an example of an accent we will use the hat:

\starttabulate[|T|c|l|c|]
\NC U+005E \NC x\char"005E x m\char"005E m\NC \tex {Hat}     \NC \im{x \char"005E x + m\char"005E m} \NC \NR %  94
\NC U+02C6 \NC x\char"02C6 x m\char"02C6 m\NC \tex {hat}     \NC \im{x \char"02C6 x + m\char"02C6 m} \NC \NR % 710
\NC U+0302 \NC x\char"0302 x m\char"0302 m\NC \tex {widehat} \NC \im{x \char"0302 x + m\char"0302 m} \NC \NR % 770
\stoptabulate

Normally the font handler will take care of anchoring \type {U+0302}, but it can
only be done properly when there are anchors defined for what are called \quote
{marks}: the official feature description is mark|-|to|-|base (or simply \type
{mark}).  The last column in the above table shows math and as we input a raw
character we don't get proper anchoring: the zero width makes it overlap.

% till here

Now wait, you will say, but why does it actually overlap? The reason is that zero
width is not actually zero width here! The glyph has a bounding box that goes
into the negative horizontal direction and therefore, when such a shape gets
injected into the output, the rendering in the viewer will move the left edge to
the left. But because the \TEX\ engine only handles positive widths and because
the width is explicitly part of a character specification anyway\footnote {The
height and depth are not: these we derive from the bounding box.} we don't
progress (advance) which is why the overlapping sort of works for the $x$ but
less so for the $m$: in math mode we need to use these \type {\hat} and \type
{\widehat} commands.

The hat and widehat assignments were those of August 2022. In plain \TEX\ we see
these definitions:

\starttyping
\def\hat    {\mathaccent"705E }
\def\widehat{\mathaccent"0362 }
\stoptyping

The \type {\mathaccent} primitive takes an integer that encodes the class, family,
and slot in the 8 bit font encoding. Here we see that the hat comes from family
0, the upright math font. The widehat comes from extensible family 3. These two
are independently defined. When you want a hat that spans the nucleus, you need to
use the widehat. In the math engine spanning actually means that we have a
delimiter and normally that means: start with a basic shape, when that is too
narrow, go to the extensible font and follow the chain with increasing sizes and
when you run out of those apply an extensible recipe. The sequence and extensible
are both optional and the important part is that we first look at what is called
the small character and then to the large one(s).

However, the \type {\mathaccent} primitives doesn't take a delimiter! It directly
starts following a chain if the given character has it (and then the character
itself is of course the first in that chain). And this is where the problems
start when we move to \OPENTYPE\ and \UNICODE\ math.

\starttabulate[|T|l|l|]
\NC U+005E \NC Hat     \NC some useless, often ugly large glyph \NC \NR %  94
\NC U+02C6 \NC hat     \NC it has width but no extensibles      \NC \NR % 710
\NC U+0302 \NC widehat \NC it has zero width and extensibles    \NC \NR % 770
\stoptabulate

Now, if we define \type {\hat} as \type {U+02C6} we don't get the extensibles,
and it basically is what was always done in \TEX\ macro packages following the
plain suggestions. If we define \type {\widehat} we start out with a glyph that
has likely zero width\footnote {Over the many years that \LUATEX\ evolved this
was not guaranteed, for instance when wide (\UNICODE) fonts were constructed from
traditional eight bit (\TEX\ encoded) fonts.} And, because \OPENTYPE\ starts with
the base glyph and {\em then} uses a set of variants of eventually a recipe of
parts, we suddenly have a different situation with \type {\mathaccent} than we
normally have, where these are decoupled. Therefore, the definition of \type {\hat}
and \type {\widehat} determines what an \OPENTYPE\ math engine will do, just as
in regular \TEX, but we might need them to be defined differently.

A solution would be to let \type {\mathaccent} (or \type {\Umathaccent}) directly
go to the variants, but that is sort of weird. Because a zero width glyph doesn't
match the criteria to span a nucleus it is likely to be skipped anyway, although
there can be a case where the next in size overruns the width of the nucleus in
which case the zero width one is used which itself is not that nice. We could
actually derive the width from the boundingbox, but that would be a bit abnormal,
and it makes no sense to burden the font machinery with that exception. Another
approach we can follow is to just copy the extensibles from \type {U+0302} to
\type {02C6} and use that one for \type {\hat} as well as \type {\widehat} and
then make \type {\widehat} an alias to \type {\hat}. After, all, the main reason
why we have two commands comes from the fact that \type {\mathaccent} doesn't
take a delimiter but single character reference (encoded in an integer).

Here is the whole list of accents:

\starttabulate[||T||T|]
\NC \tex{grave} \NC U+0060 \NC \tex{widegrave} \NC U+0300 \NC \NR
\NC \tex{ddot}  \NC U+00A8 \NC \tex{wideddot}  \NC U+0308 \NC \NR
\NC \tex{bar}   \NC U+00AF \NC \tex{widebar}   \NC U+0304 \NC \NR
\NC \tex{acute} \NC U+00B4 \NC \tex{wideacute} \NC U+0301 \NC \NR
\NC \tex{hat}   \NC U+02C6 \NC \tex{widehat}   \NC U+0302 \NC \NR
\NC \tex{check} \NC U+02C7 \NC \tex{widecheck} \NC U+030C \NC \NR
\NC \tex{breve} \NC U+02D8 \NC \tex{widebreve} \NC U+0306 \NC \NR
\NC \tex{dot}   \NC U+02D9 \NC \tex{widedot}   \NC U+0307 \NC \NR
\NC \tex{ring}  \NC U+02DA \NC \tex{widering}  \NC U+030A \NC \NR
\NC \tex{tilde} \NC U+02DC \NC \tex{widetilde} \NC U+0303 \NC \NR
\NC \tex{dddot} \NC U+20DB \NC \tex{widedddot} \NC U+20DB \NC \NR
\stoptabulate

The only accent that is an exception is the last one but is it really used? It
anyway makes no real sense to assume that users will ever directly input the
\UTF\ characters conforming the last column, so we can just go for the first one
and use the extensibles from the second and see where we end up. Neither \MATHML\
nor \TEX\ related specifications seem to cover this well, so we can just do what
suits us best.

\startbuffer
\showglyphs
\im {\widehat{a} + \widehat         {aa}} =
\im {\hat    {a} + \hat             {aa}} =
\im {\hat    {a} + \hat[stretch=yes]{aa}} =
\setupmathaccent[top][stretch=yes]
\im {\hat    {a} + \hat             {aa}}
\stopbuffer

Because all has to fit into the \CONTEXT\ user interface and because we also want
to be backward compatible (command wise), we end up with something:

\typebuffer

that gives us:

\startpacked \glyphscale = \numexpr2*\glyphscale\relax \getbuffer \stoppacked

Now, one problem, is of course that users can enter these modifiers as \UTF\
sequence in the input, just like they do with delimiters. Therefore we do support
the following feature (which is under class control so disabled by default):

\startbuffer
\Umathcode    "02C6   \mathaccentcode 0 "02C6
\edef         \HiHatA {\Uchar"02C6}
\Umathchardef \HiHatB \mathaccentcode 0 "02C6

$ \Uchar"02C6{x} + \HiHatA{xx} + \HiHatB{xx} = \widehat {xxxx} $
\stopbuffer

\typebuffer

You get this:

\start
    \pushoverloadmode \getbuffer \popoverloadmode
\stop

The only cheat here is that normally accents come after the accentee, but we can
live with that. After all, it's all about convenience.

There is another aspect of accents that we need to mention here. The hat, tilde,
and check are often used over not only single letters but also small expressions.
So how come that fonts have only very few variants defined? We can imagine that
in eight bit fonts the number of available slots plays a role but in \OPENTYPE\
fonts that is not the case. It therefore can be considered an
oversight that usage of these wide accents has not be communicated well to the
font designers.

\def\CrappyHack#1{\im{
    #1{a}       + #1{a+b}       + #1{a+b+c} +
    #1{a+b+c+d} + #1{a+b+c+d+e} + #1{a+b+c+d+e+f}
}\par}

\startpacked
\CrappyHack\widehat
\CrappyHack\widetilde
\CrappyHack\widecheck
\stoppacked

The previous lines demonstrate that we can actually cheat a little for these
three top accents: we can just scale the last variant horizontally. It was a few
lines patch to \LUAMETATEX\ to make this automatic and triggered by setting the
\type {extensible} field in a character table to \type {true} instead of a
recipe. The ingredients to get this working were already there, and it works out
quite well. The only complication was that the \type {flac} feature (that
provides flat accents for cases where the nucleus is rather high) could interfere,
but that was trivial to deal with in the code that does the goodies. \footnote
{When we were testing fonts this got us by surprise when we tested Cambria that
has these flat overloads for the tilde and check. Because supports this automatic
(hidden from the user) one doesn't look into that direction when testing
something.}

When it comes to these delimiters that have no real solution in the font, we can
consider delegating coming up with a glyph to the macro package at the time it is
needed, and we can actually do that. However, this is mostly interesting for
educational usage, where the amount of delimiters is predictable and limited.
About a decade ago some mechanism was added to the \MKIV\ math machinery that
support plugins so that we could use \METAFUN\ to generate (most noticeably)
square root symbols the way we liked. \footnote {This was a fun project of Alan
and Hans.} The main drawback is that mixing this in means matching to a font, and
that is not always trivial. But it is this kind of trickery that makes working
with \TEX\ fun. That said: what we are discussing here is more fundamental in the
sense that we try to come up with generic engine solutions that just rely on the
fonts. That way complex math with all reasonable symbols is also served.
\footnote {These \METAFUN\ plugins are still possible, but we need to adapt some
to \LMTX\ which will happen as we go.}

Interestingly there are some arrows that act like accents. There are over- and
under ones as well as combining (often zero width) accents. Fonts are not always
consistent in how these extends (the wide ones). Often the combining accents are
smaller and closer to the running text. Traditionally in \TEX\ fonts there are no
extensible arrows: they are constructed from arrow heads, minus and equal signs
with some negative spacing in between. One can therefore wonder is the smaller
combining ones are appreciated by those who want stable math. It definitely means
that we have to make choices. Even more interesting is that while \UNICODE\ has
some means to construct braces from predictable \UNICODE\ slots. there is no way
to do the same with arrows and (indeed) there are fonts out there with shaped
arrows that demand different middle and end pieces. In fact, the same is true for
rules that are not simple rectangles and radical extensions that are not flat
rules either. In all these cases the usage patterns of accents and similar
constructs has not really been fed back into the way \UNICODE\ and \OPENTYPE\
fonts support math. \footnote {One can argue that this is not what \UNICODE\ is
for but if so, then some other bits and pieces also make little sense.}

\stopsection

\startsection[title=Bullets]

In \TEX\ usage bullets are a it special. Because fonts had a limited number of slots
available, bullets in for instance itemized lists traditionally were taken from
a math font. The bullet in Computer Modern has a comfortable size and is quite
useful for that. Bullets in text fonts often were (are) relatively small so even when
they were available they were not really used. The official \UNICODE\ slot for
bullet is \type {U+2022} and in this font it shows up as \quote {•}. The \WIKIPEDIA\ page
on bullets (typography) mentions:

\startquotation
    A variant, the bullet operator (\type {U+2219} ∙ \typ {BULLET OPERATOR}) is
    used as a math symbol, akin to the dot operator. Specifically, in logic, $x •
    y$ means logical conjunction. It is the same as saying \quotation {x and y}
\stopquotation

The page also mentions that \quotation {glyphs such as {\switchtobodyfont
[stixtwo]$•$} and {\switchtobodyfont [stixtwo]$◦$}} have \quotation {reversed
variants {\switchtobodyfont [stixtwo]$◘$} and {\switchtobodyfont [stixtwo]$◙$}}
although we haven't see the reverse once in \TEX\ documents (yet), like these (we
use \STIX2\ to show them):

\starttabulate[|Tl|l|l|]
\NC U+2022 \NC \switchtobodyfont[stixtwo]$•$ \NC BULLET \NC \NR
\NC U+2023 \NC \switchtobodyfont[stixtwo]$‣$ \NC TRIANGULAR BULLET \NC \NR
\NC U+2043 \NC \switchtobodyfont[stixtwo]$⁃$\NC HYPHEN BULLET \NC \NR
\NC U+204C \NC \switchtobodyfont[stixtwo]$⁌$\NC LACK LEFTWARDS BULLET \NC \NR
\NC U+204D \NC \switchtobodyfont[stixtwo]$⁍$\NC LACK RIGHTWARDS BULLET \NC \NR
\NC U+2219 \NC \switchtobodyfont[stixtwo]$∙$ \NC BULLET OPERATOR (math) \NC \NR
\NC U+25CB \NC \switchtobodyfont[stixtwo]$○$ \NC WHITE CIRCLE \NC \NR
\NC U+25CF \NC \switchtobodyfont[stixtwo]$●$ \NC BLACK CIRCLE \NC \NR
\NC U+25D8 \NC \switchtobodyfont[stixtwo]$◘$ \NC INVERSE BULLET \NC \NR
\NC U+25E6 \NC \switchtobodyfont[stixtwo]$◦$ \NC WHITE BULLET \NC \NR
\NC U+29BE \NC \switchtobodyfont[stixtwo]$⦾$ \NC CIRCLED WHITE BULLET \NC \NR
\NC U+29BF \NC \switchtobodyfont[stixtwo]$⦿$ \NC CIRCLED BULLET \NC \NR
\stoptabulate

The reverse ones are not really reverse in \STIX2\ as they have bigger circles.
There are a few more bullets mentioned but probably only because they have the
word bullet in their description and they don't really look like bullets. Given
the already discussed lack of granularity in some math symbols with multiple
usage it is somewhat surprising that we have a math bullet. The weird looking
left- and rightward bullets are kind of hard to distinguish. Let's hope that
mathematicians don't discover these!

This brings us to the more general way of looking at these bullets because among
the popular math symbols used in text are also the triangles and (\TEX) math
fonts came with. When we have a few commands for circular shapes like \typ
{$\bullet \bigcirc \circ$} giving $\bullet \bigcirc \circ$ we have plenty of
(black) triangles.

For instance, we have \type {\triangledown} and \type {\bigtriangledown} and these
have corresponding \UNICODE\ slots \type {U+25BD} and \type {U+25BF} but when
you try these in for instance \STIX2, Pagella and Cambria you got:
▽ + ▿, ▽ + ? and ? + ?, where the question mark indicates a missing character.

It is for that reason that \type {\triangledown} and \type {\bigtriangledown} are
both defined as using the large one. This test also demonstrated us that we
didn't have to waste time looking up what \MATHML\ had to tell about it. A
typeset version of that specification was never a visual highlight and missing
glyphs only makes that worse. And, when fonts lack shapes no one uses them
anyway.

However, it makes sense to think a bit about how to deal with this properly, and
we will likely add some checking to the goodie files for it, so that when we do
have them, we use them. \footnote {Most practical is to add this information to
the character database which is a bit of work}. But even then, most troublesome
is that the size (and even positioning) of these symbols is rather inconsistent
across math fonts, but because they are seldom used it doesn't make much sense to
compensate for that (read: we just wait till users ask for it).

% {\switchtobodyfont[stixtwo]$\char"25BD+\char"25BF$}% +\triangledown+\bigtriangledown$
% {\switchtobodyfont[pagella]$\char"25BD+\char"25BF$}% +\triangledown+\bigtriangledown$
% {\switchtobodyfont[cambria]$\char"25BD+\char"25BF$}% +\triangledown+\bigtriangledown$

\stopsection

\startsection[title=Punctuation]

There are quite some punctuation symbols in \UNICODE\ but not for math where the
main troublemakers are the period, comma, colon and semicolon. The first two can
be used as separator in numbers, in which case we don't want any spacing, or they
can be part of a (pseudo) sentence in a formula, or they can separate entries in
a list (take coordinates).

\starttyping
1.1 + 1.2
(1.1, 1.2)
x + 1.1, x + 1.2
\stoptyping

When used as separator in a sentence, which is more likely in display math than
in inline math, the spacing after it can be either regular (as in text) or wide.
And the symbol can come from the math font or text (and these can actually look
different). In \CONTEXT\ (also pre \LMTX) we have some special trickery at work
for spacing comma's and periods but we leave that aside now. What should be noted
is that out|-|of|-|the|-|box spaces are ignored when math is scanned so we cannot
take that surrounding into account when dealing with spacing in the engine.

Although the \UNICODE\ specification provides a classification of characters that
includes punctuation in practice we need to deal with it ourselves. For instance,
by default a period is not considered punctuation but a command and semi colon
are, while a colon is a relation!

Take for instance $f.$ (math italic f followed by a period). Italic correction
and math glyphs have this special relationship and it also shows up in
punctuation. Imagine that we have a sequence of characters, say $fx$. These are
actually two ordinary atoms but in $f,$ we have an ordinary atom followed by a
punctuation atom so here spacing is determined by how these classes are set up.
But, given the shape if the $f$ we actually don't want italic correction here.

\startbuffer
$fx + f. +f, + f: + f; + a. +a, + a: + a; + x, +x, + x: + x;$%
\stopbuffer

\startlinecorrection
\scale[width=\textwidth]{%
    \getbuffer
}
\blank[halfline]
\scale[width=\textwidth]{%
    \showmakeup[mathglue]%
    \mathspacingmode\plusone
    \showfontitalics
    \showfontkerns
    \showglyphs
    \getbuffer
}
\stoplinecorrection

When you zoom in you can see the subtle spacing differences. We can compensate
for the semi colon being a bit higher than the period by applying some kern,
something that we can set up in the goodie file.

Actually, if we assume that periods only occur in numbers we can make it
punctuation and set it up for digit spacing but then commas etc also get done
that way. A variant is to have two punctuation classes (or cheat and put the
period in the digit class). No matter what we do, no help can be expected from
documents mentioned: it's mostly a visual thing anyway.

Let's end with the visual aspect: in most fonts the two colons \type {0x003A} and
\type {0x2236} are different: one has more distance between the periods. Which
one? Well, that depends on the font! Latin Modern has a cramped \type {0x2236}
while \STIX2 has a cramped \type {0x003A}. Cambria has square dots for the
{0x003A} and round ones slightly more cramped for \type {0x2236}. Lucida goes
extreme: it has smaller dots far apart for \type {0x2236}. If the idea is that a
reader should get from the shape what it's about one can wonder if texts get read
the way the author intended. Of maybe shapes don't matter. Of course a macro
package can obscure these inconsistencies by setting the math character code of
\type {0x003A} to \type {0x2236} but that only obscures the fact that little
attention has been paid: what one can consider bugs became features.

\stopsection

\startsection[title=Special ones]

There are quite some characters that really depend on a math renderer. Examples
are wide accents, fences, and arrows. Some constructs, like fractions use rules
and these don't come from \UNICODE\ nor fonts. A mixed case is radicals: there
is a \UNICODE\ point and fonts can provide larger variants. Normally one steps up
a slightly slanted version but when things get large the radical becomes an
extensible and therefore gets an upright shape. The engine is supposed to add a
horizontal rule at the right location. Interesting is that there is no provision
for a right end cap. The reason probably is that \TEX, being the major renderer,
has no combined horizontal and vertical extenders and \OPENTYPE\ doesn't have
that either. Some properties are driven by the fonts' math parameters which sort
of makes the radical rendering a very restricted adventure: it is supposed to be
used for roots only, either of not with a degree anchored in the right top area.
It looks like that degree is not really to extend much beyond the left edge of
the symbol.

In \UNICODE\ there is an actuarian character \type {U+20E7} and support in fonts
is not that good. We do support it because we ran into in \MATHML. However, it is
a hack. The symbol as provided by fonts is rather useless.

\startbuffer
$ \sqrt {x + 1} + \annuity{x + 1} $
\stopbuffer

\typebuffer

Let's see how it renders:

\startlinecorrection
\scale[width=.5\textwidth]{\getbuffer}
\stoplinecorrection

We take the dimensions of a radical as template and when we look at the bare
glyphs we see this:

\startlinecorrection
\scale[height=2\lineheight]{$\char"221A \enspace \char"20E7$}
\stoplinecorrection

Basically we have a right actuarian character like we have a left radical. But In
this case the rule will go left instead of right. This is implemented on top of
radicals so and driven by \type {\Udelimited} that takes two delimiters and
doesn't scan for a degree. For two-sided roots (with degree) we have \type
{\Urooted}. And like normal radicals the delimited one adapts itself to the
content:

\startbuffer
$ \sqrt {x + \frac{1}{x}} + \annuity {x + \frac{1}{x}} $
\stopbuffer

\typebuffer

So we get:

\startlinecorrection
\scale[width=.5\textwidth]{\showstruts \getbuffer}
\stoplinecorrection

For the record: in \CONTEXT\ spacing is also driven by the struts and because we
use the radicals renderer the gap and distance parameters also apply. It might
look spacy, but keep in mind that we want radicals to look similar when we have
more of them in line, and we can configure all. We have also enabled the feature
that radicals at the same level are normalized in height and depth. Here are some
variants:

\startbuffer
$ \lannuity  {x + \frac{1}{x}} +
  \rannuity  {x + \frac{1}{x}} +
  \lrannuity {x + \frac{1}{x}} $
\stopbuffer

\typebuffer

This gives:

\startlinecorrection
\scale[width=.75\textwidth]{\getbuffer}
\stoplinecorrection

So we can have a mix of left, right and both end radical like symbols that
encompass the nucleus. We're not aware of more such characters in \UNICODE\ but
when they show up we are prepared. Only real usage can result in some parameters
being fine|-|tuned.

\stopsection

% \startsection[title=Summary]
%
% Here we give a summary of some of the things that added on top of \UNICODE\ and
% \OPENTYPE\ math in order to be able to properly render these more complex atoms
% and molecules.
%
% \stopsection

\startsection[title=Final words]

This text was written in 2022 when we were working on math, extending the goodie
files with new tweaks, checking support in fonts and updating manuals. But, as we
moved forward, for instance with adapting \TYPEONE\ support of Antykwa and Iwona
to the new possibilities again we had to go back in time and figure out why
actually things were done in certain ways. And I have to admit that we had some
good laughs and quite some fun on seeing how strange and inconsistent the assumed
structured and logical \TEX\ ecosystem deals with math. A wrapup like is is never
complete and we can keep adding to it so just consider it to be a momentary
impression.

Personally I have to admit that I've always overestimated what happened outside
the \CONTEXT\ bubble, especially given the claims made. Consistency in \UNICODE\
math is probably not as good as is could have been and the same is true for
\OPENTYPE\ math support, but maybe I'm naive in expecting consistency and logic
in math related work. The mere fact that Donald Knuth pays a lot of attention to
the math in his writing doesn't automatically translate in all \TEX ies doing the
same. I don't claim that \CONTEXT\ is doing better but I do hope that its users
keep going for the best outcome.

\stopsection

\startsection[title=Resources]

\starttyping
[1] https://en.wikipedia.org/wiki/Slash_(punctuation)
[2] http://www.unicode.org/reports/tr25
[3] https://www.w3.org/TR/MathML3
[4] https://www.unicode.org/Public/math/revision-15/MathClass-15.txt
[5] https://en.wikipedia.org/wiki/Vertical_bar
[6] https://en.wikipedia.org/wiki/Dash
[7] https://en.wikipedia.org/wiki/Commercial_minus_sign
[8] https://en.wikipedia.org/wiki/Division_sign
[9] https://en.wikipedia.org/wiki/Bullet_(typography)
\stoptyping

\stopsection

% After reading the \UNICODE\ report about math I don't feel too guilty when people
% complain about the \CONTEXT\ manuals. It is a curious mix of discussing
% organization of symbols, rendering, usage, structure, exchange, parsing,
% confusion, etc. and it is clearly a mix of experiences with the web, word
% processing and \TEX\ and as such not that useable because it is just not how
% \TEX\ works with input and fonts and how users perceive matters. But it
% definitely helps to get an idea why we ended up with the current situation: the
% unification of math was more a combination of what was there and not a fresh
% start. Maybe that is not really possible anyway. If we flash forward a couple of
% pages it will all look the same to us as stone age chiseling in stone.

\stopchapter

\stopcomponent