diff options
author | Hans Hagen <pragma@wxs.nl> | 2021-10-08 20:46:55 +0200 |
---|---|---|
committer | Context Git Mirror Bot <phg@phi-gamma.net> | 2021-10-08 20:46:55 +0200 |
commit | 778f381ba6a448ab00d67994a412dd4226d43238 (patch) | |
tree | d9dade45016a572e6c22521bfb165f9829ac3192 /doc/context/presentations | |
parent | 2073fe5d88215dddd9a9e6421afaea7ab7db955a (diff) | |
download | context-778f381ba6a448ab00d67994a412dd4226d43238.tar.gz |
2021-10-08 20:07:00
Diffstat (limited to 'doc/context/presentations')
14 files changed, 2361 insertions, 0 deletions
diff --git a/doc/context/presentations/context/2021/context-2021-compactfonts.pdf b/doc/context/presentations/context/2021/context-2021-compactfonts.pdf Binary files differnew file mode 100644 index 000000000..fb3f448ba --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-compactfonts.pdf diff --git a/doc/context/presentations/context/2021/context-2021-compactfonts.tex b/doc/context/presentations/context/2021/context-2021-compactfonts.tex new file mode 100644 index 000000000..7d2d3e482 --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-compactfonts.tex @@ -0,0 +1,609 @@ +% language=us + +\usemodule[present-boring,abbreviations-logos] + + +\definehighlight[nb][style=bold,color=middlegray,define=no] + +\definecolor[maincolor][g=.3] + +\startdocument + [title={COMPACT FONTS}, + banner={a different approach to scaling}, + location={context\enspace {\bf 2021}\enspace meeting}] + +\starttitle[title=How \TEX\ uses fonts] + +\startitemize + \startitem + In a \TEX\ engine there is a 1-to-1 mapping from characters in the input + to a slot in a font. + \stopitem + \startitem + Combinations of characters can be mapped onto one shape: ligatures. + \stopitem + \startitem + Hyphenation is (also) controlled by the input encoding of a font which + imposes some limitations. + \stopitem + \startitem + In the typeset result characters (glyphs) can be (re)positioned relatively: kerns. + \stopitem + \startitem + This all happens in a very interwoven way (e.g. inside the par builder). + \stopitem +\stopitemize + +\stoptitle + +\starttitle[title=How traditional \TEX\ sees fonts] + +\startitemize + \startitem + A font is just a (binary) blob of data with information where characters + have a width, height, depth and italic correction. + \stopitem + \startitem + The engine only needs the dimensions and when the same font is used with a + different scale a copy with scaled dimensions is used. + \stopitem + \startitem + There can be recipes for ligatures and (more complex) so called math extensible + shapes (like arrows, wide accents, fences and radicals) + \stopitem + \startitem + There are pairwise specification for kerning. + \stopitem + \startitem + A handful of font parameters is provided in order to control for instance spacing. + \stopitem + \startitem + Virtual fonts are just normal fonts but they have recipes for the backend to + construct shapes: \TEX\ uses the normal metric file, the backend also the virtual + specification file. + \stopitem +\stopitemize + +\blank[2*big] + +An example of a definition and usage: + +\starttyping +\font\cs somename [somescale] test {\cs test} test +\stoptyping + +\stoptitle + +\starttitle[title=\UNICODE\ engines] + +\startitemize + \startitem + There are no fundamental differences between a traditional engine and an + \UNICODE\ aware one with respect to what \TEX\ needs from fonts: + dimensions. + \stopitem + \startitem + There can be more than 256 characters in a font. + \stopitem + \startitem + Some mechanisms have to be present for applying font features (like ligaturing and + kerning). + \stopitem + \startitem + In \LUATEX\ hyphenation, ligaturing and kerning are clearly separate steps. + \stopitem + \startitem + In \LUATEX\ callbacks can do lots of things with fonts and characters. + \stopitem + \startitem + Math fonts are handled differently and there is more info that comes from the font. + \stopitem + \startitem + In \LUATEX\ features like expansion are implemented differently, i.e.\ no + copies of fonts are made and applied expansion travels with the glyphs. + \stopitem +\stopitemize + +\stoptitle + +\starttitle[title=Overhead] + +\startitemize + \startitem + In all engines scales copies are used. For traditional \TEX\ a copy is cheap, but a + font that supports more of \UNICODE\ cna be quite large. + \stopitem + \startitem + Different feature sets in principle make for a different font (this can be different + per macro package). + \stopitem + \startitem + Every math scale needs three font instances: text, script and scriptscript. + \stopitem + \startitem + In \CONTEXT\ lots of sharing takes place and a lot of low level + optimizations happens (memory, performance). We always made sure (already + in \MKII) that the font system was not the bottleneck. + \stopitem + \startitem + In most cases font definitions are dynamic, and we delay initialization + as much as possible. + \stopitem + \startitem + In \MKIV\ font features can be dynamic which saves a lot of memory at the + cost of some extra processing overhead. + \stopitem +\stopitemize + +\stoptitle + +\starttitle[title=Design sizes] + +\startitemize + \startitem + A macro package's font handling is complicated by design sizes. The low level + code can look complex too. + \stopitem + \startitem + There are hardly any fonts that come in design sizes (basically only Computer + Modern) so we need to support that. + \stopitem + \startitem + Design sizes make sense for math fonts where very small characters and + symbols are used in scripts. + \stopitem + \startitem + In \OPENTYPE\ fonts smaller math design sizes are part of the font (that + then gets loaded three times with different \type {ssty} settings). + \stopitem +\stopitemize + +\startlinecorrection +\dontleavehmode\scale[frame=on,height=5ex]{$\textstyle 2$}\quad +\dontleavehmode\scale[frame=on,height=5ex]{$\scriptstyle 2$}\quad +\dontleavehmode\scale[frame=on,height=5ex]{$\scriptscriptstyle 2$}\quad\quad +\dontleavehmode\scale[frame=on,height=2ex]{$\textstyle 2$}\quad +\dontleavehmode\scale[frame=on,height=2ex]{$\scriptstyle 2$}\quad +\dontleavehmode\scale[frame=on,height=2ex]{$\scriptscriptstyle 2$}\quad\quad +\dontleavehmode\scale[frame=on,width=4em]{\colored[r=.6,a=1,t=.5]{$\textstyle 2$}}\hskip-4em +\dontleavehmode\scale[frame=on,width=4em]{\colored[g=.6,a=1,t=.5]{$\scriptstyle 2$}}\hskip-4em +\dontleavehmode\scale[frame=on,width=4em]{\colored[b=.6,a=1,t=.5]{$\scriptscriptstyle 2$}}\quad\quad +\dontleavehmode{$\textstyle 2$}\quad +\dontleavehmode{$\scriptstyle 2$}\quad +\dontleavehmode{$\scriptscriptstyle 2$}\quad +\stoplinecorrection + +\stoptitle + +\starttitle[title=The system] + +\startitemize + \startitem + Each bodyfont size (often a few are defined) has regular, bold, italic, + bold italic, slanted, bold slanted, etc. There can be unscaled instances + (design size) or scaled ones. + \stopitem + \startitem + Within a bodyfont size we have size variants: smaller {\tx x} and {\tx xx}, + larger {\tfa a}, {\tfb b}, {\tfc c} and {\tfd d}. + \stopitem + \startitem + In \MKII\ we inherit smallcap and oldstyle setups but in \MKIV\ one can either + do that dynamic or define an additional bodyfont instance. + \stopitem + \startitem + A document can use a mix of fonts mixed setup, so there is a namespace + used per family. + \stopitem + \startitem + In \OPENTYPE\ symbols are more naturally part of a font, as are emoji and + colored shapes. + \stopitem + \startitem + Lack of variants can result in artificially slanted of boldened fonts (more + advanced in \MKIV). Variable fonts are not special, but demand some work + when loading and embedding. + \stopitem + \startitem + Runtime virtual fonts are part of \MKIV\ as are fallback shapes. + \stopitem + \startitem + Languages and scripts can introduce an extra dimension to operate on. + \stopitem + \startitem + Math is greatly simplified by the fact that families are part of a font. + \stopitem + \startitem + Bold math fonts are provided by the boldening feature. + \stopitem +\stopitemize + +\stoptitle + +\starttitle[title={Practice}] + +\startitemize + \startitem + Due to optimizations the actual number of loaded instances is normally + small but there are exceptions. + \stopitem + \startitem + The \LUAMETATEX\ manual has 158 instances most of which are math (six per + used bodyfont for math plus a regular: Lucida, Pagella, Latin Modern, + Dejavu, Cambria). + \stopitem + \startitem + The final \PDF\ file has 21 embedded fonts (subsets). + \stopitem + \startitem + The fact that different sizes each have an instance and that for math + we need three per size plus one for r2l, while the only difference + is scale, makes one think of other solutions. + \stopitem + \startitem + Using duplicates of huge \CJK\ fonts makes it even more noticeable. + \stopitem +\stopitemize + +But there is a way out! + +\stoptitle + +\starttitle[title=Glyph scaling] + +We can scale glyphs in three ways: + +\startbuffer +\definedfont[lmroman10-regular*default]% +test {\glyphxscale 2500 test} test +test {\glyphyscale 2500 test} test +test {\glyphscale 2500 test} test +\stopbuffer + +\typebuffer {\getbuffer} + +We do need to honor grouping: + +\startbuffer +\definedfont[lmroman10-regular*default]% +e{\glyphxscale 2500 ff}icient +ef{\glyphxscale 2500 f}icient +ef{\glyphxscale 2500 fi}cient +e{\glyphxscale 2500 ffi}cient +\stopbuffer + +\typebuffer {\getbuffer} + +These scales can also be part of a font definition so we can do with less +instances. + +\stoptitle + +\starttitle[title=Implications] + +\startitemize + \startitem + The text processing part of the engine has to scale on the fly (e.g.\ + when dimensions are needed in paragraph building and (box) packing. + \stopitem + \startitem + Font processing (features) already distinguishes per instance but now also + has to differentiate by (upto three) scales (we use the same font id + for scaled instances). + \stopitem + \startitem + The math machinery has to check for smaller sizes and also scale + dimensions on the fly. + \stopitem + \startitem + For math that means two times scaling: the main scale (like glyph scale) plus + the relative scaling of script and scriptscript. + \stopitem + \startitem + When they are used font dimensions and math parameters are to be scaled as are + rules in some math extensibles. + \stopitem + \startitem + This all has to be done efficiently so that there is no significant drop in + performance. + \stopitem + \startitem + But quite a bit less memory is used and loading time has become less too. + \stopitem +\stopitemize + +Some day this will become default (which means a sentimental process of dropping +ancient code): + +\starttyping +\enableexperiments[fonts.compact] +\stoptyping + +\stoptitle + +\starttitle[title=Details] + +The tricks shown on this page and following pages have been in place and tested +for a while now. Because \TEX\ uses integers a scale value of 1000 means 1.000, +as in other mechanisms: + +\startbuffer +\definedfont[lmroman10-regular*default]% +test {\glyphscale 2500 test} test +\stopbuffer + +\typebuffer {\getbuffer} + +% \glyphtextscale +% \glyphscriptscale +% \glyphscriptscriptscale + +\page + +The font definition mechanism supports horizontal and vertical scaling: + +\startbuffer +\definefont[FooA][Serif*default @ 12pt 1800 500] +\definefont[FooB][Serif*default @ 12pt 0.85 0.4] +\definefont[FooC][Serif*default @ 12pt] + +There is also a new command: + +\definetweakedfont[runwider] [xscale=1.5] +\definetweakedfont[runtaller][yscale=2.5,xscale=.8,yoffset=-.2ex] + +{\FooA test test \runwider test test \runtaller test test}\par +{\FooB test test \runwider test test \runtaller test test}\par +{\FooC test test \runwider test test \runtaller test test}\par +\stopbuffer + +\typebuffer + +\getbuffer + +\page + +Tweaking also works in math: + +\startbuffer +\definetweakedfont[squeezed][xscale=0.9] +\definetweakedfont[squoozed][xscale=0.7] + +\startlines +$a = b^2 + \sqrt{c}$ +{\squeezed $a = b^2 + \sqrt{c}$} +{\squoozed $a = b^2 + \sqrt{c}$} +{$\squoozed a = b^2 + \sqrt{c}$} +{$\scriptstyle a = b^2 + \sqrt{c}$} +\stoplines +\stopbuffer + +\typebuffer + +\getbuffer + +\page + +\startbuffer +\startcombination[3*1] + {\bTABLE + \bTR \bTD foo \eTD \bTD[style=\squeezed] $x = 1$ \eTD \eTR + \bTR \bTD oof \eTD \bTD[style=\squeezed] $x = 2$ \eTD \eTR + \eTABLE} {local} + {\bTABLE[style=\squeezed] + \bTR \bTD $x = 1$ \eTD \bTD $x = 3$ \eTD \eTR + \bTR \bTD $x = 2$ \eTD \bTD $x = 4$ \eTD \eTR + \eTABLE} {global} + {\bTABLE[style=\squeezed\squeezed\squeezed\squeezed] + \bTR \bTD $x = 1$ \eTD \bTD $x = 3$ \eTD \eTR + \bTR \bTD $x = 2$ \eTD \bTD $x = 4$ \eTD \eTR + \eTABLE} {multiple} +\stopcombination +\stopbuffer + +\typebuffer + +\startlinecorrection +\getbuffer +\stoplinecorrection + +\page + +Tweaking can be combined with styles: + +\startbuffer +\definetweakedfont[MyLargerFontA][scale=2000,style=bold] +test {\MyLargerFontA test} test +\stopbuffer + +\typebuffer + +\getbuffer + +\page + +We can use negative scale values: + +\startbuffer +\bTABLE[align=middle] + \bTR + \bTD a{\glyphxscale 1000 \glyphyscale 1000 bc}d \eTD + \bTD a{\glyphxscale 1000 \glyphyscale -1000 bc}d \eTD + \bTD a{\glyphxscale -1000 \glyphyscale -1000 bc}d \eTD + \bTD a{\glyphxscale -1000 \glyphyscale 1000 bc}d \eTD + \eTR + \bTR + \bTD \tttf +1000 +1000 \eTD + \bTD \tttf +1000 -1000 \eTD + \bTD \tttf -1000 -1000 \eTD + \bTD \tttf -1000 +1000 \eTD + \eTR +\eTABLE +\stopbuffer + +\typebuffer + +\startlinecorrection +\getbuffer +\stoplinecorrection + +\page + +There are more glyph properties: + +\startbuffer +\ruledhbox{ + \ruledhbox{\glyph yoffset 1ex options 0 123} % left curly brace + \ruledhbox{\glyph xoffset .5em yoffset 1ex options "C0 123} + \ruledhbox{oeps{\glyphyoffset 1ex \glyphxscale 800 + \glyphyscale\glyphxscale oeps}oeps} +} +\stopbuffer + +\typebuffer \scale[width=\textwidth]{\getbuffer} + +\page + +Glyph scales are internal counter values, like any other: + +\startbuffer +\samplefile{jojomayer} +{\glyphyoffset .8ex + \glyphxscale 700 \glyphyscale\glyphxscale + \samplefile{jojomayer}} +{\glyphyscale\numexpr3*\glyphxscale/2\relax + \samplefile{jojomayer}} +{\glyphyoffset -.2ex + \glyphxscale 500 \glyphyscale\glyphxscale + \samplefile{jojomayer}} +\samplefile{jojomayer} +\stopbuffer + +\typebuffer + +\getbuffer + +\page + +There is a helper for real (float) scales: + +\startbuffer +1200 : \the\numericscale1200 +1.20 : \the\numericscale1.200 +\stopbuffer + +\typebuffer + +These lines produce the same integer: + +\startlines\tttf +\getbuffer +\stoplines + +\page + +You can do this but it's not always predictable (depends on outer scales too): + +\startbuffer[definition] +\def\UnKernedTeX + {T% + {\glyph xoffset -.2ex yoffset -.4ex `E}% + {\glyph xoffset -.4ex options "60 `X}} +\stopbuffer + +\startbuffer[example] +We use \UnKernedTeX\ and {\bf \UnKernedTeX} and {\bs \UnKernedTeX}: +the slanted version could use some more left shifting of the E. +\stopbuffer + +\typebuffer[definition,example] + +\startnarrower + \getbuffer[definition,example] +\stopnarrower + +Marks and cursive features can interfere, so this is an alternative: + +\startbuffer[definition] +\def\UnKernedTeX + {T\glyph left .2ex raise -.4ex `E\glyph left .2ex `X\relax} +\stopbuffer + +\typebuffer[definition] + +\startnarrower + \getbuffer[definition,example] +\stopnarrower + +\page + +Yet another glyph option: + +\starttyping +\multiply\glyphscale by 2 <{\darkgray \glyph `M}> +\multiply\glyphscale by 2 <{\darkgray \glyph raise 3pt `M}> +\multiply\glyphscale by 2 <{\darkgray \glyph raise -3pt `M}> +\multiply\glyphscale by 2 <{\darkgray \glyph left 3pt `M}> +\multiply\glyphscale by 2 <{\darkgray \glyph right 2pt `M}> +\multiply\glyphscale by 2 <{\darkgray \glyph left 3pt right 2pt `M}> +\multiply\glyphscale by 2 <{\darkgray \glyph left -3pt `M}> +\multiply\glyphscale by 2 <{\darkgray \glyph right -2pt `M}> +\multiply\glyphscale by 2 <{\darkgray \glyph left -3pt right -2pt `M}> +\stoptyping + +\startlinecorrection +\bTABLE[align=middle,width=.33\textwidth] + \bTR + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph `M}>\eTD + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph raise 3pt `M}>\eTD + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph raise -3pt `M}>\eTD + \eTR + \bTR[frame=off] + \bTD \tttf \eTD + \bTD \tttf raise 3pt \eTD + \bTD \tttf raise -3pt \eTD + \eTR + \bTR + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph left 3pt `M}>\eTD + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph right 2pt `M}>\eTD + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph left 3pt right 2pt `M}>\eTD + \eTR + \bTR[frame=off] + \bTD \tttf left 3pt \eTD + \bTD \tttf right 2pt\eTD + \bTD \tttf left 3pt right 2pt\eTD + \eTR + \bTR + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph left -3pt `M}>\eTD + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph right -2pt `M}>\eTD + \bTD \showglyphs \multiply\glyphscale by 2 <{\darkgray \glyph left -3pt right -2pt `M}>\eTD + \eTR + \bTR[frame=off] + \bTD \tttf left -3pt \eTD + \bTD \tttf right -2pt \eTD + \bTD \tttf left -3pt right -2pt \eTD + \eTR +\eTABLE +\stoplinecorrection + +\page + +A larger example: + +\startbuffer +\glyphscale 4000 +\vl\glyph `M\vl\quad +\vl\glyph raise .2em `M\vl\quad +\vl\glyph left .3em `M\vl\quad +\vl\glyph right .2em`M\vl\quad +\vl\glyph left -.2em right -.2em`M\vl\quad +\vl\glyph raise -.2em right .4em`M\vl +\stopbuffer + +\typebuffer + +{\getbuffer} + +\stoptitle + +\stopdocument diff --git a/doc/context/presentations/context/2021/context-2021-localcontrol.pdf b/doc/context/presentations/context/2021/context-2021-localcontrol.pdf Binary files differnew file mode 100644 index 000000000..f9f96aa41 --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-localcontrol.pdf diff --git a/doc/context/presentations/context/2021/context-2021-localcontrol.tex b/doc/context/presentations/context/2021/context-2021-localcontrol.tex new file mode 100644 index 000000000..72c1114be --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-localcontrol.tex @@ -0,0 +1,369 @@ +% language=us + +% TAKE FROM LOWLEVEL-EXPANSION.TEX + +\usemodule[system-tokens] + +\usemodule[present-boring,abbreviations-logos] + +\definehighlight[nb][style=bold,color=middlegray,define=no] + +\startdocument + [title={PROGRAMMING}, + banner={local control}, + location={context\enspace {\bf 2021}\enspace meeting}] + +\starttitle[title=Expansion] + +\TEX\ can be in several so called input reading modes: + +Users mostly see it reading from the source file(s). Characters are picked up and +interpreted. Depending on what token it becomes some action takes place. + +\starttyping +1 \setbox0\hbox to 10pt{2} \count0=3 \the\count0 \multiply\count0 by 4 +\stoptyping + +\startitemize + +\startitem + The \type {1} gets typeset because characters like that are seen as text. +\stopitem + +\startitem + The \type {\setbox} primitive triggers picking up a register number, then + goes on scanning for a box specification and that itself will typeset a + sequence of whatever until the group ends. +\stopitem + +\startitem + The \type {count} primitive triggers scanning for a register number (or + reference) and then scans for a number; the equal sign is optional. +\stopitem + +\startitem + The \type {the} primitive injects some value into the current input stream + (it does so by entering a new input level). +\stopitem + +\startitem + The \type {multiply} primitive picks up a register specification and + multiplies that by the next scanned number. The \type {by} is optional. +\stopitem + +\startitem + Printing from \LUA\ and scanning tokens with e.g.\ \type {\scantokens} is like + reading (pseudo) files. +\stopitem + +\stopitemize + +\stoptitle + +\starttitle[title=Expansion] + +\startbuffer[def] +\def\TestA {1 \setbox0\hbox{2} \count0=3 \the\count0} +\stopbuffer + +\startbuffer[edef] +\edef\TestB{1 \setbox0\hbox{2} \count0=3 \the\count0} +\stopbuffer + +\typebuffer[def] +\typebuffer[edef] + +\getbuffer[def] +\getbuffer[edef] + +\blank[2*big] + +\startcolumns[n=2] \switchtobodyfont[8pt] +\luatokentable\TestA +\column +\luatokentable\TestB +\stopcolumns + +\stoptitle + +\starttitle[title=Local control] + +\startbuffer[edef] +\edef\TestB{1 \setbox0\hbox{2} \count0=3 \the\count0} +\stopbuffer + +\startbuffer[ldef] +\edef\TestC{1 \setbox0\hbox{2} \localcontrolled{\count0=3} \the\count0} +\stopbuffer + +\typebuffer[edef] +\typebuffer[ldef] + +\getbuffer[edef] +\getbuffer[ldef] + +\blank[2*big] + +\startcolumns[n=2] \switchtobodyfont[8pt] +\luatokentable\TestB +\column +\luatokentable\TestC +\stopcolumns + +\stoptitle + +\starttitle[title=Side effects] + +\startbuffer[edef] +\edef\TestB{1 \setbox0\hbox{2} \count0=3 \the\count0} +\stopbuffer + +\startbuffer[ldef] +\edef\TestD{\localcontrolled{1 \setbox0\hbox{2} \count0=3 \the\count0}} +\stopbuffer + +\typebuffer[edef] +\typebuffer[ldef] + +\getbuffer[edef]\getbuffer[ldef]\quad{\darkgray\leftarrow\space Watch how the results end up here!} + +\blank[2*big] + +\startcolumns[n=2] \switchtobodyfont[8pt] +\luatokentable\TestB +\column +\luatokentable\TestD +\stopcolumns + +\stoptitle + +\starttitle[title=Usage] + +\startbuffer[def] +\def\WidthOf#1% + {\beginlocalcontrol + \setbox0\hbox{#1}% + \endlocalcontrol + \wd0 } +\stopbuffer + +\startbuffer[use] +\scratchdimen\WidthOf{The Rite Of Spring} + +\the\scratchdimen +\stopbuffer + +\typebuffer[def] +\typebuffer[use] + +\getbuffer[def]\getbuffer[use] + +\stoptitle + +\starttitle[title=Not always pretty] + +\startbuffer[def] +\def\WidthOf#1% + {\dimexpr + \beginlocalcontrol + \begingroup + \setbox0\hbox{#1}% + \expandafter + \endgroup + \expandafter + \endlocalcontrol + \the\wd0 + \relax} +\stopbuffer + +\startbuffer[use] +\scratchdimen\WidthOf{The Rite Of Spring} + +\the\scratchdimen +\stopbuffer + +\typebuffer[def] +\typebuffer[use] + +\getbuffer[def]\getbuffer[use] + +\stoptitle + +\starttitle[title=The \LUA\ end] + +Right from the start the way to get something into \TEX\ from \LUA\ has been the +print functions. But we can also go local (immediate). There are several methods: + +\startitemize +\startitem via a set token register \stopitem +\startitem via a defined macro \stopitem +\startitem via a string \stopitem +\stopitemize + +Among the things to keep in mind are catcodes, scope and expansion (especially in +when the result itself ends up in macros). + +\stoptitle + +\starttitle[title=Via a token register] + +% runlocal : macro|toks expand group + +\startbuffer[set] +\toks0={\setbox0\hbox{The Rite Of Spring (Igor Stravinsky)}} +\toks2={\setbox0\hbox{The Rite Of Spring (Joe Parrish)}} +\stopbuffer + +\typebuffer[set] + +\startbuffer[run] +\startluacode +tex.runlocal(0) context("[1: %p]",tex.box[0].width) +tex.runlocal(2) context("[2: %p]",tex.box[0].width) +\stopluacode +\stopbuffer + +\typebuffer[run] + +\start \getbuffer[set,run] \stop + +\stoptitle + +\starttitle[title=Via a token macro] + +\startbuffer[set] +\def\TestA{\setbox0\hbox{The Rite Of Spring (Igor Stravinsky)}} +\def\TestB{\setbox0\hbox{The Rite Of Spring (Joe Parrish)}} +\stopbuffer + +\typebuffer[set] + +\startbuffer[run] +\startluacode +tex.runlocal("TestA") context("[3: %p]",tex.box[0].width) +tex.runlocal("TestB") context("[4: %p]",tex.box[0].width) +\stopluacode +\stopbuffer + +\typebuffer[run] + +\start \getbuffer[set,run] \stop + +\stoptitle + +\starttitle[title=Via a string] + +\startbuffer[run] +\startluacode +tex.runstring([[\setbox0\hbox{The Rite Of Spring (Igor Stravinsky)}]]) + +context("[5: %p]",tex.box[0].width) + +tex.runstring([[\setbox0\hbox{The Rite Of Spring (Joe Parrish)}]]) + +context("[6: %p]",tex.box[0].width) +\stopluacode +\stopbuffer + +\typebuffer[run] + +\start \getbuffer[run] \stop + +\blank[2*big] + +A bit more high level: + +\starttyping +context.runstring([[\setbox0\hbox{(Here \bf 1.2345)}]]) +context.runstring([[\setbox0\hbox{(Here \bf %.3f)}]],1.2345) +\stoptyping + +\stoptitle + +\starttitle[title=Locked in \LUA] + +\startbuffer[run] +\startluacode +token.setmacro("TestX",[[\setbox0\hbox{The Rite Of Spring (Igor)}]]) +tex.runlocal("TestX") +context("[7: %p]",tex.box[0].width) +\stopluacode +\stopbuffer + +\typebuffer[run] + +\start \getbuffer[run] \stop + +\startbuffer[run] +\startluacode +tex.scantoks(0,tex.ctxcatcodes,[[\setbox0\hbox{The Rite Of Spring (Joe)}]]) +tex.runlocal(0) +context("[8: %p]",tex.box[0].width) +\stopluacode +\stopbuffer + +\typebuffer[run] + +\start \getbuffer[run] \stop + +\stoptitle + +\starttitle[title=Order matters] + +A lot this relates to pushing stuff into the input which is stacked. Compare: + +\startbuffer[run] +\startluacode +context("[HERE 1]") +context("[HERE 2]") +\stopluacode +\stopbuffer + +\typebuffer[run] + +\start \getbuffer[run] \stop + +with this: + +\startbuffer[run] +\startluacode +tex.pushlocal() context("[HERE 1]") tex.poplocal() +tex.pushlocal() context("[HERE 2]") tex.poplocal() +\stopluacode +\stopbuffer + +\typebuffer[run] + +\start \getbuffer[run] \stop + +% \startluacode +% tex.runlocal(function() context("[Here 1]") end) +% context("[Here 12]") +% tex.runlocal(function() context("[Here 2]") end) +% \stopluacode + +% tex.pushlocal % swaps order +% tex.poplocal +% tex.quitlocal + +% token.expandmacro + +% str : serialized +% true : wrap next {} +% table : {serialzed} {serialzed} {serialzed} +% token : inject +% number: catcode table + +% tex.expandasvalue: +% (kind, like expand_macro) + +% tex.runstring: +% [catcode] string expand grouped + +% tex.runlocal +% function|number(register)|string(macro)|userdata(token) / expand / grouped + +% mplib.expandtex +% mpx, kind, macro, arguments + +\stopdocument diff --git a/doc/context/presentations/context/2021/context-2021-luametafun.pdf b/doc/context/presentations/context/2021/context-2021-luametafun.pdf Binary files differnew file mode 100644 index 000000000..25c7b95a4 --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-luametafun.pdf diff --git a/doc/context/presentations/context/2021/context-2021-luametafun.tex b/doc/context/presentations/context/2021/context-2021-luametafun.tex new file mode 100644 index 000000000..333c68682 --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-luametafun.tex @@ -0,0 +1,362 @@ +% language=us + +\usemodule[present-boring,abbreviations-logos] + +\setuptolerance[verytolerant,stretch] + +\startdocument + [title={LUAMETAFUN}, + banner={the new interfaces}, + location={context\enspace {\bf 2021}\enspace meeting}] + +\starttitle[title=Three subsystems] + +\startitemize + +\startitem + The core of the engine in still \TEX. It all starts there. +\stopitem + +\startitem + The \TEX\ internals are opened up via \LUA. We can call out to \LUA\ from + \TEX\ and to some extend from \LUA\ to \TEX. Quite often we push back + something via the input channels. +\stopitem + +\startitem + The \METAPOST\ library is accessed from \LUA. So from within \TEX\ there is + always \LUA\ in between. Results go back via \LUA. The library can also call + out to \LUA\ and from there to \TEX. +\stopitem + +\startitem + This means that all three major components of \LUAMETATEX\ can talk to each + other and use each others capabilities. +\stopitem + +\startitem + With \LUA\ in the center, we also have access to other mechanism, for instance + fonts, graphics and libraries. +\stopitem + +\startitem + In \CONTEXT\ the \LUA\ language also permits using \XML, \JSON, \CSV, \SQL\ + databases and other input that can be dealt with programmatically. +\stopitem + +\startitem + All has been reasonably optimized for efficiency and performance. +\stopitem + +\stopitemize + +\stoptitle + +\starttitle[title=The (\LUATEX) library] + +\startitemize + +\startitem + Turning \METAPOST\ into a library has been a subproject of the \LUATEX\ project. The + semi|-|official team (Taco, Jacko, Hans, Luigi) got John Hobbies blessing. +\stopitem + +\startitem + This was a rather massive (and impressive) operation by Taco because multiple + number models were to be supported and the internals had to be made such that + different backends were possible. All with remaining perfect (DEK) compatibility. +\stopitem + +\startitem + The \METAPOST\ library serves both the stand alone program and \LUATEX. +\stopitem + +\startitem + That means the \POSTSCRIPT\ backend is built in plus some basic (\TYPEONE) + font handling. We support \PDF\ output via the \METAPOST\ \LUA\ backend (in + \MKII\ that is done by parsing the \POSTSCRIPT\ and specials). +\stopitem + +\startitem + In addition there is \PNG\ and \SVG\ output. It helps that \METAPOST\ output is + rather simple. +\stopitem + +\startitem + The \LUATEX\ engine uses the \LUA\ backend which represents the result in \LUA\ tables + resembling the \METAPOST\ internal representation. +\stopitem + +\startitem + The library supports scaled and double (internal) but also binary and decimal + number models that use (linked in) libraries. +\stopitem + +% this would force a page +% +% \startitem +% The library is quite stable and Taco transferred maintenance to Luigi. +% \stopitem + +\stopitemize + +\stoptitle + +\starttitle[title=The (\LUAMETATEX) library] + +\startitemize + +\startitem + We don't need the \POSTSCRIPT\ backend (which only does \TYPEONE\ anyway). +\stopitem + +\startitem + We also have no use for \SVG\ and \PNG\ output. +\stopitem + +\startitem + The binary number model has no advantages over the decimal one but brings + quite some dependency with it (library code). +\stopitem + +\startitem + The \TYPEONE\ font support is not used in \CONTEXT\ because we handle text + differently. +\stopitem + +\startitem + All this means that we can do with a smaller (simplified) \METAPOST\ library. +\stopitem + +\startitem + The codebase has been overhauled. We still have \type {.w} files (\CWEB) but + use a \LUA\ script to convert that to \CCODE\ which means that we have better + control over how it comes out. +\stopitem + +\startitem + As with \LUATEX\ the file \IO, message handling etc.\ now largely goes via \LUA; + it is more integrated. +\stopitem + +\startitem + The same is true for scanning interfaces and return values (injectors). That also + made for more symbolic coding. +\stopitem + +\startitem + Memory management (allocation) is under engine control (as with \TEX\ and \LUA); + we use a common high performance allocator library. +\stopitem + +\stopitemize + +\stoptitle + +\starttitle[title=The (\LUAMETATEX) library] + +\startitemize + +\startitem + Some already present mechanism have been extended, for instance clips have + pre- and postscripts. +\stopitem + +\startitem + A grouping wrapper has been added (handy for some graphic trickery supported in the + backend.) +\stopitem + +\startitem + The \type {runscript} primitive supports symbolic references to functions (of course to + be provided at the \LUA\ end). +\stopitem + +\startitem + The \type {runscript} return values can be more native, in addition to the already + present (default) \type {scantokens} support. +\stopitem + +\startitem + Internals are extended with booleans and strings. +\stopitem + +\startitem + Output (paths, clips etc) can be stacked in a different order. +\stopitem + +\startitem + There are additional statistics available. +\stopitem + +\startitem + In some places performance could be improved. +\stopitem + +\startitem + In the meantime it can be considered a major upgrade and (for various + reasons) backporting to \LUATEX\ makes no sense. And yes, all errors are + mine. +\stopitem + +\stopitemize + +\starttitle[title=The \LUA fication] + + {\em See Taco's presentation where he gives some examples.} + +\stoptitle + +\starttitle[title=Callbacks] + +We need to hook in some functions: \blank[2*big] + +\starttabulate[|T|T|T|p|] + \HL + \NC \NC find_file \NC (name,mode,kind) \NC locate a file (usually within the \TDS\ setup) \NC \NR + \NC f \NC open_file \NC (name,mode,kind) \NC open given file \NC \NR + \NC \NC close_file \NC (handle) \NC close opened file \NC \NR + \NC s \NC read_file \NC (handle,size) \NC read from file \NC \NR + \NC \NC write_file \NC (handle,str) \NC write to file \NC \NR + \HL + \NC s \NC run_script \NC (code,size,index) \NC run the given string as \LUA\ script \NC \NR + \NC s \NC make_text \NC (str,size,mode) \NC process btex/etex \NC \NR + \HL + \NC \NC run_internal \NC (action,index,kind,name) \NC act on internal definition \NC \NR + \HL + \NC n \NC run_overload \NC (property,name,mode) \NC process overload check \NC \NR + \HL + \NC \NC run_logger \NC (target,str,size) \NC process log message \NC \NR + \NC \NC run_error \NC (message,help,interaction) \NC handle error (message) \NC \NR + \NC \NC run_warning \NC (message) \NC handle warning \NC \NR + \HL +\stoptabulate + +% check_overload shipout_backend + +\stoptitle + +\starttitle[title=Two calling methods] + +The runner can be called as: + +\starttyping +runscript("mp.MyFunction()") +\stoptyping + +which implies at the \LUA\ end: + +\starttyping +function mp.MyFunction() + ... +end +\stoptyping + +Here the callback function is responsible for loading the string and executing it. + +Alternatively one can say: + +\starttyping +runscript <number> +\stoptyping + +The number can be intercepted at the \LUA\ end to do some associated action. + +\stoptitle + +\starttitle[title=Variables] + +We can do: + +\starttyping +lua.mp.MyFunction("foo",123,true) +\stoptyping + +which in the end is equivalent to: + +\starttyping +runscript("mp.MyFunction('foo',123,true)") +\stoptyping + +Alternatively one can pick up values by scanning: like \type {scannext}, \type +{scanexpression}, \type {scantoken}, \type {scansymbol}, \type {scannumeric}, +\type {scaninteger}, \type {scanboolean}, \type {scanstring}, \type {scanpair}, +\type {scancolor}, \type {scancmykcolor}, \type {scantransform}, \type +{scanpath}, \type {scanpen}, etc. + +\stoptitle + +\starttitle[title=Return values] + +The runner can return: + +\startitemize + +\startitem + a string that gets fed into the \type {scantokens} primitive +\stopitem + +\startitem + a numeric or boolean that gets injected as native \METAPOST\ object +\stopitem + +\startitem + a table that gets concatenated and fed into the \type {scantokens} primitive +\stopitem + +\startitem + \type {true} and a second argument that gets converted into a native \METAPOST\ object +\stopitem + +\startitem + in the last case the number of table elements determines the object +\stopitem + +\stopitemize + +Instead of returning a value one can inject: \type {injectnumeric}, \type +{injectinteger}, \type {injectboolean}, \type {injectstring}, \type {injectpair}, +\type {injectcolor}, \type {injectcmykcolor}, \type {injecttransform}, \type +{injectpath}, \type {injectwhatever}, etc.\ and these accept one or more values +and|/|or tables. + +These mechanisms might evolve a bit over time. Lots of examples can be found in +the \type {mlib-*.lmt} files. + +\stoptitle + +\starttitle[title=Parameters] + +\startitemize + \startitem + The new interfaces permit us to program quite robust parameter driven + interfaces that (sort of) match the way we do things at the \TEX\ end. + \stopitem + \startitem + The distribution has several examples of usage and more will be added. + \stopitem + \startitem + Macros that use the new mechanisms can be recognized by the \type {lmt_} + prefix. + \stopitem +\stopitemize + +\starttyping +lmt_mytrick [ + somestring = "test", + somenumeric = 123, + someboolean = true, + somecolor = (1, 0, 1), + somepath = fullsquare scaled 10cm, + somelist = { (0, 0), (1, 3), (8, 9) }, + sometable = [ + somenumeric = 321, + ], +] ; +\stoptyping + +{\em Show the pattern of defining these at the \LUA\ end and in \METAPOST\ files.} + +\stoptitle + +\stopdocument diff --git a/doc/context/presentations/context/2021/context-2021-math.pdf b/doc/context/presentations/context/2021/context-2021-math.pdf Binary files differnew file mode 100644 index 000000000..dd343899b --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-math.pdf diff --git a/doc/context/presentations/context/2021/context-2021-math.tex b/doc/context/presentations/context/2021/context-2021-math.tex new file mode 100644 index 000000000..fce4ca4d5 --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-math.tex @@ -0,0 +1,236 @@ +% language=us + +\usemodule[present-boring,abbreviations-logos] + +\definecolor[maincolor] [r=.4] +\definecolor[extracolor][s=.1] + +\startdocument + [title={MATH}, + banner={complete control}, + location={context\enspace {\bf 2021}\enspace meeting}] + +\starttitle[title=The benchmark] + +\startitemize + +\startitem + Traditional \TEX\ has set the benchmark for typesetting math. +\stopitem + +\startitem + It uses three categories of fonts: alphabet, symbol, extensible. +\stopitem + +\startitem + Limitations in fonts have lead to some curious handling of dimensions. +\stopitem + +\startitem + The fact that there is \type {\over} makes for multipass scanning and + processing. +\stopitem + +\startitem + The last family counts (when entering the second pass) so one has to use very + controlled font switching. +\stopitem + +\startitem + Some features (limits, integrals, primes) rely on special macros and parsing. +\stopitem + +\startitem + One can end up in tricky font switching. Also, quite some fonts are loaded in + order to set up the machinery. +\stopitem + +\stopitemize + +\stoptitle + +\starttitle[title=Todays reality] + +\startitemize + +\startitem + We now use \OPENTYPE\ math fonts. In fact, in \MKIV\ we always used only one + font instance (unless we mix font families). +\stopitem + +\startitem + We operate in the \UNICODE\ domain, so in the end there was no need to bump the + number of families in \LUATEX. +\stopitem + +\startitem + In \CONTEXT\ we use only a few font families: regular, regular r2l, bold, bold r2l. +\stopitem + +\startitem + All the tricky stuff is done with the help of \LUA. This already started very + early in the \LUATEX\ project and has not changed. +\stopitem + +\startitem + But we still run into issues because the available fonts are inconsistent, + incompatible, have issues and that is unlikely to change. +\stopitem + +\startitem + We have a virtual font system in place that was used during the transition (when no + fonts were available). +\stopitem + +\startitem + In \CONTEXT\ there are various ways to deal with shortcomings in or extensions to fonts. +\stopitem + +\startitem + At some point we need to make up our minds, accept the issues with fonts, and + just fix things runtime. +\stopitem + +\stopitemize + +\stoptitle + +\starttitle[title=The engine] + +\startitemize + +\startitem + In \LUATEX\ we started with a hybrid approach but eventually ended up with + more split code paths. +\stopitem + +\startitem + In \LUAMETATEX\ nearly all aspects of the engine had made configurable and are under + user control. +\stopitem + +\startitem + This permits experiments where we can apply old methods onto new fonts. +\stopitem + +\stopitemize + +\stoptitle + +\starttitle[title=The tricky things] + +\startitemize + +\startitem + Spacing in traditional math is a combination of widths and italic correction: + we need to add them and sometimes afterwards subtract the italic correction. + In \OPENTYPE\ math we don't lie about dimensions and apply italic correction + selectively (as we have staircase kerns). +\stopitem + +\startitem + Special symbols like primes are quite inconsistent wrt dimensions and positioning + and we need to catch that. We also need robust ways to collapse them to the proper + \UNICODE\ symbol. +\stopitem + +\startitem + Larger variants (these \type {\bigg} things) needs some attention too. +\stopitem + +\startitem + We need to map characters onto the right shaped (alphabets) because hardly + anyone will enter the \UNICODE\ math characters directly. +\stopitem + +\startitem + We might want to fix scripts, italics, kern pairs etc. +\stopitem + +\stopitemize + +\stoptitle + +\starttitle[title=Implementation] + +Among the new features (introduced over a period of time) are: + +\startitemize + +\startitem + All kind of inter|-|class spacing parameters (in addition to what \LUATEX\ + already provides). +\stopitem + +\startitem + Dozens of \OPENTYPE\ font related spacing parameters (more that traditional + \TEX). +\stopitem + +\startitem + Opened up additional font (and taste) related parameters (hard coded in + traditional \TEX). +\stopitem + +\startitem + Opened up style related parameters (hard coded in traditional \TEX). +\stopitem + +\startitem + Control codes that make the engine follow a different (the traditional) code + path (which can come in handy when testing or writing manuals). +\stopitem + +\startitem + The ability to simplify the result a bit wrt characters (the engine loves to + box a lot). +\stopitem + +\startitem + Efficient font, style and parameter scaling by reusing fonts and thereby + limiting the number of instances. +\stopitem + +\startitem + More options to special (constructed) symbols. +\stopitem + +\startitem + Let math related nodes carry around more control properties and states. +\stopitem + +\startitem + Support of local changes to math parameters. +\stopitem + +\stopitemize + +\stoptitle + +\stopdocument + +% mml +% vf chars for extensibles + +% noads.handlers.showtree +% noads.handlers.unscript +% noads.handlers.unstack +% noads.handlers.variants +% noads.handlers.relocate +% noads.handlers.families +% noads.handlers.render +% noads.handlers.collapse +% noads.handlers.fixscripts +% noads.handlers.domains +% noads.handlers.autofences +% noads.handlers.resize +% noads.handlers.respace +% noads.handlers.alternates +% noads.handlers.tags +% noads.handlers.italics +% noads.handlers.kernpairs +% noads.handlers.classes + +% builders.kernel.mlisttohlist +% noads.handlers.makeup +% noads.handlers.align diff --git a/doc/context/presentations/context/2021/context-2021-overloadprotection.pdf b/doc/context/presentations/context/2021/context-2021-overloadprotection.pdf Binary files differnew file mode 100644 index 000000000..c6335f32d --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-overloadprotection.pdf diff --git a/doc/context/presentations/context/2021/context-2021-overloadprotection.tex b/doc/context/presentations/context/2021/context-2021-overloadprotection.tex new file mode 100644 index 000000000..5143e855a --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-overloadprotection.tex @@ -0,0 +1,298 @@ +% language=us + +\usemodule[present-boring,abbreviations-logos] + + +\definehighlight[nb][style=bold,color=middlegray,define=no] + +\setuptolerance[verytolerant,stretch] + +\definecolor[maincolor][r=.3,b=.3] + +\startdocument + [title={OVERLOAD PROTECTION}, + banner={the downside of macros}, + location={context\enspace {\bf 2021}\enspace meeting}] + +% TEX MP + +\starttitle[title=Primitives] + +A \TEX\ engine comes with a whole set of primitive operations for: + +\startitemize[packed] + \startitem + accessing internal variables + \stopitem + \startitem + defining macros + \stopitem + \startitem + controlling expansion + \stopitem + \startitem + constructing boxes + \stopitem + \startitem + finalizing pages + \stopitem + \startitem + defining characters (text and math) + \stopitem + \startitem + inserting kerns, glue and penalties + \stopitem + \startitem + defining fonts + \stopitem + \startitem + dealing with languages (hyphenation) + \stopitem + \startitem + testing properties + \stopitem + \startitem + manipulating tokens + \stopitem + \startitem + managing inserts + \stopitem + \startitem + handling marks + \stopitem + \startitem + grouping + \stopitem + \startitem + mathematics + \stopitem + \startitem + tracing + \stopitem +\stopitemize + +\stoptitle + +\starttitle[title=Macros] + +\startitemize + \startitem + Macros are commands defined by the user and/or a macro package. + \stopitem + \startitem + They can overload a primitive which then can confuse the whole machinery. + \stopitem + \startitem + A macro package can alias primitives for instance \type {\relax} can be + replaced by \type {\foo_relax} after \typ {\let \foo_relax \relax}. + \stopitem + \startitem + That only when (at definition time) the \type {_} is a letter. By using such + a character some protection against overload is provided. + \stopitem + \startitem + In \CONTEXT\ we often use(d) aliases like \type {\normalrelax} but of course + these can also be overloaded. + \stopitem + \startitem + We only overload a very few primitives, for instance \type {\language}. + \stopitem + \startitem + Users who overload primitives are \quote {on their own} and \quote {without + support}. + \stopitem + \startitem + An easy way to protect yourself is using \type {\CamelCase} names. + \stopitem +\stopitemize + +\stoptitle + +\starttitle[title=Overload protection] + +\startitemize + \startitem + The \LUAMETATEX\ engine has overload protection built in for the \TEX\ engine + as well as provides means to do that for \METAPOST. + \stopitem + \startitem + In \LMTX\ all commands have been tagged accordingly (which was quite some work). + \stopitem + \startitem + Processing \type {s-system-macros.mkxl} gives an overview. + \stopitem + \startitem + Overload protection is off by default but can be turned on: + +\starttyping +\enabledirectives[overloadmode=warning] +\enabledirectives[overloadmode=error] +\stoptyping + + \stopitem + \startitem + I myself always run with the error variant and make sure that the manuals obey + the rules. + \stopitem + \startitem + Modules and/or styles (and in a few cases the core code) can cheat and use: + +\starttyping +\pushoverloadmode + ........................ + ........................ +\popoverloadmode +\stoptyping + + \stopitem +\stopitemize + +\stoptitle + +\starttitle[title=Details] + +\startitemize + \startitem + Traditional \TEX\ has a few so called prefixes: \type {\global}, \type {\outer}, + \type {\long}, and type \type {\immediate}. + \stopitem + \startitem + The \ETEX\ engine adds \type {\protected} (because we already had that in + \CONTEXT\ we use what we (also already) had: \type {\unexpanded}). + \stopitem + \startitem + In \LUATEX\ we can force macros to be always long, something that we do + in \MKIV\ (as in \MKII). + \stopitem + \startitem + In \LUAMETATEX\ the \type {\outer} and \type {\long} prefixes have been + dropped (they are ignored). + \stopitem + \startitem + In \LUAMETATEX\ the \type {\protected} prefix acts like in other engines + but protection is implemented more naturally. + \stopitem + \startitem + In addition \LUAMETATEX\ has new prefixes: \type {\frozen}, \type + {\permanent}, \type {\immutable}, \type {\mutable}, \type {\noaligned}, + \type {\instance}, \type {\untraced}, \type {\tolerant}, \type + {\overloaded}, \type {\aliased}, \type {\immediate} and an experimental + \type {\semiprotected}, + \stopitem + \startitem + Some prefixes end up as properties of macros, some influence scanning (for instance + in alignments and when calling \LUA\ functions). There is no noticeable runtime + overhead. + \stopitem + \startitem + The \type {\meaningfull} primitive is like \type {\meaning} but also reports + properties set by prefixes; there is also \type {\meaningless}. + \stopitem +\stopitemize + +\stoptitle + +\starttitle[title=Prefixes] + +Regular definitions: + +\startitemize + \startitem + \type {\global}: defines a macro or sets a register value out of scope. + \stopitem + \startitem + \type {\outer}: is used to issue a warning when a macro defined as such was + used nested (just ignored in \LUAMETATEX). + \stopitem + \startitem + \type {\long}: triggers a warning when an argument of a macro contains a + \type {\par} equivalent (just ignored in \LUAMETATEX). + \stopitem + \startitem + \type {\protected}: makes a macro unexpandable (being itself) in an \type + {\edef} equivalent situation (where it can get out of hands). + \stopitem + \startitem + \type {\semiprotected}: is like \type {\protected} but the property is + ignored when \type {\semiexpanded} is applied. + \stopitem +\stopitemize + +Special case: + +\startitemize + \startitem + \type {\immediate}: tells a backend primitive to come into action + immediately instead of creating a delayed action (via a whatsit node). In + \LUAMETATEX\ we have no built|-|in backend so there is signals a \LUA\ + interface function this property. + \stopitem +\stopitemize + +\page + +Scanning related: + +\startitemize + \startitem + \type {\noaligned}: tags a macro as valid peek ahead macro when scanning + alignments; normally that only makes sense for \type {\protected} macros. + This permits cleaner macros in for instance table mechanisms (no + unexpected expansion side effects). + \stopitem + \startitem + \type {\untraced}: this flag makes a macro report itself as primitive in + traces which is sometimes nicer that presenting a user with some + confusing meaning. + \stopitem + \startitem + \type {\tolerant}: a prefix that makes the macro argument parser accept + all kind of new argument parsing options and continue when delimited + arguments fail. This makes macros with optional arguments produce less + noise when they are traced but more important, it makes for cleaner low + level interfaces. + \stopitem +\stopitemize + + +\page + +Overload protection (primitives are protected against overloads by default): + +\startitemize + \startitem + \type {\aliased}: create a reference (using \type {\let}) that also inherits the + properties. + \stopitem + \startitem + \type {\permanent}: sets a flag that makes a macro (or definition) immune for + redefinition. + \stopitem + \startitem + \type {\frozen}: prevents overloading but one can bypass that with some + more effort. + \stopitem + \startitem + \type {\immutable}: makes a (normally variable definition) fixed, for instance + constants. + \stopitem + \startitem + \type {\mutable}: a flag showing that this macro or definition can be used for + anything (so the macro package also has to assume that). + \stopitem + \startitem + \type {\instance}: just a flag that can be used to signal that a macro (or definition) + is an instance of a more general concept. + \stopitem + \startitem + \type {\overloaded}: bypass a frozen flag (property). + \stopitem +\stopitemize + +\blank[2*big] + +{\em Show some examples in the source code and editor.} + +\stoptitle + +\stopdocument diff --git a/doc/context/presentations/context/2021/context-2021-paragraphs.pdf b/doc/context/presentations/context/2021/context-2021-paragraphs.pdf Binary files differnew file mode 100644 index 000000000..9b32a6576 --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-paragraphs.pdf diff --git a/doc/context/presentations/context/2021/context-2021-paragraphs.tex b/doc/context/presentations/context/2021/context-2021-paragraphs.tex new file mode 100644 index 000000000..7a4e566d8 --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-paragraphs.tex @@ -0,0 +1,162 @@ +% language=us + +\usemodule[present-boring,abbreviations-logos] + +\definecolor[maincolor] [r=.4,g=.4] +\definecolor[extracolor][s=.1] + +\startdocument + [title={PARAGRAPHS}, + banner={a bit of an upgrade}, + location={context\enspace {\bf 2021}\enspace meeting}] + +\starttitle[title=Note] + +\startitemize + +\startitem + Some of the following already is present for a while and has been discussed + at previous meetings. +\stopitem + +\startitem + But \unknown\ occasionally some minor tweak gets added so consider this to be + an update. +\stopitem + +\stopitemize + +\starttitle[title=Spacing] + +\startitemize + +\startitem + Spaces in \TEX\ become glue nodes (with optional stretch and shrink). +\stopitem + +\startitem + In traditional \TEX\ these glue nodes are ref counted copies of the current + spacing related variables. +\stopitem + +\startitem + In \LUATEX\ we make real copies so that when we mess with the node list + changes to glue don't affect other instances. +\stopitem + +\stopitemize + +\starttitle[title=Parameters] + +\startitemize + +\startitem + In traditional \TEX\ the paragraphs bound properties that are in effect + when \type {\par} happens are used when breaking into lines. +\stopitem + +\startitem + In \LUAMETATEX\ the paragraphs bound properties are stored with the + paragraph and can be frozen when they are set. +\stopitem + +\startitem + This gives a more predictable (and robust) way of manipulating a + paragraph. +\stopitem + +\startitem + We can for instance get rid of grouping side effects that interfere with + \type {\everypar}. +\stopitem + +\startitem + Currently three dozen parameters are tracked but they are grouped in + categories. +\stopitem + +\blank[2*big] {\em (show code and examples)} + +\stopitemize + +\stoptitle + +\starttitle[title=Wrapping] + +\startitemize + +\startitem + Doing something in front of a paragraph is taken care of by good old + \type {\everypar}. +\stopitem + +\startitem + In \LUAMETATEX\ we also have \type {\everybeforepar} but so far in \CONTEXT\ + we haven't used that. +\stopitem + +\startitem + Adding something to the end of a paragraph can be tricky so we have + a wrapper mechanism: \type {\wrapuppar}. +\stopitem + +\startitem + The \type {\wrapuppar} primitive is similar to \type {\atendofgroup} in the + sense that it accumulates tokens (so no \type {\endofpar}). +\stopitem + +\startitem + Normally these primitives are not used directly but managed by a more general + system of handling paragraphs. +\stopitem + +\stopitemize + +\blank[2*big] {\em (show code and examples)} + +\stoptitle + +\starttitle[title=Normalizing] + +\startitemize + +\startitem + In order to see consistent paragraphs at the \LUA\ end in \LUAMETATEX\ + we can normalize the lines that come from the paragraph builder. +\stopitem + +\startitem + Normalization results in: + + \startitemize + \startitem + the first line having: indent skip + \stopitem + \startitem + each line having: left hang, left skip, right skip, right hang + \stopitem + \startitem + the last line having: left parfill skip, right parfill skip + \stopitem + \stopitemize +\stopitem + +\startitem + It is controlled by \type {\normalizelinemode} which has additional flags for + swapping hanging indentation and par shapes, breaking after dir nodes, + removing margin kerns and clipping the line width. +\stopitem + +\startitem + The clipping options avoids the side effects of \TEX\ using shifts which has + the side effect of unreal dimensions. This is one of the tricks|/|properties + of the traditional engine that is perfectly fine until we open up things. +\stopitem + +\stopitemize + +\blank[2*big] {\em (show code and examples)} + +\stoptitle + +\stopdocument diff --git a/doc/context/presentations/context/2021/context-2021-programming.pdf b/doc/context/presentations/context/2021/context-2021-programming.pdf Binary files differnew file mode 100644 index 000000000..86f61bed9 --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-programming.pdf diff --git a/doc/context/presentations/context/2021/context-2021-programming.tex b/doc/context/presentations/context/2021/context-2021-programming.tex new file mode 100644 index 000000000..907a9ebfa --- /dev/null +++ b/doc/context/presentations/context/2021/context-2021-programming.tex @@ -0,0 +1,325 @@ +% language=us + +\usemodule[present-boring,abbreviations-logos] + +\definehighlight[nb][style=bold,color=middlegray,define=no] + +\startdocument + [title={PROGRAMMING}, + banner={the way \CONTEXT\ is set up}, + location={context\enspace {\bf 2021}\enspace meeting}] + +\starttitle[title=Levels] + +When you look at \CONTEXT\ bottom|--|up (engine|--|interface) you will notice: \blank[2*big] + +\startnarrower + +\startitemize[n] + +\startitem + \highlight[nb]{primitives:} this is what the engine comes with +\stopitem +\startitem + \highlight[nb]{infrastructure:} basic management of data structures +\stopitem +\startitem + \highlight[nb]{helpers:} macros that hide complexity +\stopitem +\startitem + \highlight[nb]{subsystems:} collections of macros that implement functionality +\stopitem +\startitem + \highlight[nb]{mechanisms:} these combine various subsystems +\stopitem +\startitem + \highlight[nb]{modules:} extra functionality (uses 1--5) +\stopitem +\startitem + \highlight[nb]{styles:} handling sources and layout (uses 4--6) +\stopitem + +\stopitemize + +\stopnarrower + +\blank[2*big] Users normally see \CONTEXT\ top|--|down (usage|--|hacking). + +\stoptitle + +% % % + +\starttitle[title=Styles] + +\startitemize + +\startitem + These are prebuilt solutions for common as well as rare situations. +\stopitem + +\startitem + The system comes with some styles: the \type {s-*} files. +\stopitem + +\startitem + Right from the start the idea was that you get some reasonable default. +\stopitem + +\startitem + And if you want more you stepwise define your style as you go. +\stopitem + +\startitem + It really is part of the game: exploration. +\stopitem + +\startitem + Solving your problem is a nice challenge. +\stopitem + +\startitem + If you want a completely predefined setup, shop somewhere else. +\stopitem + +\stopitemize + +\stoptitle + +% % % + +\starttitle[title=Modules] + +\startitemize + +\startitem + A lot of functionality is built in. +\stopitem + +\startitem + This helps to keep the system consistent. +\stopitem + +\startitem + We could cheat and ship thousands of few|--|liner styles but don't do that. +\stopitem + +\startitem + There are a few mechanisms that don't really fit into the core, so these are + implemented as modules that do fit into the interface: the \type {m-*} and + \type {x-*} files. +\stopitem + +\startitem + Users can build and share their solutions which has resulted in some + third party modules: the \type {t-*} files. +\stopitem + +\startitem + For (a few, often old) private files I use \type {p-*} name scheme. +\stopitem + +\stopitemize + +\stoptitle + +% % % + +\starttitle[title=Mechanisms] + +\startitemize + +\startitem + These are combinations of subsystems but often they cannot really be distinguished. +\stopitem + +\startitem + Examples are notes, that combine notations, lists, references, descriptions etc. +\stopitem + +\stopitemize + +\stoptitle + +% % % + +\starttitle[title=Subsystems] + +\startitemize + +\startitem + This is what users see and can configure +\stopitem + +\startitem + Most are (conceptually) rather old but evolved over time. There are no + fundamental differences between \MKIV\ and \LMTX, but the later is hopefully + a bit cleaner. +\stopitem + +\startitem + Examples are fonts, languages, color, structure (sectioning, lists, + constructions, itemgroups, references), spacing, graphics, bibliographies, + positioning, numbering and layout. +\stopitem + +\startitem + More hidden are the backend, export and \XML\ interfaces. +\stopitem + +\startitem + Some have subsystems themselves, like widgets that relate to a specific + backend. +\stopitem + +\startitem + There are often dependencies between subsystems which makes that it's not really a + hierarchy. A more strict separation would demand much more overhead. +\stopitem + +\stopitemize + +\stoptitle + +% % % + +\starttitle[title=Helpers] + +\startitemize + +\startitem + These provide basic programming help. +\stopitem + +\startitem + Examples are macros for comparing things, loops, list processing, + argument handling. +\stopitem + +\startitem + But more abstract box manipulations also fits in here. +\stopitem + +\startitem + Some subsystems, like \XML\ and bibliographies provide more specific low + level helpers. +\stopitem + +\stopitemize + +\stoptitle + +% % % + +\starttitle[title=Infrastructure] + +\startitemize + +\startitem + The engine provides counters, dimension and other registers that need + to be managed in order to avoid clashes in usage. +\stopitem + +\startitem + Many of the helpers, subsystems and mechanisms fall back on common rather + low level functions (\LUA) and macros (using primitives). +\stopitem + +\stopitemize + +\stoptitle + +% % % + +\starttitle[title=Primitives] + +\startitemize + +\startitem + This is what the engine provides: the built|-|in commands and features. +\stopitem + +\startitem + In addition to the visible primitives there are \LUA\ interfaces and these + permit adding extra primitives. +\stopitem + +\startitem + In \LUAMETATEX\ we have the core \TEX\ set but a few were dropped because we + don't have a backend and a different \IO\ subsystem (so they have to be + emulated). +\stopitem + +\startitem + We also have some of the \ETEX\ primitives and very few of the \PDFTEX\ ones + but I now consider for instance expansion and protrusion extensions to be + kind of \ETEX. +\stopitem + +\startitem + There are additional \LUATEX\ primitives but some were dropped, again because + of the backend, so we emulate some, and also because some were experimental. +\stopitem + +\startitem + There are quite some new primitives and existing mechanisms have been extended, + cleaned up and (hopefully) improved. +\stopitem + +\stopitemize + +% % % + +\starttitle[title=The shift] + +\startitemize + +\startitem + There have always been complaints about \TEX\ as a language (what makes me + wonder why those who complain use it.) +\stopitem + +\startitem + Although there are some extensions to the language in \ETEX, follow|-|ups have + not really succeeded in this area. +\stopitem + +\startitem + At some point I decided that code in the categories 1|--|4 cold benefit from + extensions. +\stopitem + +\startitem + That also meant that we use less of the low helpers. It makes the code look a + bit more \TEX. +\stopitem + +\startitem + It also means less clutter, in code as well in tracing. Often the code + becomes simpler too. +\stopitem + +\startitem + The idea is that \TEX\ becomes a bit more a programming language. +\stopitem + +\startitem + Of course it takes away the \quotation {Watch me, I can do real dirty \TEX\ + hacking!} brawling. +\stopitem + +\startitem + It also can take away some of the complaints. +\stopitem + +\startitem + And it definitely adds some fun. +\stopitem + +\stopitemize + +{\em During the week we show some of the implementation (in Visual Studio) and +examples of applications. We also write a small extension (the {\tttf dk} unit)} + +\stoptitle + +\stopdocument |