From 4f7f67101a808c6b6c89d64ad5ee1f1701d8f632 Mon Sep 17 00:00:00 2001 From: Hans Hagen Date: Sat, 27 Feb 2021 20:17:05 +0100 Subject: 2021-02-27 19:30:00 --- .../general/manuals/lowlevel/lowlevel-boxes.tex | 10 +- .../manuals/lowlevel/lowlevel-paragraphs.tex | 423 +++++++++++++++++++++ .../general/manuals/metafun/metafun-basics.tex | 66 +++- .../general/manuals/metafun/metafun-debugging.tex | 163 +++++++- .../general/manuals/metafun/metafun-examples.tex | 253 ++++++++++++ .../general/manuals/metafun/metafun-lua.tex | 276 ++++++++++++++ .../general/manuals/metafun/metafun-macros.tex | 13 +- 7 files changed, 1188 insertions(+), 16 deletions(-) create mode 100644 doc/context/sources/general/manuals/lowlevel/lowlevel-paragraphs.tex (limited to 'doc/context/sources') diff --git a/doc/context/sources/general/manuals/lowlevel/lowlevel-boxes.tex b/doc/context/sources/general/manuals/lowlevel/lowlevel-boxes.tex index 9de79c5ee..9a097b878 100644 --- a/doc/context/sources/general/manuals/lowlevel/lowlevel-boxes.tex +++ b/doc/context/sources/general/manuals/lowlevel/lowlevel-boxes.tex @@ -9,9 +9,7 @@ [title=boxes, color=middlered] -\startsection[title=Preamble] - -\startsubsection[title=Introduction] +\startsection[title=Introduction] An average \CONTEXT\ user will not use the low level box primitives but a basic understanding of how \TEX\ works doesn't hurt. In fact, occasionally using a box @@ -25,9 +23,9 @@ about all kind of glues, kerns and penalties, just boxes it is. This explanation will be extended when I feel the need (or users have questions that can be answered here). -\stopsubsection +\stopsection -\startsubsection[title=Boxes] +\startsection[title=Boxes] This paragraph of text is made from lines that contain words that themselves contain symbolic representations of characters. Each line is wrapped in a so @@ -68,7 +66,7 @@ other hand wraps a linked list of so called nodes: glyphs, kerns, glue, penalties, rules, boxes, etc. It is a container with properties like width, height, depth and shift. -\stopsubsection +\stopsection \stopsection diff --git a/doc/context/sources/general/manuals/lowlevel/lowlevel-paragraphs.tex b/doc/context/sources/general/manuals/lowlevel/lowlevel-paragraphs.tex new file mode 100644 index 000000000..a7a7fddf3 --- /dev/null +++ b/doc/context/sources/general/manuals/lowlevel/lowlevel-paragraphs.tex @@ -0,0 +1,423 @@ +% language=us + +\environment lowlevel-style + +\startdocument + [title=paragraphs, + color=middlecyan] + +\startsection[title=Introduction] + +This manual is mostly discussing a few wrappers around low level \TEX\ features. +Its writing is triggered by an update to the \METAFUN\ and \LUAMETAFUN\ manuals +where we mess a bit with shapes. It gave a good reason to also cover some more +paragraph related topics but it might take a while to complete. Remind me if you +feel that takes too much time. + +\stopsection + +\startsection[title=Properties] + +A paragraph is just a collection of lines that result from one input line that +got broken. This process of breaking into lines is influenced by quite some +parameters. In traditional \TEX\ and also in \LUAMETATEX\ by default the values +that are in effect when the end of the paragraph is met are used. So, when you +change them in a group and then ends the paragraph after the group, the values +you've set in the group are not used. + +However, in \LUAMETATEX\ we can optionally store them with the paragraph. When +that happens the values current at the start are frozen. You can still overload +them but that has to be done explicitly then. The advantage is that grouping no +longer interferes with the line break algorithm. The magic primitive is \type +{\snapshotpar}. + +\starttabulate +\NC \type {\hsize} \NC \NC \NR +\NC \type {\leftskip} \NC \NC \NR +\NC \type {\rightskip} \NC \NC \NR +\NC \type {\hangindent} \NC \NC \NR +\NC \type {\hangafter} \NC \NC \NR +\NC \type {\parindent} \NC \NC \NR +\NC \type {\parfillleftskip} \NC \NC \NR +\NC \type {\parfillrightskip} \NC \NC \NR +\NC \type {\adjustspacing} \NC \NC \NR +\NC \type {\protrudechars} \NC \NC \NR +\NC \type {\pretolerance} \NC \NC \NR +\NC \type {\tolerance} \NC \NC \NR +\NC \type {\emergencystretch} \NC \NC \NR +\NC \type {\looseness} \NC \NC \NR +\NC \type {\lastlinefit} \NC \NC \NR +\NC \type {\linepenalty} \NC \NC \NR +\NC \type {\interlinepenalty} \NC \NC \NR +\NC \type {\clubpenalty} \NC \NC \NR +\NC \type {\widowpenalty} \NC \NC \NR +\NC \type {\displaywidowpenalty} \NC \NC \NR +\NC \type {\brokenpenalty} \NC \NC \NR +\NC \type {\adjdemerits} \NC \NC \NR +\NC \type {\doublehyphendemerits} \NC \NC \NR +\NC \type {\finalhyphendemerits} \NC \NC \NR +\NC \type {\parshape} \NC \NC \NR +\NC \type {\interlinepenalties} \NC \NC \NR +\NC \type {\clubpenalties} \NC \NC \NR +\NC \type {\widowpenalties} \NC \NC \NR +\NC \type {\displaywidowpenalties} \NC \NC \NR +\NC \type {\baselineskip} \NC \NC \NR +\NC \type {\lineskip} \NC \NC \NR +\NC \type {\lineskiplimit} \NC \NC \NR +\NC \type {\adjustspacingstep} \NC \NC \NR +\NC \type {\adjustspacingshrink} \NC \NC \NR +\NC \type {\adjustspacingstretch} \NC \NC \NR +\NC \type {\hyphenationmode} \NC \NC \NR +\stoptabulate + +There are more paragraph related parameters than in for instance \PDFTEX\ and +\LUATEX\ and these are (to be) explained in the \LUAMETATEX\ manual. You can +imagine that keeping this around with the paragraph adds some extra overhead to +the machinery but most users won't notice that because is is compensated by gains +elsewhere. + +In \LMTX\ taking these snapshots is turned on by default and because it thereby +fundamentally influences the par builder, users can run into compatibility issues +but in practice there has been no complaints (and this feature has been in use +quite a while before this document was written). One reason for users not +noticing is that one of the big benefits is probably handled by tricks mentioned on the +mailing list. Imagine that you have this: + +\starttyping[option=TEX] +{\bf watch out:} here is some text +\stoptyping + +In this small example the result will be as expected. But what if something magic +with the start of a paragraph is done? Like this: + +\starttyping[option=TEX] +\placefigure[left]{A cow!}{\externalfigure[cow.pdf]} + +{\bf watch out:} here is some text ... of course much more is needed to + get a flow around the figure! +\stoptyping + +The figure will hang at the left side of the paragraph but it is put there when +the text starts and that happens inside the bold group. It means that the +properties we set in order to get the shape around the figure are lost as soon as +we're at \quote{\type {here is some text}} and definitely is wrong when the +paragraph ends and the par builder has to use them to get the shape right. We get +text overlapping the figure. A trick to overcome this is: + +\starttyping[option=TEX] +\dontleavehmode {\bf watch out:} here is some text ... of course much + more is needed to get a flow around the figure! +\stoptyping + +where the first macro makes sure we already start a paragraph before the group is +entered (using a \type {\strut} also works). It's not nice and I bet users have +been bitten by this and by now know the tricks. But, with snapshots such fuzzy +hacks are not needed any more! The same is true with this: + +\starttyping[option=TEX] +{\leftskip 1em some text \par} +\stoptyping + +where we had to explicitly end the paragraph inside the group in order to retain +the skip. I suppose that users normally use the high level environments so they +never had to worry about this. It's also why users probably won't notice that +this new mechanism has been active for a while. Actually, when you now change a +parameter inside the paragraph will not be applied (unless you prefix it with +\type {\frozen}) but no one did that anyway. + +{\em todo: freeze categories, overloading, turning on and off, etc} + +\stopsection + +\startsection[title=Wraping up] + +In \CONTEXT\ \LMTX\ we have a mechanism to exercise macros (or content) before a +paragraph ends. This is implemented using the \type {\wrapuppar} primitive. The +to be wrapped up material is bound to the current paragraph which in order to +get this done has to be started when this primitive is used. + +Although the high level interface has been around for a while it still needs a +bit more testing (read: use cases are needed). In the few cases where we already +use it application can be different because again it relates to snapshots. This +because in the past we had to use tricks that also influenced the user interface +of some macros (which made them less natural as one would expect). So the +question is: where do we apply it in old mechanisms and where not. + +{\em todo: accumulation, interference, where applied, limitations} + +% \vbox {vbox : \wrapuppar{1}test\par x\wrapuppar{2}test}\blank +% \vtop {vtop : \wrapuppar{1}test\par x\wrapuppar{2}test}\blank +% \vcenter{vcenter : \wrapuppar{1}test\par x\wrapuppar{2}test}\blank +% $$x = \vcenter{vcenter : \wrapuppar{1}test\par x\wrapuppar{2}test}$$\blank +% x\vadjust{vadjust : \wrapuppar{1}test\par x\wrapuppar{2}test}x\blank + +\stopsection + +\startsection[title=Shapes] + +In \CONTEXT\ we don't use \type {\parshape} a lot. It is used in for instance +side floats but even then not in all cases. It's more meant for special +applications. This means that in \MKII\ and \MKIV\ we don't have some high level +interface. However, when \METAFUN\ got upgraded to \LUAMETAFUN, and the manual +also needed an update, one of the examples in that manual that used shapes also +got done differently (read: nicer). And that triggered the arrival of a new high +level shape mechanism. + +One important property of the \type {\parshape} mechanism is that it works per +paragraph. You define a shape in terms of a left margin and width of a line. The +shape has a fixed number of such pairs and when there is more content, the last one +is used for the rest of the lines. When the paragraph is finished, the shape is +forgotten. + +{\em Not discussed here is a variant that will end up in \LUAMETATEX\ that works +with the progression, i.e.\ takes the height of the content so far into account. +This is somewhat tricky because for that to work vertical skips need to be +frozen, which is no real big deal but has to be done careful in the code.} + +The high level interface is a follow up on the example in the \METAFUN\ manual and +uses shapes that carry over to the next paragraph. In addition we can cycle over +a shape. In this interface shapes are defined using keyword. Here are some +examples: + +\starttyping[option=TEX] +\startparagraphshape[test] + left 1mm right 1mm + left 5mm right 5mm +\stopparagraphshape +\stoptyping + +This shape has only two entries so the first line will have a 1mm margin while +later lines will get 5mm margins. This translates into a \type {\parshape} like: + +\starttyping[option=TEX] +\parshape 2 + 1mm \dimexpr\hsize-1mm\relax + 5mm \dimexpr\hsize-5mm\relax +\stoptyping + +Watch the number \type {2}: it tells how many specification lines follow. As you +see, we need to calculate the width. + +\starttyping[option=TEX] +\startparagraphshape[test] + left 1mm right 1mm + left 5mm right 5mm + repeat +\stopparagraphshape +\stoptyping + +This variant will alternate between 1mm and 5mm margins. The repeating feature is +translated as follows. Maybe at some point I will introduce a few more options. + +\starttyping[option=TEX] +\parshape 2 options 1 + 1mm \dimexpr\hsize-1mm\relax + 5mm \dimexpr\hsize-5mm\relax +\stoptyping + +A shape can have some repetition, and we can save keystrokes by copying the last +entry. The resulting \type {\parshape} becomes rather long. + +\starttyping[option=TEX] +\startparagraphshape[test] + left 1mm right 1mm + left 2mm right 2mm + left 3mm right 3mm + copy 8 + left 4mm right 4mm + left 5mm right 5mm + left 5mm hsize 10cm +\stopparagraphshape +\stoptyping + +Also watch the \type {hsize} keyword: we don't calculate the hsize from the \type +{left} and \type {right} values but explicitly set it. + +\starttyping[option=TEX] +\startparagraphshape[test] + left 1mm right 1mm + right 3mm + left 5mm right 5mm + repeat +\stopparagraphshape +\stoptyping + +When a \type {right} keywords comes first the \type {left} is assumed to be zero. +In the examples that follow we will use a couple of definitions: + +\startbuffer[setup] +\startparagraphshape[test] + both 1mm both 2mm both 3mm both 4mm both 5mm both 6mm + both 7mm both 6mm both 5mm both 4mm both 3mm both 2mm +\stopparagraphshape +\stopbuffer + +\startbuffer[setup-repeat] +\startparagraphshape[test-repeat] + both 1mm both 2mm both 3mm both 4mm both 5mm both 6mm + both 7mm both 6mm both 5mm both 4mm both 3mm both 2mm + repeat +\stopparagraphshape +\stopbuffer + +\typebuffer[setup,setup-repeat][option=TEX] + +The last one could also be defines as: + +\starttyping[option=TEX] +\startparagraphshape[test-repeat] + \rawparagraphshape{test} repeat +\stopparagraphshape +\stoptyping + +In the previous code we already introduced the \type {repeat} option. This will +make the shape repeat at the engine level when the shape runs out of specified +lines. In the application of a shape definition we can specify a \type {method} +to be used and that determine if the next paragraph will start where we left off +and discard afterwards (\type {shift}) or that we move the discarded lines up +front so that we never run out of lines (\type {cycle}). It sounds complicated +but just keep in mind that \type {repeat} is part of the \type {\parshape} and +act within a paragraph while \type {shift} and \type {cycle} are applied when a +new paragraph is started. + +\startbuffer[demo] +\startshapedparagraph[list=test] + \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par} +\stopshapedparagraph +\stopbuffer + +\startbuffer[demo-repeat] +\startshapedparagraph[list=test-repeat] + \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par} +\stopshapedparagraph +\stopbuffer + +In \in {figure} [fig:shape:discard] you see the following applied: + +\typebuffer[demo,demo-repeat][option=TEX] + +\startplacefigure[title=Discarded shaping,reference=fig:shape:discard] +\startcombination[nx=2,ny=2] + {\typesetbuffer[setup,demo][page=1,width=.4\textwidth,frame=on]} {discard, finite shape, page 1} + {\typesetbuffer[setup,demo][page=2,width=.4\textwidth,frame=on]} {discard, finite shape, page 2} + {\typesetbuffer[setup-repeat,demo-repeat][page=1,width=.4\textwidth,frame=on]} {discard, repeat in shape, page 1} + {\typesetbuffer[setup-repeat,demo-repeat][page=2,width=.4\textwidth,frame=on]} {discard, repeat in shape, page 2} +\stopcombination +\stopplacefigure + +In \in {figure} [fig:shape:shift] we use this instead: + +\startbuffer[demo] +\startshapedparagraph[list=test,method=shift] + \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par} +\stopshapedparagraph +\stopbuffer + +\startbuffer[demo-shift] +\startshapedparagraph[list=test-repeat,method=shift] + \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par} +\stopshapedparagraph +\stopbuffer + +\typebuffer[demo,demo-repeat][option=TEX] + +\startplacefigure[title=Shifted shaping,,reference=fig:shape:shift] +\startcombination[nx=2,ny=2] + {\typesetbuffer[setup,demo][page=1,width=.4\textwidth,frame=on]} {shift, finite shape, page 1} + {\typesetbuffer[setup,demo][page=2,width=.4\textwidth,frame=on]} {shift, finite shape, page 2} + {\typesetbuffer[setup-repeat,demo-shift][page=1,width=.4\textwidth,frame=on]} {shift, repeat in shape, page 1} + {\typesetbuffer[setup-repeat,demo-shift][page=2,width=.4\textwidth,frame=on]} {shift, repeat in shape, page 2} +\stopcombination +\stopplacefigure + +Finally, in \in {figure} [fig:shape:cycle] we use: + +\startbuffer[demo] +\startshapedparagraph[list=test,method=cycle] + \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par} +\stopshapedparagraph +\stopbuffer + +\startbuffer[demo-cycle] +\startshapedparagraph[list=test-repeat,method=cycle] + \dorecurse{8}{\showparagraphshape\samplefile{tufte} \par} +\stopshapedparagraph +\stopbuffer + +\typebuffer[demo,demo-repeat][option=TEX] + +\startplacefigure[title=Cycled shaping,reference=fig:shape:cycle] +\startcombination[nx=2,ny=2] + {\typesetbuffer[setup,demo][page=1,width=.4\textwidth,frame=on]} {cycle, finite shape, page 1} + {\typesetbuffer[setup,demo][page=2,width=.4\textwidth,frame=on]} {cycle, finite shape, page 2} + {\typesetbuffer[setup-repeat,demo-cycle][page=1,width=.4\textwidth,frame=on]} {cycle, repeat in shape, page 1} + {\typesetbuffer[setup-repeat,demo-cycle][page=2,width=.4\textwidth,frame=on]} {cycle, repeat in shape, page 2} +\stopcombination +\stopplacefigure + +These examples are probably too small to see the details but you can run them +yourself or zoom in on the details. In the margin we show the values used. Here +is a simple example of (non) poetry. There are other environments that can be +used instead but this makes a good example anyway. + +\startbuffer +\startparagraphshape[test] + left 0em right 0em + left 1em right 0em + repeat +\stopparagraphshape + +\startshapedparagraph[list=test,method=cycle] + verse line 1.1\crlf verse line 2.1\crlf + verse line 3.1\crlf verse line 4.1\par + verse line 1.2\crlf verse line 2.2\crlf + verse line 3.2\crlf verse line 4.2\crlf + verse line 5.2\crlf verse line 6.2\par +\stopshapedparagraph +\stopbuffer + +\typebuffer[option=TEX] + +\getbuffer + +{\em todo: move the new (still in {\em \type {meta-imp-txt.mkxl})} code into the +core and integrate it in {\em \type {\startshapedparagraph}} as method {\em \type +{mp}} in which case the list is a list of graphics.} + +\starttyping[option=TEX] +\startshapedparagraph[list={test 1,test 2,test 3,test 4},method=mp] + ..... +\stopshapedparagraph +\stoptyping + +{\em So methods then become kind of plugins.} + +A mechanism like this is often never completely automatic in the sense that you +need to keep an eye on the results. Depending on user demands more features can +be added. With weird shapes you might want to set up the alignment to be \type +{tolerant} and have some \type {stretch}. + +\stopsection + +% \startsection[title=Linebreaks] +\startsection[title=Modes] + +% \ruledvbox{1\ifhmode\writestatus{!}{HMODE 1}\fi} % hsize +% \ruledvbox{\hbox{\strut 2}\ifhmode\writestatus{!}{HMODE 2}\fi} % fit +% \ruledvbox{\hbox{\strut 3}\hbox{\strut 3}\ifhmode\writestatus{!}{HMODE 3}\fi} % fit +% \ruledvbox{\hbox{\strut 4}4\ifhmode\writestatus{!}{HMODE 4}\fi} % hsize +% \ruledvbox{\hbox{\strut 5}5\hbox{\strut 5}\ifhmode\writestatus{!}{HMODE 5}\fi} % hsize +% \ruledvbox{6\hbox{\strut 6}\ifhmode\writestatus{!}{HMODE 6}\fi} % hsize + +{\em todo: some of the side effects of so called modes} + +\stopsection + +\startsection[title=Normalization] + +{\em todo: users don't need to bother about this but it might be interesting anyway} + +\stopsection + +\stopdocument + diff --git a/doc/context/sources/general/manuals/metafun/metafun-basics.tex b/doc/context/sources/general/manuals/metafun/metafun-basics.tex index 27d2f5fdf..92d2f2c07 100644 --- a/doc/context/sources/general/manuals/metafun/metafun-basics.tex +++ b/doc/context/sources/general/manuals/metafun/metafun-basics.tex @@ -463,7 +463,7 @@ draw boundingcircle p withpen pencircle scaled 1mm withcolor .625yellow ; You can consider the \type {boundingcircle} to be a round boundingbox. -\startlinecorrection +\startlinecorrection[blank] \startcombination[nx=3,ny=1,location=middle] {\processMPbuffer[a]} {square} {\processMPbuffer[b]} {circle} @@ -3596,6 +3596,70 @@ a default value of 20. \stopsection +\startsection[title=Grouping] + +The grouping model in \METAPOST\ is kind of special. Basically anything that +happens in a group is processed in some nested action and doesn't interfere with +the outside. However, the last value put (back) into the input is picked up after +the group. A \type {vardef} acts that way: + +\startbuffer +vardef foo (expr i, j) = + save ii, jj ; ii := 2*i ; jj :=j/2 ; + (i, j) -- (ii,jj) +enddef ; + +draw (foo(1,2) .. foo(5,1) .. foo(12,3)) + ysized 1cm + withpen pencircle scaled 2mm + withcolor darkred ; + +draw boundingbox currentpicture withcolor darkyellow ; +\stopbuffer + +\typebuffer + +This weird shape comes out: + +\startlinecorrection[blank] + \processMPbuffer +\stoplinecorrection +We save two variables, and then give new local numerics with their names some +values. Then we \quote {return} a path. A \type {vardef} automatically starts +with a \type {begingroup} and ends with an \type {endgroup}. The next example +shows how we can use that grouping trick directly. The \type {--} has to come +before the \type {begingroup}. + +\startbuffer +vardef turtle expr p = + save a ; pair a ; a := point 0 of p ; a + for i = 1 upto length(p) if cycle p : - 1 fi : + -- begingroup a := a shifted point i of p ; a endgroup + endfor + if cycle p : -- cycle fi +enddef ; + +draw ((0, 0) -- (10, 0) -- (100, 10) -- (-5,20) -- cycle) + withpen pencircle scaled 2 withcolor darkred ; +draw turtle ((0, 0) -- (10, 0) -- (100, 10) -- (-5,20) -- cycle) + withpen pencircle scaled 2 withcolor darkyellow ; +\stopbuffer + +\typebuffer + +Turtle graphics use relative positions. Actually they use one number and switch +direction but the above is okay too. Effectively we loop over the path and use +each point as a relative move, so we recalculate it. + +\startlinecorrection[blank] + \processMPbuffer +\stoplinecorrection + +We could have said \type {-- hide(a := a shifted point i of p) a} because the +\type {hide} macro does that the grouping trick. + +\stopsection + \stopchapter \stopcomponent diff --git a/doc/context/sources/general/manuals/metafun/metafun-debugging.tex b/doc/context/sources/general/manuals/metafun/metafun-debugging.tex index 4174d34e1..de863aea0 100644 --- a/doc/context/sources/general/manuals/metafun/metafun-debugging.tex +++ b/doc/context/sources/general/manuals/metafun/metafun-debugging.tex @@ -56,9 +56,8 @@ parent point with thin lines. \processMPbuffer \stoplinecorrection -You can deduce the direction of a path from the way the -points are numbered, but using an arrow to indicate the -direction is more clear. +You can deduce the direction of a path from the way the points are numbered, but +using an arrow to indicate the direction is more clear. \startbuffer path p ; p := fullcircle xscaled 4cm yscaled 3cm ; @@ -378,6 +377,164 @@ When we overlay these three we get. The envelope only returns the outer curve. \stopsection +\startsection[title=Performance] + +On the average performance of \METAPOST\ is quite okay. The original program uses +scaled numbers, which are floats packed into an integer. The library also +supports doubles, decimal and binary number models. In \CONTEXT\ we only support +scaled, double and decimal. Because the library has to support multiple models +there is more overhead and therefore it is also slower. There's also more dynamic +memory allocation going on. In the transition from \MKII\ to \MKIV\ some of the +critical code (like the code involved in passing \TEX\ states to \METAPOST) had +to be optimized, although when the \LUA\ interface was added, betters ways became +possible. We have to accept the penalty in performance and often can gain back a +lot because we have the \LUA\ interface. + +One of the main bottlenecks in storing quantities. \footnote {Recently, Taco +Hoekwater has done some excellent explanations about the way \METAPOST\ scans the +input and create variables and you can find his presentations at meetings on the +\CONTEXT\ garden.} When we see something \type {a[1]} and \type {a[3]} the \type +{a} is a root variable and the \type {1} and {3} are entries in a linked list +from that root. It's not an array in the sense that there is some upper bound and +that there's also a slot \type {2}. There is order but the list is sparse. When +access is needed, for instance to do some calculations, a linear lookup (from the +head of the list) takes place. This is quite okay performance wise because +normally these list are small. The same is true for a path, which is also a +linked list. If you need point 25, it is looked up by starting at the first knot +of the path. The longer the path, the more time it takes to reach arbitrary +points. In the \LUA\ chapter we give an example of how to get around that +limitation. + +Concerning the arrays, here is s trick to get around a performance bottleneck: + +\starttyping +numeric foo[]; + +def set_foo(expr c, s) = + foo[c] := s ; +enddef ; + +def get_foo(expr c) = + foo[c] +enddef ; +\stoptyping + +If you use this as follows: + +\starttyping +numeric n ; n = 123 ; + +for i=1 upto 20000 : + set_foo(i,n) ; +endfor ; + +for i=1 upto 20000 : + n := get_foo(i) ; +endfor ; +\stoptyping + +the runtime can (for instance) be 3.3 seconds, but when you use the following +variant, it goes down to 0.13 seconds. + +\starttyping +numeric foo[][][][]; % 12345 : 1 12 123 44 instead of 12344 + +def set_foo(expr c, s) = + foo[c div 10000][c div 1000][c div 100][c] := s ; +enddef ; +def get_foo(expr c) = + foo[c div 10000][c div 1000][c div 100][c] +enddef ; +\stoptyping + +This time the lookup is not split into phases each being relatively fast. So, in +order to reach slot 1234 the engine doesn't have to check and jump over what +comes before that. You basically create a tree here: 0 (hit), 1000 (hit in one), +200 (hit in two), 34 (hit in 34). We could go to a single digit but that doesn't +save much. Before we had ways to store data at the \LUA\ end we used this a few +times in macros that dealt with data (like Alan Braslau's node and graphics +modules). This is typically something one can figure out by looking at the (non +trivial) source code. + +Here is another example. In \LUA\ we can easily create a large file, like this: + +\starttyping +\startluacode + local t = { } + for i=1,10000 do + t[i] = string.rep( + "here we have number " .. + tostring(i) .. + " out of the 10000 numbers that we will test" + ,100) + end + t = table.concat(t,"\n") + io.savedata("foo1.tmp",t) + io.savedata("foo2.tmp",t) + io.savedata("foo3.tmp",t) +\stopluacode +\stoptyping + +We make two copies because we do two experiments and we want to treat them equal with +respect to caching. + +\starttyping +\startMPcode + string f ; f := "foo1.tmp" ; + string s[] ; + numeric n ; n := 0 ; + for i=1 upto 10000 : + s[i] := readfrom f ; + exitif s[i] = EOF ; + n := n + 1 ; + endfor ; +\stopMPcode +\stoptyping + +Say that this runs in 2.2 seconds, how come that the next one runs in 1.7 seconds +instead? + +\starttyping +\startMPcode + string f ; f := "foo2.tmp" ; + string s[] ; + string ss ; + numeric n ; n := 0 ; + for i=1 upto 10000 : + ss := readfrom f ; + exitif ss = EOF ; + s[i] := ss ; + n := n + 1 ; + endfor ; +\stopMPcode +\stoptyping + +The main reason is that the first case we have two lookups in the linked list +that determines variable \type {s} and the longer the list, the more time it will +take. In the second case we use an intermediate variable. Although that means +extra memory (de)allocation it still pays of. In practice you don't need to worry +too much about it but of course we can again follow the tree approach: + +\startMPcode + string f ; f := "foo3.tmp" ; + string s[][][] ; + string ss ; + numeric n ; n := 0 ; + for i=1 upto 10000 : + ss := readfrom f ; + exitif ss = EOF ; + s[i div 1000][i div 100][i] := ss ; + n := n + 1 ; + endfor ; +\stopMPcode + +This time we go down to 1.5 second. Timings could be a bit different in \MKIV\ and +\LMTX\ because in \LUAMETATEX\ all \METAPOST\ file \IO\ goes through \LUA\ but the +relative performance gains are the same. With \LUATEX\ and \MKIV\ I measures +2.9, 2.5 and 2.1 and with \LUAMETATEX\ and \LMTX\ I got 2.3, 1.7 and 1.5. + +\stopsection + \stopchapter \stopcomponent diff --git a/doc/context/sources/general/manuals/metafun/metafun-examples.tex b/doc/context/sources/general/manuals/metafun/metafun-examples.tex index 4e5e0eed3..5549a3bdd 100644 --- a/doc/context/sources/general/manuals/metafun/metafun-examples.tex +++ b/doc/context/sources/general/manuals/metafun/metafun-examples.tex @@ -3264,6 +3264,259 @@ We get: \stopsection +\startsection[title=Educational] + +I made this example long ago, when some family member had to learn tables by +heart. For some reason, at school, this is made into a complex issue, with tricks +and such, even it if only involves only a few numbers. My own experience (if I +remember right) was that some of these numbers are trivial, and that there is +quite some symmetry, so in practice only a quarter needs to be remembered. And, +assuming that you can easily deduct the trivial cases, one can just calculate the +rest if needed. + +\startbuffer +\start \ttbf \startMPcode + def MyDraw(expr i, j, c) = + fill fullsquare shifted (i,j) withcolor c withtransparency (1,.5) ; + enddef ; + + for i = 1 upto 10 : + for j = 1 upto 10 : MyDraw( i, -j, "middlered" ) ; endfor ; endfor ; + + for j = 1 upto 10 : MyDraw( 1, -j, "middleblue" ) ; endfor ; + for i = 1 upto 10 : MyDraw( i, -10, "middlegreen" ) ; endfor ; + for i = 1 upto 10 : MyDraw( i, -1, "middleyellow") ; endfor ; + for j = 1 upto 10 : MyDraw(10, -j, "middlecyan" ) ; endfor ; + for j = 1 upto 10 : MyDraw( 5, -j, "middlegray" ) ; endfor ; + for j = 1 upto 10 : MyDraw( j, -j, "middlegray" ) ; endfor ; + + draw image ( for i = 1 upto 10 : for j = 1 upto 10 : + draw textext(decimal (i*j)) ysized .25 shifted (i,-j) ; + endfor ; endfor ; ) withcolor white ; + + currentpicture := currentpicture ysized 10cm ; +\stopMPcode \stop +\stopbuffer + +\typebuffer + +\startplacefigure[title=Overlapping glyphs,reference=fig:tentable] + \getbuffer +\stopplacefigure + +Here we use both transparency and colors to stress the reduction of cases. The +named colors resolve to ones defined at the \TEX\ end. We see the redndering +\in {figure} [fig:tentable]. + +\startsection[title=Glyph magic] + +The next example is the result of a tread on the mailing list. After Henri Menke +posted a some glyph overlap code, I made a variant that more suited the way we do +it in \METAFUN. In the meantime Floris van Maanen had found out that some glyphs +need a more inventive solution so after that the code evolved. By then the \type +{outlinetext}, \type {drawoutlinetext} and \type {filloutlinetext} helpers had +been added to the code base. + +Because this is complicated stuff, we just show the two solutions. The first one +is a relative simple one, the second one uses an approach suggested by Alan +Braslau and therefore uses some of the code that can be found in the \type +{crossingunder} macro. + +\startbuffer +\startMPdefinitions +def ShowOverlapInOutlinesA(expr first, second) = + path p_i, p_j, s_i, s_j ; + numeric n_i, n_j, index ; + pair found ; + index := 0 ; + for i within first : + for j within second : + p_i := pathpart i ; n_i := length(p_i) ; + p_j := pathpart j ; n_j := length(p_j) ; + for ii = 0 upto n_i - 1 : + s_i := subpath(ii,ii+1) of p_i ; + for jj = 0 upto n_j - 1 : + s_j := subpath(jj,jj+1) of p_j ; + found := s_i intersection_point s_j ; + if intersection_found : + index := index + 1 ; + drawdot found + withpen pencircle scaled 4 withcolor white ; + draw textext("\strut\ttbf " & decimal index) ysized 3 + shifted found ; + fi ; + endfor ; + endfor ; + endfor ; + endfor ; +enddef ; +\stopMPdefinitions +\stopbuffer + +\typebuffer \getbuffer + +This is the solution based on \type {crossingunder}, a macro that has been +introduced as part of Alan's neat node module. + +\startbuffer +\startMPdefinitions +def ShowOverlapInOutlinesB(expr first, second) = + begingroup ; + save p, q, n, t, a, b, c, bcuttings, hold, found ; + path p, q ; + numeric n, hold ; + path a, b, c, bcuttings ; + pair found ; + c := makepath(currentpen scaled crossingscale) ; + for f within first : + numeric t[]; + path hold[]; + t[0] := n := hold := 0 ; + for s within second : + p := pathpart f ; + q := pathpart s ; + a := p ; + for i=1 upto crossingnumbermax : % safeguard + clearxy ; z = a intersectiontimes q ; + if x < 0 : + exitif hold < 1 ; + a := hold[hold] ; hold := hold - 1 ; + clearxy ; z = a intersectiontimes q ; + fi + (t[incr n], whatever) = p intersectiontimes point x of a ; + if x = 0 : + a := a cutbefore c shifted point x of a ; + elseif x = length a : + a := a cutafter c shifted point x of a ; + else : % before or after? + b := subpath (0,x) of a cutafter c shifted point x of a ; + bcuttings := cuttings ; + a := subpath (x,length a) of a cutbefore c shifted point x of a ; + clearxy ; z = a intersectiontimes q ; + if x < 0 : + a := b ; + cuttings := bcuttings ; + elseif length bcuttings > 0 : + clearxy ; z = b intersectiontimes q ; + if x >= 0 : + hold[incr hold] := b ; + fi + fi + fi + if length cuttings = 0 : + exitif hold < 1 ; + a := hold[hold] ; hold := hold - 1 ; + fi + endfor ; + endfor ; + t[incr n] = length p ; + for i=1 upto n : + found := point t[i] of p ; + drawdot found + withpen pencircle scaled 4 withcolor white ; + draw textext("\strut\ttbf " & decimal i) ysized 3 + shifted found ; + endfor ; + endfor ; + endgroup ; +enddef ; +\stopMPdefinitions +\stopbuffer + +\typebuffer \getbuffer + +We demonstrate the differences with an example. The result can be seen in +\in {figure} [fig:overlapping:a]. + +\startbuffer +\startcombination + {\startMPcode + picture first, second ; + first := outlinetext.p("N") ; first := first scaled 10 ; + second := outlinetext.p("T") ; second := second scaled 10 ; + second := second rotatedaround(center second, 5) shifted (1,-1) ; + filloutlinetext(first ) withcolor .5[darkblue,white] ; + filloutlinetext(second) withcolor .5[darkred,white] ; + drawoutlinetext(first ) ; + drawoutlinetext(second) ; + ShowOverlapInOutlinesA(first, second) ; + addbackground withcolor darkgray ; + currentpicture := currentpicture scaled 2.5 ; + \stopMPcode} {Method A} + {\startMPcode + picture first, second ; + first := outlinetext.p("N") ; first := first scaled 10 ; + second := outlinetext.p("T") ; second := second scaled 10 ; + second := second rotatedaround(center second, 5) shifted (1,-1) ; + filloutlinetext(first ) withcolor .5[darkgreen,white] ; + filloutlinetext(second) withcolor .5[darkyellow,white] ; + drawoutlinetext(first ) ; + drawoutlinetext(second) ; + ShowOverlapInOutlinesB(first, second) ; + addbackground withcolor darkgray ; + currentpicture := currentpicture scaled 2.5 ; + \stopMPcode} {Method B} +\stopcombination +\stopbuffer + +\typebuffer + +\startplacefigure[title=Overlapping glyphs,reference=fig:overlapping:a] + \getbuffer +\stopplacefigure + +We duplicate some code because the pictures will change in the process of +analyzing. Let's make a helper for that: + +\startbuffer +\startMPdefinitions +def ShowOverlap(expr f, s, m) = + picture first, second ; + first := outlinetext.p(f) ; first := first scaled 10 ; + second := outlinetext.p(s) ; second := second scaled 10 ; + + filloutlinetext(first ) withcolor darkblue ; + drawoutlinetext(first ) ; + + filloutlinetext(second) withcolor darkred ; + drawoutlinetext(second) ; + + if m == 2 : + ShowOverlapInOutlinesB + else : + ShowOverlapInOutlinesA + fi (first, second) ; + + addbackground withcolor darkgray ; + currentpicture := currentpicture ysized 4cm ; +enddef ; +\stopMPdefinitions +\stopbuffer + +\typebuffer \getbuffer + +Again we demonstrate the differences with some examples. The result can be seen in +\in {figure} [fig:overlapping:b]. + +\startbuffer + \startcombination[nx=3,ny=2] + {\startMPcode ShowOverlap("N","T",1) ; \stopMPcode} {Method 1} + {\startMPcode ShowOverlap("\$","Q",1) ; \stopMPcode} {Method 1} + {\startMPcode ShowOverlap("\tttf ABC","\tttf PQR",1) ; \stopMPcode} {Method 1} + {\startMPcode ShowOverlap("N","T",2) ; \stopMPcode} {Method 2} + {\startMPcode ShowOverlap("\$","Q",2) ; \stopMPcode} {Method 2} + {\startMPcode ShowOverlap("\tttf ABC","\tttf PQR",2) ; \stopMPcode} {Method 2} + \stopcombination +\stopbuffer + +\typebuffer + +\startplacefigure[title=Overlapping glyphs,reference=fig:overlapping:b] + \getbuffer +\stopplacefigure + +\stopsection + \stopchapter \stopcomponent diff --git a/doc/context/sources/general/manuals/metafun/metafun-lua.tex b/doc/context/sources/general/manuals/metafun/metafun-lua.tex index 7cd915005..e445cef53 100644 --- a/doc/context/sources/general/manuals/metafun/metafun-lua.tex +++ b/doc/context/sources/general/manuals/metafun/metafun-lua.tex @@ -1211,6 +1211,282 @@ just mimicking drawing the path. \processMPbuffer \stoplinecorrection +\stopsection + +\startsection[title=Acessing \TEX] + +In \MKIV\ and \LMTX\ it is possible to access \TEX\ registers and macros from the +\METAPOST\ end. Let's first define and set some: + +\startbuffer +\newdimen\MyMetaDimen \MyMetaDimen = 2mm +\newcount\MyMetaCount \MyMetaCount = 10 +\newtoks \MyMetaToks \MyMetaToks = {\bfd \TeX} + \def\MyMetaMacro {not done} +\stopbuffer + +\typebuffer \getbuffer + +\startbuffer +\startMPcode + for i=1 upto getcount("MyMetaCount") : + draw fullcircle scaled (i * getdimen("MyMetaDimen")) ; + endfor ; + draw textext(gettoks("MyMetaToks")) xsized 15mm withcolor darkred ; + setglobaldimen("MyMetaDimen", bbwidth(currentpicture)) ; + setglobalmacro("MyMetaMacro", "done") ; +\stopMPcode +\stopbuffer + +\typebuffer + +\startlinecorrection[blank] + \getbuffer +\stoplinecorrection + +We can now look at the two updated globals where \type {\MyMetaMacro: \the\MyMetaDimen} +typesets: {\tttf \MyMetaMacro: \the\MyMetaDimen}. As demonstrated you can best define your +own registers but in principle you can also access system ones, like \type {\scratchdimen} +and friends. + +\stopsection + +\startsection[title=Abstraction] + +We will now stepwise implement some simple helpers for accessing data in files. +The examples are kind of useless but demonstrate how interfaces evolved. The +basic command to communicate with \LUA\ is \type {runscript}. In this example +we will load a (huge) file and run over the lines. + +\starttyping +\startMPcode{doublefun} + save q ; string q ; q := "'\\" & ditto & "'" ; + runscript ( + "GlobalData = string.splitlines(io.loaddata('foo.tmp')) return ''" + ) ; + numeric l ; l = runscript ( + "return string.format('\letterpercent q',\letterhash GlobalData)" + ); + for i=1 step 1 until l : + l := length ( runscript ( + "return string.format('\letterpercent q',GlobalData[" & decimal i & "])" + ) ) ; + endfor ; + draw textext(decimal l); +\stopMPcode +\stoptyping + +The \type {runscript} primitive takes a string and should return a string (in +\LUAMETATEX\ you can also return nothing). This low level solution will serve as +our benchmark: it takes 2.04 seconds on the rather large (64MB) test file with +10.000 lines. + +The code looks somewhat clumsy. This is because in \METAPOST\ escaping is not +built in so one has to append a double quote character using \type {char 34} and +the \type {ditto} string is defined as such. This mess is why in \CONTEXT\ we +have an interface: + +\starttyping +\startMPcode{doublefun} + lua("GlobalData = string.splitlines(io.loaddata('foo.tmp'))") ; + numeric l ; + for i=1 step 1 until lua("mp.print(\#GlobalData)") : + l := length(lua("mp.quoted(GlobalData[" & decimal i & "])")) ; + endfor ; + draw textext(decimal l); +\stopMPcode +\stoptyping + +As expected we pay a price for the additional overhead, so this time we need 2.28 +seconds to process the file. The return value of a run is a string that is fed +into \type {scantokens}. Here \type {print} function prints the number as string +and that gets scanned back to a number. The \type {quoted} function returns a +string in a string so when we're back in \METAPOST\ that gets scanned as string. + +When code is used more frequently, we can make a small library, like this: + +\starttyping +\startluacode + local MyData = { } + function mp.LoadMyData(filename) + MyData = string.splitlines(io.loaddata(filename)) + end + local mpprint = mp.print + local mpquoted = mp.quoted + function mp.MyDataSize() + mpprint(#MyData) + end + function mp.MyDataString(i) + mpquoted(MyData[i] or "") + end +\stopluacode +\stoptyping + +It is not that hard to imagine a more advanced mechanisms where data from multiple +files can be handled at the same time. This code is used as: + +\starttyping +\startMPcode{doublefun} + lua.mp.LoadMyData("foo.tmp") ; + numeric l ; + for i=1 step 1 until lua.mp.MyDataSize() : + l := length(lua.mp.MyDataString(i)) ; + endfor ; + draw textext(decimal l); +\stopMPcode +\stoptyping + +The \type {mp} namespace at the \LUA\ end is a subnamespace at the \METAPOST\ +end. This solution needs 2.20 seconds so we're still slower than the first one, +but in \LUAMETATEX\ with \LMTX we can do better. First the \LUA\ code: + +\starttyping +\startluacode + local injectnumeric = mp.inject.numeric + local injectstring = mp.inject.string + local MyData = { } + function mp.LoadMyData(filename) + MyData = string.splitlines(io.loaddata(filename)) + end + function mp.MyDataSize() + injectnumeric(#MyData) + end + function mp.MyDataString(i) + injectstring(MyData[i] or "") + end +\stopluacode +\stoptyping + +This time we use injectors. The mentioned \type {print} helpers serialize data so +numbers, pairs, colors etc are converted to a string that represents them that is +fed back to \METAPOST\ after the snippet is run. Multiple prints are collected +into one string. An injecter follows a more direct route: it pushes back a proper +\METAPOST\ data type. + +\starttyping +\startMPcode{doublefun} + lua.mp.LoadMyData("foo.tmp") ; + numeric l ; + for i=1 step 1 until lua.mp.MyDataSize() : + l := length(lua.mp.MyDataString(i)) ; + endfor ; + draw textext(decimal l); +\stopMPcode +\stoptyping + +This usage brings us down to 1.14 seconds, so we're still not good. The next +variant is performing similar: 1.05 seconds. + +\starttyping +\startMPcode{doublefun} + runscript("mp.LoadMyData('foo.tmp')") ; + numeric l ; + for i=1 step 1 until runscript("mp.MyDataSize()") : + l := length(runscript("mp.MyDataString(" & decimal i & ")")) ; + endfor ; + draw textext(decimal l); +\stopMPcode +\stoptyping + +We will now delegate scanning to the \LUA\ end. + +\starttyping +\startluacode + local injectnumeric = mp.inject.numeric + local injectstring = mp.inject.string + local scannumeric = mp.scan.numeric + local scanstring = mp.scan.string + local MyData = { } + function mp.LoadMyData() + MyData = string.splitlines(io.loaddata(scanstring())) + end + function mp.MyDataSize() + injectnumeric(#MyData) + end + function mp.MyDataString() + injectstring(MyData[scannumeric()] or "") + end +\stopluacode +\stoptyping + +This time we are faster than the clumsy code we started with: 0.87 seconds. + +\starttyping +\startMPcode{doublefun} + runscript("mp.LoadMyData()") "foo.tmp" ; + numeric l ; + for i=1 step 1 until runscript("mp.MyDataSize()") : + l := length(runscript("mp.MyDataString()") i) ; + endfor ; + draw textext(decimal l); +\stopMPcode +\stoptyping + +In \LMTX\ we can add some more abstraction. Performance is about the same and +sometimes a bit faster but that depends on extreme usage: you need thousands of +call to notice. + +\starttyping +\startluacode + local injectnumeric = mp.inject.numeric + local injectstring = mp.inject.string + local scannumeric = mp.scan.numeric + local scanstring = mp.scan.string + local MyData = { } + metapost.registerscript("LoadMyData", function() + MyData = string.splitlines(io.loaddata(scanstring())) + end) + metapost.registerscript("MyDataSize", function() + injectnumeric(#MyData) + end) + metapost.registerscript("MyDataString", function() + injectstring(MyData[scannumeric()] or "") + end) +\stopluacode +\stoptyping + +We have the same scripts but we register them. At the \METAPOST\ end we resolve +the registered scripts and then call \type {runscript} with the (abstract) numeric +value: + +\starttyping +\startMPcode{doublefun} + newscriptindex my_script_LoadMyData ; + newscriptindex my_script_MyDataSize ; + newscriptindex my_script_MyDataString ; + + my_script_LoadMyData := scriptindex "LoadMyData" ; + my_script_MyDataSize := scriptindex "MyDataSize" ; + my_script_MyDataString := scriptindex "MyDataString" ; + + runscript my_script_LoadMyData "foo.tmp" ; + numeric l ; + for i=1 step 1 until runscript my_script_MyDataSize : + l := length(my_script_MyDataString i) ; + endfor ; + draw textext(decimal l); +\stopMPcode +\stoptyping + +This is of course nicer: + +\starttyping +\startMPcode{doublefun} + def LoadMyData (expr s) = runscript my_script_LoadMyData s enddef ; + def MyDataSize = runscript my_script_MyDataSize enddef ; + def MyDataString(expr i) = runscript my_script_MyDataString i enddef ; + + LoadMyData("foo.tmp") ; + numeric l ; + for i=1 step 1 until MyDataSize : + l := length(MyDataString(i)) ; + endfor ; + draw textext(decimal l); +\stopMPcode +\stoptyping + +So, to sumarize, there are many ways to look at this: verbose direct ones +but also nicely abstract ones. \stopsection diff --git a/doc/context/sources/general/manuals/metafun/metafun-macros.tex b/doc/context/sources/general/manuals/metafun/metafun-macros.tex index e1df13c92..969e7d1b1 100644 --- a/doc/context/sources/general/manuals/metafun/metafun-macros.tex +++ b/doc/context/sources/general/manuals/metafun/metafun-macros.tex @@ -31,12 +31,13 @@ There are several ways to use the power of \METAFUN, either or not using \stopitem \startitem - You can embed the graphic in a \type {\startMPpage} construct and process it - with \CONTEXT\ \MKIV. In that case you have the full \METAFUN\ functionality - available. If for some reason you still want to use \MKII, you need to use - \TEXEXEC\ as before processing the file, it will do a couple of checks on the - file. It will also make sure that the temporary files (\type {mpt} for \type - {textext}'s and \type {mpo} for outline fonts) are taken care of too. + You can embed the graphic in a \type {\startMPpage} \unknown \type + {\stopMPpage} construct and process it with \CONTEXT\ \MKIV. In that case you + have the full \METAFUN\ functionality available. If for some reason you still + want to use \MKII, you need to use \TEXEXEC\ as before processing the file, + it will do a couple of checks on the file. It will also make sure that the + temporary files (\type {mpt} for \type {textext}'s and \type {mpo} for + outline fonts) are taken care of too. \stopitem \startitem -- cgit v1.2.3