diff options
Diffstat (limited to 'doc/context/sources/general/manuals/mk/mk-mplib.tex')
| -rw-r--r-- | doc/context/sources/general/manuals/mk/mk-mplib.tex | 924 |
1 files changed, 924 insertions, 0 deletions
diff --git a/doc/context/sources/general/manuals/mk/mk-mplib.tex b/doc/context/sources/general/manuals/mk/mk-mplib.tex new file mode 100644 index 000000000..78e7b8f97 --- /dev/null +++ b/doc/context/sources/general/manuals/mk/mk-mplib.tex @@ -0,0 +1,924 @@ +% language=uk + +\useMPlibrary[dum] + +\startcomponent mk-mplib + +\environment mk-environment + +\definetextbackground + [sideline] + [mp=mpos:par:sideline, + framecolor=mkcolor, + frameoffset=5mm] + +\startuseMPgraphic{mpos:par:sideline}{linecolor,lineoffset} + for i=1 upto nofmultipars : + fill leftboundary multipars[i] + shifted (-\MPvar{lineoffset},0) + rightenlarged 1mm withcolor \MPvar{linecolor} ; + endfor ; +\stopuseMPgraphic + +\chapter{The MetaPost Library} + +% \subject{Hans Hagen \& Taco Hoekwater} \blank[3*big] + +This chapter is written by Taco and Hans around the time +that \MPLIB\ was integrated into \LUATEX. +It is part of our torture test. + +\subject{introduction} + +If \METAPOST\ support had not been as tightly integrated into +\CONTEXT\ as it is, at least half of the projects \PRAGMA\ has +been doing in the last decade could not have been done at all. +\starttextbackground[sideline]Take for instance +backgrounds behind text or graphic markers alongside text. These are +probably the most complex mechanisms in \CONTEXT: positions are +stored, and positional information is passed on to +\METAPOST, where intersections between the text areas and the running +text are converted into graphics that are then positioned in the +background of the text.\stoptextbackground{} Underlining of text +(sometimes used in the educational documents that we typeset) and +change bars (in the margins) are implemented using the same +mechanism because those are basically a background with only one of the +frame sides drawn. + +You can probably imagine that a 300 page document with several such +graphics per page takes a while to process. A nice example of such +integrated graphics is the \LUATEX\ reference manual, that has an +unique graphic at each page: a stylized image of a revolving moon. + +\startuseMPgraphic{lualogo-x}{angle} +color luaplanetcolor ; luaplanetcolor := .5blue ; +color luaholecolor ; luaholecolor := white ; +numeric luaextraangle ; luaextraangle := \MPvar{angle} ; + +vardef lualogo = image ( + % Graphic design by A. Nakonechnyj. Copyright (c) 1998, All rights reserved. + save luaorbitcolor, d, r, p ; color luaorbitcolor ; numeric d, r, p ; + luaorbitcolor := .5luaholecolor ; d := sqrt(2)/4 ; r := 1/4 ; p := r/8 ; + fill fullcircle scaled 1 withcolor luaplanetcolor ; + draw fullcircle rotated 40.5 scaled (1+r) dashed evenly scaled p withpen pencircle scaled (p/2) withcolor luaorbitcolor ; + fill fullcircle scaled r shifted (d+1/8,d+1/8) rotated luaextraangle withcolor luaplanetcolor ; + fill fullcircle scaled r shifted (d-1/8,d-1/8) withcolor luaholecolor ; + setbounds currentpicture to fullsquare scaled 1.3; +) enddef ; + +draw lualogo scaled 1cm ; +\stopuseMPgraphic + +\startlinecorrection[blank] +\hbox to \hsize + {\hss\dostepwiserecurse{0}{360}{60}{\useMPgraphic{lualogo-x}{angle=\recurselevel}\hss}} +\stoplinecorrection + +Most of the running time integrating such graphics seemed to be +caused by the mechanics of the process: starting the separate +\METAPOST\ interpreter and having to deal with a number of +temporary files. Therefore our expectations were high with regards +to integrating \METAPOST\ more tightly into \LUATEX. Besides the +speed gain, it also true that the simpler the process of using +such use of graphics becomes, the more modern a \TEX\ runs looks +and the less problems new users will have with understanding how +all the processes cooperate. + +This article will not discuss the application interface of the +\MPLIB\ library in detail, for that there is the \LUATEX\ manual. In short, +using the embedded \METAPOST\ interpreter in \LUATEX\ boils +down to the following: + +\startitemize[packed] +\item Open an instance using \type {mplib.new}, either to process + images with a format to be loaded, or to create such a format. + This function returns a library object. +\item Execute sequences of \METAPOST\ commands, using the object's + \type{execute} method. This returns a result. +\item Check if the result is valid and (if it is okay) request the list + of objects. Do whatever you want with them, most probably + convert them to some output format. You can also request a + string representation of a graphic in \POSTSCRIPT\ format. +\stopitemize + +There is no need to close the library object. As long as you didn't make +any fatal errors, the library recovers well and can stay alive during +the entire \LUATEX\ run. + +Support for \MPLIB\ depends on a few components: integration, +conversion and extensions. This article shows some of the code +involved in supporting the library. Let's start with the conversion. + +\subject{conversion} + +The result of a \METAPOST\ run traditionally is a \POSTSCRIPT\ +language description of the generated graphic(s). When +\PDF\ is needed, that \POSTSCRIPT\ code has to be converted to the target +format. This includes embedded text as well as penshapes used for +drawing. To demonstrate, here is a simple example graphic: + +\startluacode +document.sample_mp_graphic = [[ + draw fullcircle + scaled 2cm + withpen pencircle xscaled 1mm yscaled .5mm rotated 30 + withcolor .75red ; +]] +\stopluacode + +\startbuffer[mpexample] +draw fullcircle + scaled 2cm + withpen pencircle xscaled 1mm yscaled .5mm rotated 30 + withcolor .75red ; +\stopbuffer + +\placefigure + [left] + {} + {\startMPcode \ctxlua{tex.sprint(document.sample_mp_graphic)}\stopMPcode} + +\typebuffer[mpexample] + +Notice how the pen is not a circle but a rotated ellipse. Later on it +will become clear what the consequences of that are for the conversion. + +How does this output look in \POSTSCRIPT ? If the preamble is left out +it looks like this: + +\startbuffer +\startluacode +do + local mpx = metapost.format("metafun") + local result = mpx:execute(string.format("beginfig(0);%s;endfig;",document.sample_mp_graphic)) + local str = result.fig[1]:postscript() + str = str:gsub("%%BeginProlog.-%%EndProlog","%% <<stripped preamble was here>>") + tex.sprint(tex.ctxcatcodes,"\\starttyping\n") + tex.sprint((str:gsub("[\n\r]$",""))) + tex.sprint("\\stoptyping") +end +\stopluacode +\stopbuffer + +\getbuffer + +The most prominent code here concerns the path. The numbers in brackets define +the transformation matrix for the pen we used. The \PDF\ variant looks as follows: + +\startbuffer +\startluacode +do + local mpx = metapost.format("metafun") + local result = mpx:execute(string.format("beginfig(0);%s;endfig;",document.sample_mp_graphic)) + local list = metapost.pdfliterals(result) + tex.sprint(tex.ctxcatcodes,"\\starttyping") + tex.sprint(table.concat(list,"\n")) + tex.sprint("\\stoptyping") +end +\stopluacode +\stopbuffer + +\getbuffer + +The operators don't look much different from the \POSTSCRIPT, which is +mostly due to the fact that in the \POSTSCRIPT\ code, the preamble +defines shortcuts like \type {c} for \type {curveto}. Again, most code +involves the path. However, this time the numbers are different and +the transformation comes before the path. + +In the case of \PDF\ output, we could use \TEX\ itself to do the +conversion: a generic converter is implemented in \type +{supp-pdf.tex}, while a converter optimized for \CONTEXT\ \MKII\ is +defined in the files whose names start with \type {meta-pdf}. But in +\CONTEXT\ \MKIV\ we use \LUA\ code for the conversion instead. Thanks to +\LUA's powerful \LPEG\ parsing library, this gives cleaner code and is +also faster. This converter currently lives in \type {mlib-pdf.lua}. + +Now, with the embedded \METAPOST\ library, conversion goes different +still because now it is possible to request the drawn result and +associated information in the form of \LUA\ tables. + +\startbuffer +\startluacode +do + local mpx = metapost.format("metafun") + local result = mpx:execute(string.format("beginfig(0);%s;endfig;",document.sample_mp_graphic)) + local str = table.serialize(metapost.totable(result), "figure") + tex.sprint(tex.ctxcatcodes,"\\starttyping") + tex.sprint((str:gsub("[\n\r]$",""))) + tex.sprint("\\stoptyping") +end +\stopluacode +\stopbuffer % $<-emacs font-lock + +% \startsimplecolumns[n=3,width=18.5cm] +% \getbuffer +% \stopsimplecolumns + +\getbuffer + +This means that instead of parsing \POSTSCRIPT\ output, we now can operate on +a proper datastructure and get code like the following: + +\starttyping +function convertgraphic(result) + if result then + local figures = result.fig + if figures then + for fig in ipairs(figures) do + local llx, lly, urx, ury = unpack(fig:boundingbox()) + if urx > llx then + startgraphic(llx, lly, urx, ury) + for object in ipairs(fig:objects()) do + if object.type == "..." then + ... + flushgraphic(...) + ... + else + ... + end + end + finishgraphic() + end + end + end + end +end +\stoptyping + +Here \type {result} is what the library returns when one or more +graphics are processed. As you can deduce from this snippet, a +result can contain multiple figures. Each figure corresponds with a +\type {beginfig} \type {...} \type {endfig}. The graphic operators that +the converter generates (so called \PDF\ literals) have to be +encapsulated in a proper box so this is why we have: + +\startitemize[packed] +\item \type {startgraphic}: start packaging the graphic +\item \type {flushgraphic}: pipe literals to \TEX +\item \type {finishgraphic}: finish packaging the graphic +\stopitemize + +It does not matter what number you passed to \type {beginfig}, the +graphics come out in the natural order. + +Little over half a dozen different object types are possible. The example +\METAPOST\ \type{draw} command from above results in an \type {outline} object. +This object contains not only path information but also carries +rendering data, like the color and the pen. So, in the end we will +flush code like \type {1 M} which sets the \type {miterlimit} to one +or \type {.5 g} which sets the color to 50\% gray, in addition to a +path. + +Because objects are returned in a way that closely resembles a +\METAPOST's internals, some extra work needs to be done in order to +calculate paths with elliptical pens. An example of a helper function +in somewhat simplified form is shown next: + +\starttyping +function pen_characteristics(object) + local p = object.pen[1] + local wx, wy, width + if p.right_x == p.x_coord and p.left_y == p.y_coord then + wx = abs(p.left_x - p.x_coord) + wy = abs(p.right_y - p.y_coord) + else -- pyth: sqrt(a^2 +b^2) + wx = pyth(p.left_x - p.x_coord, p.right_x - p.x_coord) + wy = pyth(p.left_y - p.y_coord, p.right_y - p.y_coord) + end + if wy/coord_range_x(object.path, wx) >= + wx/coord_range_y(object.path, wy) then + width = wy + else + width = wx + end + local sx, sy = p.left_x, p.right_y + local rx, ry = p.left_y, p.right_x + local tx, ty = p.x_coord, p.y_coord + if width ~= 1 then + if width == 0 then + sx, sy = 1, 1 + else + rx, ry, sx, sy = rx/width, ry/width, sx/width, sy/width + end + end + if abs(sx) < eps then sx = eps end + if abs(sy) < eps then sy = eps end + return sx, rx, ry, sy, tx, ty, width +end +\stoptyping + +If \type {sx} and \type {sy} are 1, there is no need to transform +the path, otherwise a suitable transformation matrix is calculated +and returned. The function itself uses a few helpers that make the +calculations even more obscure. This kind of code does not fall in +the category trivial and as already mentioned, these basic +algorithms were derived from the \METAPOST\ sources. Even so, +these snippets demonstrate that interfacing using \LUA\ does not +look that bad. + +In the actual \MKIV\ code things look a bit different because it does +a bit more and uses optimized code. There you will also find the code +dealing with the actual transformation, of which these helpers are +just a portion. + +If you compare the \POSTSCRIPT\ and the \PDF\ code you will notice +that the paths looks different. This is because the use and +application of a transformation matrix in \PDF\ is different from how +it is handled in \POSTSCRIPT. In \PDF\ more work is assumed to be +done by the \PDF\ generating application. This is why in both the +\TEX\ and the \LUA\ based converters you will find transformation code +and the library follows the same pattern. In that respect \PDF\ +differs fundamentally from \POSTSCRIPT. + +Within the \TEX\ based converter there was the problem of keeping the +needed calculations within \TEX's accuracy, which fortunately permits +larger values that \METAPOST\ can produce. This plus the parsing code +resulted in a not-that-easy to follow bunch of \TEX\ code. The \LUA\ +based parser is more readable, but since it also operates on +\POSTSCRIPT\ code it is kind of unnatural too, but at least there are +less problems with keeping the calculations sane. The \MPLIB\ based +converter is definitely the cleanest and least sensitive to future +changes in the \POSTSCRIPT\ output. Does this mean that there is no +ugly code left? Alas, as we will see in the next section, dealing with +extensions is still somewhat messy. In practice users will not be +bothered with such issues, because writing a converter is a one time +job by macro package writers. + +\subject{extensions} + +In \METAFUN, which is the \METAPOST\ format used with \CONTEXT, +a few extensions are provided, like: + +\startitemize[packed] +\item cmyk, spot and multitone colors +\item including external graphics +\item lineair and circulair shades +\item texts converted to outlines +\item inserting arbitrary texts +\stopitemize + +Until now, most of these extensions have been implemented by using +specially coded colors and by injecting so called specials (think of +them as comments) into the output. On one of our trips to a \TEX\ +conference, we discussed ways to pass information along with +paths and eventually we arrived at associating text strings with paths +as a simple and efficient solution. As a result, recently \METAPOST\ +was extended by \type {withprescript} and \type {withpostscript} +directives. For those who are unfamiliar with these new scripts, +they are used as follows: + +\starttyping +draw fullcircle withprescript "hello" withpostscript "world" ; +\stoptyping + +In the \POSTSCRIPT\ output these scripts end up before and after the +path, but in the \PDF\ converter they can be overloaded to implement +extensions, and that works reasonably well. However, at the moment +there cannot be multiple pre- and postscripts associated with a single +path inside the \METAPOST\ internals. This means that for the moment, +the scripts mechanism is only used for a few of the extensions. +Future versions of \MPLIB\ may provide more sophisticated methods +for carrying information around. + +The \MKIV\ conversion mechanism uses scripts for graphic inclusion, +shading and text processing but unfortunately cannot use them for +more advanced color support. + +A nasty complication is that the color spaces in \METAPOST\ don't +cast, which means that one cannot assign any color to a color +variables: each colorspace has it's own type of variable. + +\starttyping +color one ; one := (1,1,0) ; % correct +cmykcolor two ; two := (1,0,0,1) ; % correct +one := two ; % error +fill fullcircle scaled 1cm withcolor .5[one,two] ; % error +\stoptyping + +In \CONTEXT\ we use constructs like this: + +\starttyping +\startreusableMPgraphic{test} + fill fullcircle scaled 1cm withcolor \MPcolor{mycolor} ; +\stopreusableMPgraphic + +\reuseMPgraphic{test} +\stoptyping + +Because \type {withcolor} is clever enough to understand what color +type it receives, this is ok, but how about: + +\starttyping +\startreusableMPgraphic{test} + color c ; c := \MPcolor{mycolor} ; + fill fullcircle scaled 1cm withcolor c ; +\stopreusableMPgraphic +\stoptyping + +Here the color variable only accepts an \RGB\ color and because in +\CONTEXT\ there is mixed color space support combined with +automatic colorspace conversions, it doesn't know in advance what type +it is going to get. By implementing color spaces other than \RGB\ +using special colors (as before) such type mismatches can be avoided. + +The two techniques (coding specials in colors and pre|/|postscripts) +cannot be combined because a script is associated with a path and +cannot be bound to a variable like \type{c}. So this again is an argument +for using special colors that remap onto \CMYK\, spot or +multi|-|tone colors. + +Another area of extensions is text. In previous versions of +\CONTEXT\ the text processing was already isolated: text ended +up in a separate file and was processed in an separate run. More +recent versions of \CONTEXT\ use a more abstract model of boxes that +are preprocessed before a run, which avoids the external run(s). In +the new approach everything can be kept internal. The conversion +even permits constructs like: + +\starttyping +for i=1 upto 100 : + draw btex oeps etex rotated i ; +endfor ; +\stoptyping + +but since this construct is kind of obsolete (at least in the library +version of \METAPOST) it is better to use: + +\starttyping +for i=1 upto 100 : + draw textext("cycle " & decimal i) rotated i ; +endfor ; +\stoptyping + +Internally a trial pass is done so that indeed 100 different texts will +be drawn. The throughput of texts is so high that in practice one will +not even notice that this happens. + +Dealing with text is yet another example of using \LPEG. The following +snippet of code sheds some light on how text in graphics is dealt with. +Actually this is a variation on a previous implementation. That one +was slightly faster but looked more complex. It was also not robust for +complex texts defined in macros in a format. + +\starttyping +local P, S, V, Cs = lpeg.P, lpeg.S, lpeg.V, lpeg.Cs + +local btex = P("btex") +local etex = P(" etex") +local vtex = P("verbatimtex") +local ttex = P("textext") +local gtex = P("graphictext") +local spacing = S(" \n\r\t\v")^0 +local dquote = P('"') + +local found = false + +local function convert(str) + found = true + return "textext(\"" .. str .. "\")" +end +local function ditto(str) + return "\" & ditto & \"" +end +local function register() + found = true +end + +local parser = P { + [1] = Cs((V(2)/register + V(3)/convert + 1)^0), + [2] = ttex + gtex, + [3] = (btex + vtex) * spacing * + Cs((dquote/ditto + (1-etex))^0) * etex, +} + +function metapost.check_texts(str) + found = false + return parser:match(str), found +end +\stoptyping + +If you are unfamiliar with \LPEG\ it may take a while to see what +happens here: we replace the text between \type {btex} and \type {etex} +by a call to \type {textext}, a macro. Special care is given to +embedded double quotes. + +When text is found, the graphic is processed two times. The definition +of \type {textext} is different for each run. The first run we have: + +\starttyping +vardef textext(expr str) = + image ( + draw unitsquare + withprescript "tf" + withpostscript str ; + ) +enddef ; +\stoptyping + +After the first run the result is not really converted, but just +the outlines with the \type {tf} prescript are filtered. In the +loop over the object there is code like: + +\starttyping +local prescript = object.prescript +if prescript then + local special = metapost.specials[prescript] + if special then + special(object.postscript,object) + end +end +\stoptyping + +Here, \type {metapost} is just the namespace used by the +converter. The prescript tag \type {tf} triggers a function: + +\starttyping +function metapost.specials.tf(specification,object) + tex.sprint(tex.ctxcatcodes,format("\\MPLIBsettext{%s}{%s}", + metapost.textext_current,specification)) + if metapost.textext_current < metapost.textext_last then + metapost.textext_current = metapost.textext_current + 1 + end + ... +end +\stoptyping + +Again, you can forget about the details of this function. Important is +that there is a call out to \TEX\ that will process the text. Each snippet +gets the number of the box that holds the content. The macro that is +called just puts stuff in a box: + +\starttyping +\def\MPLIBsettext#1#2% + {\global\setbox#1\hbox{#2}} +\stoptyping + +In the next processing cycle of the \METAPOST\ code, the \type {textext} +macro does something different : + +\starttyping +vardef textext(expr str) = + image ( + _tt_n_ := _tt_n_ + 1 ; + draw unitsquare + xscaled _tt_w_[_tt_n_] + yscaled (_tt_h_[_tt_n_] + _tt_d_[_tt_n_]) + withprescript "ts" + withpostscript decimal _tt_n_ ; + ) +enddef ; +\stoptyping + +This time the by then known dimensions of the box that is used to +store the snippet are used. These are stored in the \type {_tt_w_}, +\type {_tt_h_} and \type {_tt_d_} arrays. The arrays are defined by +\LUA\ using information about the boxes, and passed to the library +before the second run. The result from the second \METAPOST\ run +is converted, and again the prescript is used as trigger: + +\starttyping +function metapost.specials.ts(specification,object,result) + local op = object.path + local first, second, fourth = op[1], op[2], op[4] + local tx, ty = first.x_coord , first.y_coord + local sx, sy = second.x_coord - tx, fourth.y_coord - ty + local rx, ry = second.y_coord - ty, fourth.x_coord - tx + if sx == 0 then sx = 0.00001 end + if sy == 0 then sy = 0.00001 end + metapost.flushfigure(result) + tex.sprint(tex.ctxcatcodes,format( + "\\MPLIBgettext{%f}{%f}{%f}{%f}{%f}{%f}{%s}", + sx,rx,ry,sy,tx,ty,metapost.textext_current)) + ... +end +\stoptyping + +At this point the converter is actually converting the graphic and +passing \PDF\ literals to \TEX. As soon as it encounters a text, +it flushes the \PDF\ code collected so far and injects some \TEX\ +code. The \TEX\ macro looks like: + +\starttyping +\def\MPLIBgettext#1#2#3#4#5#6#7% + {\ctxlua{metapost.sxsy(\number\wd#7,\number\ht#7,\number\dp#7)}% + \pdfliteral{q #1 #2 #3 #4 #5 #6 cm}% + \vbox to \zeropoint{\vss\hbox to \zeropoint + {\scale[sx=\sx,sy=\sy]{\raise\dp#7\box#7}\hss}}% + \pdfliteral{Q}} +\stoptyping + +Because text can be transformed, it needs to be scale back to the right +dimensions, using both the original box dimensions and the transformation +of the unitquare associated with the text. + +\starttyping +local factor = 65536*(7200/7227) + +function metapost.sxsy(wd,ht,dp) -- helper for text + commands.edef("sx",(wd ~= 0 and 1/( wd /(factor))) or 0) + commands.edef("sy",(wd ~= 0 and 1/((ht+dp)/(factor))) or 0) +end +\stoptyping + +So, in fact there are the following two processing alternatives: + +\startitemize[packed] +\item tex: calls a \LUA\ function that processed the graphic +\item lua: parse the \METAPOST\ code for texts and decide if two + runs are needed +\stopitemize + +Now, if there was no text to be found, the continuation is: + +\startitemize[packed] +\item lua: process the code using the library +\item lua: convert the resulting graphic (if needed) and check + if texts are used +\stopitemize + +Otherwise, the next steps are: + +\startitemize[packed] +\item lua: process the code using the library +\item lua: parse the resulting graphic for texts (in the postscripts) + and signal \TEX\ to process these texts afterwards +\item tex: process the collected text and put the result in boxes +\item lua: process the code again using the library but this time let + the unitsquare be transformed using the text dimensions +\item lua: convert the resulting graphic and replace the transformed + unitsquare by the boxes with text +\stopitemize + +The processor itself is used in the \MKIV\ graphic function that takes +care of the multiple passes mentioned before. To give you an idea of +how it works, here is how the main graphic processing function roughly +looks. + +\starttyping +local current_format, current_graphic + +function metapost.graphic_base_pass(mpsformat,str,preamble) + local prepared, done = metapost.check_texts(str) + metapost.textext_current = metapost.first_box + if done then + current_format, current_graphic = mpsformat, prepared + metapost.process(mpsformat, { + preamble or "", + "beginfig(1); ", + "_trial_run_ := true ;", + prepared, + "endfig ;" + }, true ) -- true means: trialrun + tex.sprint(tex.ctxcatcodes, + "\\ctxlua{metapost.graphic_extra_pass()}") + else + metapost.process(mpsformat, { + preamble or "", + "beginfig(1); ", + "_trial_run_ := false ;", + str, + "endfig ;" + } ) + end +end + +function metapost.graphic_extra_pass() + metapost.textext_current = metapost.first_box + metapost.process(current_format, { + "beginfig(0); ", + "_trial_run_ := false ;", + table.concat(metapost.texttextsdata()," ;\n"), + current_graphic, + "endfig ;" + }) +end +\stoptyping + +The box information is generated as follows: + +\starttyping +function metapost.texttextsdata() + local t, n = { }, 0 + for i = metapost.first_box, metapost.last_box do + n = n + 1 + local box_i = tex.box[i] + if box_i then + t[#t+1] = format( + "_tt_w_[%i]:=%f;_tt_h_[%i]:=%f;_tt_d_[%i]:=%f;", + n, box_i.width /factor, + n, box_i.height/factor, + n, box_i.depth /factor + ) + else + break + end + end + return t +end +\stoptyping + +This is a typical example of accessing information available inside +\TEX\ from \LUA, in this case information about boxes. + +The \type {trial_run} flag is used at the \METAPOST\ end, in fact the +\type {textext} macro looks as follows: + +\starttyping +vardef textext(expr str) = + if _trial_run_ : + % see first variant above + else : + % see second variant above + fi +enddef ; +\stoptyping + +This trickery is not new. We used it already in \CONTEXT\ for some +time, but until now the multiple runs took way more time and from +the perspective of the user this all looked much more complex. + +It may not be that obvious, but: in the case of a trial run (for +instance when texts are found), after the first processing stage, +and during the parsing of the result, the commands that typeset the +content will be printed to \TEX. After processing, the command to do +an extra pass is printed to \TEX\ also. So, once control is passed +back to \TEX, at some point \TEX\ itself will pass control back to +\LUA\ and do the extra pass. + +The base function is called in: + +\starttyping +function metapost.graphic(mpsformat,str,preamble) + local mpx = metapost.format(mpsformat or "metafun") + metapost.graphic_base_pass(mpx,str,preamble) +end +\stoptyping + +The \type {metapost.format} function is part of \type {mlib-run}. +It loads the \type{metafun} format, possibly after (re|)|generating it. + +Now, admittedly all this looks a bit messy, but in pure \TEX\ macros +it would be even more so. Sometime in the future, the postponed calls to +\tex{ctxlua} and the explicit \tex{pdfliteral}s can and will be +replaced by using direct node generation, but that requires a rewrite +of the internal \LUATEX\ support for \PDF\ literals. + +The snippets are part of the \type {mlib-*} files of \MKIV. These files are +tagged as experimental and will stay that way for a while yet. This is +proved by the fact that by now we use a slightly different approach. + +Summarizing the impact of \MPLIB\ on extensions, we can conclude that +some are done better and some more or less the same. There are some +conceptual problems that prohibit using pre- and postscripts for +everything (at least currently). + +\subject{integrating} + +The largest impact of \MPLIB\ is processing graphics at runtime. +In \MKII\ there are two methods: real runtime processing (each +graphic triggered a call to \METAPOST) and collective processing +(between \TEX\ runs). The first method slows down the \TEX\ run, +the second method generates a whole lot of intermediate \POSTSCRIPT\ +files. In both cases there is a lot of file \IO\ involved. + +In \MKIV, the integrated library is capable of processing +thousands of graphics per second, including conversion. The +preliminary tests (which involved no extensions) involved graphics +with 10 random circles drawn with penshapes in random colors, and +the thoughput was around 2000 such graphics per second on a +2.3 MHz Core Duo: + +\startlinecorrection[blank] +\hbox to \textwidth{\strut\dorecurse{5}{\startMPcode +numeric w ; w := \the\textwidth/6 ; +for k:=1 upto 10 : + draw fullcircle + scaled uniformdeviate(w) + withpen pencircle xscaled (w/20) yscaled (w/40) rotated 30 + withcolor (red/(k/4)) ; +endfor ; +setbounds currentpicture to fullsquare scaled w ; +\stopMPcode\hss}\unskip} +\stoplinecorrection + +In practice there will be some more overhead involved than in the +tests. For instance, in \CONTEXT\ information about the current state +of \TEX\ has to be passed on also: page dimensions, font information, +typesetting related parameters, preamble code, etc. + +The whole \TEX\ interface is written around one process function: + +\starttyping +metapost.graphic(metapost.format("metafun"),"mp code") +\stoptyping + +optionally a preamble can be passed as the third argument. +This one function is used in several other macros, like: + +\starttyping +\startMPcode ... \stopMPcode +\startMPpage ... \stopMPpage +\startuseMPgraphic {name} ... \stopuseMPgraphic +\startreusableMPgraphic{name} ... \stopreusableMPgraphic +\startuniqueMPgraphic {name} ... \stopuniqueMPgraphic + +\useMPgraphic{name} +\reuseMPgraphic{name} +\uniqueMPgraphic{name} +\stoptyping + +The user interface is downward compatible: in \MKIV\ the same +top-level commands are provided as in \MKII. However, the +(previously required) configuration macros and flags are obsolete. + +This time, the conclusion is that the impact on \CONTEXT\ is immense: +The code for embedding graphics is very clean, and the running time +for graphics inclusion is now negligible. Support for text in graphics is +more natural now, and takes no runtime either (in \MKII\ some +parsing in \TEX\ takes place, and if needed long lines are split; +all this takes time). + +In the styles that \PRAGMA\ uses internally, there is support for the +generation of placeholders for missing graphics. These placeholders +are \METAPOST\ graphics that have some 60 randomly scaled circles with randomized +colors. The time involved in generating 50 such graphics is (on Hans' +machine) some 14 seconds, while in \LUATEX\ only half a second is needed. + +\startlinecorrection[blank] +\hbox to \textwidth \bgroup +\hss\dorecurse{4}{\externalfigure[dummy][width=.2\textwidth]\hss}% +\egroup +\stoplinecorrection + +Because \LUATEX\ needs more startup time and deals with larger fonts +resources, \PDFTEX\ is generally faster, but now that we have \MPLIB, +\LUATEX\ suddenly is the winner. + +% We end this article by showing a few graphics. \CONTEXT\ ships with a module +% that permits tracking of resource usage. Users can add the following line +% to their document: + +% \starttyping +% \usemodule[timing] +% \stoptyping + +% After that one can use commands like + +% \starttyping +% \ShowNamedUsage{\jobname-luatex-progress}{luastate_bytes}{elapsed_time} +% \ShowNamedUsage{\jobname-luatex-progress}{dyn_used}{elapsed_time} +% \ShowNamedUsage{\jobname-luatex-progress}{str_ptr}{elapsed_time} +% \ShowNamedUsage{\jobname-luatex-progress}{pdf_literal}{elapsed_time} +% \ShowNamedUsage{\jobname-luatex-progress}{glyph}{elapsed_time} +% \stoptyping + +% In this document, \LUA\ memory usage is as follows. The blue lines +% represent runtime, on Hans' machine some 3.5 seconds, including +% startuptime, which clearly shows in the graphic. Keep in mind that +% garbage collection only happens at certain moment, so \LUA\ memory +% usage is normally a sawtooth graphic. The horizontal axis reflects the +% number of pages. + +% \startlinecorrection[blank] +% \ShowNamedUsage{\jobname-luatex-progress}{luastate_bytes}{elapsed_time} +% \stoplinecorrection + +% Dynamic memory usage of \TEX\ grows a bit because at the \TEX\ end we need +% to store data as well. + +% \startlinecorrection[blank] +% \ShowNamedUsage{\jobname-luatex-progress}{dyn_used}{elapsed_time} +% \stoplinecorrection + +% Currently we use \type {\pdfliteral}, which means that all +% literals go through \TEX's tokenizer and string pool. With large +% graphics this can be a real memory hog. + +% Unfortunately not all this memory is freed, so eventually Hartmut +% Henkel (the third member of the \LUATEX\ team) will reimplement +% this part of the backend. + +% \startlinecorrection[blank] +% \ShowNamedUsage{\jobname-luatex-progress}{str_ptr}{elapsed_time} +% \stoplinecorrection + +% Just to complete this picture, we show the usage of literals next. Although +% the converter collects as much data as possible before flushing, we see +% some peaks. + +% \startlinecorrection[blank] +% \ShowNamedUsage{\jobname-luatex-progress}{pdf_literal}{elapsed_time} +% \stoplinecorrection + +% Compare this to the glyph usage. Indeed: the more graphics we use, the +% less glyphs we encounter. + +% \startlinecorrection[blank] +% \ShowNamedUsage{\jobname-luatex-progress}{glyph}{elapsed_time} +% \stoplinecorrection + +% Of course these graphics were integrated, generated and converted +% using \MPLIB. + +\stopcomponent |