diff options
20 files changed, 258 insertions, 18 deletions
diff --git a/doc/context/sources/general/manuals/still/still-acrobat-math.png b/doc/context/sources/general/manuals/still/still-acrobat-math.png Binary files differnew file mode 100644 index 000000000..e2f842a36 --- /dev/null +++ b/doc/context/sources/general/manuals/still/still-acrobat-math.png diff --git a/doc/context/sources/general/manuals/still/still-acrobat-text.png b/doc/context/sources/general/manuals/still/still-acrobat-text.png Binary files differnew file mode 100644 index 000000000..1c0d97e91 --- /dev/null +++ b/doc/context/sources/general/manuals/still/still-acrobat-text.png diff --git a/doc/context/sources/general/manuals/still/still-math-stripe.png b/doc/context/sources/general/manuals/still/still-math-stripe.png Binary files differnew file mode 100644 index 000000000..606ea7961 --- /dev/null +++ b/doc/context/sources/general/manuals/still/still-math-stripe.png diff --git a/doc/context/sources/general/manuals/still/still-math.tex b/doc/context/sources/general/manuals/still/still-math.tex index 473d664b7..9918b5c79 100644 --- a/doc/context/sources/general/manuals/still/still-math.tex +++ b/doc/context/sources/general/manuals/still/still-math.tex @@ -416,7 +416,7 @@ appearance depends on the font used but operating systems tend to consult multiple fonts when the core font of the editor doesn't provide a glyph. \startlinecorrection - \externalfigure[math-stripe.png][width=\textwidth] + \externalfigure[still-math-stripe.png][width=\textwidth] \stoplinecorrection Here I show a small portion of the stripe with buttons that inject the shown diff --git a/doc/context/sources/general/manuals/still/still-one.tex b/doc/context/sources/general/manuals/still/still-one.tex new file mode 100644 index 000000000..7d0c392c7 --- /dev/null +++ b/doc/context/sources/general/manuals/still/still-one.tex @@ -0,0 +1,234 @@ +% language=uk + +\usemodule[art-01,abr-02] \setupbodyfont[11pt] + +\starttext + +\startchapter[title=\LUATEX\ going stable] + +\startsection[title=Introduction] + +We're closing in on version 1.0 of \LUATEX\ and at the time of this writing (mid +April 2016) we're at version 0.95. The last decade we've reported on a regular +basis about progress in user group journals, \CONTEXT\ related documents and the +\LUATEX\ manual and it makes no sense to repeat ourselves. + +So where do we stand now? I will not go into details about what is available in +\LUATEX, for that you consult the manual but will stick to the larger picture +instead. + +\stopsection + +\startsection[title=What is it] + +First of all, as the name suggests, \LUATEX\ has the \LUA\ scripting engine on +board. Currently we're still at version 5.2 and the reason for not going 5.3 is +mainly because it has a different implementation of numbers and we cannot foresee +side effects. We will test this when we move on to \LUATEX\ version 2.0. + +The second part of the name indicates that we have some kind of \TEX\ and we +think we managed to remain largely compatible with the traditional engine. We +took most of \ETEX, much of \PDFTEX\ and some from \ALEPH\ (\OMEGA). On top of +that we added a few new primitives and extended others. + +If you look at the building blocks of \TEX, you can roughly recognize these: + +\startitemize +\startitem + an input parser (tokenizer) that includes macro expansion; its working is + well described, of course in the \TEX\ book, but more than three decades of + availability has made \TEX's behaviour rather well documented +\stopitem +\startitem + a list builder that links basic elements like characters (tagged with font + information), rules, boxes, glue and kerns together in a double linked + list of so called nodes (and noads in intermediate math lists) +\stopitem +\startitem + a language subsystem that is responsible for hyphenating words using so called + patterns and exceptions +\stopitem +\startitem + a font subsystem that provides information about glyphs properties, and that + also makes it possible to construct math symbols from snippets; it also makes + sure that the backend knows what to embed +\stopitem +\startitem + a paragraph builder that breaks a long list into lines and a page builder + that splits of chunks that can be wrapped into pages; this is all done within + given constraints using a model of rewards and penalties +\stopitem +\startitem + a first class math renderer that set the standard and has inspired modern + math font technology +\stopitem +\startitem + mechanisms for dealing with floating data, marking page related info, wrapping + stuff in boxes, adding glue, penalties and special information +\stopitem +\startitem + a backend that is responsible for wrapping everything typeset in a format that + can be printed and viewed +\stopitem +\stopitemize + +So far we're still talking of a rather generic variant of \TEX\ with \LUA\ as +extension language. Next we zoom in on some details. + +\stopsection + +\startsection[title=Where it differs] + +Given experiences with discussing extensions to the engine and given the fact +that there is never really an agreement about what makes sense or not, the +decission was made to not extend the engine any more than really needed but to +provide hooks to do that in \LUA. And, time has proven that this is a feasible +approach. On the one hand we are as good as possible faithful to the original, +and at the same time we can deal with todays and near future demands. + +Tokenization still happens as before but we can also write input parsers +ourselves. You can intercept the raw input when it gets read from file, but you +can also create scanners that you can sort of plug into the parser. Both are a +compromise between convenience and speed but powerful enough. At the input end we +now can group catcode changes (catcodes are properties of characters that control +how they are interpreted) into tables so that switching between regimes is fast. + +You can in great detail influence how data gets read from files because the \IO\ +subsystem is opened up. In fact, you have the full power of \LUA\ available when +doing so. At the same time you can print back from \LUA\ into the input stream. + +The input that makes in into \TEX, either or not intercepted and manipulated +beforehand, is to be in \UTF8. What comes out to the terminal and log is also +\UTF8, and internally all codepaths work with wide characters. Some memory +constraints have been lifted, and character related commands accept large +numbers. This comes at a price, which means that in practice the \LUATEX\ engine +can be several times slower than the 8|-|bit \PDFTEX, but of course in practice +performance is mostly determined by the efficiency of macro package, so it might +actually be faster in situations that would stress its ancestors. + +Node lists travel through \TEX\ and can be intercepted at many points. That way +you can add additional manipulations. You can for instance rely on \TEX\ for +hyphenation, ligature building and kerning but you can also plug in alternatives. +For this purpose these stages are clearly separated and less integrated (deep +down) than in traditional \TEX. There are helpers for accessing lists of nodes, +individual nodes and you can box those lists too (this is called packing). You +can adapt, create and destroy node lists at will, as long as you make sure you +feed back into \TEX\ something that makes sense. + +In order to control (or communicate with) nodes from the \TEX\ end, an attribute +mechanism was added that makes it possible to bind properties to nodes when they +get added to lists. At the \TEX\ end you can set an attribute that then gets +assigned to the currently injected nodes, while at the \LUA\ end you can query +the node for these attributes and their values. + +The language subsystem is re|-|implemented and behaves mostly the same as in the +original \TEX\ program. It has a few extensions and permits runtime loading of +patterns. In addition to language support we also have basic script support, that +is: directional information is now part of the stream and contrary to \ALEPH\ +that wraps this into extension whatsits, in \LUATEX\ we have directional nodes as +core nodes. + +The font subsystem is opened up in such a way that you can pass your own fonts to +the core. You can even construct virtual fonts. This open approach makes it +possible to support \OPENTYPE\ fonts and whatever format will show up in the +future. Of course the backend needs to embed the right data in the result file +but by then the hard work is already done. This approach fits into the always +present wish of users (and package writers) to be able to implement whatever +crazy thought one comes up with. + +The paragraph builder is a somewhat cleaned up variant of the \PDFTEX\ one, +combined with directional and boundary support from \ALEPH. The protrusion and +expansion mechanism have been redone in such a way that the front- and backend +code is better separated and is somewhat more efficient now. As one can intercept +the paragraph builder, additional functionality can be injected before, after or +at some stages in the process. + +Of course we have kept the math engine but, because we now need to support +\OPENTYPE\ math, alternative code paths have been added to deal with the kind of +information that such fonts provide. We also took the opportunity to open up the +math machinery a bit so that one can control rendering of some more complex +elements and set the spacing between elements. Because \TEX\ users are quite +traditional we had to stop somewhere, simply because legacy code has to be dealt +with. + +Most mentioned auxiliary mechanisms can be accessed via the node lists, for +instance you can locate inserts and marks in them. The backend related whatsit +nodes can be recognized as well. At any time one can query and set \TEX\ +registers and intercept boxed material. Of course some knowledge of the inner +working of \TEX\ helps here. + +The backend code is as much as possible separated from the frontend code (but +there is still some work to do there). As in \PDFTEX\ you can of course inject +arbitrary \PDF\ code and make feature rich documents. This flexibility keeps +\TEX\ current. + +\stopsection + +\startsection[title=Extras] + +Is that all? No, apart from some minor extensions that might help to make +programming somewhat easier \TEX, there are a few more fundamental additions. + +Images and reusable content (boxes) are now part of the core instead of them +being wrapped into backend specific whatsits, although of course the backend has +to provide support for it. This is more natural in the frontend (and user +interface) and also more consistent in the engine itself. All backend +functionality is now collected in three primitives that take arguments. This +permits a cleaner separation between front- and backend. + +Then there is the \METAPOST\ library, a feature already present for many years +now. It provides \TEX\ with some graphic capabilities that, given the origin, +fits nicely into the whole. The \LUATEX\ and \MPLIB\ project started about the +same time and right from the start it was our plan to combine both. + +One of the extras is of course \LUA. It not only permits us to interface to the +internals of \TEX, but it also provides the user with a way to manipulate data. +Even if you never use \LUA\ to access internals, it might still be found useful +for occasionally doing things that are hard to accomplish using the macro +langage. + +In addition to stock \LUA\ we include the \LPEG\ library, an image reading +library (related to the backend) including read access to \PDF\ files via the +used poppler library, parsing of \PDF\ content streams, zip compression, access +to the file system, the ability to run commands and socket support. Some of this +might become external libraries at some point, as we want to keep the expected +core functionality lean and mean. A nice extra is that we provide \LUAJITTEX, a +compatible variant that has a faster \LUA\ virtual machine on board. + +\stopsection + +\startsection[title=Follow up] + +The interfaces that we have now have to a large extent evolved to what we had in +mind. We started with simple experiments: just \LUA\ plus a bit of access to +registers. Then the Oriental \TEX\ project (with Idris Samawi Hamid) made it +possible to speed up development and conversion to \CCODE\ and opening up took +off. After that we gradually moved forward. + +That doesn't mean that we're done yet. The \LUATEX\ 1.0 engine will not change +much. We might add a few things, and for sure we will keep working on the code +base. The move from \PASCAL\ to \CCODE\ \WEB\ (an impressive job by itself), as +well as merging functionality of engines (kind of a challenge when you want to +remain compatible), opening up via \LUA\ (which possibilities even surprised us), +and experimenting (\CONTEXT\ users paid the price for that) took quite some time, +also because we played with proofs of concept. It helped that we used the engine +exclusively for real typesetting related work ourselves. + +We will continue to clean up and document the source and stepwise improve the +manual. If you followed the development of \CONTEXT, you will have noticed that +\MKIV\ is heavily relying on the \LUA\ interface so stability is important +(although we can relatively easy adapt to future developments as we did in the +past). However, the fact that other packages support \LUATEX\ means that we also +need to keep the 1.0 engine stable. Our challenge is to provide stability on the +one hand, but not limit ourselves to much on the other. We'll keep you posted on +what comes next. + +\blank + +Hans, Hartmut, Luigi, Taco + +\stopsection + +\stopchapter + +\stoptext diff --git a/doc/context/sources/general/manuals/still/still-sumatra-math.png b/doc/context/sources/general/manuals/still/still-sumatra-math.png Binary files differnew file mode 100644 index 000000000..0eb1417dc --- /dev/null +++ b/doc/context/sources/general/manuals/still/still-sumatra-math.png diff --git a/doc/context/sources/general/manuals/still/still-sumatra-text.png b/doc/context/sources/general/manuals/still/still-sumatra-text.png Binary files differnew file mode 100644 index 000000000..191c1a0ef --- /dev/null +++ b/doc/context/sources/general/manuals/still/still-sumatra-text.png diff --git a/doc/context/sources/general/manuals/still/still.tex b/doc/context/sources/general/manuals/still/still.tex index 28bc60c79..5fe279672 100644 --- a/doc/context/sources/general/manuals/still/still.tex +++ b/doc/context/sources/general/manuals/still/still.tex @@ -34,6 +34,7 @@ \component still-profiling %component still-fonts.tex % unfinished %component still-newlines.tex % unfinished + \component still-viewing %component still-plain \component still-opentypemath \component still-expanding diff --git a/tex/context/base/context-version.pdf b/tex/context/base/context-version.pdf Binary files differindex 69116a012..ffa980a47 100644 --- a/tex/context/base/context-version.pdf +++ b/tex/context/base/context-version.pdf diff --git a/tex/context/base/mkiv/cont-new.mkiv b/tex/context/base/mkiv/cont-new.mkiv index 06fa9865c..0886f6e81 100644 --- a/tex/context/base/mkiv/cont-new.mkiv +++ b/tex/context/base/mkiv/cont-new.mkiv @@ -11,7 +11,7 @@ %C therefore copyrighted by \PRAGMA. See mreadme.pdf for %C details. -\newcontextversion{2016.08.24 21:16} +\newcontextversion{2016.08.25 14:14} %D This file is loaded at runtime, thereby providing an excellent place for %D hacks, patches, extensions and new features. diff --git a/tex/context/base/mkiv/context.mkiv b/tex/context/base/mkiv/context.mkiv index 83a0d0162..1ca5dbb5e 100644 --- a/tex/context/base/mkiv/context.mkiv +++ b/tex/context/base/mkiv/context.mkiv @@ -39,7 +39,7 @@ %D up and the dependencies are more consistent. \edef\contextformat {\jobname} -\edef\contextversion{2016.08.24 21:16} +\edef\contextversion{2016.08.25 14:14} \edef\contextkind {beta} %D For those who want to use this: diff --git a/tex/context/base/mkiv/file-lib.lua b/tex/context/base/mkiv/file-lib.lua index b8ba69427..62cf938ba 100644 --- a/tex/context/base/mkiv/file-lib.lua +++ b/tex/context/base/mkiv/file-lib.lua @@ -48,12 +48,7 @@ function resolvers.uselibrary(specification) -- todo: reporter end for i=1,#files do local filename = files[i] - if loaded[filename] then - -- next one - else - if onlyonce then - loaded[filename] = true -- todo: base this on return value - end + if not loaded[filename] then local foundname = nil local barename = removesuffix(filename) -- direct search (we have an explicit suffix) @@ -76,10 +71,18 @@ function resolvers.uselibrary(specification) -- todo: reporter end end end - if foundname then - action(name,foundname) - elseif failure then - failure(name) + if not loaded[foundname] then + if foundname then + action(name,foundname) + if onlyonce then + loaded[foundname] = true -- todo: base this on return value + end + elseif failure then + failure(name) + end + if onlyonce then + loaded[filename] = true -- todo: base this on return value + end end end end diff --git a/tex/context/base/mkiv/font-mps.lua b/tex/context/base/mkiv/font-mps.lua index d56c25731..39c17a1cc 100644 --- a/tex/context/base/mkiv/font-mps.lua +++ b/tex/context/base/mkiv/font-mps.lua @@ -289,7 +289,7 @@ function metapost.output(kind,font,char,advance,shift,ex) if glyphs then local glyf = glyphs[index] if glyf then - local units = shapedata.units or 1000 + local units = 1000 -- factor already takes shapedata.units into account local yfactor = (sc/units) * parameters[font].factor / 655.36 local xfactor = yfactor local shift = shift or 0 diff --git a/tex/context/base/mkiv/meta-imp-outlines.mkiv b/tex/context/base/mkiv/meta-imp-outlines.mkiv index 199dadb32..7629c0c92 100644 --- a/tex/context/base/mkiv/meta-imp-outlines.mkiv +++ b/tex/context/base/mkiv/meta-imp-outlines.mkiv @@ -67,7 +67,7 @@ function metapost.showglyph(specification) end local glyph = shapeglyphs[index] if glyph and (glyph.segments or glyph.sequence) then - local units = data.fontheader and data.fontheader.emsize or 1000 + local units = shapedata.units or 1000 local factor = 100/units local paths = metapost.paths(glyph,factor) if #paths > 0 and glyph.boundingbox and glyph.width then diff --git a/tex/context/base/mkiv/publ-imp-apa.mkvi b/tex/context/base/mkiv/publ-imp-apa.mkvi index 5d11ef62f..9492b44e8 100644 --- a/tex/context/base/mkiv/publ-imp-apa.mkvi +++ b/tex/context/base/mkiv/publ-imp-apa.mkvi @@ -318,7 +318,9 @@ [apa:\s!cite:author] [\c!left={(}, \c!right={)}, - \c!inbetween={\btxcomma}] + \c!inbetween={\btxcomma}, + \c!separator:names:3={\btxcomma\texampersand\space}, + \c!separator:names:4={\btxspace\texampersand\space}] \definebtx [apa:\s!cite:default] @@ -326,7 +328,7 @@ \definebtx [apa:\s!cite:authoryears] - [apa:\s!cite:authoryear] + [apa:\s!cite:author] [\c!left=, \c!right=, \c!inbetween={\btxspace}] diff --git a/tex/context/base/mkiv/status-files.pdf b/tex/context/base/mkiv/status-files.pdf Binary files differindex b1184e1eb..50eaedba1 100644 --- a/tex/context/base/mkiv/status-files.pdf +++ b/tex/context/base/mkiv/status-files.pdf diff --git a/tex/context/base/mkiv/status-lua.pdf b/tex/context/base/mkiv/status-lua.pdf Binary files differindex 829a8576a..61861f2bd 100644 --- a/tex/context/base/mkiv/status-lua.pdf +++ b/tex/context/base/mkiv/status-lua.pdf diff --git a/tex/context/interface/mkiv/i-context.pdf b/tex/context/interface/mkiv/i-context.pdf Binary files differindex 86a044447..d2bf8cfbd 100644 --- a/tex/context/interface/mkiv/i-context.pdf +++ b/tex/context/interface/mkiv/i-context.pdf diff --git a/tex/context/interface/mkiv/i-readme.pdf b/tex/context/interface/mkiv/i-readme.pdf Binary files differindex 0f70e266c..dc1a443e2 100644 --- a/tex/context/interface/mkiv/i-readme.pdf +++ b/tex/context/interface/mkiv/i-readme.pdf diff --git a/tex/generic/context/luatex/luatex-fonts-merged.lua b/tex/generic/context/luatex/luatex-fonts-merged.lua index 2bac536cd..d86763204 100644 --- a/tex/generic/context/luatex/luatex-fonts-merged.lua +++ b/tex/generic/context/luatex/luatex-fonts-merged.lua @@ -1,6 +1,6 @@ -- merged file : c:/data/develop/context/sources/luatex-fonts-merged.lua -- parent file : c:/data/develop/context/sources/luatex-fonts.lua --- merge date : 08/24/16 21:16:06 +-- merge date : 08/25/16 14:14:14 do -- begin closure to overcome local limits and interference |
