diff options
Diffstat (limited to 'doc/context/sources/general/manuals/cld')
23 files changed, 10074 insertions, 0 deletions
diff --git a/doc/context/sources/general/manuals/cld/cld-abitoflua.tex b/doc/context/sources/general/manuals/cld/cld-abitoflua.tex new file mode 100644 index 000000000..e61507929 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-abitoflua.tex @@ -0,0 +1,869 @@ +% language=uk + +\startcomponent cld-abitoflua + +\environment cld-environment + +\startchapter[title=A bit of Lua] + +\startsection[title=The language] + +\index[lua]{\LUA} + +Small is beautiful and this is definitely true for the programming language \LUA\ +(moon in Portuguese). We had good reasons for using this language in \LUATEX: +simplicity, speed, syntax and size to mention a few. Of course personal taste +also played a role and after using a couple of scripting languages extensively +the switch to \LUA\ was rather pleasant. + +As the \LUA\ reference manual is an excellent book there is no reason to discuss +the language in great detail: just buy \quote {Programming in \LUA} by the \LUA\ +team. Nevertheless I will give a short summary of the important concepts but +consult the book if you want more details. + +\stopsection + +\startsection[title=Data types] + +\index{functions} +\index{variables} +\index{strings} +\index{numbers} +\index{booleans} +\index{tables} + +The most basic data type is \type {nil}. When we define a variable, we don't need +to give it a value: + +\starttyping +local v +\stoptyping + +Here the variable \type {v} can get any value but till that +happens it equals \type {nil}. There are simple data types like +\type {numbers}, \type {booleans} and \type {strings}. Here are +some numbers: + +\starttyping +local n = 1 + 2 * 3 +local x = 2.3 +\stoptyping + +Numbers are always floats \footnote {This is true for all versions upto 5.2 but +following version can have a more hybrid model.} and you can use the normal +arithmetic operators on them as well as functions defined in the math library. +Inside \TEX\ we have only integers, although for instance dimensions can be +specified in points using floats but that's more syntactic sugar. One reason for +using integers in \TEX\ has been that this was the only way to guarantee +portability across platforms. However, we're 30 years along the road and in \LUA\ +the floats are implemented identical across platforms, so we don't need to worry +about compatibility. + +Strings in \LUA\ can be given between quotes or can be so called long strings +forced by square brackets. + +\starttyping +local s = "Whatever" +local t = s .. ' you want' +local u = t .. [[ to know]] .. [[--[ about Lua!]--]] +\stoptyping + +The two periods indicate a concatenation. Strings are hashed, so when you say: + +\starttyping +local s = "Whatever" +local t = "Whatever" +local u = t +\stoptyping + +only one instance of \type {Whatever} is present in memory and this fact makes +\LUA\ very efficient with respect to strings. Strings are constants and therefore +when you change variable \type {s}, variable \type {t} keeps its value. When you +compare strings, in fact you compare pointers, a method that is really fast. This +compensates the time spent on hashing pretty well. + +Booleans are normally used to keep a state or the result from an expression. + +\starttyping +local b = false +local c = n > 10 and s == "whatever" +\stoptyping + +The other value is \type {true}. There is something that you need +to keep in mind when you do testing on variables that are yet +unset. + +\starttyping +local b = false +local n +\stoptyping + +The following applies when \type {b} and \type {n} are defined this way: + +\starttabulate[|Tl|Tl|] +\NC b == false \NC true \NC \NR +\NC n == false \NC false \NC \NR +\NC n == nil \NC true \NC \NR +\NC b == nil \NC false \NC \NR +\NC b == n \NC false \NC \NR +\NC n == nil \NC true \NC \NR +\stoptabulate + +Often a test looks like: + +\starttyping +if somevar then + ... +else + ... +end +\stoptyping + +In this case we enter the else branch when \type {somevar} is either \type {nil} +or \type {false}. It also means that by looking at the code we cannot beforehand +conclude that \type {somevar} equals \type {true} or something else. If you want +to really distinguish between the two cases you can be more explicit: + +\starttyping +if somevar == nil then + ... +elseif somevar == false then + ... +else + ... +end +\stoptyping + +or + +\starttyping +if somevar == true then + ... +else + ... +end +\stoptyping + +but such an explicit test is seldom needed. + +There are a few more data types: tables and functions. Tables are very important +and you can recognize them by the same curly braces that make \TEX\ famous: + +\starttyping +local t = { 1, 2, 3 } +local u = { a = 4, b = 9, c = 16 } +local v = { [1] = "a", [3] = "2", [4] = false } +local w = { 1, 2, 3, a = 4, b = 9, c = 16 } +\stoptyping + +The \type {t} is an indexed table and \type {u} a hashed table. Because the +second slot is empty, table \type {v} is partially indexed (slot 1) and partially +hashed (the others). There is a gray area there, for instance, what happens when +you nil a slot in an indexed table? In practice you will not run into problems as +you will either use a hashed table, or an indexed table (with no holes), so table +\type {w} is not uncommon. + +We mentioned that strings are in fact shared (hashed) but that an assignment of a +string to a variable makes that variable behave like a constant. Contrary to +that, when you assign a table, and then copy that variable, both variables can be +used to change the table. Take this: + +\starttyping +local t = { 1, 2, 3 } +local u = t +\stoptyping + +We can change the content of the table as follows: + +\starttyping +t[1], t[3] = t[3], t[1] +\stoptyping + +Here we swap two cells. This is an example of a parallel assigment. However, the +following does the same: + +\starttyping +t[1], t[3] = u[3], u[1] +\stoptyping + +After this, both \type {t} and \type {u} still share the same table. This kind of +behaviour is quite natural. Keep in mind that expressions are evaluated first, so + +\starttyping +t[#t+1], t[#t+1] = 23, 45 +\stoptyping + +Makes no sense, as the values end up in the same slot. There is no gain in speed +so using parallel assignments is mostly a convenience feature. + +There are a few specialized data types in \LUA, like \type {coroutines} (built +in), \type {file} (when opened), \type {lpeg} (only when this library is linked +in or loaded). These are called \quote {userdata} objects and in \LUATEX\ we have +more userdata objects as we will see in later chapters. Of them nodes are the +most noticeable: they are the core data type of the \TEX\ machinery. Other +libraries, like \type {math} and \type {bit32} are just collections of functions +operating on numbers. + +Functions look like this: + +\starttyping +function sum(a,b) + print(a, b, a + b) +end +\stoptyping + +or this: + +\starttyping +function sum(a,b) + return a + b +end +\stoptyping + +There can be many arguments of all kind of types and there can be multiple return +values. A function is a real type, so you can say: + +\starttyping +local f = function(s) print("the value is: " .. s) end +\stoptyping + +In all these examples we defined variables as \type {local}. This is a good +practice and avoids clashes. Now watch the following: + +\starttyping +local n = 1 + +function sum(a,b) + n = n + 1 + return a + b +end + +function report() + print("number of summations: " .. n) +end +\stoptyping + +Here the variable \type {n} is visible after its definition and accessible for +the two global functions. Actually the variable is visible to all the code +following, unless of course we define a new variable with the same name. We can +hide \type {n} as follows: + +\starttyping +do + local n = 1 + + sum = function(a,b) + n = n + 1 + return a + b + end + + report = function() + print("number of summations: " .. n) + end +end +\stoptyping + +This example also shows another way of defining the function: by assignment. + +The \typ {do ... end} creates a so called closure. There are many places where +such closures are created, for instance in function bodies or branches like \typ +{if ... then ... else}. This means that in the following snippet, variable \type +{b} is not seen after the end: + +\starttyping +if a > 10 then + local b = a + 10 + print(b*b) +end +\stoptyping + +When you process a blob of \LUA\ code in \TEX\ (using \type {\directlua} or \type +{\latelua}) it happens in a closure with an implied \typ {do ... end}. So, \type +{local} defined variables are really local. + +\stopsection + +\startsection[title=\TEX's data types] + +We mentioned \type {numbers}. At the \TEX\ end we have counters as well as +dimensions. Both are numbers but dimensions are specified differently + +\starttyping +local n = tex.count[0] +local m = tex.dimen.lineheight +local o = tex.sp("10.3pt") -- sp or 'scaled point' is the smallest unit +\stoptyping + +The unit of dimension is \quote {scaled point} and this is a pretty small unit: +10 points equals to 655360 such units. + +Another accessible data type is tokens. They are automatically converted to +strings and vice versa. + +\starttyping +tex.toks[0] = "message" +print(tex.toks[0]) +\stoptyping + +Be aware of the fact that the tokens are letters so the following will come out +as text and not issue a message: + +\starttyping +tex.toks[0] = "\message{just text}" +print(tex.toks[0]) +\stoptyping + +\stopsection + +\startsection[title=Control structures] + +\index{loops} + +Loops are not much different from other languages: we have \typ {for ... do}, +\typ {while ... do} and \typ {repeat ... until}. We start with the simplest case: + +\starttyping +for index=1,10 do + print(index) +end +\stoptyping + +You can specify a step and go downward as well: + +\starttyping +for index=22,2,-2 do + print(index) +end +\stoptyping + +Indexed tables can be traversed this way: + +\starttyping +for index=1,#list do + print(index, list[index]) +end +\stoptyping + +Hashed tables on the other hand are dealt with as follows: + +\starttyping +for key, value in next, list do + print(key, value) +end +\stoptyping + +Here \type {next} is a built in function. There is more to say about this +mechanism but the average user will use only this variant. Slightly less +efficient is the following, more readable variant: + +\starttyping +for key, value in pairs(list) do + print(key, value) +end +\stoptyping + +and for an indexed table: + +\starttyping +for index, value in ipairs(list) do + print(index, value) +end +\stoptyping + +The function call to \type {pairs(list)} returns \typ {next, list} so there is an +(often neglectable) extra overhead of one function call. + +The other two loop variants, \type {while} and \type {repeat}, are similar. + +\starttyping +i = 0 +while i < 10 do + i = i + 1 + print(i) +end +\stoptyping + +This can also be written as: + +\starttyping +i = 0 +repeat + i = i + 1 + print(i) +until i = 10 +\stoptyping + +Or: + +\starttyping +i = 0 +while true do + i = i + 1 + print(i) + if i = 10 then + break + end +end +\stoptyping +\stopsection + +Of course you can use more complex expressions in such constructs. + +\startsection[title=Conditions] + +\index{expressions} + +Conditions have the following form: + +\starttyping +if a == b or c > d or e then + ... +elseif f == g then + ... +else + ... +end +\stoptyping + +Watch the double \type {==}. The complement of this is \type {~=}. Precedence is +similar to other languages. In practice, as strings are hashed. Tests like + +\starttyping +if key == "first" then + ... +end +\stoptyping + +and + +\starttyping +if n == 1 then + ... +end +\stoptyping + +are equally efficient. There is really no need to use numbers to identify states +instead of more verbose strings. + +\stopsection + +\startsection[title=Namespaces] + +\index{namespaces} + +Functionality can be grouped in libraries. There are a few default libraries, +like \type {string}, \type {table}, \type {lpeg}, \type {math}, \type {io} and +\type {os} and \LUATEX\ adds some more, like \type {node}, \type {tex} and \type +{texio}. + +A library is in fact nothing more than a bunch of functionality organized using a +table, where the table provides a namespace as well as place to store public +variables. Of course there can be local (hidden) variables used in defining +functions. + +\starttyping +do + mylib = { } + + local n = 1 + + function mylib.sum(a,b) + n = n + 1 + return a + b + end + + function mylib.report() + print("number of summations: " .. n) + end +end +\stoptyping + +The defined function can be called like: + +\starttyping +mylib.report() +\stoptyping + +You can also create a shortcut, This speeds up the process because there are less +lookups then. In the following code multiple calls take place: + +\starttyping +local sum = mylib.sum + +for i=1,10 do + for j=1,10 do + print(i, j, sum(i,j)) + end +end + +mylib.report() +\stoptyping + +As \LUA\ is pretty fast you should not overestimate the speedup, especially not +when a function is called seldom. There is an important side effect here: in the +case of: + +\starttyping + print(i, j, sum(i,j)) +\stoptyping + +the meaning of \type {sum} is frozen. But in the case of + +\starttyping + print(i, j, mylib.sum(i,j)) +\stoptyping + +The current meaning is taken, that is: each time the interpreter will access +\type {mylib} and get the current meaning of \type {sum}. And there can be a good +reason for this, for instance when the meaning is adapted to different +situations. + +In \CONTEXT\ we have quite some code organized this way. Although much is exposed +(if only because it is used all over the place) you should be careful in using +functions (and data) that are still experimental. There are a couple of general +libraries and some extend the core \LUA\ libraries. You might want to take a look +at the files in the distribution that start with \type {l-}, like \type +{l-table.lua}. These files are preloaded.\footnote {In fact, if you write scripts +that need their functionality, you can use \type {mtxrun} to process the script, +as \type {mtxrun} has the core libraries preloaded as well.} For instance, if you +want to inspect a table, you can say: + +\starttyping +local t = { "aap", "noot", "mies" } +table.print(t) +\stoptyping + +You can get an overview of what is implemented by running the following command: + +\starttyping +context s-tra-02 --mode=tablet +\stoptyping + +{\em todo: add nice synonym for this module and also add helpinfo at the to so +that we can do \type {context --styles}} + +\stopsection + +\startsection[title=Comment] + +\index{comment} + +You can add comments to your \LUA\ code. There are basically two methods: one +liners and multi line comments. + +\starttyping +local option = "test" -- use this option with care + +local method = "unknown" --[[comments can be very long and when entered + this way they and span multiple lines]] +\stoptyping + +The so called long comments look like long strings preceded by \type {--} and +there can be more complex boundary sequences. + +\stopsection + +\startsection[title=Pitfalls] + +Sometimes \type {nil} can bite you, especially in tables, as they have a dual nature: +indexed as well as hashed. + +\startbuffer +\startluacode +local n1 = # { nil, 1, 2, nil } -- 3 +local n2 = # { nil, nil, 1, 2, nil } -- 0 + +context("n1 = %s and n2 = %s",n1,n2) +\stopluacode +\stopbuffer + +\typebuffer + +results in: \getbuffer + +So, you cannot really depend on the length operator here. On the other hand, with: + +\startbuffer +\startluacode +local function check(...) + return select("#",...) +end + +local n1 = check ( nil, 1, 2, nil ) -- 4 +local n2 = check ( nil, nil, 1, 2, nil ) -- 5 + +context("n1 = %s and n2 = %s",n1,n2) +\stopluacode +\stopbuffer + +\typebuffer + +we get: \getbuffer, so the \type {select} is quite useable. However, that function also +has its specialities. The following example needs some close reading: + +\startbuffer +\startluacode +local function filter(n,...) + return select(n,...) +end + +local v1 = { filter ( 1, 1, 2, 3 ) } +local v2 = { filter ( 2, 1, 2, 3 ) } +local v3 = { filter ( 3, 1, 2, 3 ) } + +context("v1 = %+t and v2 = %+t and v3 = %+t",v1,v2,v3) +\stopluacode +\stopbuffer + +\typebuffer + +We collect the result in a table and show the concatination: + +\getbuffer + +So, what you effectively get is the whole list starting with the given offset. + +\startbuffer +\startluacode +local function filter(n,...) + return (select(n,...)) +end + +local v1 = { filter ( 1, 1, 2, 3 ) } +local v2 = { filter ( 2, 1, 2, 3 ) } +local v3 = { filter ( 3, 1, 2, 3 ) } + +context("v1 = %+t and v2 = %+t and v3 = %+t",v1,v2,v3) +\stopluacode +\stopbuffer + +\typebuffer + +Now we get: \getbuffer. The extra \type {()} around the result makes sure that +we only get one return value. + +Of course the same effect can be achieved as follows: + +\starttyping +local function filter(n,...) + return select(n,...) +end + +local v1 = filter ( 1, 1, 2, 3 ) +local v2 = filter ( 2, 1, 2, 3 ) +local v3 = filter ( 3, 1, 2, 3 ) + +context("v1 = %s and v2 = %s and v3 = %s",v1,v2,v3) +\stoptyping + +\stopsection + +\startsection[title={A few suggestions}] + +You can wrap all kind of functionality in functions but sometimes it makes no +sense to add the overhead of a call as the same can be done with hardly any code. + +If you want a slice of a table, you can copy the range needed to a new table. A +simple version with no bounds checking is: + +\starttyping +local new = { } for i=a,b do new[#new+1] = old[i] end +\stoptyping + +Another, much faster, variant is the following. + +\starttyping +local new = { unpack(old,a,b) } +\stoptyping + +You can use this variant for slices that are not extremely large. The function +\type {table.sub} is an equivalent: + +\starttyping +local new = table.sub(old,a,b) +\stoptyping + +An indexed table is empty when its size equals zero: + +\starttyping +if #indexed == 0 then ... else ... end +\stoptyping + +Sometimes this is better: + +\starttyping +if indexed and #indexed == 0 then ... else ... end +\stoptyping + +So how do we test if a hashed table is empty? We can use the +\type {next} function as in: + +\starttyping +if hashed and next(indexed) then ... else ... end +\stoptyping + +Say that we have the following table: + +\starttyping +local t = { a=1, b=2, c=3 } +\stoptyping + +The call \type {next(t)} returns the first key and value: + +\starttyping +local k, v = next(t) -- "a", 1 +\stoptyping + +The second argument to \type {next} can be a key in which case the +following key and value in the hash table is returned. The result +is not predictable as a hash is unordered. The generic for loop +uses this to loop over a hashed table: + +\starttyping +for k, v in next, t do + ... +end +\stoptyping + +Anyway, when \type {next(t)} returns zero you can be sure that the table is +empty. This is how you can test for exactly one entry: + +\starttyping +if t and not next(t,next(t)) then ... else ... end +\stoptyping + +Here it starts making sense to wrap it into a function. + +\starttyping +function table.has_one_entry(t) + t and not next(t,next(t)) +end +\stoptyping + +On the other hand, this is not that usefull, unless you can spent the runtime on +it: + +\starttyping +function table.is_empty(t) + return not t or not next(t) +end +\stoptyping + +\stopsection + +\startsection[title=Interfacing] + +We have already seen that you can embed \LUA\ code using commands like: + +\starttyping +\startluacode + print("this works") +\stopluacode +\stoptyping + +This command should not be confused with: + +\starttyping +\startlua + print("this works") +\stoplua +\stoptyping + +The first variant has its own catcode regime which means that tokens between the start +and stop command are treated as \LUA\ tokens, with the exception of \TEX\ commands. The +second variant operates under the regular \TEX\ catcode regime. + +Their short variants are \type {\ctxluacode} and \type {\ctxlua} as in: + +\starttyping +\ctxluacode{print("this works")} +\ctxlua{print("this works")} +\stoptyping + +In practice you will probably use \type {\startluacode} when using or defining % \stopluacode +a blob of \LUA\ and \type {\ctxlua} for inline code. Keep in mind that the +longer versions need more initialization and have more overhead. + +There are some more commands. For instance \type {\ctxcommand} can be used as +an efficient way to access functions in the \type {commands} namespace. The +following two calls are equivalent: + +\starttyping +\ctxlua {commands.thisorthat("...")} +\ctxcommand {thisorthat("...")} +\stoptyping + +There are a few shortcuts to the \type {context} namespace. Their use can best be +seen from their meaning: + +\starttyping +\cldprocessfile#1{\directlua{context.runfile("#1")}} +\cldloadfile #1{\directlua{context.loadfile("#1")}} +\cldcontext #1{\directlua{context(#1)}} +\cldcommand #1{\directlua{context.#1}} +\stoptyping + +The \type {\directlua{}} command can also be implemented using the token parser +and \LUA\ itself. A variant is therefore \type {\luascript{}} which can be +considered an alias but with a bit different error reporting. A variant on this +is the \type {\luathread {name} {code}} command. Here is an example of their +usage: + +\startbuffer +\luascript { context("foo 1:") context(i) } \par +\luathread {test} { i = 10 context("bar 1:") context(i) } \par +\luathread {test} { context("bar 2:") context(i) } \par +\luathread {test} {} % resets +\luathread {test} { context("bar 3:") context(i) } \par +\luascript { context("foo 2:") context(i) } \par +\stopbuffer + +\typebuffer + +These commands result in: + +\startpacked \getbuffer \stoppacked + +% \testfeatureonce{100000}{\directlua {local a = 10 local a = 10 local a = 10}} % 0.53s +% \testfeatureonce{100000}{\luascript {local a = 10 local a = 10 local a = 10}} % 0.62s +% \testfeatureonce{100000}{\luathread {test} {local a = 10 local a = 10 local a = 10}} % 0.79s + +The variable \type {i} is local to the thread (which is not really a thread in +\LUA\ but more a named piece of code that provides an environment which is shared +over the calls with the same name. You will probably never need these. + +Each time a call out to \LUA\ happens the argument eventually gets parsed, converted +into tokens, then back into a string, compiled to bytecode and executed. The next +example code shows a mechanism that avoids this: + +\starttyping +\startctxfunction MyFunctionA + context(" A1 ") +\stopctxfunction + +\startctxfunctiondefinition MyFunctionB + context(" B2 ") +\stopctxfunctiondefinition +\stoptyping + +The first command associates a name with some \LUA\ code and that code can be +executed using: + +\starttyping +\ctxfunction{MyFunctionA} +\stoptyping + +The second definition creates a command, so there we do: + +\starttyping +\MyFunctionB +\stoptyping + +There are some more helpers but for use in document sources they make less sense. You +can always browse the source code for examples. + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-afewdetails.tex b/doc/context/sources/general/manuals/cld/cld-afewdetails.tex new file mode 100644 index 000000000..6c0cf3afa --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-afewdetails.tex @@ -0,0 +1,398 @@ +% language=uk + +\startcomponent cld-afewdetails + +\environment cld-environment + +\startchapter[title=A few Details] + +\startsection[title=Variables] + +\index{user interface} + +Normally it makes most sense to use the English version of \CONTEXT. The +advantage is that you can use English keywords, as in: + +\starttyping +context.framed( { + frame = "on", + }, + "some text" +) +\stoptyping + +If you use the Dutch interface it looks like this: + +\starttyping +context.omlijnd( { + kader = "aan", + }, + "wat tekst" +) +\stoptyping + +A rather neutral way is: + +\starttyping +context.framed( { + frame = interfaces.variables.on, + }, + "some text" +) +\stoptyping + +But as said, normally you will use the English user interface so you can forget +about these matters. However, in the \CONTEXT\ core code you will often see the +variables being used this way because there we need to support all user +interfaces. + +\stopsection + +\startsection[title=Modes] + +\index{modes} +\index{systemmodes} +\index{constants} + +Context carries a concept of modes. You can use modes to create conditional +sections in your style (and|/|or content). You can control modes in your styles +or you can set them at the command line or in job control files. When a mode test +has to be done at processing time, then you need constructs like the following: + +\starttyping +context.doifmodeelse( "screen", + function() + ... -- mode == screen + end, + function() + ... -- mode ~= screen + end +) +\stoptyping + +However, often a mode does not change during a run, and then we can use the +following method: + +\starttyping +if tex.modes["screen"] then + ... +else + ... +end +\stoptyping + +Watch how the \type {modes} table lives in the \type {tex} namespace. We also +have \type {systemmodes}. At the \TEX\ end these are mode names preceded by a +\type {*}, so the following code is similar: + +\starttyping +if tex.modes["*mymode"] then + -- this is the same +elseif tex.systemmodes["mymode"] then + -- test as this +else + -- but not this +end +\stoptyping + +Inside \CONTEXT\ we also have so called constants, and again these can be +consulted at the \LUA\ end: + +\starttyping +if tex.constants["someconstant'] then + ... +else + ... +end +\stoptyping + +But you will hardly need these and, as they are often not public, their +meaning can change, unless of course they {\em are} documented as public. + +\stopsection + +\startsection[title={Token lists}] + +\index{tokens} + +There is normally no need to mess around with nodes and tokens at the \LUA\ end +yourself. However, if you do, then you might want to flush them as well. Say that +at the \TEX\ end we have said: + +\startbuffer +\toks0 = {Don't get \inframed{framed}!} +\stopbuffer + +\typebuffer \getbuffer + +Then at the \LUA\ end you can say: + +\startbuffer +context(tex.toks[0]) +\stopbuffer + +\typebuffer + +and get: \ctxluabuffer\ In fact, token registers are exposed as strings so here, +register zero has type \type {string} and is treated as such. + +\startbuffer +context("< %s >",tex.toks[0]) +\stopbuffer + +\typebuffer + +This gives: \ctxluabuffer. But beware, if you go the reverse way, you don't get +what you might expect: + +\startbuffer +tex.toks[0] = [[\framed{oeps}]] +\stopbuffer + +\typebuffer \ctxluabuffer + +If we now say \type{\the\toks0} we will get {\tttf \the\toks0} as +all tokens are considered to be letters. + +\stopsection + +\startsection[title={Node lists}] + +\index{nodes} + +If you're not deep into \TEX\ you will never feel the need to manipulate node +lists yourself, but you might want to flush boxes. As an example we put something +in box zero (one of the scratch boxes). + +\startbuffer +\setbox0 = \hbox{Don't get \inframed{framed}!} +\stopbuffer + +\typebuffer \getbuffer + +At the \TEX\ end you can flush this box (\type {\box0}) or take a copy +(\type{\copy0}). At the \LUA\ end you would do: + +\starttyping +context.copy() +context.direct(0) +\stoptyping + +or: + +\starttyping +context.copy(false,0) +\stoptyping + +but this works as well: + +\startbuffer +context(node.copy_list(tex.box[0])) +\stopbuffer + +\typebuffer + +So we get: \ctxluabuffer\ If you do: + +\starttyping +context(tex.box[0]) +\stoptyping + +you also need to make sure that the box is freed but let's not go into those +details now. + +Here is an example if messing around with node lists that get seen before a +paragraph gets broken into lines, i.e.\ when hyphenation, font manipulation etc +take place. First we define some colors: + +\startbuffer +\definecolor[mynesting:0][r=.6] +\definecolor[mynesting:1][g=.6] +\definecolor[mynesting:2][r=.6,g=.6] +\stopbuffer + +\typebuffer \getbuffer + +Next we define a function that colors nodes in such a way that we can see the +different processing stages. + +\startbuffer +\startluacode +local enabled = false +local count = 0 +local setcolor = nodes.tracers.colors.set + +function userdata.processmystuff(head) + if enabled then + local color = "mynesting:" .. (count % 3) + -- for n in node.traverse(head) do + for n in node.traverse_id(nodes.nodecodes.glyph,head) do + setcolor(n,color) + end + count = count + 1 + return head, true + end + return head, false +end + +function userdata.enablemystuff() + enabled = true +end + +function userdata.disablemystuff() + enabled = false +end +\stopluacode +\stopbuffer + +\typebuffer \getbuffer + +We hook this function into the normalizers category of the processor callbacks: + +\startbuffer +\startluacode +nodes.tasks.appendaction("processors", "normalizers", "userdata.processmystuff") +\stopluacode +\stopbuffer + +\typebuffer \getbuffer + +We now can enable this mechanism and show an example: + +\startbuffer +\startbuffer +Node lists are processed \hbox {nested from \hbox{inside} out} which is not +what you might expect. But, \hbox{coloring} does not \hbox {happen} really +nested here, more \hbox {in} \hbox {the} \hbox {order} \hbox {of} \hbox +{processing}. +\stopbuffer + +\ctxlua{userdata.enablemystuff()} +\par \getbuffer \par +\ctxlua{userdata.disablemystuff()} +\stopbuffer + +\typebuffer + +The \type {\par} is needed because otherwise the processing is already disabled +before the paragraph gets seen by \TEX. + +\blank \getbuffer \blank + +\startbuffer +\startluacode +nodes.tasks.disableaction("processors", "userdata.processmystuff") +\stopluacode +\stopbuffer + +\typebuffer + +Instead of using an boolean to control the state, we can also do this: + +\starttyping +\startluacode +local count = 0 +local setcolor = nodes.tracers.colors.set + +function userdata.processmystuff(head) + count = count + 1 + local color = "mynesting:" .. (count % 3) + for n in node.traverse_id(nodes.nodecodes.glyph,head) do + setcolor(n,color) + end + return head, true +end + +nodes.tasks.appendaction("processors", "after", "userdata.processmystuff") +\stopluacode +\stoptyping + +\startbuffer +\startluacode +nodes.tasks.disableaction("processors", "userdata.processmystuff") +\stopluacode +\stopbuffer + +Disabling now happens with: + +\typebuffer \getbuffer + +As you might want to control these things in more details, a simple helper +mechanism was made: markers. The following example code shows the way: + +\startbuffer +\definemarker[mymarker] +\stopbuffer + +\typebuffer \getbuffer + +Again we define some colors: + +\startbuffer +\definecolor[mymarker:1][r=.6] +\definecolor[mymarker:2][g=.6] +\definecolor[mymarker:3][r=.6,g=.6] +\stopbuffer + +\typebuffer \getbuffer + +The \LUA\ code like similar to the code presented before: + +\startbuffer +\startluacode +local setcolor = nodes.tracers.colors.setlist +local getmarker = nodes.markers.get +local hlist_code = nodes.codes.hlist +local traverse_id = node.traverse_id + +function userdata.processmystuff(head) + for n in traverse_id(hlist_code,head) do + local m = getmarker(n,"mymarker") + if m then + setcolor(n.list,"mymarker:" .. m) + end + end + return head, true +end + +nodes.tasks.appendaction("processors", "after", "userdata.processmystuff") +nodes.tasks.disableaction("processors", "userdata.processmystuff") +\stopluacode +\stopbuffer + +\typebuffer \getbuffer + +This time we disabled the processor (if only because in this document we don't +want the overhead. + +\startbuffer +\startluacode +nodes.tasks.enableaction("processors", "userdata.processmystuff") +\stopluacode + +Node lists are processed \hbox \boxmarker{mymarker}{1} {nested from \hbox{inside} +out} which is not what you might expect. But, \hbox {coloring} does not \hbox +{happen} really nested here, more \hbox {in} \hbox \boxmarker{mymarker}{2} {the} +\hbox {order} \hbox {of} \hbox \boxmarker{mymarker}{3} {processing}. + +\startluacode +nodes.tasks.disableaction("processors", "userdata.processmystuff") +\stopluacode +\stopbuffer + +\typebuffer + +The result looks familiar: + +\getbuffer + +% We don't want the burden of this demo to cary on: + +% {\em If there's enough interest I will expand this section with some basic +% information on what nodes are.} + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-backendcode.tex b/doc/context/sources/general/manuals/cld/cld-backendcode.tex new file mode 100644 index 000000000..9c1284baa --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-backendcode.tex @@ -0,0 +1,388 @@ +% language=uk + +\startcomponent cld-backendcode + +\environment cld-environment + +% derived from hybrid + +\startchapter[title={Backend code}] + +\startsection [title={Introduction}] + +In \CONTEXT\ we've always separated the backend code in so called driver files. +This means that in the code related to typesetting only calls to the \API\ take +place, and no backend specific code is to be used. Currently a \PDF\ backend is +supported as well as an \XML\ export. \footnote {This chapter is derived from an +article on these matters. You can find nore information in \type {hybrid.pdf}.} + +Some \CONTEXT\ users like to add their own \PDF\ specific code to their styles or +modules. However, such extensions can interfere with existing code, especially +when resources are involved. Therefore the construction of \PDF\ data structures +and resources is rather controlled and has to be done via the official helper +macros. + +\stopsection + +\startsection [title={Structure}] + +A \PDF\ file is a tree of indirect objects. Each object has a number and the file +contains a table (or multiple tables) that relates these numbers to positions in +a file (or position in a compressed object stream). That way a file can be viewed +without reading all data: a viewer only loads what is needed. + +\starttyping +1 0 obj << + /Name (test) /Address 2 0 R +>> +2 0 obj [ + (Main Street) (24) (postal code) (MyPlace) +] +\stoptyping + +For the sake of the discussion we consider strings like \type {(test)} also to be +objects. In the next table we list what we can encounter in a \PDF\ file. There +can be indirect objects in which case a reference is used (\type{2 0 R}) and +direct ones. + +It all starts in the document's root object. From there we access the page tree +and resources. Each page carries its own resource information which makes random +access easier. A page has a page stream and there we find the to be rendered +content as a mixture of (\UNICODE) strings and special drawing and rendering +operators. Here we will not discuss them as they are mostly generated by the +engine itself or dedicated subsystems like the \METAPOST\ converter. There we use +literal or \type {\latelua} whatsits to inject code into the current stream. + +\stopsection + +\startsection [title={Data types}] + +There are several datatypes in \PDF\ and we support all of them one way or the +other. + +\starttabulate[|l|l|p|] +\FL +\NC \bf type \NC \bf form \NC \bf meaning \NC \NR +\TL +\NC constant \NC \type{/...} \NC A symbol (prescribed string). \NC \NR +\NC string \NC \type{(...)} \NC A sequence of characters in pdfdoc + encoding \NC \NR +\NC unicode \NC \type{<...>} \NC A sequence of characters in utf16 + encoding \NC \NR +\NC number \NC \type{3.1415} \NC A number constant. \NC \NR +\NC boolean \NC \type{true/false} \NC A boolean constant. \NC \NR +\NC reference \NC \type{N 0 R} \NC A reference to an object \NC \NR +\NC dictionary \NC \type{<< ... >>} \NC A collection of key value pairs + where the value itself is an (indirect) object. + \NC \NR +\NC array \NC \type{[ ... ]} \NC A list of objects or references to + objects. \NC \NR +\NC stream \NC \NC A sequence of bytes either or not packaged with + a dictionary that contains descriptive data. \NC \NR +\NC xform \NC \NC A special kind of object containing an reusable + blob of data, for example an image. \NC \NR +\LL +\stoptabulate + +While writing additional backend code, we mostly create dictionaries. + +\starttyping +<< /Name (test) /Address 2 0 R >> +\stoptyping + +In this case the indirect object can look like: + +\starttyping +[ (Main Street) (24) (postal code) (MyPlace) ] +\stoptyping + +The \LUATEX\ manual mentions primitives like \type {\pdfobj}, \type {\pdfannot}, +\type {\pdfcatalog}, etc. However, in \MKIV\ no such primitives are used. You can +still use many of them but those that push data into document or page related +resources are overloaded to do nothing at all. + +In the \LUA\ backend code you will find function calls like: + +\starttyping +local d = lpdf.dictionary { + Name = lpdf.string("test"), + Address = lpdf.array { + "Main Street", "24", "postal code", "MyPlace", + } +} +\stoptyping + +Equaly valid is: + +\starttyping +local d = lpdf.dictionary() +d.Name = "test" +\stoptyping + +Eventually the object will end up in the file using calls like: + +\starttyping +local r = lpdf.immediateobject(tostring(d)) +\stoptyping + +or using the wrapper (which permits tracing): + +\starttyping +local r = lpdf.flushobject(d) +\stoptyping + +The object content will be serialized according to the formal specification so +the proper \type {<< >>} etc.\ are added. If you want the content instead you can +use a function call: + +\starttyping +local dict = d() +\stoptyping + +An example of using references is: + +\starttyping +local a = lpdf.array { + "Main Street", "24", "postal code", "MyPlace", +} +local d = lpdf.dictionary { + Name = lpdf.string("test"), + Address = lpdf.reference(a), +} +local r = lpdf.flushobject(d) +\stoptyping + +\stopsection + +We have the following creators. Their arguments are optional. + +\starttabulate[|l|p|] +\FL +\NC \bf function \NC \bf optional parameter \NC \NR +\TL +\NC \type{lpdf.null} \NC \NC \NR +\NC \type{lpdf.number} \NC number \NC \NR +\NC \type{lpdf.constant} \NC string \NC \NR +\NC \type{lpdf.string} \NC string \NC \NR +\NC \type{lpdf.unicode} \NC string \NC \NR +\NC \type{lpdf.boolean} \NC boolean \NC \NR +\NC \type{lpdf.array} \NC indexed table of objects \NC \NR +\NC \type{lpdf.dictionary} \NC hash with key/values \NC \NR +%NC \type{lpdf.stream} \NC indexed table of operators \NC \NR +\NC \type{lpdf.reference} \NC string \NC \NR +\NC \type{lpdf.verbose} \NC indexed table of strings \NC \NR +\LL +\stoptabulate + +\ShowLuaExampleString{tostring(lpdf.null())} +\ShowLuaExampleString{tostring(lpdf.number(123))} +\ShowLuaExampleString{tostring(lpdf.constant("whatever"))} +\ShowLuaExampleString{tostring(lpdf.string("just a string"))} +\ShowLuaExampleString{tostring(lpdf.unicode("just a string"))} +\ShowLuaExampleString{tostring(lpdf.boolean(true))} +\ShowLuaExampleString{tostring(lpdf.array { 1, lpdf.constant("c"), true, "str" })} +\ShowLuaExampleString{tostring(lpdf.dictionary { a=1, b=lpdf.constant("c"), d=true, e="str" })} +%ShowLuaExampleString{tostring(lpdf.stream("whatever"))} +\ShowLuaExampleString{tostring(lpdf.reference(123))} +\ShowLuaExampleString{tostring(lpdf.verbose("whatever"))} + +\stopsection + +\startsection[title={Managing objects}] + +Flushing objects is done with: + +\starttyping +lpdf.flushobject(obj) +\stoptyping + +Reserving object is or course possible and done with: + +\starttyping +local r = lpdf.reserveobject() +\stoptyping + +Such an object is flushed with: + +\starttyping +lpdf.flushobject(r,obj) +\stoptyping + +We also support named objects: + +\starttyping +lpdf.reserveobject("myobject") + +lpdf.flushobject("myobject",obj) +\stoptyping + +A delayed object is created with: + +\starttyping +local ref = pdf.delayedobject(data) +\stoptyping + +The data will be flushed later using the object number that is returned (\type +{ref}). When you expect that many object with the same content are used, you can +use: + +\starttyping +local obj = lpdf.shareobject(data) +local ref = lpdf.shareobjectreference(data) +\stoptyping + +This one flushes the object and returns the object number. Already defined +objects are reused. In addition to this code driven optimization, some other +optimization and reuse takes place but all that happens without user +intervention. Only use this when it's really needed as it might consume more +memory and needs more processing time. + +\startsection [title={Resources}] + +While \LUATEX\ itself will embed all resources related to regular typesetting, +\MKIV\ has to take care of embedding those related to special tricks, like +annotations, spot colors, layers, shades, transparencies, metadata, etc. Because +third party modules (like tikz) also can add resources we provide some macros +that makes sure that no interference takes place: + +\starttyping +\pdfbackendsetcatalog {key}{string} +\pdfbackendsetinfo {key}{string} +\pdfbackendsetname {key}{string} + +\pdfbackendsetpageattribute {key}{string} +\pdfbackendsetpagesattribute{key}{string} +\pdfbackendsetpageresource {key}{string} + +\pdfbackendsetextgstate {key}{pdfdata} +\pdfbackendsetcolorspace {key}{pdfdata} +\pdfbackendsetpattern {key}{pdfdata} +\pdfbackendsetshade {key}{pdfdata} +\stoptyping + +One is free to use the \LUA\ interface instead, as there one has more +possibilities but when code is shared with other macro packages the macro +interface makes more sense. The names of the \LUA\ functions are similar, like: + +\starttyping +lpdf.addtoinfo(key,anything_valid_pdf) +\stoptyping + +Currently we expose a bit more of the backend code than we like and +future versions will have a more restricted access. The following +function will stay public: + +\starttyping +lpdf.addtopageresources (key,value) +lpdf.addtopageattributes (key,value) +lpdf.addtopagesattributes(key,value) + +lpdf.adddocumentextgstate(key,value) +lpdf.adddocumentcolorspac(key,value) +lpdf.adddocumentpattern (key,value) +lpdf.adddocumentshade (key,value) + +lpdf.addtocatalog (key,value) +lpdf.addtoinfo (key,value) +lpdf.addtonames (key,value) +\stoptyping + +\stopsection + +\startsection [title={Annotations}] + +You can use the \LUA\ functions that relate to annotations etc.\ but normally you +will use the regular \CONTEXT\ user interface. You can look into some of the +\type {lpdf-*} modules to see how special annotations can be dealt with. + +\stopsection + +\startsection [title={Tracing}] + +There are several tracing options built in and some more will be added in due +time: + +\starttyping +\enabletrackers + [backend.finalizers, + backend.resources, + backend.objects, + backend.detail] +\stoptyping + +As with all trackers you can also pass them on the command line, for example: + +\starttyping +context --trackers=backend.* yourfile +\stoptyping + +The reference related backend mechanisms have their own trackers. When you write +code that generates \PDF, it also helps to look in the \PDF\ file so see if +things are done right. In that case you need to disable compression: + +\starttyping +\nopdfcompression +\stoptyping + +\stopsection + +\startsection[title={Analyzing}] + +The \type {epdf} library that comes with \LUATEX\ offers a userdata interface to +\PDF\ files. On top of that \CONTEXT\ provides a more \LUA-ish access, using +tables. You can open a \PDF\ file with: + +\starttyping +local mypdf = lpdf.epdf.load(filename) +\stoptyping + +When opening is successful, you have access to a couple of tables: + +\starttyping +\NC \type{pages} \NC indexed \NC \NR +\NC \type{destinations} \NC hashed \NC \NR +\NC \type{javascripts} \NC hashed \NC \NR +\NC \type{widgets} \NC hashed \NC \NR +\NC \type{embeddedfiles} \NC hashed \NC \NR +\NC \type{layers} \NC indexed \NC \NR +\stoptyping + +These provide efficient access to some data that otherwise would take a bit of +code to deal with. Another top level table is the for \PDF\ characteristic \type +{Catalog}. Watch the capitalization: as with other native \PDF\ data structures, +keys are case sensitive and match the standard. + +Here is an example of usage: + +\starttyping +local MyDocument = lpdf.epdf.load("somefile.pdf") + +context.starttext() + + local pages = MyDocument.pages + local nofpages = pages.n + + context.starttabulate { "|c|c|c|" } + + context.NC() context("page") + context.NC() context("width") + context.NC() context("height") context.NR() + + for i=1, nofpages do + local page = pages[i] + local bbox = page.CropBox or page.MediaBox + context.NC() context(i) + context.NC() context(bbox[4]-bbox[2]) + context.NC() context(bbox[3]-bbox[1]) context.NR() + end + + context.stoptabulate() + +context.stoptext() +\stoptyping + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-callbacks.tex b/doc/context/sources/general/manuals/cld/cld-callbacks.tex new file mode 100644 index 000000000..c449af864 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-callbacks.tex @@ -0,0 +1,240 @@ +% language=uk + +\startcomponent cld-callbacks + +\environment cld-environment + +\startchapter[title={Callbacks}] + +\startsection [title={Introduction}] + +\index {callbacks} + +The \LUATEX\ engine provides the usual basic \TEX\ functionality plus a bit more. +It is a deliberate choice not to extend the core engine too much. Instead all +relevant processes can be overloaded by new functionality written in \LUA. In +\CONTEXT\ callbacks are wrapped in a protective layer: on the one hand there is +extra functionality (usually interfaced through macros) and on the other hand +users can pop in their own handlers using hooks. Of course a plugged in function +has to do the right thing and not mess up the data structures. In this chapter +the layer on top of callbacks is described. + +\stopsection + +\startsection [title={Actions}] + +\index {nodelists} + +Nearly all callbacks in \LUATEX\ are used in \CONTEXT. In the following list the +callbacks tagged with \type {enabled} are used and frozen, the ones tagged \type +{disabled} are blocked and never used, while the ones tagged \type {undefined} +are yet unused. + +% \ctxlua{callbacks.table()} % \ctxlua{callbacks.report()} +\ctxcommand{showcallbacks()} + +Eventually all callbacks will be used so don't rely on undefined callbacks not +being protected. Some callbacks are only set when certain functionality is +enabled. + +It may sound somewhat harsh but if users kick in their own code, we cannot +guarantee \CONTEXT's behaviour any more and support becomes a pain. If you really +need to use a callback yourself, you should use one of the hooks and make sure +that you return the right values. + +All callbacks related to file handling, font definition and housekeeping are +frozen and cannot be overloaded. A reason for this are that we need some kind of +protection against misuse. Another reason is that we operate in a well defined +environment, the so called \TEX\ directory structure, and we don't want to mess +with that. And of course, the overloading permits \CONTEXT\ to provide extensions +beyond regular engine functionality. + +So as a fact we only open up some of the node list related callbacks and these +are grouped as follows: + +\starttabulate[|l|l|p|] +\FL +\NC \bf category \NC \bf callback \NC \bf usage \NC \NR +\TL +\NC \type{processors} \NC \type{pre_linebreak_filter} \NC called just before the paragraph is broken into lines \NC \NR +\NC \NC \type{hpack_filter} \NC called just before a horizontal box is constructed \NC \NR +\NC \type{finalizers} \NC \type{post_linebreak_filter} \NC called just after the paragraph has been broken into lines \NC \NR +\NC \type{shipouts} \NC \type{no callback yet} \NC applied to the box (or xform) that is to be shipped out \NC \NR +\NC \type{mvlbuilders} \NC \type{buildpage_filter} \NC called after some material has been added to the main vertical list \NC \NR +\NC \type{vboxbuilders} \NC \type{vpack_filter} \NC called when some material is added to a vertical box \NC \NR +%NC \type{parbuilders} \NC \type{linebreak_filter} \NC called when a paragraph is to be broken into lines \NC \NR +%NC \type{pagebuilders} \NC \type{pre_output_filter} \NC called when a page it fed into the output routing \NC \NR +\NC \type{math} \NC \type{mlist_to_hlist} \NC called just after the math list is created, before it is turned into an horizontal list \NC \NR +\BL +\stoptabulate + +Each category has several subcategories but for users only two make sense: \type +{before} and \type {after}. Say that you want to hook some tracing into the \type +{mvlbuilder}. This is how it's done: + +\starttyping +function third.mymodule.myfunction(where) + nodes.show_simple_list(tex.lists.contrib_head) +end + +nodes.tasks.appendaction("processors", "before", "third.mymodule.myfunction") +\stoptyping + +As you can see, in this case the function gets no \type {head} passed (at least +not currently). This example also assumes that you know how to access the right +items. The arguments and return values are given below. \footnote {This interface +might change a bit in future versions of \CONTEXT. Therefore we will not discuss +the few more optional arguments that are possible.} + +\starttabulate[|l|l|p|] +\FL +\NC \bf category \NC \bf arguments \NC \bf return value \NC \NR +\TL +\NC \type{processors} \NC \type{head, ...} \NC \type{head, done} \NC \NR +\NC \type{finalizers} \NC \type{head, ...} \NC \type{head, done} \NC \NR +\NC \type{shipouts} \NC \type{head} \NC \type{head, done} \NC \NR +\NC \type{mvlbuilders} \NC \NC \type{done} \NC \NR +\NC \type{vboxbuilders} \NC \type{head, ...} \NC \type{head, done} \NC \NR +\NC \type{parbuilders} \NC \type{head, ...} \NC \type{head, done} \NC \NR +\NC \type{pagebuilders} \NC \type{head, ...} \NC \type{head, done} \NC \NR +\NC \type{math} \NC \type{head, ...} \NC \type{head, done} \NC \NR +\LL +\stoptabulate + +\stopsection + +\startsection [title={Tasks}] + +\index {tasks} + +In the previous section we already saw that the actions are in fact tasks and +that we can append (and therefore also prepend) to a list of tasks. The \type +{before} and \type {after} task lists are valid hooks for users contrary to the +other tasks that can make up an action. However, the task builder is generic +enough for users to be used for individual tasks that are plugged into the user +hooks. + +Of course at some point, too many nested tasks bring a performance penalty with +them. At the end of a run \MKIV\ reports some statistics and timings and these +can give you an idea how much time is spent in \LUA. + +The following tables list all the registered tasks for the processors actions: + +\ctxlua{nodes.tasks.table("processors")} + +Some of these do have subtasks and some of these even more, so you can imagine +that quite some action is going on there. + +The finalizer tasks are: + +\ctxlua{nodes.tasks.table("finalizers")} + +Shipouts concern: + +\ctxlua{nodes.tasks.table("shipouts")} + +There are not that many mvlbuilder tasks currently: + +\ctxlua{nodes.tasks.table("mvlbuilders")} + +The vboxbuilder perform similar tasks: + +\ctxlua{nodes.tasks.table("vboxbuilders")} + +In the future we expect to have more parbuilder tasks. Here again there are +subtasks that depend on the current typesetting environment, so this is the right +spot for language specific treatments. + +\ctxlua{nodes.tasks.table("parbuilders")} + +The following actions are applied just before the list is passed on the the +output routine. The return value is a vlist. + +\ctxlua{nodes.tasks.table("pagebuilders")} + +{\em Both the parbuilders and pagebuilder tasks are unofficial and not yet meant +for users.} + +Finally, we have tasks related to the math list: + +\ctxlua{nodes.tasks.table("math")} + +As \MKIV\ is developed in sync with \LUATEX\ and code changes from experimental +to more final and reverse, you should not be too surprised if the registered +function names change. + +You can create your own task list with: + +\starttyping +nodes.tasks.new("mytasks",{ "one", "two" }) +\stoptyping + +After that you can register functions. You can append as well as prepend them +either or not at a specific position. + +\starttyping +nodes.tasks.appendaction ("mytask","one","bla.alpha") +nodes.tasks.appendaction ("mytask","one","bla.beta") + +nodes.tasks.prependaction("mytask","two","bla.gamma") +nodes.tasks.prependaction("mytask","two","bla.delta") + +nodes.tasks.appendaction ("mytask","one","bla.whatever","bla.alpha") +\stoptyping + +Functions can also be removed: + +\starttyping +nodes.tasks.removeaction("mytask","one","bla.whatever") +\stoptyping + +As removal is somewhat drastic, it is also possible to enable and disable +functions. From the fact that with these two functions you don't specify a +category (like \type {one} or \type {two}) you can conclude that the function +names need to be unique within the task list or else all with the same name +within this task will be disabled. + +\starttyping +nodes.tasks.enableaction ("mytask","bla.whatever") +nodes.tasks.disableaction("mytask","bla.whatever") +\stoptyping + +The same can be done with a complete category: + +\starttyping +nodes.tasks.enablegroup ("mytask","one") +nodes.tasks.disablegroup("mytask","one") +\stoptyping + +There is one function left: + +\starttyping +nodes.tasks.actions("mytask",2) +\stoptyping + +This function returns a function that when called will perform the tasks. In this +case the function takes two extra arguments in addition to \type {head}. +\footnote {Specifying this number permits for some optimization but is not really +needed} + +Tasks themselves are implemented on top of sequences but we won't discuss them +here. + +\stopsection + +\startsection [title={Paragraph and page builders}] + +Building paragraphs and pages is implemented differently and has no user hooks. +There is a mechanism for plugins but the interface is quite experimental. + +\stopsection + +\startsection [title={Some examples}] + +{\em todo} + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-contents.tex b/doc/context/sources/general/manuals/cld/cld-contents.tex new file mode 100644 index 000000000..132da7dff --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-contents.tex @@ -0,0 +1,11 @@ +\startcomponent cld-contents + +\environment cld-environment + +\starttitle[title=Contents] + + \placelist[chapter,section][criterium=text,aligntitle=yes] + +\stoptitle + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-ctxfunctions.tex b/doc/context/sources/general/manuals/cld/cld-ctxfunctions.tex new file mode 100644 index 000000000..11600b847 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-ctxfunctions.tex @@ -0,0 +1,786 @@ +% language=uk + +\startcomponent cld-ctxfunctions + +\environment cld-environment + +\startchapter[title={The \LUA\ interface code}] + +\startsection[title={Introduction}] + +There is a lot of \LUA\ code in \MKIV. Much is not exposed and a lot of what is +exposed is not meant to be used directly at the \LUA\ end. But there is also +functionality and data that can be accessed without side effects. + +In the following sections a subset of the built in functionality is discussed. +There are often more functions alongside those presented but they might change or +disappear. So, if you use undocumented features, be sure to tag them somehow in +your source code so that you can check them out when there is an update. Best +would be to have more functionality defined local so that it is sort of hidden +but that would be unpractical as for instance functions are often used in other +modules and or have to be available at the \TEX\ end. + +It might be tempting to add your own functions to namespaces created by \CONTEXT\ +or maybe overload some existing ones. Don't do this. First of all, there is no +guarantee that your code will not interfere, nor that it overloads future +functionality. Just use your own namespace. Also, future versions of \CONTEXT\ +might have a couple of protection mechanisms built in. Without doubt the +following sections will be extended as soon as interfaces become more stable. + +\stopsection + +\startsection[title={Characters}] + +% not discussed: +% +% characters.filters.utf.addgrapheme() +% characters.filters.utf.collapse() +% characters.getrange() +% characters.bidi[] +% tex.uprint() +% utf.string() +% characters.flush() + +There are quite some data tables defined but the largest is the character +database. You can consult this table any time you want but you're not supposed to +add or change its content if only because changes will be overwritten when you +update \CONTEXT. Future versions may carry more information. The table can be +accessed using an unicode number. A relative simple entry looks as follows: + +\ShowLuaExampleTableHex{characters.data[0x00C1]} + +Much of this is rather common information but some of it is specific for use with +\CONTEXT. Some characters have even more information, for instance those that +deal with mathematics: + +\ShowLuaExampleTableHex{characters.data[0x2190]} + +Not all characters have a real entry. For instance most \CJK\ characters are +virtual and share the same data: + +\ShowLuaExampleTableHex{characters.data[0x3456]} + +You can also access the table using \UTF\ characters: + +\ShowLuaExampleTable{characters.data["ä"]} + +A more verbose string access is also supported: + +\ShowLuaExampleTableHex{characters.data["U+0070"]} + +Another (less usefull) table contains information about ranges in this character +table. You can access this table using rather verbose names, or you can use +collapsed lowercase variants. + +\ShowLuaExampleTableHex{characters.blocks["CJK Compatibility Ideographs"]} + +\ShowLuaExampleTableHex{characters.blocks["hebrew"]} + +\ShowLuaExampleTableHex{characters.blocks["combiningdiacriticalmarks"]} + +Some fields can be accessed using functions. This can be handy when you need that +information for tracing purposes or overviews. There is some overhead in the +function call, but you get some extra testing for free. You can use characters as +well as numbers as index. + +\ShowLuaExampleString{characters.contextname("ä")} +\ShowLuaExampleString{characters.adobename(228)} +\ShowLuaExampleString{characters.description("ä")} + +The category is normally a two character tag, but you can also ask for a more +verbose variant: + +\ShowLuaExampleString{characters.category(228)} +\ShowLuaExampleString{characters.category(228,true)} + +The more verbose category tags are available in a table: + +\ShowLuaExampleString{characters.categorytags["lu"]} + +There are several fields in a character entry that help us to remap a character. +The \type {lccode} indicates the lowercase code point and the \type {uccode} to +the uppercase code point. The \type {shcode} refers to one or more characters +that have a similar shape. + +\ShowLuaExampleString{characters.shape ("ä")} +\ShowLuaExampleString{characters.uccode("ä")} +\ShowLuaExampleString{characters.lccode("ä")} + +\ShowLuaExampleString{characters.shape (100)} +\ShowLuaExampleString{characters.uccode(100)} +\ShowLuaExampleString{characters.lccode(100)} + +You can use these function or access these fields directly in an +entry, but we also provide a few virtual tables that avoid +accessing the whole entry. This method is rather efficient. + +\ShowLuaExampleString{characters.lccodes["ä"]} +\ShowLuaExampleString{characters.uccodes["ä"]} +\ShowLuaExampleString{characters.shcodes["ä"]} +\ShowLuaExampleString{characters.lcchars["ä"]} +\ShowLuaExampleString{characters.ucchars["ä"]} +\ShowLuaExampleString{characters.shchars["ä"]} + +As with other tables, you can use a number instead of an \UTF\ character. Watch +how we get a table for multiple shape codes but a string for multiple shape +characters. + +\ShowLuaExampleString{characters.lcchars[0x00C6]} +\ShowLuaExampleString{characters.ucchars[0x00C6]} +\ShowLuaExampleString{characters.shchars[0x00C6]} +\ShowLuaExampleTable {characters.shcodes[0x00C6]} + +These codes are used when we manipulate strings. Although there +are \type {upper} and \type {lower} functions in the \type +{string} namespace, the following ones are the real ones to be +used in critical situations. + +\ShowLuaExampleString{characters.lower("ÀÁÂÃÄÅàáâãäå")} +\ShowLuaExampleString{characters.upper("ÀÁÂÃÄÅàáâãäå")} +\ShowLuaExampleString{characters.shaped("ÀÁÂÃÄÅàáâãäå")} + +A rather special one is the following: + +\ShowLuaExampleString{characters.lettered("Only 123 letters + count!")} + +With the second argument is true, spaces are kept and collapsed. Leading and +trailing spaces are stripped. + +\ShowLuaExampleString{characters.lettered("Only 123 letters + count!",true)} + +Access to tables can happen by number or by string, although there are some +limitations when it gets too confusing. Take for instance the number \type {8} +and string \type {"8"}: if we would interpret the string as number we could never +access the entry for the character eight. However, using more verbose hexadecimal +strings works okay. The remappers are also available as functions: + +\ShowLuaExampleString{characters.tonumber("a")} +\ShowLuaExampleString{characters.fromnumber(100)} +\ShowLuaExampleString{characters.fromnumber(0x0100)} +\ShowLuaExampleString{characters.fromnumber("0x0100")} +\ShowLuaExampleString{characters.fromnumber("U+0100")} + +In addition to the already mentioned category information you can also use a more +direct table approach: + +\ShowLuaExampleString{characters.categories["ä"]} +\ShowLuaExampleString{characters.categories[100]} + +In a similar fashion you can test if a given character is in a specific category. +This can save a lot of tests. + +\ShowLuaExampleBoolean{characters.is_character[characters.categories[67]]} +\ShowLuaExampleBoolean{characters.is_character[67]} +\ShowLuaExampleBoolean{characters.is_character[characters.data[67].category]} +\ShowLuaExampleBoolean{characters.is_letter[characters.data[67].category]} +\ShowLuaExampleBoolean{characters.is_command[characters.data[67].category]} + +Another virtual table is the one that provides access to special information, for +instance about how a composed character is made up of components. + +\ShowLuaExampleString{characters.specialchars["ä"]} +\ShowLuaExampleString{characters.specialchars[100]} + +The outcome is often similar to output that uses the shapecode information. + +Although not all the code deep down in \CONTEXT\ is meant for use at the user +level, it sometimes can eb tempting to use data and helpers that are available as +part of the general housekeeping. The next table was used when looking into +sorting Korean. For practical reasons we limit the table to ten entries; +otherwise we would have ended up with hundreds of pages. + +\startbuffer +\startluacode +local data = characters.data +local map = characters.hangul.remapped + +local first, last = characters.getrange("hangulsyllables") + +last = first + 9 -- for now + +context.start() + +context.definedfont { "file:unbatang" } + +context.starttabulate { "|T||T||T||T||T|" } +for unicode = first, last do + local character = data[unicode] + local specials = character.specials + if specials then + context.NC() + context.formatted("%04V",unicode) + context.NC() + context.formatted("%c",unicode) + for i=2,4 do + local chr = specials[i] + if chr then + chr = map[chr] or chr + context.NC() + context.formatted("%04V",chr) + context.NC() + context.formatted("%c",chr) + else + context.NC() + context.NC() + end + end + context.NC() + context(character.description) + context.NC() + context.NR() + end +end +context.stoptabulate() + +context.stop() +\stopluacode +\stopbuffer + +\getbuffer \typebuffer + +\stopsection + +\startsection[title={Fonts}] + +% not discussed (as not too relevant for users): +% +% cache cache_version +% nomath +% units units_per_em +% direction embedding encodingbytes +% boundarychar boundarychar_label +% has_italic has_math +% tounicode sub +% colorscheme (will probably become a hash) +% language script +% spacer +% MathConstants and a few split_names +% +% tables.baselines + +There is a lot of code that deals with fonts but most is considered to be a black +box. When a font is defined, its data is collected and turned into a form that +\TEX\ likes. We keep most of that data available at the \LUA\ end so that we can +later use it when needed. In this chapter we discuss some of the possibilities. +More details can be found in the font manual(s) so we don't aim for completeness +here. + +A font instance is identified by its id, which is a number where zero is reserved +for the so called \type {nullfont}. The current font id can be requested by the +following function. + +\ShowLuaExampleString{fonts.currentid()} + +The \type {fonts.current()} call returns the table with data related to the +current id. You can access the data related to any id as follows: + +\starttyping +local tfmdata = fonts.identifiers[number] +\stoptyping + +Not all entries in the table make sense for the user as some are just meant to +drive the font initialization at the \TEX\ end or the backend. The next table +lists the most important ones. Some of the tables are just shortcuts to en entry +in one of the \type {shared} subtables. + +\starttabulate[|l|Tl|p|] +\NC \type{ascender} \NC number \NC the height of a line conforming the font \NC \NR +\NC \type{descender} \NC number \NC the depth of a line conforming the font \NC \NR +\NC \type{italicangle} \NC number \NC the angle of the italic shapes (if present) \NC \NR +\NC \type{designsize} \NC number \NC the design size of the font (if known) \NC \NR +\ML +\NC \type{size} \NC number \NC the size in scaled points if the font instance \NC \NR +\NC \type{factor} \NC number \NC the multiplication factor for unscaled dimensions \NC \NR +\NC \type{hfactor} \NC number \NC the horizontal multiplication factor \NC \NR +\NC \type{vfactor} \NC number \NC the vertical multiplication factor \NC \NR +\NC \type{extend} \NC number \NC the horizontal scaling to be used by the backend \NC \NR +\NC \type{slant} \NC number \NC the slanting to be applied by the backend \NC \NR +\ML +\NC \type{characters} \NC table \NC the scaled character (glyph) information (tfm) \NC \NR +\NC \type{descriptions} \NC table \NC the original unscaled glyph information (otf, afm, tfm) \NC \NR +\NC \type{indices} \NC table \NC the mapping from unicode slot to glyph index \NC \NR +\NC \type{unicodes} \NC table \NC the mapoing from glyph names to unicode \NC \NR +\NC \type{marks} \NC table \NC a hash table with glyphs that are marks as entry \NC \NR +\NC \type{parameters} \NC table \NC the font parameters as \TEX\ likes them \NC \NR +\NC \type{mathconstants} \NC table \NC the \OPENTYPE\ math parameters \NC \NR +\NC \type{mathparameters} \NC table \NC a reference to the \type {MathConstants} table \NC \NR +\NC \type{shared} \NC table \NC a table with information shared between instances \NC \NR +\NC \type{unique} \NC table \NC a table with information unique for this instance \NC \NR +\NC \type{unscaled} \NC table \NC the unscaled (intermediate) table \NC \NR +\NC \type{goodies} \NC table \NC the \CONTEXT\ specific extra font information \NC \NR +\NC \type{fonts} \NC table \NC the table with references to other fonts \NC \NR +\NC \type{cidinfo} \NC table \NC a table with special information for the backend \NC \NR +\ML +\NC \type{filename} \NC string \NC the full path of the loaded font \NC \NR +\NC \type{fontname} \NC string \NC the font name as specified in the font (limited in size) \NC \NR +\NC \type{fullname} \NC string \NC the complete font name as specified in the font \NC \NR +\NC \type{name} \NC string \NC the (short) name of the font \NC \NR +\NC \type{psname} \NC string \NC the (unique) name of the font as used by the backend \NC \NR +\ML +\NC \type{hash} \NC string \NC the hash that makes this instance unique \NC \NR +\NC \type{id} \NC number \NC the id (number) that \TEX\ will use for this instance \NC \NR +\ML +\NC \type{type} \NC string \NC an idicator if the font is \type {virtual} or \type {real} \NC \NR +\NC \type{format} \NC string \NC a qualification for this font, e.g.\ \type {opentype} \NC \NR +\NC \type{mode} \NC string \NC the \CONTEXT\ processing mode, \type {node} or \type {base} \NC \NR +\ML +\stoptabulate + +The \type {parameters} table contains variables that are used by \TEX\ itself. +You can use numbers as index and these are equivalent to the so called \type +{\fontdimen} variables. More convenient is is to access by name: + +\starttabulate[|l|p|] +\NC \type{slant} \NC the slant per point (seldom used) \NC \NR +\NC \type{space} \NC the interword space \NC \NR +\NC \type{spacestretch} \NC the interword stretch \NC \NR +\NC \type{spaceshrink} \NC the interword shrink \NC \NR +\NC \type{xheight} \NC the x|-|height (not per se the heigth of an x) \NC \NR +\NC \type{quad} \NC the so called em|-|width (often the width of an emdash)\NC \NR +\NC \type{extraspace} \NC additional space added in specific situations \NC \NR +\stoptabulate + +The math parameters are rather special and explained in the \LUATEX\ manual. +Quite certainly you never have to touch these parameters at the \LUA\ end. + +En entry in the \type {characters} table describes a character if we have entries +within the \UNICODE\ range. There can be entries in the private area but these +are normally variants of a shape or special math glyphs. + +\starttabulate[|l|p|] +\NC \type{name} \NC the name of the character \NC \NR +\NC \type{index} \NC the index in the raw font table \NC \NR +\NC \type{height} \NC the scaled height of the character \NC \NR +\NC \type{depth} \NC the scaled depth of the character \NC \NR +\NC \type{width} \NC the scaled height of the character \NC \NR +\NC \type{tounicode} \NC a \UTF-16 string representing the conversion back to unicode \NC \NR +\NC \type{expansion_factor} \NC a multiplication factor for (horizontal) font expansion \NC \NR +\NC \type{left_protruding} \NC a multiplication factor for left side protrusion \NC \NR +\NC \type{right_protruding} \NC a multiplication factor for right side protrusion \NC \NR +\NC \type{italic} \NC the italic correction \NC \NR +\NC \type{next} \NC a pointer to the next character in a math size chain \NC \NR +\NC \type{vert_variants} \NC a pointer to vertical variants conforming \OPENTYPE\ math \NC \NR +\NC \type{horiz_variants} \NC a pointer to horizontal variants conforming \OPENTYPE\ math \NC \NR +\NC \type{top_accent} \NC information with regards to math top accents \NC \NR +\NC \type{mathkern} \NC a table describing stepwise math kerning (following the shape) \NC \NR +\NC \type{kerns} \NC a table with intercharacter kerning dimensions \NC \NR +\NC \type{ligatures} \NC a (nested) table describing ligatures that start with this character \NC \NR +\NC \type{commands} \NC a table with commands that drive the backend code for a virtual shape \NC \NR +\stoptabulate + +Not all entries are present for each character. Also, in so called \type {node} +mode, the \type {ligatures} and \type {kerns} tables are empty because in that +case they are dealt with at the \LUA\ end and not by \TEX. + +% \startluacode +% local tfmdata = fonts.current() +% context.starttabulate{ "|l|pl|" } +% for k, v in table.sortedhash(tfmdata) do +% local tv = type(v) +% if tv == "string" or tv == "number" or tv == "boolean" then +% context.NC() +% string.tocontext(k) +% context.NC() +% string.tocontext(tostring(v)) +% context.NC() +% context.NR() +% end +% end +% context.stoptabulate() +% \stopluacode + +% \ShowLuaExampleTable{table.sortedkeys(fonts.current())} + +Say that you run into a glyph node and want to access the data related to that +glyph. Given that variable \type {n} points to the node, the most verbose way of +doing that is: + +\starttyping +local g = fonts.identifiers[n.id].characters[n.char] +\stoptyping + +Given the speed of \LUATEX\ this is quite fast. Another method is the following: + +\starttyping +local g = fonts.characters[n.id][n.char] +\stoptyping + +For some applications you might want faster access to critical +parameters, like: + +\starttyping +local quad = fonts.quads [n.id][n.char] +local xheight = fonts.xheights[n.id][n.char] +\stoptyping + +but that only makes sense when you don't access more than one such variable at +the same time. + +Among the shared tables is the feature specification: + +\ShowLuaExampleTable{fonts.current().shared.features} + +As features are a prominent property of \OPENTYPE\ fonts, there are a few +datatables that can be used to get their meaning. + +\ShowLuaExampleString{fonts.handlers.otf.tables.features['liga']} +\ShowLuaExampleString{fonts.handlers.otf.tables.languages['nld']} +\ShowLuaExampleString{fonts.handlers.otf.tables.scripts['arab']} + +There is a rather extensive font database built in but discussing its interface +does not make much sense. Most usage happens automatically when you use the \type +{name:} and \type {spec:} methods of defining fonts and the \type {mtx-fonts} +script is built on top of it. + +\ctxlua{fonts.names.load()} % could be metatable driven + +\ShowLuaExampleTable{table.sortedkeys(fonts.names.data)} + +You can load the database (if it's not yet loaded) with: + +\starttyping +names.load(reload,verbose) +\stoptyping + +When the first argument is true, the database will be rebuild. The second +arguments controls verbosity. + +Defining a font normally happens at the \TEX\ end but you can also do it in \LUA. + +\starttyping +local id, fontdata = fonts.definers.define { + lookup = "file", -- use the filename (file spec name) + name = "pagella-regular", -- in this case the filename + size = 10*65535, -- scaled points + global = false, -- define the font globally + cs = "MyFont", -- associate the name \MyFont + method = "featureset", -- featureset or virtual (* or @) + sub = nil, -- no subfont specifier + detail = "whatever", -- the featureset (or whatever method applies) +} +\stoptyping + +In this case the \type {detail} variable defines what featureset has to be +applied. You can define such sets at the \LUA\ end too: + +\starttyping +fonts.definers.specifiers.presetcontext ( + "whatever", + "default", + { + mode = "node", + dlig = "yes", + } +) +\stoptyping + +The first argument is the name of the featureset. The second argument can be an +empty string or a reference to an existing featureset that will be taken as +starting point. The final argument is the featureset. This can be a table or a +string with a comma separated list of key|/|value pairs. + +\stopsection + +\startsection[title={Nodes}] + +Nodes are the building blocks that make a document reality. Nodes are linked into +lists and at various moments in the typesetting process you can manipulate them. +Deep down in \CONTEXT\ we use quite some \LUA\ magic to manipulate lists of +nodes. Therefore it is no surprise that we have some tracing available. Take the +following box. + +\startbuffer +\setbox0\hbox{It's in \hbox{\bf all} those nodes.} +\stopbuffer + +\typebuffer \getbuffer + +This box contains characters and glue between the words. The box is already +constructed. There can also be kerns between characters, but of course only if +the font provides such a feature. Let's inspect this box: + +\ShowLuaExampleString{nodes.toutf(tex.box[0])} +\ShowLuaExampleString{nodes.toutf(tex.box[0].list)} + +This tracer returns the text and spacing and recurses into nested lists. The next +tracer does not do this and marks non glyph nodes as \type {[-]}: + +\ShowLuaExampleString{nodes.listtoutf(tex.box[0])} +\ShowLuaExampleString{nodes.listtoutf(tex.box[0].list)} + +A more verbose tracer is the next one. It does show a bit more detailed +information about the glyphs nodes. + +\ShowLuaExampleString{nodes.tosequence(tex.box[0])} +\ShowLuaExampleString{nodes.tosequence(tex.box[0].list)} + +The fourth tracer does not show that detail and collapses sequences of similar +node types. + +\ShowLuaExampleString{nodes.idstostring(tex.box[0])} +\ShowLuaExampleString{nodes.idstostring(tex.box[0].list)} + +The number of nodes in a list is identified with the \type {countall} function. +Nested nodes are counted too. + +\ShowLuaExampleString{nodes.countall(tex.box[0])} +\ShowLuaExampleString{nodes.countall(tex.box[0].list)} + +There are a lot of helpers in the \type {nodes} namespace. In fact, we map all the +helpers provided by the engine itself under \type {nodes} too. These are described +in the \LUATEX\ manual. There are for instance functions to check node types and +node id's: + +\starttyping +local str = node.type(1) +local num = node.id("vlist") +\stoptyping + +These are basic \LUATEX\ functions. In addition to those we also provide a few more +helpers as well as +mapping tables. There are two tables that map node id's to strings and backwards: + +\starttabulate +\NC \type{nodes.nodecodes} \NC regular nodes, some fo them are sort of private to the engine \NC \NR +\NC \type{nodes.noadcodes} \NC math nodes that later on are converted into regular nodes \NC \NR +\stoptabulate + +Nodes can have subtypes. Again we have tables that map the subtype numbers onto +meaningfull names and reverse. + +\starttabulate +\NC \type{nodes.listcodes} \NC subtypes of \type {hlist} and \type {vlist} nodes \NC \NR +\NC \type{nodes.kerncodes} \NC subtypes of \type {kern} nodes \NC \NR +\NC \type{nodes.gluecodes} \NC subtypes of \type {glue} nodes (skips) \NC \NR +\NC \type{nodes.glyphcodes} \NC subtypes of \type {glyph} nodes, the subtype can change \NC \NR +\NC \type{nodes.mathcodes} \NC math specific subtypes \NC \NR +\NC \type{nodes.fillcodes} \NC these are not really subtypes but indicate the strength of the filler \NC \NR +\NC \type{nodes.whatsitcodes} \NC subtypes of a rather large group of extension nodes \NC \NR +\stoptabulate + +Some of the names of types and subtypes have underscores but you can omit them +when you use these tables. You can use tables like this as follows: + +\starttyping +local glyph_code = nodes.nodecodes.glyph +local kern_code = nodes.nodecodes.kern +local glue_code = nodes.nodecodes.glue + +for n in nodes.traverse(list) do + local id == n.id + if id == glyph_code then + ... + elseif id == kern_code then + ... + elseif id == glue_code then + ... + else + ... + end +end +\stoptyping + +You only need to use such temporary variables in time critical code. In spite of +what you might think, lists are not that long and given the speed of \LUA\ (and +successive optimizations in \LUATEX) looping over a paragraphs is rather fast. + +Nodes are created using \type {node.new}. If you study the \CONTEXT\ code you +will notice that there are quite some functions in the \type {nodes.pool} +namespace, like: + +\starttyping +local g = nodes.pool.glyph(fnt,chr) +\stoptyping + +Of course you need to make sure that the font id is valid and that the referred +glyph in in the font. You can use the allocators but don't mess with the code in +the \type {pool} namespace as this might interfere with its usage all over +\CONTEXT. + +The \type {nodes} namespace provides a couple of helpers and some of them are +similar to ones provided in the \type {node} namespace. This has practical as +well as historic reasons. For instance some were prototypes functions that were +later built in. + +\starttyping +local head, current = nodes.before (head, current, new) +local head, current = nodes.after (head, current, new) +local head, current = nodes.delete (head, current) +local head, current = nodes.replace(head, current, new) +local head, current, old = nodes.remove (head, current) +\stoptyping + +Another category deals with attributes: + +\starttyping +nodes.setattribute (head, attribute, value) +nodes.unsetattribute (head, attribute) +nodes.setunsetattribute (head, attribute, value) +nodes.setattributes (head, attribute, value) +nodes.unsetattributes (head, attribute) +nodes.setunsetattributes(head, attribute, value) +nodes.hasattribute (head, attribute, value) +\stoptyping + +% context(typesetters.hpack("Hello World!")) +% context(typesetters.hpack("Hello World!",1,100*1024*10)) + +% nodes.firstchar +% nodes.firstcharinbox + +% maybe node-tst +% tasks and so +% number.points (to numbers) + +\stopsection + +% \startsection[title={Core}] +% {\em todo} +% \stopsection + +\startsection[title={Resolvers}] + +All \IO\ is handled by functions in the \type {resolvers} namespace. Most of the +code that you find in the \type {data-*.lua} files is of litle relevance for +users, especially at the \LUA\ end, so we won't discuss it here in great detail. + +The resolver code is modelled after the \KPSE\ library that itself implements the +\TEX\ Directory Structure in combination with a configuration file. However, we +go a bit beyond this structure, for instance in integrating support for other +resources that file systems. We also have our own configuration file. But +important is that we still support a similar logic too so that regular +configurations are dealt with. + +During a run \LUATEX\ needs files of a different kind: source files, font files, +images, etc. In practice you will probably only deal with source files. The most +fundamental function is \type {findfile}. The first argument is the filename to +be found. A second optional argument indicates the file type. + +The following table relates so called formats to suffixes and variables in the +configuration file. + +\startluacode +context.starttabulate { "|lp|lp|l|" } +context.NC() context.bold("variable") +context.NC() context.bold("format") +context.NC() context.bold("suffix") +context.NC() context.NR() +context.ML() +for k, v in table.sortedpairs(resolvers.relations.core) do + local names = v.names + local variable = v.variable + local suffixes = v.suffixes + context.NC() + if variable then + context.type(variable) + end + context.NC() + if names then + for i=1,#names do + context.type(names[i]) + context.par() + end + end + context.NC() + if suffixes then + context.type(table.concat(suffixes, " ")) + end + context.NC() + context.NR() +end +context.stoptabulate() +\stopluacode + +There are a couple of more formats but these are not that relevant in the +perspective of \CONTEXT. + +When a lookup takes place, spaces are ignored and formats are normalized to +lowercase. + +\ShowLuaExampleString{file.strip(resolvers.findfile("context.tex"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("context.mkiv"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("context"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("data-res.lua"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("lmsans10-bold"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("lmsans10-bold.otf"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("lmsans10-bold","otf"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("lmsans10-bold","opentype"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("lmsans10-bold","opentypefonts"),"tex/")} +\ShowLuaExampleString{file.strip(resolvers.findfile("lmsans10-bold","opentype fonts"),"tex/")} + +The plural variant of this function returns one or more matches. + +\ShowLuaExampleTable{resolvers.findfiles("texmfcnf.lua","cnf")} +\ShowLuaExampleTable{resolvers.findfiles("context.tex","")} + +% table.print(resolvers.instance.environment) +% table.print(resolvers.instance.variables) +% table.print(resolvers.instance.expansions) +% +% resolvers.expandbraces +% resolvers.expandpath +% resolvers.expandvar +% resolvers.showpath +% resolvers.var_value +% +% resolvers.getenv +% resolvers.variable() +% resolvers.expansion() +% resolvers.is_variable +% resolvers.is_expansion +% +% resolvers.unexpandedpathlist(str) +% resolvers.unexpandedpath(str) +% resolvers.cleanpathlist +% resolvers.expandpath +% resolvers.expandedpath +% resolvers.expandedpathlistfromvariable +% resolvers.expandpathfromvariable +% resolvers.expandbraces +% +% resolvers.findpath +% resolvers.findgivenfiles +% resolvers.findgivenfile +% resolvers.findwildcardfiles +% resolvers.findwildcardfile +% resolvers.showpath + +% data-tre as example +% schemes (data-she) +% caching (containers) +% findbinfile (open|load) +% variables / environment +% findtexfile opentexfile loadtexfile +% file:// + +% supp + +\stopsection + +\startsection[title={Mathematics (math)}] + {\em todo} +\stopsection + +\startsection[title={Graphics (grph)}] + {\em is a separate chapter} +\stopsection + +\startsection[title={Languages (lang)}] + {\em todo} +\stopsection + +\startsection[title={MetaPost (mlib)}] + {\em todo} +\stopsection + +\startsection[title={Lua\TeX\ (luat)}] + {\em todo} +\stopsection + +\startsection[title={Tracing (trac)}] + {\em todo} +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-environment.tex b/doc/context/sources/general/manuals/cld/cld-environment.tex new file mode 100644 index 000000000..1355110bd --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-environment.tex @@ -0,0 +1,224 @@ +% language=uk + +\startenvironment cld-environment + +\usemodule[abr-04] + +\setuplayout + [width=middle, + height=middle, + backspace=2cm, + topspace=1cm, + footer=0pt, + bottomdistance=1cm, + bottom=1cm, + bottomspace=2cm] + +\setuppagenumbering + [alternative=doublesided] + +\definecolor[darkred] [r=.5] +\definecolor[darkgreen][g=.5] +\definecolor[darkblue] [b=.5] + +\definecolor[red] [darkred] +\definecolor[green][darkgreen] +\definecolor[blue] [darkblue] + +\definetype + [boldtypebig] + [style=\ttbfa] + +\definetype + [boldtype] + [style=\ttbf] + +\definetyping + [smalltyping] + [bodyfont=small] + +\setuptype + [color=blue] + +\setuptyping + [color=blue] + +\setupbodyfont + [palatino,11pt] + +\setuphead + [chapter] + [style=\bfc, + color=blue] + +\setuphead + [section] + [style=\bfb, + color=blue] + +\definehead + [summary] + [subsubsubsubject] + +\setuphead + [summary] + [style=, + deeptextcommand=\boldtypebig, + color=blue] + +\definehead + [subsummary] + [subsubsubsubsubject] + +\setuphead + [subsummary] + [style=, + before=\blank, + after=\blank, + deeptextcommand=\type, + command=\MySubSummaryHead, + color=blue] + +\unexpanded\def\MySummaryHead#1#2% + {\framed + [frame=off, + bottomframe=on, + offset=0cm] + {#2}} + +\unexpanded\def\MySubSummaryHead#1#2% + {\framed + [frame=off, + bottomframe=on, + offset=0cm] + {#2}} + +\setupwhitespace + [big] + +\setupheadertexts + [] + +\setupheadertexts + [] + [{\getmarking[chapter]\quad\pagenumber}] + [{\pagenumber\quad\getmarking[chapter]}] + [] + +\setupheader + [color=darkblue] + +\setuplist + [chapter,title] + [color=darkblue, + style=bold] + +\setupbottom + [style=\bfx, + color=darkred] + +\setupbottomtexts + [preliminary, uncorrected version -- \currentdate] + +% special functions + +\unexpanded\def\ShowLuaExampleOne#1#2#3% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1.#2(#3)}] + \ctxlua{table.tocontext(#3)} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleTwo#1#2#3% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1.#2(#3)}] + \ctxlua{table.tocontext(#1.#2(#3))} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleThree#1#2#3% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1.#2(#3)}] + \ctxlua{string.tocontext(tostring(#1.#2(#3)))} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleFour#1#2#3#4% + {\bgroup + \obeyluatokens + \startsubsummary[title={t=#3 #1.#2(t#4)}] + \ctxlua{local t = #3 #1.#2(t#4) table.tocontext(t)} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleFive#1#2% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1.#2}] + \ctxlua{string.tocontext(tostring(#1.#2))} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleSix#1#2#3% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1.#2(#3)}] + \ctxlua{string.tocontext(#1.#2(#3))} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleSeven#1#2#3% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1.#2(#3)}] + \ctxlua{string.tocontext(table.concat({#1.#2(#3)}," "))} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleTable#1% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1}] + \ctxlua{table.tocontext(#1,false)} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleTableHex#1% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1}] + \ctxlua{table.tocontext(#1,false,false,true,true)} % name, reduce, noquotes, hex + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleString#1% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1}] + \ctxlua{string.tocontext(#1)} + \stopsubsummary + \egroup} + +\unexpanded\def\ShowLuaExampleBoolean#1% + {\bgroup + \obeyluatokens + \startsubsummary[title={#1}] + \ctxlua{boolean.tocontext(#1)} + \stopsubsummary + \egroup} + +% interaction + +\setupinteraction + [state=start, + color=, + contrastcolor=] + +\setuplist + [chapter,section] + [interaction=all] + +\stopenvironment diff --git a/doc/context/sources/general/manuals/cld/cld-files.tex b/doc/context/sources/general/manuals/cld/cld-files.tex new file mode 100644 index 000000000..38a7322b1 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-files.tex @@ -0,0 +1,78 @@ +% language=uk + +\startcomponent cld-macros + +\environment cld-environment + +\startchapter[title=Files] + +\startsection[title={Preprocessing}] + +Although this option must be used with care, it is possible to preprocess files +before they enter \TEX. The following example shows this. + +\starttyping +local function showline(str,filename,linenumber,noflines) + logs.simple("[lc] file: %s, line: %s of %s, length: %s", + file.basename(filename),linenumber,noflines,#str) +end + +local function showfile(str,filename) + logs.simple("[fc] file: %s, length: %s", + file.basename(filename),#str) +end + +resolvers.installinputlinehandler(showline) +resolvers.installinputfilehandler(showfile) +\stoptyping + +Preprocessors like this are rather innocent. If you want to manipulate the +content you need to be aware of the fact that modules and such also pass your +code, and manipulating them can give unexpected side effects. So, the following +code will not make \CONTEXT\ happy. + +\starttyping +local function foo() + return "bar" +end + +resolvers.installinputlinehandler(foo) +\stoptyping + +But, as we pass the filename, you can base your preprocessing on names. + +There can be multiple handlers active at the same time, and although more +detailed control is possible, the current interface does not provide that, simply +because having too many handlers active is asking for trouble anyway. What you +can do, is putting your handler in front or after the built in handlers. + +\starttyping +resolvers.installinputlinehandler("before",showline) +resolvers.installinputfilehandler("after", showfile) +\stoptyping + +Of course you can also preprocess files outside this mechanism, which in most +cases might be a better idea. However, the following example code is quite +efficient and robust. + +\starttyping +local function MyHandler(str,filename) + if file.suffix(filename) == "veryspecial" then + logs.simple("preprocessing file '%s',filename) + return MyConverter(str) + else + return str + end +end + +resolvers.installinputfilehandler("before",MyHandler) +\stoptyping + +In this case only files that have a suffix \type {.veryspecial} will get an extra +treatment. + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-gettingstarted.tex b/doc/context/sources/general/manuals/cld/cld-gettingstarted.tex new file mode 100644 index 000000000..5c7e1c263 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-gettingstarted.tex @@ -0,0 +1,437 @@ +% language=uk + +\startcomponent cld-gettingstarted + +\environment cld-environment + +\startchapter[title=Getting started] + +\startsection[title=Some basics] + +\index{processing} + +I assume that you have either the so called \CONTEXT\ standalone (formerly known +as minimals) installed or \TEXLIVE. You only need \LUATEX\ and can forget about +installing \PDFTEX\ or \XETEX, which saves you some megabytes and hassle. Now, +from the users perspective a \CONTEXT\ run goes like: + +\starttyping +context yourfile +\stoptyping + +and by default a file with suffix \type {tex}, \type {mkvi} or \type {mkvi} will +be processed. There are however a few other options: + +\starttyping +context yourfile.xml +context yourfile.rlx --forcexml +context yourfile.lua +context yourfile.pqr --forcelua +context yourfile.cld +context yourfile.xyz --forcecld +context yourfile.mp +context yourfile.xyz --forcemp +\stoptyping + +When processing a \LUA\ file the given file is loaded and just processed. This +options will seldom be used as it is way more efficient to let \type {mtxrun} +process that file. However, the last two variants are what we will discuss here. +The suffix \type {cld} is a shortcut for \CONTEXT\ \LUA\ Document. + +A simple \type {cld} file looks like this: + +\starttyping +context.starttext() +context.chapter("Hello There!") +context.stoptext() +\stoptyping + +So yes, you need to know the \CONTEXT\ commands in order to use this mechanism. +In spite of what you might expect, the codebase involved in this interface is not +that large. If you know \CONTEXT, and if you know how to call commands, you +basically can use this \LUA\ method. + +The examples that I will give are either (sort of) standalone, i.e.\ they are +dealt with from \LUA, or they are run within this document. Therefore you will +see two patterns. If you want to make your own documentation, then you can use +this variant: + +\starttyping +\startbuffer +context("See this!") +\stopbuffer + +\typebuffer \ctxluabuffer +\stoptyping + +I use anonymous buffers here but you can also use named ones. The other variant +is: + +\starttyping +\startluacode +context("See this!") +\stopluacode +\stoptyping + +This will process the code directly. Of course we could have encoded this +document completely in \LUA\ but that is not much fun for a manual. + +\stopsection + +\startsection[title=The main command] + +There are a few rules that you need to be aware of. First of all no syntax +checking is done. Second you need to know what the given commands expects in +terms of arguments. Third, the type of your arguments matters: + +\starttabulate[|||] +\NC \type{nothing} \EQ just the command, no arguments \NC \NR +\NC \type{string} \EQ an argument with curly braces \NC \NR +\NC \type{array} \EQ a list between square backets (sometimes optional) \NC \NR +\NC \type{hash} \EQ an assignment list between square brackets \NC \NR +\NC \type{boolean} \EQ when \type {true} a newline is inserted \NC \NR +\NC \EQ when \type {false}, omit braces for the next argument \NC \NR +\stoptabulate + +In the code above you have seen examples of this but here are some more: + +\starttyping +context.chapter("Some title") +context.chapter({ "first" }, "Some title") +context.startchapter({ title = "Some title", label = "first" }) +\stoptyping + +This blob of code is equivalent to: + +\starttyping +\chapter{Some title} +\chapter[first]{Some title} +\startchapter[title={Some title},label=first] +\stoptyping + +You can simplify the third line of the \LUA\ code to: + +\starttyping +context.startchapter { title = "Some title", label = "first" } +\stoptyping + +In case you wonder what the distinction is between square brackets and curly +braces: the first category of arguments concerns settings or lists of options or +names of instances while the second category normally concerns some text to be +typeset. + +Strings are interpreted as \TEX\ input, so: + +\starttyping +context.mathematics("\\sqrt{2^3}") +\stoptyping + +and if you don't want to escape: + +\starttyping +context.mathematics([[\sqrt{2^3}]]) +\stoptyping + +are both correct. As \TEX\ math is a language in its own and a de-facto standard +way of inputting math this is quite natural, even at the \LUA\ end. + +\stopsection + +\startsection[title=Spaces and Lines] + +\index{spaces} +\index{lines} + +In a regular \TEX\ file, spaces and newline characters are collapsed into one +space. At the \LUA\ end the same happens. Compare the following examples. First +we omit spaces: + +\startbuffer +context("left") +context("middle") +context("right") +\stopbuffer + +\typebuffer \ctxluabuffer + +Next we add spaces: + +\startbuffer +context("left") +context(" middle ") +context("right") +\stopbuffer + +\typebuffer \ctxluabuffer + +We can also add more spaces: + +\startbuffer +context("left ") +context(" middle ") +context(" right") +\stopbuffer + +\typebuffer \ctxluabuffer + +In principle all content becomes a stream and after that the \TEX\ parser will do +its normal work: collapse spaces unless configured to do otherwise. Now take the +following code: + +\startbuffer +context("before") +context("word 1") +context("word 2") +context("word 3") +context("after") +\stopbuffer + +\typebuffer \ctxluabuffer + +Here we get no spaces between the words at all, which is what we expect. So, how +do we get lines (or paragraphs)? + +\startbuffer +context("before") +context.startlines() +context("line 1") +context("line 2") +context("line 3") +context.stoplines() +context("after") +\stopbuffer + +\typebuffer \ctxluabuffer + +This does not work out well, as again there are no lines seen at the \TEX\ end. +Newline tokens are injected by passing \type {true} to the \type {context} +command: + +\startbuffer +context("before") +context.startlines() +context("line 1") context(true) +context("line 2") context(true) +context("line 3") context(true) +context.stoplines() +context("after") +\stopbuffer + +\typebuffer \ctxluabuffer + +Don't confuse this with: + +\startbuffer +context("before") context.par() +context("line 1") context.par() +context("line 2") context.par() +context("line 3") context.par() +context("after") context.par() +\stopbuffer + +\typebuffer \ctxluabuffer + +There we use the regular \type {\par} command to finish the current paragraph and +normally you will use that method. In that case, when set, whitespace will be +added between paragraphs. + +This newline issue is a somewhat unfortunate inheritance of traditional \TEX, +where \type {\n} and \type {\r} mean something different. I'm still not sure if +the \CLD\ do the right thing as dealing with these tokens also depends on the +intended effect. Catcodes as well as the \LUATEX\ input parser also play a role. +Anyway, the following also works: + +\startbuffer +context.startlines() +context("line 1\n") +context("line 2\n") +context("line 3\n") +context.stoplines() +\stopbuffer + +\typebuffer + +\stopsection + +\startsection[title=Direct output] + +\index{direct output} +\index{verbose} + +The \CONTEXT\ user interface is rather consistent and the use of special input +syntaxes is discouraged. Therefore, the \LUA\ interface using tables and strings +works quite well. However, imagine that you need to support some weird macro (or +a primitive) that does not expect its argument between curly braces or brackets. +The way out is to precede an argument by another one with the value \type +{false}. We call this the direct interface. This is demonstrated in the following +example. + +\startbuffer +\unexpanded\def\bla#1{[#1]} + +\startluacode +context.bla(false,"***") +context.par() +context.bla("***") +\stopluacode +\stopbuffer + +\typebuffer + +This results in: + +\getbuffer + +Here, the first call results in three \type {*} being passed, and \type {#1} +picks up the first token. The second call to \type {bla} gets \type {{***}} +passed so here \type {#1} gets the triplet. In practice you will seldom need the +direct interface. + +In \CONTEXT\ for historical reasons, combinations accept the following syntax: + +\starttyping +\startcombination % optional specification, like [2*3] + {\framed{content one}} {caption one} + {\framed{content two}} {caption two} +\stopcombination +\stoptyping + +You can also say: + +\starttyping +\startcombination + \combination {\framed{content one}} {caption one} + \combination {\framed{content two}} {caption two} +\stopcombination +\stoptyping + +When coded in \LUA, we can feed the first variant as follows: + +\startbuffer +context.startcombination() + context.direct("one","two") + context.direct("one","two") +context.stopcombination() +\stopbuffer + +\typebuffer + +To give you an idea what this looks like, we render it: + +\startlinecorrection[blank] +\ctxluabuffer +\stoplinecorrection + +So, the \type {direct} function is basically a no|-|op and results in nothing by +itself. Only arguments are passed. An equivalent but bit more ugly looking is: + +\starttyping +context.startcombination() + context(false,"one","two") + context(false,"one","two") +context.stopcombination() +\stoptyping + +\stopsection + +\startsection[title=Catcodes] + +\index{catcodes} + +If you are familiar with the inner working of \TEX, you will know that characters +can have special meanings. This meaning is determined by their catcodes. + +\startbuffer +context("$x=1$") +\stopbuffer + +\typebuffer + +This gives: \ctxluabuffer\ because the dollar tokens trigger inline math mode. If +you think that this is annoying, you can do the following: + +\startbuffer +context.pushcatcodes("text") +context("$x=1$") +context.popcatcodes() +\stopbuffer + +\typebuffer + +Now we get: \ctxluabuffer. There are several catcode regimes of +which only a few make sense in the perspective of the cld +interface. + +\starttabulate[|Tl|l|] +\NC ctx, ctxcatcodes, context \NC the normal \CONTEXT\ catcode regime \NC \NR +\NC prt, prtcatcodes, protect \NC the \CONTEXT\ protected regime, used for modules \NC \NR +\NC tex, texcatcodes, plain \NC the traditional (plain) \TEX\ regime \NC \NR +\NC txt, txtcatcodes, text \NC the \CONTEXT\ regime but with less special characters \NC \NR +\NC vrb, vrbcatcodes, verbatim \NC a regime specially meant for verbatim \NC \NR +\NC xml, xmlcatcodes \NC a regime specially meant for \XML\ processing \NC \NR +\stoptabulate + +In the second case you can still get math: + +\starttyping +context.pushcatcodes("text") +context.mathematics("x=1") +context.popcatcodes() +\stoptyping + +When entering a lot of math you can also consider this: + +\starttyping +context.startimath() +context("x") +context("=") +context("1") +context.stopimath() +\stoptyping + +Module writers of course can use \type {unprotect} and \type {protect} as they do +at the \TEX\ end. + +As we've seen, a function call to \type {context} acts like a print, as in: + +\startbuffer +context("test ") +context.bold("me") +context(" first") +\stopbuffer + +\typebuffer \ctxluabuffer + +When more than one argument is given, the first argument is considered a format +conforming the \type {string.format} function. + +\startbuffer +context.startimath() +context("%s = %0.5f",utf.char(0x03C0),math.pi) +context.stopimath() +\stopbuffer + +\typebuffer \ctxluabuffer + +This means that when you say: + +\starttyping +context(a,b,c,d,e,f) +\stoptyping + +the variables \type {b} till \type {f} are passed to the format and when the +format does not use them, they will not end up in your output. + +\starttyping +context("%s %s %s",1,2,3) +context(1,2,3) +\stoptyping + +The first line results in the three numbers being typeset, but in the second case +only the number~1 is typeset. + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-goodies.tex b/doc/context/sources/general/manuals/cld/cld-goodies.tex new file mode 100644 index 000000000..d5b4b5c9c --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-goodies.tex @@ -0,0 +1,621 @@ +% language=uk + +\startcomponent cld-macros + +% \usemodule[man-01] +% \setvariables[document][title=Font Goodies, author=Hans Hagen] +% \setups[titlepage] + +\environment cld-environment + +\startchapter[title=Font goodies] + +\startsection[title=Introduction] + +One of the interesting aspects of \TEX\ is that it provides control over fonts +and \LUATEX\ provides quite some. In \CONTEXT\ we support basic functionality, +like \OPENTYPE\ features, as well as some extra functionality. We also have a +mechanism for making virtual fonts which is mostly used for the transition from +\TYPEONE\ math fonts to \OPENTYPE\ math fonts. Instead of hard coding specific +details in the core \LUA\ code, we use so called \LUA\ Font Goodies to control +them. These goodies are collected in tables and live in files. When a font is +loaded, one or more such goodie files can be loaded alongside. + +In the following typescript we load a goodies file that defines a virtual Lucida +math font. The goodie file is loaded immediately and some information in the +table is turned into a form that permits access later on: the virtual font id +\type {lucida-math} that is used as part of the font specification. + +\starttyping +\starttypescript [math] [lucida] + \loadfontgoodies[lucida-math] + \definefontsynonym[MathRoman][lucidamath@lucida-math] +\stoptypescript +\stoptyping + +Not all information is to be used directly. Some can be accessed when needed. In +the following case the file \type {dingbats.lfg} gets loaded (only once) when the +font is actually used. In that file, there is information that is used by the +\type {unicoding} feature. + +\starttyping +\definefontfeature + [dingbats] + [mode=base, + goodies=dingbats, + unicoding=yes] + +\definefont[dingbats][file:dingbats][features=dingbats] +\stoptyping + +In the following sections some aspects of goodies are discussed. We don't go into +details of what these goodies are, but just stick to the \LUA\ side of the +specification. + +\stopsection + +\startsection[title=Virtual math fonts] + +A virtual font is defined using the \type {virtuals} entry in the \type +{mathematics} subtable. As \TYPEONE\ fonts are used, an additional table \type +{mapfiles} is needed to specify the files that map filenames onto real files. + +\startsmalltyping +return { + name = "px-math", + version = "1.00", + comment = "Goodies that complement px math.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + mapfiles = { + "mkiv-px.map", + }, + virtuals = { + ["px-math"] = { + { name = "texgyrepagella-regular.otf", features = "virtualmath", main = true }, + { name = "rpxr.tfm", vector = "tex-mr" } , + { name = "rpxmi.tfm", vector = "tex-mi", skewchar=0x7F }, + { name = "rpxpplri.tfm", vector = "tex-it", skewchar=0x7F }, + { name = "pxsy.tfm", vector = "tex-sy", skewchar=0x30, parameters = true } , + { name = "pxex.tfm", vector = "tex-ex", extension = true } , + { name = "pxsya.tfm", vector = "tex-ma" }, + { name = "pxsyb.tfm", vector = "tex-mb" }, + { name = "texgyrepagella-bold.otf", vector = "tex-bf" } , + { name = "texgyrepagella-bolditalic.otf", vector = "tex-bi" } , + { name = "lmsans10-regular.otf", vector = "tex-ss", optional=true }, + { name = "lmmono10-regular.otf", vector = "tex-tt", optional=true }, + }, + } + } +} +\stopsmalltyping + +Here the \type {px-math} virtual font is defined. A series of fonts is loaded and +combined into one. The \type {vector} entry is used to tell the builder how to +map the glyphs onto \UNICODE. Additional vectors can be defined, for instance: + +\starttyping +fonts.encodings.math["mine"] = { + [0x1234] = 0x56, +} +\stoptyping + +Eventually these specifications wil be replaced by real \OPENTYPE\ fonts, but +even then we will keep the virtual definitions around. + +\startsection[title=Math alternates] + +In addition to the official \type {ssty} feature for enforcing usage of script +and scriptscript glyphs, some stylistic alternates can be present. + +\startsmalltyping +return { + name = "xits-math", + version = "1.00", + comment = "Goodies that complement xits (by Khaled Hosny).", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + alternates = { + cal = { + feature = 'ss01', + value = 1, + comment = "Mathematical Calligraphic Alphabet" + }, + greekssup = { + feature = 'ss02', + value = 1, + comment = "Mathematical Greek Sans Serif Alphabet" + }, + greekssit = { + feature = 'ss03', + value = 1, + comment = "Mathematical Italic Sans Serif Digits" + }, + monobfnum = { + feature = 'ss04', + value = 1, + comment = "Mathematical Bold Monospace Digits" + }, + mathbbbf = { + feature = 'ss05', + value = 1, + comment = "Mathematical Bold Double-Struck Alphabet" + }, + mathbbit = { + feature = 'ss06', + value = 1, + comment = "Mathematical Italic Double-Struck Alphabet" + }, + mathbbbi = { + feature = 'ss07', + value = 1, + comment = "Mathematical Bold Italic Double-Struck Alphabet" + }, + upint = { + feature = 'ss08', + value = 1, + comment = "Upright Integrals" + }, + } + } +} +\stopsmalltyping + +These can be activated (in math mode) with the \type {\mathalternate} command +like: + +\starttyping +$\mathalternate{cal}Z$ +\stoptyping + +\stopsection + +\startsection[title=Math parameters] + +Another goodie related to math is the overload of some parameters (part of the +font itself) and variables (used in making virtual shapes). + +\startsmalltyping +return { + name = "lm-math", + version = "1.00", + comment = "Goodies that complement latin modern math.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + mapfiles = { + "lm-math.map", + "lm-rm.map", + "mkiv-base.map", + }, + virtuals = { + ["lmroman5-math"] = five, + ["lmroman6-math"] = six, + ["lmroman7-math"] = seven, + ["lmroman8-math"] = eight, + ["lmroman9-math"] = nine, + ["lmroman10-math"] = ten, + ["lmroman10-boldmath"] = ten_bold, + ["lmroman12-math"] = twelve, + ["lmroman17-math"] = seventeen, + }, + variables = { + joinrelfactor = 3, -- default anyway + }, + parameters = { -- test values + -- FactorA = 123.456, + -- FactorB = false, + -- FactorC = function(value,target,original) return 7.89 * target.factor end, + -- FactorD = "Hi There!", + }, + } +} +\stopsmalltyping + +In this example you see several virtuals defined which is due to the fact that +Latin Modern has design sizes. The values (like \type {twelve} are tables defined +before the return happens and are not shown here. The variables are rather +\CONTEXT\ specific, and the parameters are those that come with regular +\OPENTYPE\ math fonts (so the example names are invalid). + +In the following example we show two wasy to change parameters. In this case we +have a regular \OPENTYPE\ math font. First we install a patch to the font itself. +That change will be cached. We could also have changed that parameter using the +goodies table. The first method is the oldest. + +\startsmalltyping +local patches = fonts.handlers.otf.enhancers.patches + +local function patch(data,filename,threshold) + local m = data.metadata.math + if m then + local d = m.DisplayOperatorMinHeight or 0 + if d < threshold then + patches.report("DisplayOperatorMinHeight(%s -> %s)",d,threshold) + m.DisplayOperatorMinHeight = threshold + end + end +end + +patches.register( + "after", + "check math parameters", + "asana", + function(data,filename) + patch(data,filename,1350) + end +) + +local function less(value,target,original) + return 0.25 * value +end + +return { + name = "asana-math", + version = "1.00", + comment = "Goodies that complement asana.", + author = "Hans Hagen", + copyright = "ConTeXt development team", + mathematics = { + parameters = { + StackBottomDisplayStyleShiftDown = less, + StackBottomShiftDown = less, + StackDisplayStyleGapMin = less, + StackGapMin = less, + StackTopDisplayStyleShiftUp = less, + StackTopShiftUp = less, + StretchStackBottomShiftDown = less, + StretchStackGapAboveMin = less, + StretchStackGapBelowMin = less, + StretchStackTopShiftUp = less, + } + } +} +\stopsmalltyping + +We use a function so that the scaling is taken into account as the values passed +are those resulting from the scaling of the font to the requested size. + +\stopsection + +\startsection[title=Unicoding] + +We still have to deal with existing \TYPEONE\ fonts, and some of them have an +encoding that is hard to map onto \UNICODE\ without additional information. The +following goodie does that. The keys in the \type {unicodes} table are the glyph +names. Keep in mind that this only works with simple fonts. The \CONTEXT\ code +takes care of kerns but that's about it. + +\startsmalltyping +return { + name = "dingbats", + version = "1.00", + comment = "Goodies that complement dingbats (funny names).", + author = "Hans Hagen", + copyright = "ConTeXt development team", + remapping = { + tounicode = true, + unicodes = { + a1 = 0x2701, + a10 = 0x2721, + a100 = 0x275E, + a101 = 0x2761, + ............. + a98 = 0x275C, + a99 = 0x275D, + }, + }, +} +\stopsmalltyping + +The \type {tounicode} option makes sure that additional information ends ip in +the output so that cut|-|and|-|paste becomes more trustworthy. + +\stopsection + +\startsection[title=Typescripts] + +Some font collections, like antykwa, come with so many variants that defining +them all in typescripts becomes somewhat of a nuisance. While a regular font has +a typescript of a few lines, antykwa needs way more lines. This is why we provide +a nother way as well, using goodies. + +\startsmalltyping +return { + name = "antykwapoltawskiego", + version = "1.00", + comment = "Goodies that complement Antykwa Poltawskiego", + author = "Hans & Mojca", + copyright = "ConTeXt development team", + files = { + name = "antykwapoltawskiego", -- shared + list = { + ["AntPoltLtCond-Regular.otf"] = { + -- name = "antykwapoltawskiego", + weight = "light", + style = "regular", + width = "condensed", + }, + ["AntPoltLtCond-Italic.otf"] = { + weight = "light", + style = "italic", + width = "condensed", + }, + ["AntPoltCond-Regular.otf"] = { + weight = "normal", + style = "regular", + width = "condensed", + }, + + ....... + + + ["AntPoltExpd-BoldItalic.otf"] = { + weight = "bold", + style = "italic", + width = "expanded", + }, + }, + }, + typefaces = { -- for Mojca (experiment, names might change) + ["antykwapoltawskiego-light"] = { + shortcut = "rm", + shape = "serif", + fontname = "antykwapoltawskiego", + normalweight = "light", + boldweight = "medium", + width = "normal", + size = "default", + features = "default", + }, + + ....... + + }, +} +\stopsmalltyping + +This is a typical example of when a goodies file is loaded directly: + +\starttyping +\loadfontgoodies[antykwapoltawskiego] +\stoptyping + +A bodyfont is now defined by choosing from the defined combinations: + +\starttyping +\definetypeface + [name=mojcasfavourite, + preset=antykwapoltawskiego, + normalweight=light, + boldweight=bold, + width=expanded] + +\setupbodyfont + [mojcasfavourite] +\stoptyping + +This mechanism is a follow up on a discussion at a \CONTEXT\ conference, still +somewhat experimental, and a playground for Mojca. + +\stopsection + +\startsection[title=Font strategies] + +This goodie is closely related to the Oriental \TEX\ project where a dedicated +paragraph optimizer can be used. A rather advanced font is used (husayni) and its +associated goodie file is rather extensive. It defines stylistic features, +implements a couple of feature sets, provides colorschemes and most of all, +defines some strategies for making paragraphs look better. Some of the goodie +file is shown here. + +\startsmalltyping +local yes = "yes" + +local basics = { + analyze = yes, + mode = "node", + language = "dflt", + script = "arab", +} + +local analysis = { + ccmp = yes, + init = yes, medi = yes, fina = yes, +} + +local regular = { + rlig = yes, calt = yes, salt = yes, anum = yes, + ss01 = yes, ss03 = yes, ss07 = yes, ss10 = yes, ss12 = yes, ss15 = yes, ss16 = yes, + ss19 = yes, ss24 = yes, ss25 = yes, ss26 = yes, ss27 = yes, ss31 = yes, ss34 = yes, + ss35 = yes, ss36 = yes, ss37 = yes, ss38 = yes, ss41 = yes, ss42 = yes, ss43 = yes, + js16 = yes, +} + +local positioning = { + kern = yes, curs = yes, mark = yes, mkmk = yes, +} + +local minimal_stretching = { + js11 = yes, js03 = yes, +} + +local medium_stretching = { + js12=yes, js05=yes, +} + +local maximal_stretching= { + js13 = yes, js05 = yes, js09 = yes, +} + +local wide_all = { + js11 = yes, js12 = yes, js13 = yes, js05 = yes, js09 = yes, +} + +local shrink = { + flts = yes, js17 = yes, ss05 = yes, ss11 = yes, ss06 = yes, ss09 = yes, +} + +local default = { + basics, analysis, regular, positioning, -- xxxx = yes, yyyy = 2, +} + +return { + name = "husayni", + version = "1.00", + comment = "Goodies that complement the Husayni font by Idris Samawi Hamid.", + author = "Idris Samawi Hamid and Hans Hagen", + featuresets = { -- here we don't have references to featuresets + default = { + default, + }, + minimal_stretching = { + default, + js11 = yes, js03 = yes, + }, + medium_stretching = { + default, + js12=yes, js05=yes, + }, + maximal_stretching= { + default, + js13 = yes, js05 = yes, js09 = yes, + }, + wide_all = { + default, + js11 = yes, js12 = yes, js13 = yes, js05 = yes, js09 = yes, + }, + shrink = { + default, + flts = yes, js17 = yes, ss05 = yes, ss11 = yes, ss06 = yes, ss09 = yes, + }, + }, + solutions = { -- here we have references to featuresets, so we use strings! + experimental = { + less = { + "shrink" + }, + more = { + "minimal_stretching", + "medium_stretching", + "maximal_stretching", + "wide_all" + }, + }, + }, + stylistics = { + ...... + ss03 = "level-1 stack over Jiim, initial entry only", + ss04 = "level-1 stack over Jiim, initial/medial entry", + ...... + ss54 = "chopped finals", + ss55 = "idgham-tanwin", + ...... + js11 = "level-1 stretching", + js12 = "level-2 stretching", + ...... + js21 = "Haa.final_alt2", + }, + colorschemes = { + default = { + [1] = { + "Onedotabove", "Onedotbelow", ... + }, + [2] = { + "Fathah", "Dammah", "Kasrah", ... + }, + [3] = { + "Ttaa.waqf", "SsLY.waqf", "QLY.waqf", ... + }, + [4] = { + "ZeroArabic.ayah", "OneArabic.ayah", "TwoArabic.ayah", ... + }, + [5] = { + "Ayah", "Ayah.alt1", "Ayah.alt2", ... + } + } + } +} +\stopmalltyping + +Discussion of these goodies is beyond this document and happens elsewhere. + +\stopsection + +\startsection[title=Composition] + +The \type {compose} features extends a font with additional (virtual) shapes. +This is mostly used with \TYPEONE\ fonts that lack support for eastern european +languages. The type {compositions} subtable is used to control placement of +accents. This can be done per font. + +\startmalltyping +local defaultunits = 193 - 30 + +-- local compose = { +-- DY = defaultunits, +-- [0x010C] = { DY = defaultunits }, -- Ccaron +-- [0x02C7] = { DY = defaultunits }, -- textcaron +-- } + +-- fractions relative to delta(X_height - x_height) + +local defaultfraction = 0.85 + +local compose = { + DY = defaultfraction, -- uppercase compensation +} + +return { + name = "lucida-one", + version = "1.00", + comment = "Goodies that complement lucida.", + author = "Hans and Mojca", + copyright = "ConTeXt development team", + compositions = { + ["lbr"] = compose, + ["lbi"] = compose, + ["lbd"] = compose, + ["lbdi"] = compose, + } +} +\stopsmalltyping + +\stopsection + +\startsection[title=Postprocessing] + +You can hook postprocessors into the scaler. Future versions might provide more +control over where this happens. + +\startsmalltyping +local function statistics(tfmdata) + commands.showfontparameters(tfmdata) +end + +local function squeeze(tfmdata) + for k, v in next, tfmdata.characters do + v.height = 0.75 * (v.height or 0) + v.depth = 0.75 * (v.depth or 0) + end +end + +return { + name = "demo", + version = "1.00", + comment = "An example of goodies.", + author = "Hans Hagen", + postprocessors = { + statistics = statistics, + squeeze = squeeze, + }, +} +\stopsmalltyping + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-graphics.tex b/doc/context/sources/general/manuals/cld/cld-graphics.tex new file mode 100644 index 000000000..93ab80c2c --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-graphics.tex @@ -0,0 +1,342 @@ +% language=uk + +\startcomponent cld-graphics + +\environment cld-environment + +\startchapter[title=Graphics] + +\startsection[title=The regular interface] + +If you are familiar with \CONTEXT, which by now probably is the case, you will +have noticed that it integrates the \METAPOST\ graphic subsystem. Drawing a +graphic is not that complex: + +\startbuffer +context.startMPcode() +context [[ + draw + fullcircle scaled 1cm + withpen pencircle scaled 1mm + withcolor .5white + dashed dashpattern (on 2mm off 2mm) ; + ]] +context.stopMPcode() +\stopbuffer + +\typebuffer + +We get a gray dashed circle rendered with an one millimeter thick line: + +\startlinecorrection +\ctxluabuffer +\stoplinecorrection + +So, we just use the regular commands and pass the drawing code as strings. +Although \METAPOST\ is a rather normal language and therefore offers loops and +conditions and the lot, you might want to use \LUA\ for anything else than the +drawing commands. Of course this is much less efficient, but it could be that you +don't care about speed. The next example demonstrates the interface for building +graphics piecewise. + +\startbuffer +context.resetMPdrawing() + +context.startMPdrawing() +context([[fill fullcircle scaled 5cm withcolor (0,0,.5) ;]]) +context.stopMPdrawing() + +context.MPdrawing("pickup pencircle scaled .5mm ;") +context.MPdrawing("drawoptions(withcolor white) ;") + +for i=0,50,5 do + context.startMPdrawing() + context("draw fullcircle scaled %smm ;",i) + context.stopMPdrawing() +end + +for i=0,50,5 do + context.MPdrawing("draw fullsquare scaled " .. i .. "mm ;") +end + +context.MPdrawingdonetrue() + +context.getMPdrawing() +\stopbuffer + +\typebuffer + +This gives: + +\startlinecorrection +\ctxluabuffer +\stoplinecorrection + +I the first loop we can use the format options associated with the simple \type +{context} call. This will not work in the second case. Even worse, passing more +than one argument will definitely give a faulty graphic definition. This is why +we have a special interface for \METAFUN. The code above can also be written as: + +\startbuffer +local metafun = context.metafun + +metafun.start() + +metafun("fill fullcircle scaled 5cm withcolor %s ;", + metafun.color("darkblue")) + +metafun("pickup pencircle scaled .5mm ;") +metafun("drawoptions(withcolor white) ;") + +for i=0,50,5 do + metafun("draw fullcircle scaled %smm ;",i) +end + +for i=0,50,5 do + metafun("draw fullsquare scaled %smm ;",i) +end + +metafun.stop() +\stopbuffer + +\typebuffer + +Watch the call to \type {color}, this will pass definitions at the \TEX\ end to +\METAPOST. Of course you really need to ask yourself \quotation {Do I want to use +\METAPOST\ this way?}. Using \LUA\ loops instead of \METAPOST\ ones makes much +more sense in the following case: + +\startbuffer +local metafun = context.metafun + +function metafun.barchart(t) + metafun.start() + local t = t.data + for i=1,#t do + metafun("draw unitsquare xyscaled(%s,%s) shifted (%s,0);", + 10, t[i]*10, i*10) + end + metafun.stop() +end + +local one = { 1, 4, 6, 2, 3, } +local two = { 8, 1, 3, 5, 9, } + +context.startcombination() + context.combination(metafun.delayed.barchart { data = one }, "one") + context.combination(metafun.delayed.barchart { data = two }, "two") +context.stopcombination() +\stopbuffer + +\typebuffer + +We get two barcharts alongside: + +\startlinecorrection +\ctxluabuffer +\stoplinecorrection + +\startbuffer +local template = [[ + path p, q ; color c[] ; + c1 := \MPcolor{darkblue} ; + c2 := \MPcolor{darkred} ; + p := fullcircle scaled 50 ; + l := length p ; + n := %s ; + q := subpath (0,%s/n*l) of p ; + draw q withcolor c2 withpen pencircle scaled 1 ; + fill fullcircle scaled 5 shifted point length q of q withcolor c1 ; + setbounds currentpicture to unitsquare shifted (-0.5,-0.5) scaled 60 ; + draw boundingbox currentpicture withcolor c1 ; + currentpicture := currentpicture xsized(1cm) ; +]] + +local function steps(n) + for i=0,n do + context.metafun.start() + context.metafun(template,n,i) + context.metafun.stop() + if i < n then + context.quad() + end + end +end + +context.hbox(function() steps(10) end) +\stopbuffer + +\typebuffer + +\startlinecorrection +\ctxluabuffer +\stoplinecorrection + +Using a template is quite convenient but at some point you can loose track of the +replacement values. Also, adding an extra value can force you to adapt the +following ones which enlarges the change for making an error. An alternative is +to use the template mechanism. Although this mechanism was originally made for +other purposes, you can use it for whatever you like. + +\startbuffer +local template = [[ + path p ; p := fullcircle scaled 4cm ; + draw p withpen pencircle scaled .5mm withcolor red ; + freedotlabel ("%lefttop%", point 1 of p,origin) ; + freedotlabel ("%righttop%", point 3 of p,origin) ; + freedotlabel ("%leftbottom%", point 5 of p,origin) ; + freedotlabel ("%rightbottom%",point 7 of p,origin) ; +]] + +local variables = { + lefttop = "one", + righttop = "two", + leftbottom = "three", + rightbottom = "four" , +} + +context.metafun.start() + context.metafun(utilities.templates.replace(template,variables)) +context.metafun.stop() +\stopbuffer + +\typebuffer + +Here we use named placeholders and pass a table with associated values to the +replacement function. Apart from convenience it's also more readable. And the +overhead is rather minimal. + +\startlinecorrection +\ctxluabuffer +\stoplinecorrection + +To some extent we fool ourselves with this kind of \LUA fication of \METAPOST\ +code. Of course we can make a nice \METAPOST\ library and put the code in a macro +instead. In that sense, doing this in \CONTEXT\ directly often gives better and +more efficient code. + +Of course you can use all relevant commands in the \LUA\ interface, like: + +\starttyping +context.startMPpage() + context("draw origin") + for i=0,100,10 do + context("..{down}(%d,0)",i) + end + context(" withcolor \\MPcolor{darkred} ;") +context.stopMPpage() +\stoptyping + +to get a graphic that has its own page. Don't use the \type {metafun} namespace +here, as it will not work here. This drawing looks like: + +\startlinecorrection +\startluacode +context.startMPcode() + context("draw origin") + for i=0,100,10 do + context("..{down}(%d,0)",i) + end + context(" withcolor red ;") +context.stopMPcode() +\stopluacode +\stoplinecorrection + +\stopsection + +\startsection[title=The \LUA\ interface] + +Messing around with graphics is normally not needed and if you do it, you'd +better know what you're doing. For \TEX\ a graphic is just a black box: a +rectangle with dimensions. You specify a graphic, in a format that the backend +can deal with, either or not apply some scaling and from then on a reference to +that graphic, normally wrapped in a normal \TEX\ box, enters the typesetting +machinery. Because the backend, the part that is responsible for translating +typeset content onto a viewable or printable format like \PDF, is built into +\LUATEX, at some point the real image has to be injected and the backend can only +handle a few image formats: \PNG, \JPG, \JBIG\ and \PDF. + +In \CONTEXT\ some more image formats are supported but in practice this boils +down to converting the image to a format that the backend can handle. Such a +conversion depends on an external programs and in order not to redo the +conversion each run \CONTEXT\ keeps track of the need to redo it. + +Some converters are built in, for example one that deals with \GIF\ images. This +is normally not a preferred format, but it happens that we have to deal with it +in cases where organizations use that format (if only because they use the web). +Here is how this works at the \LUA\ end: + +\starttyping +figures.converters.gif = { + pdf = function(oldname,newname) + os.execute(string.format("gm convert %s %s",oldname,newname)) + end +} +\stoptyping + +We use \type {gm} (Graphic Magic) for the conversion and pass the old and new +names. Given this definition at the \TEX\ end we can say: + +\starttyping +\externalfigure[whatever.gif][width=4cm] +\stoptyping + +Here is a another one: + +\starttyping +figures.converters.bmp = { + pdf = function(oldname,newname) + os.execute(string.format("gm convert %s %s",oldname,newname)) + end +} +\stoptyping + +In both examples we convert to \PDF\ because including this filetype is quite +fast. But you can also go to other formats: + +\starttyping +figures.converters.png = { + png = function(oldname,newname,resolution) + local command = string.format('gm convert -depth 1 "%s" "%s"',oldname,newname) + logs.report(string.format("running command %s",command)) + os.execute(command) + end +} +\stoptyping + +Instead of directly defining such a table, you can better do this: + +\starttyping +figures.converters.png = figures.converters.png or { } + +figures.converters.png.png = function(oldname,newname,resolution) + local command = string.format('gm convert -depth 1 "%s" "%s"',oldname,newname) + logs.report(string.format("running command %s",command)) + os.execute(command) +end +\stoptyping + +Here we check if a table exists and if not we extend the table. Such converters +work out of the box if you specify the suffix, but you can also opt for a simple: + +\starttyping +\externalfigure[whatever][width=4cm] +\stoptyping + +In this case \CONTEXT\ will check for all known supported formats, which is not +that efficient when no graphic can be found. In order to let for instance files +with suffix \type {bmp} can be included you have to register it as follows. The +second argument is the target. + +\starttyping +figures.registersuffix("bmp","bmp") +\stoptyping + +At some point more of the graphic inclusion helpers will be opened up for general +use but for now this is what you have available. + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-introduction.tex b/doc/context/sources/general/manuals/cld/cld-introduction.tex new file mode 100644 index 000000000..82fcb8007 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-introduction.tex @@ -0,0 +1,55 @@ +% language=uk + +\startcomponent cld-introduction + +\environment cld-environment + +\startchapter[title=Introduction] + +Sometimes you hear folks complain about the \TEX\ input language, i.e.\ the +backslashed commands that determine your output. Of course, when alternatives are +being discussed every one has a favourite programming language. In practice +coding a document in each of them triggers similar sentiments with regards to +coding as \TEX\ itself does. + +So, just for fun, I added a couple of commands to \CONTEXT\ \MKIV\ that permit +coding a document in \LUA. In retrospect it has been surprisingly easy to +implement a feature like this using metatables. Of course it's a bit slower than +using \TEX\ as input language but sometimes the \LUA\ interface is more readable +given the problem at hand. + +After a while I decided to use that interface in non|-|critical core \CONTEXT\ +code and in styles (modules) and solutions for projects. Using the \LUA\ approach +is sometimes more convenient, especially if the code mostly manipulates data. For +instance, if you process \XML\ files of database output you can use the interface +that is available at the \TEX\ end, or you can use \LUA\ code to do the work, or +you can use a combination. So, from now on, in \CONTEXT\ you can code your style +and document source in (a mixture of) \TEX, \XML, \METAPOST\ and in \LUA. + +In the following chapters I will introduce typesetting in \LUA, but as we rely on +\CONTEXT\ it is unavoidable that some regular \CONTEXT\ code shows up. The fact +that you can ignore backslashes does not mean that you can do without knowledge +of the underlying system. I expect that the user is somewhat familiar with this +macro package. Some chapters are follow ups on articles or earlier publications. + +Some information (and mechanism) show up in more than one chapter. This is a side +effect of \LUA\ being integrated in many places, so an isolated discussion is a +bit hard. + +In the meantime most of the code is rather stable and proven. However, this +manual will never be complete. You can find examples all over the code base, and +duplicating everything here makes no sense. If you find errors, please let me +know. If you think that something is missing, you can try to convince me to add +it. It's hard to keep up with what gets added so input is welcome. + +\blank[2*big] + +\startlines +Hans Hagen +Hasselt NL +2009 \emdash\ 2016 +\stoplines + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-logging.tex b/doc/context/sources/general/manuals/cld/cld-logging.tex new file mode 100644 index 000000000..cbb904a69 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-logging.tex @@ -0,0 +1,91 @@ +% language=uk + +% maybe this will become a section instead + +\startcomponent cld-logging + +\environment cld-environment + +\startchapter[title={Logging}] + +Logging and localized messages have always been rather standardized in \CONTEXT, +so upgrading the related mechanism had been quite doable. In \MKIV\ for a while +we had two systems in parallel: the old one, mostly targeted at messages at the +\TEX\ end, and a new one used at the \LUA\ end. But when more and more hybrid +code showed up, integrating both systems made sense. + +Most logging concerns tracing and can be turned on and off on demand. This kind +of control is now possible for all messages. Given that the right interfaces are +used, you can turn off all messages: + +\starttyping +context --silent +\stoptyping + +This was already possible in \MKII, but there \TEX's own messages still were +visible. More important is that we have control: + +\starttyping +context --silent=structure*,resolve*,font* +\stoptyping + +This will disable all reporting for these three categories. It is also possible +to only disable messages to the console: + +\starttyping +context --noconsole +\stoptyping + +In \CONTEXT\ you can use directives: + +\starttyping +\enabledirectives[logs.blocked=structure*,resolve*,font*] +\enabledirectives[logs.target=file] +\stoptyping + +As all logging is under \LUA\ control and because this (and other) kind of +control has to kick in early in the initialization the code might look somewhat +tricky. Users won't notice this because they only deal with the formal interface. +Here we will only discuss the \LUA\ interfaces. + +Messages related to tracing are done as follows: + +\starttyping +local report_whatever = logs.reporter("modules","whatever") + +report_whatever("not found: %s","this or that") +\stoptyping + +The first line defined a logger in the category \type {modules}. You can give a +second argument as well, the subcategory. Both will be shown as part of the +message, of which an example is given in the second line. + +These messages are shown directly, that is, when the function is called. However, +when you generate \TEX\ code, as we discuss in this document, you need to make +sure that the message is synchronized with that code. This can be done with a +messenger instead of a reporter. + +\starttyping +local report_numbers = logs.reporter("numbers","check") +local status_numbers = logs.messenger("numbers","check") + +status_numbers("number 1: %s, number 2: %s",123,456) +report_numbers("number 1: %s, number 2: %s",456,123) +\stoptyping + +Both reporters and messages are localized when the pattern given as first +argument can be found in the \type {patterns} subtable of the interface messages. +Categories and subcategories are also translated, but these are looked up in the +\type {translations} subtable. So in the case of + +\starttyping +report_whatever("found: %s",filename) +report_whatever("not found: %s",filename) +\stoptyping + +you should not be surprised if it gets translated. Of course the category and +subcategory provide some contextual information. + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-luafunctions.tex b/doc/context/sources/general/manuals/cld/cld-luafunctions.tex new file mode 100644 index 000000000..b2760e05b --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-luafunctions.tex @@ -0,0 +1,2367 @@ +% language=uk + +% table.unnest : only used in special cases +% table.derive : set metatable if unset +% table.compact : remove empty subtables + +\environment cld-environment + +\startcomponent cld-luafunctions + +\startchapter[title=Lua Functions] + +\startsection[title={Introduction}] + +When you run \CONTEXT\ you have some libraries preloaded. If you look into the +\LUA\ files you will find more than is discussed here, but keep in mind that what +is not documented, might be gone or done different one day. Some extensions live +in the same namespace as those provided by stock \LUA\ and \LUATEX, others have +their own. There are many more functions and the more obscure (or never being +used) ones will go away. + +The \LUA\ code in \CONTEXT\ is organized in quite some modules. Those with names +like \type {l-*.lua} are rather generic and are automatically available when you +use \type {mtxrun} to run a \LUA\ file. These are discusses in this chapter. A +few more modules have generic properties, like some in the categories \type +{util-*.lua}, \type {trac-*.lua}, \type {luat-*.lua}, \type {data-*.lua} and +\type {lxml-*.lua}. They contain more specialized functions and are discussed +elsewhere. + +Before we move on the the real code, let's introduce a handy helper: + +\starttyping +inspect(somevar) +\stoptyping + +Whenever you feel the need to see what value a variable has you can insert this +function to get some insight. It knows how to deal with several data types. + +\stopsection + +\startsection[title={Tables}] + +\startsummary[title={[lua] concat}] + +These functions come with \LUA\ itself and are discussed in detail in the \LUA\ +reference manual so we stick to some examples. The \type {concat} function +stitches table entries in an indexed table into one string, with an optional +separator in between. If can also handle a slice of the table + +\starttyping +local str = table.concat(t) +local str = table.concat(t,separator) +local str = table.concat(t,separator,first) +local str = table.concat(t,separator,first,last) +\stoptyping + +Only strings and numbers can be concatenated. + +\ShowLuaExampleThree {table} {concat} {{"a","b","c","d","e"}} +\ShowLuaExampleThree {table} {concat} {{"a","b","c","d","e"},"+"} +\ShowLuaExampleThree {table} {concat} {{"a","b","c","d","e"},"+",2,3} + +\stopsummary + +\startsummary[title={[lua] insert remove}] + +You can use \type {insert} and \type {remove} for adding or replacing entries in +an indexed table. + +\starttyping +table.insert(t,value,position) +value = table.remove(t,position) +\stoptyping + +The position is optional and defaults to the last entry in the table. For +instance a stack is built this way: + +\starttyping +table.insert(stack,"top") +local top = table.remove(stack) +\stoptyping + +Beware, the \type {insert} function returns nothing. You can provide an +additional position: + +\starttyping +table.insert(list,"injected in slot 2",2) +local thiswastwo = table.remove(list,2) +\stoptyping + +\stopsummary + +\startsummary[title={[lua] unpack}] + +You can access entries in an indexed table as follows: + +\starttyping +local a, b, c = t[1], t[2], t[3] +\stoptyping + +but this does the same: + +\starttyping +local a, b, c = table.unpack(t) +\stoptyping + +This is less efficient but there are situations where \type {unpack} +comes in handy. + +\stopsummary + +\startsummary[title={[lua] sort}] + +Sorting is done with \type {sort}, a function that does not return a value but +operates on the given table. + +\starttyping +table.sort(t) +table.sort(t,comparefunction) +\stoptyping + +The compare function has to return a consistent equivalent of \type {true} or +\type {false}. For sorting more complex data structures there is a specialized +sort module available. + +\ShowLuaExampleFour {table} {sort} {{"a","b","c"}} {} +\ShowLuaExampleFour {table} {sort} {{"a","b","c"}} {,function(x,y) return x > y end} +\ShowLuaExampleFour {table} {sort} {{"a","b","c"}} {,function(x,y) return x < y end} + +\stopsummary + +\startsummary[title={sorted}] + +The built|-|in \type {sort} function does not return a value but sometimes it can be +if the (sorted) table is returned. This is why we have: + +\starttyping +local a = table.sorted(b) +\stoptyping + +\stopsummary + +% table.strip + +\startsummary[title={keys sortedkeys sortedhashkeys sortedhash}] + +The \type {keys} function returns an indexed list of keys. The order is undefined +as it depends on how the table was constructed. A sorted list is provided by +\type {sortedkeys}. This function is rather liberal with respect to the keys. If +the keys are strings you can use the faster alternative \type {sortedhashkeys}. + +\starttyping +local s = table.keys (t) +local s = table.sortedkeys (t) +local s = table.sortedhashkeys (t) +\stoptyping + +Because a sorted list is often processed there is also an iterator: + +\starttyping +for key, value in table.sortedhash(t) do + print(key,value) +end +\stoptyping + +There is also a synonym \type {sortedpairs} which sometimes looks more natural +when used alongside the \type {pairs} and \type {ipairs} iterators. + +\ShowLuaExampleTwo {table} {keys} {{ [1] = 2, c = 3, [true] = 1 }} +\ShowLuaExampleTwo {table} {sortedkeys} {{ [1] = 2, c = 3, [true] = 1 }} +\ShowLuaExampleTwo {table} {sortedhashkeys} {{ a = 2, c = 3, b = 1 }} + +\stopsummary + +\startsummary[title={serialize print tohandle tofile}] + +The \type {serialize} function converts a table into a verbose representation. +The \type {print} function does the same but prints the result to the console +which is handy for tracing. The \type {tofile} function writes the table to a +file, using reasonable chunks so that less memory is used. The fourth variant +\type {tohandle} takes a handle so that you can do whatever you like with the +result. + +\starttyping +table.serialize (root, name, reduce, noquotes, hexify) +table.print (root, name, reduce, noquotes, hexify) +table.tofile (filename, root, name, reduce, noquotes, hexify) +table.tohandle (handle, root, name, reduce, noquotes, hexify) +\stoptyping + +The serialization can be controlled in several ways. Often only the first two +options makes sense: + +\ShowLuaExampleOne {table} {serialize} {{ a = 2 }} +\ShowLuaExampleOne {table} {serialize} {{ a = 2 }, "name"} +\ShowLuaExampleOne {table} {serialize} {{ a = 2 }, true} +\ShowLuaExampleOne {table} {serialize} {{ a = 2 }, false} +\ShowLuaExampleOne {table} {serialize} {{ a = 2 }, "return"} +\ShowLuaExampleOne {table} {serialize} {{ a = 2 }, 12} + +\ShowLuaExampleOne {table} {serialize} {{ a = 2, [3] = "b", [true] = "6" }, nil, true} +\ShowLuaExampleOne {table} {serialize} {{ a = 2, [3] = "b", [true] = "6" }, nil, true, true} +\ShowLuaExampleOne {table} {serialize} {{ a = 2, [3] = "b", [true] = "6" }, nil, true, true, true} + +In \CONTEXT\ there is also a \type {tocontext} function that typesets the table +verbose. This is handy for manuals and tracing. + +\stopsummary + +\startsummary[title={identical are_equal}] + +These two function compare two tables that have a similar structure. The \type +{identical} variant operates on a hash while \type {are_equal} assumes an indexed +table. + +\starttyping +local b = table.identical (one, two) +local b = table.are_equal (one, two) +\stoptyping + +\ShowLuaExampleThree {table} {identical} {{ a = { x = 2 } }, { a = { x = 3 } }} +\ShowLuaExampleThree {table} {identical} {{ a = { x = 2 } }, { a = { x = 2 } }} + +\ShowLuaExampleThree {table} {are_equal} {{ a = { x = 2 } }, { a = { x = 3 } }} +\ShowLuaExampleThree {table} {are_equal} {{ a = { x = 2 } }, { a = { x = 2 } }} + +\ShowLuaExampleThree {table} {identical} {{ "one", "two" }, { "one", "two" }} +\ShowLuaExampleThree {table} {identical} {{ "one", "two" }, { "two", "one" }} + +\ShowLuaExampleThree {table} {are_equal} {{ "one", "two" }, { "one", "two" }} +\ShowLuaExampleThree {table} {are_equal} {{ "one", "two" }, { "two", "one" }} + +\stopsummary + +\startsummary[title={tohash fromhash swapped swaphash reversed reverse mirrored}] + +We use \type {tohash} quite a lot in \CONTEXT. It converts a list into a hash so +that we can easily check if (a string) is in a given set. The \type {fromhash} +function does the opposite: it creates a list of keys from a hashed table where +each value that is not \type {false} or \type {nil} is present. + +\starttyping +local hashed = table.tohash (indexed) +local indexed = table.fromhash(hashed) +\stoptyping + +The function \type {swapped} turns keys into values vise versa while the \type +{reversed} and \type {reverse} reverses the values in an indexed table. The last +one reverses the table itself (in|-|place). + +\starttyping +local swapped = table.swapped (indexedtable) +local reversed = table.reversed (indexedtable) +local reverse = table.reverse (indexedtable) +local mirrored = table.mirrored (hashedtable) +\stoptyping + +\ShowLuaExampleTwo {table} {tohash} {{ "a", "b", "c" }} +\ShowLuaExampleTwo {table} {fromhash} {{ a = true, b = false, c = true }} +\ShowLuaExampleTwo {table} {swapped} {{ "a", "b", "c" }} +\ShowLuaExampleTwo {table} {reversed} {{ "a", "b", "c" }} +\ShowLuaExampleTwo {table} {reverse} {{ 1, 2, 3, 4 }} +\ShowLuaExampleTwo {table} {mirrored} {{ a = "x", b = "y", c = "z" }} + +\stopsummary + +\startsummary[title={append prepend}] + +These two functions operate on a pair of indexed tables. The first table gets +appended or prepended by the second. The first table is returned as well. + +\starttyping +table.append (one, two) +table.prepend(one, two) +\stoptyping + +The functions are similar to loops using \type {insert}. + +\ShowLuaExampleTwo {table} {append} {{ "a", "b", "c" }, { "d", "e" }} +\ShowLuaExampleTwo {table} {prepend} {{ "a", "b", "c" }, { "d", "e" }} + +\stopsummary + +\startsummary[title={merge merged imerge imerged}] + +You can merge multiple hashes with \type {merge} and indexed tables with \type +{imerge}. The first table is the target and is returned. + +\starttyping +table.merge (one, two, ...) +table.imerge (one, two, ...) +\stoptyping + +The variants ending with a \type {d} merge the given list of tables and return +the result leaving the first argument untouched. + +\starttyping +local merged = table.merged (one, two, ...) +local merged = table.imerged (one, two, ...) +\stoptyping + +\ShowLuaExampleTwo {table} {merge} {{ a = 1, b = 2, c = 3 }, { d = 1 }, { a = 0 }} +\ShowLuaExampleTwo {table} {imerge} {{ "a", "b", "c" }, { "d", "e" }, { "f", "g" }} + +% \ShowLuaExampleTwo {table} {merged} {{ a = 1, b = 2, c = 3 }, { d = 1 }, { a = 0 }} +% \ShowLuaExampleTwo {table} {imerged} {{ "a", "b", "c" }, { "d", "e" }, { "f", "g" }} + +\stopsummary + +\startsummary[title={copy fastcopy}] + +When copying a table we need to make a real and deep copy. The \type {copy} +function is an adapted version from the \LUA\ wiki. The \type {fastopy} is faster +because it does not check for circular references and does not share tables when +possible. In practice using the fast variant is okay. + +\starttyping +local copy = table.copy (t) +local copy = table.fastcopy(t) +\stoptyping + +\stopsummary + +\startsummary[title={flattened}] + +A nested table can be unnested using \type {flattened}. Normally you will only +use this function if the content is somewhat predictable. Often using one of the +merge functions does a similar job. + +\starttyping +local flattened = table.flatten(t) +\stoptyping + +\ShowLuaExampleTwo {table} {flattened} {{ a = 1, b = 2, { c = 3 }, d = 4}} +\ShowLuaExampleTwo {table} {flattened} {{ 1, 2, { 3, { 4 } }, 5}} +\ShowLuaExampleTwo {table} {flattened} {{ 1, 2, { 3, { 4 } }, 5}, 1} +\ShowLuaExampleTwo {table} {flattened} {{ a = 1, b = 2, { c = 3 }, d = 4}} +\ShowLuaExampleTwo {table} {flattened} {{ 1, 2, { 3, { c = 4 } }, 5}} +\ShowLuaExampleTwo {table} {flattened} {{ 1, 2, { 3, { c = 4 } }, 5}, 1} + +\stopsummary + +\startsummary[title={loweredkeys}] + +The name says it all: this function returns a new table with the keys being lower +case. This is handy in cases where the keys have a change to be inconsistent, as +can be the case when users input keys and values in less controlled ways. + +\starttyping +local normalized = table.loweredkeys { a = "a", A = "b", b = "c" } +\stoptyping + +\ShowLuaExampleTwo {table} {loweredkeys} {{ a = 1, b = 2, C = 3}} + +\stopsummary + +\startsummary[title={contains}] + +This function works with indexed tables. Watch out, when you look for a match, +the number \type {1} is not the same as string \type {"1"}. The function returns +the index or \type {false}. + +\starttyping +if table.contains(t, 5 ) then ... else ... end +if table.contains(t,"5") then ... else ... end +\stoptyping + +\ShowLuaExampleThree {table} {contains} {{ "a", 2, true, "1"}, 1} +\ShowLuaExampleThree {table} {contains} {{ "a", 2, true, "1"}, "1"} + +\stopsummary + +\startsummary[title={unique}] + +When a table (can) contain duplicate entries you can get rid of them by using the +\type {unique} helper: + +\starttyping +local t = table.unique { 1, 2, 3, 4, 3, 2, 5, 6 } +\stoptyping + +\ShowLuaExampleTwo {table} {unique} { { "a", "b", "c", "a", "d" } } + +\stopsummary + +\startsummary[title={count}] + +The name speaks for itself: this function counts the number of entries in the +given table. For an indexed table \type {#t} is faster. + +\starttyping +local n = table.count(t) +\stoptyping + +\ShowLuaExampleThree {table} {count} {{ 1, 2, [4] = 4, a = "a" }} + +\stopsummary + +\startsummary[title={sequenced}] + +Normally, when you trace a table, printing the serialized version is quite +convenient. However, when it concerns a simple table, a more compact variant is: + +\starttyping +print(table.sequenced(t, separator)) +\stoptyping + +% beware: by default sequences has | as separator + +\ShowLuaExampleThree {table} {sequenced} {{ 1, 2, 3, 4}} +\ShowLuaExampleThree {table} {sequenced} {{ 1, 2, [4] = 4, a = "a" }, ", "} + +\stopsummary + +\stopsection + +\startsection[title=Math] + +In addition to the built-in math function we provide: \type {round}, \type {odd}, +\type {even}, \type {div}, \type {mod}, \type {sind}, \type {cosd} and +\type {tand}. + +At the \TEX\ end we have a helper \type {luaexpr} that you can use to do +calculations: + +\startbuffer + \luaexpr{1 + 2.3 * 4.5 + math.pi} = \cldcontext{1 + 2.3 * 4.5 + math.pi} +\stopbuffer + +\typebuffer + +Both calls return the same result, but the first one is normally faster than the +\type {context} command which has quite some overhead. + +\blank \getbuffer \blank + +The \type {\luaexpr} command can also better deal with for instance conditions, +where it returns \type {true} or \type {false}, while \type {\cldcontext} would +interpret the boolean value as a special signal. + +\stopsection + +\startsection[title=Booleans] + +\startsummary[title={tonumber}] + +This function returns the number one or zero. You will seldom need this function. + +\starttyping +local state = boolean.tonumber(str) +\stoptyping + +\ShowLuaExampleThree {boolean} {tonumber} {true} + +\stopsummary + +\startsummary[title={toboolean}] + +When dealing with configuration files or tables a bit flexibility in setting a +state makes sense, if only because in some cases it's better to say \type {yes} +than \type {true}. + +\starttyping +local b = toboolean(str) +local b = toboolean(str,tolerant) +\stoptyping + +When the second argument is true, the strings \type {true}, \type {yes}, \type +{on}, \type {1}, \type {t} and the number \type {1} all turn into \type {true}. +Otherwise only \type {true} is honoured. This function is also defined in the +global namespace. + +\ShowLuaExampleThree {string} {toboolean} {"true"} +\ShowLuaExampleThree {string} {toboolean} {"yes"} +\ShowLuaExampleThree {string} {toboolean} {"yes",true} + +\stopsummary + +\startsummary[title={is_boolean}] + +This function is somewhat similar to the previous one. It interprets the strings +\type {true}, \type {yes}, \type {on} and \type {t} as \type {true} and +\type{false}, \type {no}, \type {off} and \type {f} as \type {false}. Otherwise +\type {nil} is returned, unless a default value is given, in which case that is +returned. + +\starttyping +if is_boolean(str) then ... end +if is_boolean(str,default) then ... end +\stoptyping + +\ShowLuaExampleThree {string} {is_boolean} {"true"} +\ShowLuaExampleThree {string} {is_boolean} {"off"} +\ShowLuaExampleThree {string} {is_boolean} {"crap",true} + +\stopsummary + +\stopsection + +\startsection[title=Strings] + +\LUA\ strings are simply sequences of bytes. Of course in some places special +treatment takes place. For instance \type {\n} expands to one or more characters +representing a newline, depending on the operating system, but normally, as long +as you manipulate strings in the perspective of \LUATEX, you don't need to worry +about such issues too much. As \LUATEX\ is a \UTF-8 engine, strings normally are +in that encoding but again, it does not matter much as \LUA\ is quite agnostic +about the content of strings: it does not care about three characters reflecting +one \UNICODE\ character or not. This means that when you use for instance the +functions discussed here, or use libraries like \type {lpeg} behave as you +expect. + +Versions later than 0.75 are likely to have some basic \UNICODE\ support on board +but we can easily adapt to that. At least till \LUATEX\ version 0.75 we provided +the \type {slunicode} library but users cannot assume that that will be present for +ever. If you want to mess around with \UTF\ string, use the \type {utf} library +instead as that is the one we provide in \MKIV. It presents the stable interface to +whatever \LUA\ itself provides and|/|or what \LUATEX\ offers and|/|or what +is there because \MKIV\ implements it. + +\startsummary[title={[lua] byte char}] + +As long as we're dealing with \ASCII\ characters we can use these two functions to +go from numbers to characters and vise versa. + +\ShowLuaExampleSeven {string} {byte} {"luatex"} +\ShowLuaExampleSeven {string} {byte} {"luatex",1,3} +\ShowLuaExampleSeven {string} {byte} {"luatex",-3,-1} + +\ShowLuaExampleSeven {string} {char} {65} +\ShowLuaExampleSeven {string} {char} {65,66,67} + +\stopsummary + +\startsummary[title={[lua] sub}] + +You cannot directly access a character in a string but you can take any slice you +want using \type {sub}. You need to provide a start position and negative values +will count backwards from the end. + +\starttyping +local slice = string.sub(str,first,last) +\stoptyping + +\ShowLuaExampleThree {string} {sub} {"abcdef",2} +\ShowLuaExampleThree {string} {sub} {"abcdef",2,3} +\ShowLuaExampleThree {string} {sub} {"abcdef",-3,-2} + +\stopsummary + +\startsummary[title={[lua] gsub}] + +There are two ways of analyzing the content of a string. The more modern and +flexible approach is to use \type {lpeg}. The other one uses some functions in +the \type {string} namespace that accept so called patterns for matching. While +\type {lpeg} is more powerfull than regular expressions, the pattern matching is +less powerfull but sometimes faster and also easier to specify. In many cases it +can do the job quite well. + +\starttyping +local new, count = string.gsub(old,pattern,replacement) +\stoptyping + +The replacement can be a function. Often you don't want the number +of matches, and the way to avoid this is either to store the result +in a variable: + +\starttyping +local new = string.gsub(old,"lua","LUA") +print(new) +\stoptyping + +or to use parentheses to signal the interpreter that only one value +is return. + +\starttyping +print((string.gsub(old,"lua","LUA")) +\stoptyping + +Patterns can be more complex so you'd better read the \LUA\ manual if you want to +know more about them. + +\ShowLuaExampleThree {string} {gsub} {"abcdef","b","B"} +\ShowLuaExampleThree {string} {gsub} {"abcdef","[bc]",string.upper} + +An optional fourth argument specifies how often the replacement has to happen + +\ShowLuaExampleThree {string} {gsub} {"textextextex","tex","abc"} +\ShowLuaExampleThree {string} {gsub} {"textextextex","tex","abc",1} +\ShowLuaExampleThree {string} {gsub} {"textextextex","tex","abc",2} + +\stopsummary + +\startsummary[title={[lua] find}] + +The \type {find} function returns the first and last position of the match: + +\starttyping +local first, last = find(str,pattern) +\stoptyping + +If you're only interested if there is a match at all, it's enough to know that +there is a first position. No match returns \type {nil}. So, + +\starttyping +if find("luatex","tex") then ... end +\stoptyping + +works out okay. You can pass an extra argument to \type {find} that indicates the +start position. So you can use this function to loop over all matches: just start +again at the end of the last match. + +A fourth optional argument is a boolean that signals not to interpret the pattern +but use it as|-|is. + +\ShowLuaExampleThree {string} {find} {"abc.def","c\letterpercent.d",1,false} +\ShowLuaExampleThree {string} {find} {"abc.def","c\letterpercent.d",1,true} +\ShowLuaExampleThree {string} {find} {"abc\letterpercent.def","c\letterpercent.d",1,false} +\ShowLuaExampleThree {string} {find} {"abc\letterpercent.def","c\letterpercent.d",1,true} + +\stopsummary + +\startsummary[title={[lua] match gmatch}] + +With \type {match} you can split of bits and pieces of a string. The parenthesis +indicate the captures. + +\starttyping +local a, b, c, ... = string.match(str,pattern) +\stoptyping + +The \type {gmatch} function is used to loop over a string, for instance the +following code prints the elements in a comma separated list, ignoring spaces +after commas. + +\starttyping +for s in string.gmatch(str,"([^,%s])+") do + print(s) +end +\stoptyping + +A more detailed description can be found in the \LUA\ reference manual, so we +only mention the special directives. Characters are grouped in classes: + +\starttabulate[|lT|l|] +\HL +\NC \letterpercent a \NC letters \NC \NR +\NC \letterpercent l \NC lowercase letters \NC \NR +\NC \letterpercent u \NC uppercase letters \NC \NR +\NC \letterpercent d \NC digits \NC \NR +\NC \letterpercent w \NC letters and digits \NC \NR +\NC \letterpercent c \NC control characters \NC \NR +\NC \letterpercent p \NC punctuation \NC \NR +\NC \letterpercent x \NC hexadecimal characters \NC \NR +\NC \letterpercent s \NC space related characters \NC \NR +\HL +\stoptabulate + +You can create sets too: + +\starttabulate[|lT|l|] +\HL +\NC [\letterpercent l\letterpercent d] \NC lowercase letters and digits \NC \NR +\NC [^\letterpercent d\letterpercent p] \NC all characters except digits and punctuation \NC \NR +\NC [p-z] \NC all characters in the range \type {p} upto \type {z} \NC \NR +\NC [pqr] \NC all characters \type {p}, \type {q} and \type {r} \NC \NR +\HL +\stoptabulate + +There are some characters with special meanings: + +\starttabulate[|lT|l|] +\HL +\NC \letterhat \NC the beginning of a string \NC \NR +\NC \letterdollar \NC end of a string \NC \NR +\NC . \NC any character \NC \NR +\NC * \NC zero or more of the preceding specifier, greedy \NC \NR +\NC - \NC zero or more of the preceding specifier, least possible \NC \NR +\NC + \NC one or more of the preceding specifier \NC \NR +\NC ? \NC zero or one of the preceding specifier \NC \NR +\NC ( ) \NC encapsulate capture \NC \NR +\NC \letterpercent b \NC capture all between the following two characters \NC \NR +\HL +\stoptabulate + +You can use whatever you like to be matched: + +\starttabulate[|lT|l|] +\HL +\NC pqr \NC the sequence \type {pqr} \NC \NR +\NC my name is (\letterpercent w) \NC the word following \type {my name is} \NC \NR +\HL +\stoptabulate + +If you want to specify such a token as it is, then you can precede it with a +percent sign, so to get a percent, you need two in a row. + +\ShowLuaExampleThree {string} {match} {"before:after","^(.-):"} +\ShowLuaExampleThree {string} {match} {"before:after","^([^:])"} +\ShowLuaExampleThree {string} {match} {"before:after","bef(.*)ter"} +\ShowLuaExampleThree {string} {match} {"abcdef","[b-e]+"} +\ShowLuaExampleThree {string} {match} {"abcdef","[b-e]*"} +\ShowLuaExampleThree {string} {match} {"abcdef","b-e+"} +\ShowLuaExampleThree {string} {match} {"abcdef","b-e*"} + +\stopsummary + +Such patterns should not be confused with regular expressions, although to some +extent they can do the same. If you really want to do complex matches, you should +look into \LPEG. + +\startsummary[title={[lua] lower upper}] + +These two function spreak for themselves. + +\ShowLuaExampleThree {string} {lower} {"LOW"} +\ShowLuaExampleThree {string} {upper} {"upper"} + +\stopsummary + +\startsummary[title={[lua] format}] + +The \type {format} function takes a template as first argument and one or more +additional arguments depending on the format. The template is similar to the one +used in \CCODE\ but it has some extensions. + +\starttyping +local s = format(format, str, ...) +\stoptyping + +The following table gives an overview of the possible format directives. The +\type {s} is the most probably candidate and can handle numbers well as strings. +Watch how the minus sign influences the alignment. \footnote {There can be +differences between platforms although so far we haven't run into problems. Also, +\LUA\ 5.2 does a bit more checking on correct arguments and \LUA\ 5.3 is more +picky on integers.} + +\starttabulate[|lB|lT|lT|lT|] +\HL +\NC integer \NC \letterpercent i \NC 12345 \NC \cldcontext{string.format("\letterpercent i", 12345 )} \NC \NR +\NC integer \NC \letterpercent d \NC 12345 \NC \cldcontext{string.format("\letterpercent d", 12345 )} \NC \NR +\NC unsigned \NC \letterpercent u \NC -12345 \NC \cldcontext{string.format("\letterpercent u", 12345 )} \NC \NR +\NC character \NC \letterpercent c \NC 123 \NC \cldcontext{string.format("\letterpercent c", 89 )} \NC \NR +\NC hexadecimal \NC \letterpercent x \NC 123 \NC \cldcontext{string.format("\letterpercent x", 123 )} \NC \NR +\NC \NC \letterpercent X \NC 123 \NC \cldcontext{string.format("\letterpercent X", 123 )} \NC \NR +\NC octal \NC \letterpercent o \NC 12345 \NC \cldcontext{string.format("\letterpercent o", 12345 )} \NC \NR +\HL +\NC string \NC \letterpercent s \NC abc \NC \cldcontext{string.format("\letterpercent s", "abcd")} \NC \NR +\NC \NC \letterpercent -8s \NC 123 \NC \cldcontext{string.format("\letterpercent -8s", 123 )} \NC \NR +\NC \NC \letterpercent 8s \NC 123 \NC \cldcontext{string.format("\letterpercent 8s", 123 )} \NC \NR +\HL +\NC float \NC \letterpercent 0.2f \NC 12.345 \NC \cldcontext{string.format("\letterpercent 0.2f",12.345)} \NC \NR +\NC exponential \NC \letterpercent 0.2e \NC 12.345 \NC \cldcontext{string.format("\letterpercent 0.2e",12.345)} \NC \NR +\NC \NC \letterpercent 0.2E \NC 12.345 \NC \cldcontext{string.format("\letterpercent 0.2E",12.345)} \NC \NR +\NC autofloat \NC \letterpercent 0.2g \NC 12.345 \NC \cldcontext{string.format("\letterpercent 0.2g",12.345)} \NC \NR +\NC \NC \letterpercent 0.2G \NC 12.345 \NC \cldcontext{string.format("\letterpercent 0.2G",12.345)} \NC \NR +\HL +\stoptabulate + +\startasciimode +\ShowLuaExampleThree {string} {format} {"U+\letterpercent 05X",2010} +\stopasciimode + +\stopsummary + +\startsummary[title={striplines}] + +The \type {striplines} function can strip leading and trailing empty lines, +collapse or delete intermediate empty lines and strips leading and trailing +spaces. We will demonstrate this with string \type {str}: + +\startluacode +local str = table.concat( { +" ", +" aap", +" noot mies", +" ", +" ", +" wim zus jet", +"teun vuur gijs", +" lam kees bok weide", +" ", +"does hok duif schapen ", +" ", +}, "\n") + +document.TempString = str + +function document.ShowStrippedString(str) + str = string.gsub(str," ","\\allowbreak<sp>\\allowbreak ") + str = string.gsub(str,"([\010])","\\allowbreak<lf>\\allowbreak ") + context.startalign { "flushleft,verytolerant" } + context("{\\tttf %s}",str) + context.stopalign() +end + +function document.ShowStrippedBuffer(name,str) + context.tobuffer(name,str) + context.typebuffer( { name }, { numbering = "line" }) + context.resetbuffer { name } +end + +function document.ShowStrippedCommand(option) + context.type( { style = "ttbf" }, [[utilities.strings.striplines(str,"]] .. option .. [[")]]) +end + +context.blank { "big" } +document.ShowStrippedString(str) +document.ShowStrippedBuffer("dummy",str) + +\stopluacode + +The different options for stripping are demonstrated below, We use verbose +descriptions instead of vague boolean flags. + +\startluacode +local str = document.TempString ; document.TempString = nil + +for option in table.sortedhash(utilities.strings.striplinepatterns) do + local s = utilities.strings.striplines(str,option) + context.blank() + document.ShowStrippedCommand(option) + context.blank { "big,samepage" } + document.ShowStrippedString(s) + context.blank { "big,samepage" } + document.ShowStrippedBuffer(option,str) +end +\stopluacode + +You can of course mix usage with the normal \type {context} helper commands, for +instance put them in buffers. Buffers normally will prune leading and trailing +empty lines anyway. + +\starttyping +context.tobuffer("dummy",utilities.strings.striplines(str)) +context.typebuffer( { "dummy" }, { numbering = "line" }) +\stoptyping + +\stopsummary + +\startsummary[title={formatters}] + +The \type {format} function discussed before is the built|-|in. As an alternative +\CONTEXT\ provides an additional formatter that has some extensions. Interesting +is that that one is often more efficient, although there are cases where the +speed is comparable. As we run out of keys, some extra ones are a bit counter +intuitive, like \type {l} for booleans (logical). + +\start \setuptype[color=] + +\starttabulate[|lB|lT|lT|lT|] +\HL +\NC utf character \NC \letterpercent c \NC 322 \NC \cldcontext{"\letterpercent c",322} \NC \NR +\HL +\NC string \NC \letterpercent s \NC foo \NC \cldcontext{"\letterpercent s","foo"} \NC \NR +\NC force tostring \NC \letterpercent S \NC nil \NC \cldcontext{"\letterpercent S",nil} \NC \NR +\NC quoted string \NC \letterpercent q \NC foo \NC \cldcontext{"\letterpercent q","foo"} \NC \NR +\NC force quoted string \NC \letterpercent Q \NC nil \NC \cldcontext{"\letterpercent Q",nil} \NC \NR +\NC \NC \letterpercent N \NC 0123 \NC \cldcontext{"\letterpercent N","0123"} \NC \NR +\NC automatic quoted \NC \letterpercent a \NC true \NC \cldcontext{"\letterpercent a",true} \NC \NR\NC \NR +\NC \NC \letterpercent A \NC true \NC \cldcontext{"\letterpercent A",true} \NC \NR\NC \NR +\NC left aligned utf \NC \letterpercent 30< \NC xx½xx \NC \cldcontext{"\letterpercent 30<","xx½xx"} \NC \NR\NC \NR +\NC right aligned utf \NC \letterpercent 30> \NC xx½xx \NC \cldcontext{"\letterpercent 30>","xx½xx"} \NC \NR\NC \NR +\HL +\NC integer \NC \letterpercent i \NC 1234 \NC \cldcontext{"\letterpercent i",1234} \NC \NR +\NC integer \NC \letterpercent d \NC 1234 \NC \cldcontext{"\letterpercent d",1234} \NC \NR +\NC signed number \NC \letterpercent I \NC 1234 \NC \cldcontext{"\letterpercent I",1234} \NC \NR +\NC rounded number \NC \letterpercent r \NC 1234.56 \NC \cldcontext{"\letterpercent r",1234.56} \NC \NR +\NC stripped number \NC \letterpercent N \NC 000123 \NC \cldcontext{"\letterpercent N","000123"} \NC \NR +\NC comma/period float \NC \letterpercent m \NC 12.34 \NC \cldcontext{"\letterpercent m",12.34} \NC \NR +\NC period/comma float \NC \letterpercent M \NC 12.34 \NC \cldcontext{"\letterpercent M",12.34} \NC \NR +\HL +\NC hexadecimal \NC \letterpercent x \NC 1234 \NC \cldcontext{"\letterpercent x",1234} \NC \NR +\NC \NC \letterpercent X \NC 1234 \NC \cldcontext{"\letterpercent X",1234} \NC \NR +\NC octal \NC \letterpercent o \NC 1234 \NC \cldcontext{"\letterpercent o",1234} \NC \NR +\HL +\NC float \NC \letterpercent 0.2f \NC 12.345 \NC \cldcontext{"\letterpercent 0.2f",12.345} \NC \NR +\NC formatted float \NC \letterpercent 2.3k \NC 12.3456 \NC \cldcontext{"\letterpercent 2.3f",12.3456} \NC \NR +\NC checked float \NC \letterpercent 0.2F \NC 12.30 \NC \cldcontext{"\letterpercent 0.2F",12.3} \NC \NR +\NC exponential \NC \letterpercent .2e \NC 12.345e120 \NC \cldcontext{"\letterpercent 0.2j",12.345e120} \NC \NR +\NC \NC \letterpercent .2E \NC 12.345e120 \NC \cldcontext{"\letterpercent 0.2J",12.345e120} \NC \NR +\NC sparse exp \NC \letterpercent 0.2j \NC 12.345e120 \NC \cldcontext{"\letterpercent 0.2j",12.345e120} \NC \NR +\NC \NC \letterpercent 0.2J \NC 12.345e120 \NC \cldcontext{"\letterpercent 0.2J",12.345e120} \NC \NR +\NC autofloat \NC \letterpercent g \NC 12.345 \NC \cldcontext{"\letterpercent 0.2J",12.345} \NC \NR +\NC \NC \letterpercent G \NC 12.345 \NC \cldcontext{"\letterpercent 0.2J",12.345} \NC \NR +\HL +\NC unicode value 0x \NC \letterpercent h \NC ł 1234 \NC \cldcontext{"\letterpercent v \letterpercent v", "ł",1234} \NC \NR +\NC \NC \letterpercent H \NC ł 1234 \NC \cldcontext{"\letterpercent V \letterpercent V", "ł",1234} \NC \NR +\NC unicode value U+ \NC \letterpercent u \NC ł 1234 \NC \cldcontext{"\letterpercent u \letterpercent u", "ł",1234} \NC \NR +\NC \NC \letterpercent U \NC ł 1234 \NC \cldcontext{"\letterpercent U \letterpercent U", "ł",1234} \NC \NR +\HL +\NC points \NC \letterpercent p \NC 1234567 \NC \cldcontext{"\letterpercent p",1234567} \NC \NR +\NC basepoints \NC \letterpercent b \NC 1234567 \NC \cldcontext{"\letterpercent b",1234567} \NC \NR +\HL +\NC table concat \NC \letterpercent t \NC \arg{1,2,3} \NC \cldcontext{"\letterpercent t",{1,2,3}} \NC \NR +\NC \NC \letterpercent *t \NC \arg{1,2,3} \NC \cldcontext{"\letterpercent *t",{1,2,3}} \NC \NR +\NC \NC \letterpercent \arg{ AND }t \NC \arg{a=1,b=3} \NC \cldcontext{"\letterpercent +{ AND }T",{a=1,b=2}} \NC \NR +\NC table serialize \NC \letterpercent T \NC \arg{1,2,3} \NC \cldcontext{"\letterpercent *t",{1,2,3}} \NC \NR +\NC \NC \letterpercent T \NC \arg{a=1,b=3} \NC \let|\relax\cldcontext{"\letterpercent T",{a=1,b=2}} \NC \NR +\NC \NC \letterpercent +T \NC \arg{a=1,b=3} \NC \cldcontext{"\letterpercent [+T]",{a=1,b=2}} \NC \NR +\HL +\NC boolean (logic) \NC \letterpercent l \NC "a" == "b" \NC \cldcontext{"\letterpercent l","a"=="b"} \NC \NR +\NC \NC \letterpercent L \NC "a" == "b" \NC \cldcontext{"\letterpercent L","a"=="b"} \NC \NR +\HL +\NC whitespace \NC \letterpercent w \NC 3 \NC \obeyspaces\vl\cldcontext{"\letterpercent w",3}\vl \NC \NR +\NC \NC \letterpercent 2w \NC 3 \NC \obeyspaces\vl\cldcontext{"\letterpercent 2w",3}\vl \NC \NR +\NC \NC \letterpercent 4W \NC \NC \obeyspaces\vl\cldcontext{"\letterpercent 4W"}\vl \NC \NR +\HL +\NC skip \NC \letterpercent 2z \NC 1,2,3,4 \NC \obeyspaces\vl\cldcontext{"\letterpercent s\letterpercent 2z\letterpercent s",1,2,3,4}\vl \NC \NR +\HL +\stoptabulate + +\stop + +The generic formatters \type {a} and \type {A} convert the argument into a string +and deals with strings, number, booleans, tables and whatever. We mostly use +these in tracing. The lowercase variant uses single quotes, and the uppercase +variant uses double quotes. + +A special one is the alignment formatter, which is a variant on the \type {s} one +that also takes an optional positive of negative number: + +\startbuffer +\startluacode +context.start() +context.tttf() +context.verbatim("[[% 30<]]","xxaxx") context.par() +context.verbatim("[[% 30<]]","xx½xx") context.par() +context.verbatim("[[% 30>]]","xxaxx") context.par() +context.verbatim("[[% 30>]]","xx½xx") context.par() +context.verbatim("[[%-30<]]","xxaxx") context.par() +context.verbatim("[[%-30<]]","xx½xx") context.par() +context.verbatim("[[%-30>]]","xxaxx") context.par() +context.verbatim("[[%-30>]]","xx½xx") context.par() +context.stop() +\stopluacode +\stopbuffer + +\typebuffer \getbuffer + +There are two more formatters plugged in: \type {!xml!} and \type {!tex!}. These +are best demonstrated with an example: + +\starttyping +local xf = formatter["xml escaped: %!xml!"] +local xr = formatter["tex escaped: %!tex!"] + +print(xf("x > 1 && x < 10")) +print(xt("this will cost me $123.00 at least")) +\stoptyping + +weird, this fails when cld-verbatim is there as part of the big thing: +catcodetable 4 suddenly lacks the comment being a other + +The \type {context} command uses the formatter so one can say: + +\startbuffer +\startluacode +context("first some xml: %!xml!, and now some %!tex!", + "x > 1 && x < 10", "this will cost me $123.00 at least") +\stopluacode +\stopbuffer + +\typebuffer + +This renders as follows: + +\blank \getbuffer \blank + +You can extend the formatter but we advise you not to do that unless you're sure +what you're doing. You never know what \CONTEXT\ itself might add for its own +benefit. + +However, you can define your own formatter and add to that without interference. +In fact, the main formatter is just defined that way. This is how it works: + +\startbuffer[definition] +local MyFormatter = utilities.strings.formatters.new() + +utilities.strings.formatters.add ( + MyFormatter, + "upper", + "global.string.upper(%s)" +) +\stopbuffer + +\typebuffer[definition] + +Now you can use this one as: + +\startbuffer[usage] +context.bold(MyFormatter["It's %s or %!upper!."]("this","that")) +\stopbuffer + +\typebuffer[usage] + +\blank \ctxluabuffer[definition,usage] \blank + +Because we're running inside \CONTEXT, a better definition would be this: + +\startbuffer +local MyFormatter = utilities.strings.formatters.new() + +utilities.strings.formatters.add ( + MyFormatter, + "uc", + "myupper(%s)", + -- "local myupper = global.characters.upper" + { myupper = global.characters.upper } +) + +utilities.strings.formatters.add ( + MyFormatter, + "lc", + "mylower(%s)", + -- "local mylower = global.characters.lower" + { mylower = global.characters.lower } +) + +utilities.strings.formatters.add ( + MyFormatter, + "sh", + "myshaped(%s)", + -- "local myshaped = global.characters.shaped" + { myshaped = global.characters.shaped } +) + +context(MyFormatter["Uppercased: %!uc!"]("ÀÁÂÃÄÅàáâãäå")) +context.par() +context(MyFormatter["Lowercased: %!lc!"]("ÀÁÂÃÄÅàáâãäå")) +context.par() +context(MyFormatter["Reduced: %!sh!"]("ÀÁÂÃÄÅàáâãäå")) +\stopbuffer + +\typebuffer + +The last arguments creates shortcuts. As expected we get: + +\blank \ctxluabuffer \blank + +Of course you can also apply the casing functions directly so in practice you +shouldn't use formatters without need. Among the advantages of using formatters +are: + +\startitemize[packed] +\startitem They provide a level of abstraction. \stopitem +\startitem They can replace multiple calls to \type {\context}. \stopitem +\startitem Sometimes they make source code look better. \stopitem +\startitem Using them is often more efficient and faster. \stopitem +\stopitemize + +The last argument might sound strange but considering the overhead involved in +the \type {context} (related) functions, doing more in one step has benefits. +Also, formatters are implemented quite efficiently, so their overhead can be +neglected. + +In the examples you see that a formatter extension is itself a template. + +\startbuffer +local FakeXML = utilities.strings.formatters.new() + +utilities.strings.formatters.add(FakeXML,"b",[["<" ..%s..">" ]]) +utilities.strings.formatters.add(FakeXML,"e",[["</"..%s..">" ]]) +utilities.strings.formatters.add(FakeXML,"n",[["<" ..%s.."/>"]]) + +context(FakeXML["It looks like %!b!xml%!e! doesn't it?"]("it","it")) +\stopbuffer + +\typebuffer + +This gives: \ctxluabuffer. Of course we could go over the top here: + +\startbuffer +local FakeXML = utilities.strings.formatters.new() + +local stack = { } + +function document.f_b(s) + table.insert(stack,s) + return "<" .. s .. ">" +end + +function document.f_e() + return "</" .. table.remove(stack) .. ">" +end + +utilities.strings.formatters.add(FakeXML,"b",[[global.document.f_b(%s)]]) +utilities.strings.formatters.add(FakeXML,"e",[[global.document.f_e()]]) + +context(FakeXML["It looks like %1!b!xml%0!e! doesn't it?"]("it")) +\stopbuffer + +\typebuffer + +This gives: \ctxluabuffer. Such a template look horrible, although it's not too +far from the regular format syntax: just compare \type {%1f} with \type {%1!e!}. +The zero trick permits us to inject information that we've put on the stack. As +this kind of duplicate usage might occur most, a better solution is available: + +\startbuffer +local FakeXML = utilities.strings.formatters.new() + +utilities.strings.formatters.add(FakeXML,"b",[["<" .. %s .. ">"]]) +utilities.strings.formatters.add(FakeXML,"e",[["</" .. %s .. ">"]]) + +context(FakeXML["It looks like %!b!xml%-1!e! doesn't it?"]("it")) +\stopbuffer + +\typebuffer + +We get: \ctxluabuffer. Anyhow, in most cases you will never feel the need for +such hackery and the regular formatter works fine. Adding this extension +mechanism was rather trivial and it doesn't influence the performance. + +In \CONTEXT\ we have a few more extensions: + +\starttyping +utilities.strings.formatters.add ( + strings.formatters, "unichr", + [["U+" .. format("%%05X",%s) .. " (" .. utfchar(%s) .. ")"]] +) + +utilities.strings.formatters.add ( + strings.formatters, "chruni", + [[utfchar(%s) .. " (U+" .. format("%%05X",%s) .. ")"]] +) +\stoptyping + +This one is used in messages: + +\startbuffer +context("Missing character %!chruni! in font.",234) context.par() +context("Missing character %!unichr! in font.",234) +\stopbuffer + +\typebuffer + +This shows up as: + +\blank \getbuffer \blank + +If you look closely to the definition, you will notice that we use \type {%s} +twice. This is a feature of the definer function: if only one argument is +picked up (which is default) then the replacement format can use that two +times. Because we use a format in the constructor, we need to escape the +percent sign there. + +\stopsummary + +\startsummary[title={strip}] + +This function removes any leading and trailing whitespace characters. + +\starttyping +local s = string.strip(str) +\stoptyping + +\ShowLuaExampleThree {string} {strip} {" lua + tex = luatex "} + +\stopsummary + +\startsummary[title={split splitlines checkedsplit}] + +The line splitter is a special case of the generic splitter. The \type {split} +function can get a string as well an \type {lpeg} pattern. The \type +{checkedsplit} function removes empty substrings. + +\starttyping +local t = string.split (str, pattern) +local t = string.split (str, lpeg) +local t = string.checkedsplit (str, lpeg) +local t = string.splitlines (str) +\stoptyping + +\start \let\ntex\relax % hack + +\ShowLuaExampleTwo {string} {split} {"a, b,c, d", ","} +\ShowLuaExampleTwo {string} {split} {"p.q,r", lpeg.S(",.")} +\ShowLuaExampleTwo {string} {checkedsplit} {";one;;two", ";"} +\ShowLuaExampleTwo {string} {splitlines} {"lua\ntex nic"} + +\stop + +\stopsummary + +\startsummary[title={quoted unquoted}] + +You will hardly need these functions. The \type {quoted} function can normally be +avoided using the \type {format} pattern \type {%q}. The \type {unquoted} +function removes single or double quotes but only when the string starts and ends +with the same quote. + +\starttyping +local q = string.quoted (str) +local u = string.unquoted(str) +\stoptyping + +\ShowLuaExampleThree {string} {quoted} {[[test]]} +\ShowLuaExampleThree {string} {quoted} {[[test"test]]} +\ShowLuaExampleThree {string} {unquoted} {[["test]]} +\ShowLuaExampleThree {string} {unquoted} {[["t\"est"]]} +\ShowLuaExampleThree {string} {unquoted} {[["t\"est"x]]} +\ShowLuaExampleThree {string} {unquoted} {"\'test\'"} + +\stopsummary + +\startsummary[title={count}] + +The function \type {count} returns the number of times that a given pattern +occurs. Beware: if you want to deal with \UTF\ strings, you need the variant that +sits in the \type {lpeg} namespace. + +\starttyping +local n = count(str,pattern) +\stoptyping + +\ShowLuaExampleThree {string} {count} {"test me", "e"} + +\stopsummary + +\startsummary[title={limit}] + +This function can be handy when you need to print messages that can be rather +long. By default, three periods are appended when the string is chopped. + +\starttyping +print(limit(str,max,sentinel) +\stoptyping + +\ShowLuaExampleThree {string} {limit} {"too long", 6} +\ShowLuaExampleThree {string} {limit} {"too long", 6, " (etc)"} + +\stopsummary + +\startsummary[title={is_empty}] + +A string considered empty by this function when its length is zero or when it +only contains spaces. + +\starttyping +if is_empty(str) then ... end +\stoptyping + +\ShowLuaExampleThree {string} {is_empty} {""} +\ShowLuaExampleThree {string} {is_empty} {" "} +\ShowLuaExampleThree {string} {is_empty} {" ? "} + +\stopsummary + +\startsummary[title={escapedpattern topattern}] + +These two functions are rather specialized. They come in handy when you need to +escape a pattern, i.e.\ prefix characters with a special meaning by a \type {%}. + +\starttyping +local e = escapedpattern(str, simple) +local p = topattern (str, lowercase, strict) +\stoptyping + +The simple variant does less escaping (only \type {-.?*} and is for instance used +in wildcard patterns when globbing directories. The \type {topattern} function +always does the simple escape. A strict pattern gets anchored to the beginning +and end. If you want to see what these functions do you can best look at their +implementation. + +\stopsummary + +% strings.tabtospace(str,n) +% strings.striplong + +\stopsection + +\startsection[title=\UTF] + +We used to have the \type {slunicode} library available but as most of it is not +used and because it has a somewhat fuzzy state, we will no longer rely on it. In +fact we only used a few functions in the \type {utf} namespace so as \CONTEXT\ +user you'd better stick to what is presented here. You don't have to worry how +they are implemented. Depending on the version of \LUATEX\ it can be that a +library, a native function, or \LPEG is used. + +\startsummary[title={char byte}] + +As \UTF\ is a multibyte encoding the term char in fact refers to a \LUA\ +string of one upto four 8|-|bit characters. + +\starttyping +local b = utf.byte("å") +local c = utf.char(0xE5) +\stoptyping + +The number of places in \CONTEXT\ where do such conversion is not that large: +it happens mostly in tracing messages. + +\starttyping +logs.report("panic","the character U+%05X is used",utf.byte("æ")) +\stoptyping + +\ShowLuaExampleThree {utf} {byte} {"æ"} +\ShowLuaExampleThree {utf} {char} {0xE6} + +\stopsummary + +\startsummary[title={sub}] + +If you need to take a slice of an \UTF\ encoded string the \type {sub} function +can come in handy. This function takes a string and a range defined by two +numbers. Negative numbers count from the end of the string. + +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",1,7} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",0,7} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",0,9} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",4} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",0} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",0,0} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",4,4} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",4,0} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",-3,0} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",0,-3} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",-5,-3} +\ShowLuaExampleThree {utf} {sub} {"123456àáâãäå",-3} + +\stopsummary + +\startsummary[title={len}] + +There are probably not that many people that can instantly see how many bytes the +string in the following example takes: + +\starttyping +local l = utf.len("ÀÁÂÃÄÅàáâãäå") +\stoptyping + +Programming languages use \ASCII\ mostly so there each characters takes one byte. +In \CJK\ scripts however, you end up with much longer sequences. If you ever did +some typesetting of such scripts you have noticed that the number of characters +on a page is less than in the case of a Latin script. As information is coded +in less characters, effectively the source of a Latin or \CJK\ document will not +differ that much. + +\ShowLuaExampleThree {utf} {len} {"ÒÓÔÕÖòóôõö"} + +\stopsummary + +\startsummary[title={values characters}] + +There are two iterators that deal with \UTF. In \LUATEX\ these are extensions to +the \type {string} library but for consistency we've move them to the \type {utf} +namespace. + +The following function loops over the \UTF\ characters in a string and returns +the \UNICODE\ number in \type {u}: + +\starttyping +for u in utf.values(str) do + ... -- u is a number +end +\stoptyping + +The next one returns a string \type {c} that has one or more characters as \UTF\ +characters can have upto 4 bytes. + +\starttyping +for c in utf.characters(str) do + ... -- c is a string +end +\stoptyping + +\stopsummary + +\startsummary[title={ustring xstring tocodes}] + +These functions are mostly useful for logging where we want to see the \UNICODE\ +number. + +\ShowLuaExampleThree {utf} {ustring} {0xE6} +\ShowLuaExampleThree {utf} {ustring} {"ù"} +\ShowLuaExampleThree {utf} {xstring} {0xE6} +\ShowLuaExampleThree {utf} {xstring} {"à"} +\ShowLuaExampleThree {utf} {tocodes} {"ùúü"} +\ShowLuaExampleThree {utf} {tocodes} {"àáä",""} +\ShowLuaExampleThree {utf} {tocodes} {"òóö","+"} + +\stopsummary + +\startsummary[title={split splitlines totable}] + +The \type {split} function splits a sequence of \UTF\ characters into a table +which one character per slot. The \type {splitlines} does the same but each slot +has a line instead. The \type {totable} function is similar to \type {split}, but +the later strips an optionally present \UTF\ bom. + +\ShowLuaExampleThree {utf} {split} {"òóö"} + +\stopsummary + +\startsummary[title={count}] + +This function counts the number of times that a given substring occurs in a +string. The patterns can be a string or an \LPEG\ pattern. + +\ShowLuaExampleThree {utf} {count} {"òóöòóöòóö","ö"} +\ShowLuaExampleThree {utf} {count} {"äáàa",lpeg.P("á") + lpeg.P("à")} + +\stopsummary + +\startsummary[title={remapper replacer substituter}] + +With \type {remapper} you can create a remapping function that remaps a given +string using a (hash) table. + +\starttyping +local remap = utf.remapper { a = 'd', b = "c", c = "b", d = "a" } + +print(remap("abcd 1234 abcd")) +\stoptyping + +A remapper checks each character against the given mapping table. Its cousin +\type {replacer} is more efficient and skips non matches. The \type {substituter} +function only does a quick check first and avoids building a string with no +replacements. That one is much faster when you expect not that many replacements. + +The \type {replacer} and \type {substituter} functions take table as argument +and an indexed as well as hashed one are acceptable. In fact you can even do +things like this: + +\starttyping +local rep = utf.replacer { [lpeg.patterns.digit] = "!" } +\stoptyping + +\stopsummary + +\startsummary[title={is_valid}] + +This function returns false if the argument is no valid \UTF\ string. As \LUATEX\ +is pretty strict with respect to the input, this function is only useful when +dealing with external files. + +\starttyping +function checkfile(filename) + local data = io.loaddata(filename) + if data and data ~= "" and not utf.is_valid(data) then + logs.report("error","file %q contains invalid utf",filename) + end +end +\stoptyping + +\stopsummary + +% not that relevant: +% +% -- utf.filetype +% -- string.toutf + +\stopsection + +\startsection[title=Numbers and bits] + +In the \type {number} namespace we collect some helpers that deal with numbers as +well as bits. Starting with \LUA\ 5.2 a library \type {bit32} is but the language +itself doesn't provide for them via operators: the library uses functions to +manipulate numbers upto 2\high{32}. In the latest \LUATEX\ you can use the new +bit related operators. + +% For advanced bit manipulations you should use the \type {bit32} library, otherwise +% it's best to stick to the functions described here. +% +% \startsummary[title={hasbit setbit clearbit}] +% +% As bitsets are numbers you will also use numbers to qualify them. So, if you want to +% set bits 1, 4 and 8, you can to that using the following specification: +% +% \starttyping +% local b = 1 + 4 + 8 -- 0x1 + 0x4 + 0x8 +% local b = 13 -- or 0xC +% \stoptyping +% +% However, changing one bit by adding a number to an existing doesn't work out that well +% if that number already has that bit set. Instead we use: +% +% \starttyping +% local b = number.setbit(b,0x4) +% \stoptyping +% +% In a similar fashion you can turn of a bit: +% +% \starttyping +% local b = number.clearbit(b,0x4) +% \stoptyping +% +% Testing for a bit(set) is done as follows: +% +% \starttyping +% local okay = number.hasbit(b,0x4) +% \stoptyping +% +% \stopsummary +% +% \startsummary[title={bit}] +% +% Where the previously mentioned helpers work with numbers representing one or more +% bits, it is sometimes handy to work with positions. The \type {bit} function +% returns the associated number value. +% +% \ShowLuaExampleThree {number} {bit} {5} +% +% \stopsummary + +\startsummary[title={tobitstring}] + +There is no format option to go from number to bits in terms of zeros and ones so +we provide a helper: \type {tobitsting}. + +\ShowLuaExampleThree {number} {tobitstring} {2013} +\ShowLuaExampleThree {number} {tobitstring} {2013,3} +\ShowLuaExampleThree {number} {tobitstring} {2013,1} + +\stopsummary + +% \startsummary[title={bits}] +% +% If you ever want to convert a bitset into a table containing the set bits you can +% use this function. +% +% \ShowLuaExampleTwo {number} {bits} {11} +% +% \stopsummary +% +% \startsummary[title={toset}] +% +% A string or number can be split into digits with \type {toset}. Beware, this +% function does not return a function but multiple numbers +% +% \starttyping +% local a, b, c, d = number.toset("1001") +% \stoptyping +% +% The returned values are either numbers or \type {nil} when an valid digit is +% seen. +% +% \ShowLuaExampleSeven {number} {toset} {100101} +% \ShowLuaExampleSeven {number} {toset} {"100101"} +% \ShowLuaExampleSeven {number} {toset} {"21546"} +% +% \stopsummary + +\startsummary[title={valid}] + +This function can be used to check or convert a number, for instance in user +interfaces. + +\ShowLuaExampleThree {number} {valid} {12} +\ShowLuaExampleThree {number} {valid} {"34"} +\ShowLuaExampleThree {number} {valid} {"ab",56} + +\stopsummary + +\stopsection + +\startsection[title=\LPEG\ patterns] + +For \LUATEX\ and \CONTEXT\ \MKIV\ the \type {lpeg} library came at the right +moment as we can use it in lots of places. An in|-|depth discussion makes no +sense as it's easier to look into \type {l-lpeg.lua}, so we stick to an overview. +\footnote {If you search the web for \type {lua lpeg} you will end up at the +official documentation and tutorial.} Most functions return an \type {lpeg} +object that can be used in a match. In time critical situations it's more +efficient to use the match on a predefined pattern that to create the pattern new +each time. Patterns are cached so there is no penalty in predefining a pattern. +So, in the following example, the \type {splitter} that splits at the asterisk +will only be created once. + +\starttyping +local splitter_1 = lpeg.splitat("*") +local splitter_2 = lpeg.splitat("*") + +local n, m = lpeg.match(splitter_1,"2*4") +local n, m = lpeg.match(splitter_2,"2*4") +\stoptyping + +\startsummary[title={[lua] match print P R S V C Cc Cs ...}] + +The \type {match} function does the real work. Its first argument is a \type +{lpeg} object that is created using the functions with the short uppercase names. + +\starttyping +local P, R, C, Ct = lpeg.P, lpeg.R, lpeg.C, lpeg.Ct + +local pattern = Ct((P("[") * C(R("az")^0) * P(']') + P(1))^0) + +local words = lpeg.match(pattern,"a [first] and [second] word") +\stoptyping + +In this example the words between square brackets are collected in a table. There +are lots of examples of \type {lpeg} in the \CONTEXT\ code base. + +\stopsummary + +\startsummary[title={anywhere}] + +\starttyping +local p = anywhere(pattern) +\stoptyping + +\ShowLuaExampleTwo {lpeg} {match} {lpeg.Ct((lpeg.anywhere("->")/"!")^0), "oeps->what->more"} + +\stopsummary + +\startsummary[title={splitter splitat firstofsplit secondofsplit}] + +The \type {splitter} function returns a pattern where each match gets an action +applied. The action can be a function, table or string. + +\starttyping +local p = splitter(pattern, action) +\stoptyping + +The \type {splitat} function returns a pattern that will return the split off +parts. Unless the second argument is \type {true} the splitter keeps splitting + +\starttyping +local p = splitat(separator,single) +\stoptyping + +When you need to split off a prefix (for instance in a label) you can use: + +\starttyping +local p = firstofsplit(separator) +local p = secondofsplit(separator) +\stoptyping + +The first function returns the original when there is no match but the second +function returns \type {nil} instead. + +\ShowLuaExampleTwo {lpeg} {match} {lpeg.Ct(lpeg.splitat("->",false)), "oeps->what->more"} +\ShowLuaExampleTwo {lpeg} {match} {lpeg.Ct(lpeg.splitat("->",false)), "oeps"} +\ShowLuaExampleTwo {lpeg} {match} {lpeg.Ct(lpeg.splitat("->",true)), "oeps->what->more"} +\ShowLuaExampleTwo {lpeg} {match} {lpeg.Ct(lpeg.splitat("->",true)), "oeps"} + +\ShowLuaExampleThree {lpeg} {match} {lpeg.firstofsplit(":"), "before:after"} +\ShowLuaExampleThree {lpeg} {match} {lpeg.firstofsplit(":"), "whatever"} +\ShowLuaExampleThree {lpeg} {match} {lpeg.secondofsplit(":"), "before:after"} +\ShowLuaExampleThree {lpeg} {match} {lpeg.secondofsplit(":"), "whatever"} + +\stopsummary + +\startsummary[title={split checkedsplit}] + +The next two functions have counterparts in the \type {string} namespace. They +return a table with the split parts. The second function omits empty parts. + +\starttyping +local t = split (separator,str) +local t = checkedsplit(separator,str) +\stoptyping + +\ShowLuaExampleTwo {lpeg} {split} {",","a,b,c"} +\ShowLuaExampleTwo {lpeg} {split} {",",",a,,b,c,"} +\ShowLuaExampleTwo {lpeg} {checkedsplit} {",",",a,,b,c,"} + +\stopsummary + +\startsummary[title={stripper keeper replacer}] + +These three functions return patterns that manipulate a string. The \type +{replacer} gets a mapping table passed. + +\starttyping +local p = stripper(str or pattern) +local p = keeper (str or pattern) +local p = replacer(mapping) +\stoptyping + +\ShowLuaExampleThree {lpeg} {match} {lpeg.stripper(lpeg.R("az")), "[-a-b-c-d-]"} +\ShowLuaExampleThree {lpeg} {match} {lpeg.stripper("ab"), "[-a-b-c-d-]"} +\ShowLuaExampleThree {lpeg} {match} {lpeg.keeper(lpeg.R("az")), "[-a-b-c-d-]"} +\ShowLuaExampleThree {lpeg} {match} {lpeg.keeper("ab"), "[-a-b-c-d-]"} +\ShowLuaExampleThree {lpeg} {match} {lpeg.replacer{{"a","p"},{"b","q"}}, "[-a-b-c-d-]"} + +\stopsummary + +\startsummary[title={balancer}] + +One of the nice things about \type {lpeg} is that it can handle all kind of +balanced input. So, a function is provided that returns a balancer pattern: + +\starttyping +local p = balancer(left,right) +\stoptyping + +\ShowLuaExampleTwo {lpeg} {match} {lpeg.Ct((lpeg.C(lpeg.balancer("{","}"))+1)^0),"{a} {b{c}}"} +\ShowLuaExampleTwo {lpeg} {match} {lpeg.Ct((lpeg.C(lpeg.balancer("((","]"))+1)^0),"((a] ((b((c]]"} + +\stopsummary + +\startsummary[title={counter}] + +The \type {counter} function returns a function that returns the length of a +given string. The \type {count} function differs from its counterpart living in +the \type {string} namespace in that it deals with \UTF\ and accepts strings as +well as patterns. + +\starttyping +local fnc = counter(lpeg.P("á") + lpeg.P("à")) +local len = fnc("äáàa") +\stoptyping + +\stopsummary + +\startsummary[title={UP US UR}] + +In order to make working with \UTF-8 input somewhat more convenient a few helpers +are provided. + +\starttyping +local p = lpeg.UP(utfstring) +local p = lpeg.US(utfstring) +local p = lpeg.UR(utfpair) +local p = lpeg.UR(first,last) +\stoptyping + +\ShowLuaExampleThree {utf} {count} {"äáàa",lpeg.UP("áà")} +\ShowLuaExampleThree {utf} {count} {"äáàa",lpeg.US("àá")} +\ShowLuaExampleThree {utf} {count} {"äáàa",lpeg.UR("aá")} +\ShowLuaExampleThree {utf} {count} {"äáàa",lpeg.UR("àá")} +\ShowLuaExampleThree {utf} {count} {"äáàa",lpeg.UR(0x0000,0xFFFF)} + +\startsummary[title={patterns}] + +The following patterns are available in the \type {patterns} table in the \type +{lpeg} namespace: + +\startluacode +context.startalignment { "flushleft" } +local done = false +for k, v in table.sortedpairs(lpeg.patterns) do + if done then + context.space() + else + done = true + end + context.type(k) +end +context.stopalignment() +\stopluacode + +There will probably be more of them in the future. + +\stopsummary + +\stopsection + +\startsection[title=IO] + +The \type {io} library is extended with a couple of functions as well and +variables but first we mention a few predefined functions. + +\startsummary[title={[lua] open popen...}] + +The IO library deals with in- and output from the console and +files. + +\starttyping +local f = io.open(filename) +\stoptyping + +When the call succeeds \type {f} is a file object. You close this file +with: + +\starttyping +f:close() +\stoptyping + +Reading from a file is done with \type {f:read(...)} and writing to a file with +\type {f:write(...)}. In order to write to a file, when opening a second argument +has to be given, often \type {wb} for writing (binary) data. Although there are +more efficient ways, you can use the \type {f:lines()} iterator to process a file +line by line. + +You can open a process with \type {io.popen} but dealing with this one depends a +bit on the operating system. + +\stopsummary + +\startsummary[title={fileseparator pathseparator}] + +The value of the following two strings depends on the operating system that is +used. + +\starttyping +io.fileseparator +io.pathseparator +\stoptyping + +\ShowLuaExampleFive {io} {fileseparator} +\ShowLuaExampleFive {io} {pathseparator} + +\stopsummary + +\startsummary[title={loaddata savedata}] + +These two functions save you some programming. The first function loads a whole +file in a string. By default the file is loaded in binary mode, but when the +second argument is \type {true}, some interpretation takes place (for instance +line endings). In practice the second argument can best be left alone. + +\starttyping +io.loaddata(filename,textmode) +\stoptyping + +Saving the data is done with: + +\starttyping +io.savedata(filename,str) +io.savedata(filename,tab,joiner) +\stoptyping + +When a table is given, you can optionally specify a string that +ends up between the elements that make the table. + +\stopsummary + +\startsummary[title={exists size noflines}] + +These three function don't need much comment. + +\starttyping +io.exists(filename) +io.size(filename) +io.noflines(fileobject) +io.noflines(filename) +\stoptyping + +\stopsummary + +\startsummary[title={characters bytes readnumber readstring}] + +When I wrote the icc profile loader, I needed a few helpers for reading strings +of a certain length and numbers of a given width. Both accept five values of +\type {n}: \type {-4}, \type {-2}, \type {1}, \type {2} and \type {4} where the +negative values swap the characters or bytes. + +\starttyping +io.characters(f,n) -- +io.bytes(f,n) +\stoptyping + +The function \type {readnumber} accepts five sizes: \type {1}, \type {2}, \type +{4}, \type {8}, \type {12}. The string function handles any size and strings zero +bytes from the string. + +\starttyping +io.readnumber(f,size) +io.readstring(f,size) +\stoptyping + +Optionally you can give the position where the reading has to start: + +\starttyping +io.readnumber(f,position,size) +io.readstring(f,position,size) +\stoptyping + +\stopsummary + +\startsummary[title={ask}] + +In practice you will probably make your own variant of the following function, +but at least a template is there: + +\starttyping +io.ask(question,default,options) +\stoptyping + +For example: + +\starttyping +local answer = io.ask("choice", "two", { "one", "two" }) +\stoptyping + +\stopsummary + +\stopsection + +\startsection[title=File] + +The file library is one of the larger core libraries that comes with +\CONTEXT. + +\startsummary[title={dirname basename extname nameonly}] + +We start with a few filename manipulators. + +\starttyping +local path = file.dirname(name,default) +local base = file.basename(name) +local suffix = file.extname(name,default) -- or file.suffix +local name = file.nameonly(name) +\stoptyping + +\ShowLuaExampleThree {file} {dirname} {"/data/temp/whatever.cld"} +\ShowLuaExampleThree {file} {dirname} {"c:/data/temp/whatever.cld"} +\ShowLuaExampleThree {file} {basename} {"/data/temp/whatever.cld"} +\ShowLuaExampleThree {file} {extname} {"c:/data/temp/whatever.cld"} +\ShowLuaExampleThree {file} {nameonly} {"/data/temp/whatever.cld"} + +\stopsummary + +\startsummary[title={addsuffix replacesuffix}] + +These functions are used quite often: + +\starttyping +local filename = file.addsuffix(filename, suffix, criterium) +local filename = file.replacesuffix(filename, suffix) +\stoptyping + +The first one adds a suffix unless one is present. When \type {criterium} is +\type {true} no checking is done and the suffix is always appended. The second +function replaces the current suffix or add one when there is none. + +\ShowLuaExampleThree {file} {addsuffix} {"whatever","cld"} +\ShowLuaExampleThree {file} {addsuffix} {"whatever.tex","cld"} +\ShowLuaExampleThree {file} {addsuffix} {"whatever.tex","cld",true} + +\ShowLuaExampleThree {file} {replacesuffix} {"whatever","cld"} +\ShowLuaExampleThree {file} {replacesuffix} {"whatever.tex","cld"} + +\stopsummary + +\startsummary[title={is_writable is_readable}] + +These two test the nature of a file: + +\starttyping +file.is_writable(name) +file.is_readable(name) +\stoptyping + +\stopsummary + +\startsummary[title={splitname join collapsepath}] + +Instead of splitting off individual components you can get them all in one go: + +\starttyping +local drive, path, base, suffix = file.splitname(name) +\stoptyping + +The \type {drive} variable is empty on operating systems other than \MSWINDOWS. +Such components are joined with the function: + +\starttyping +file.join(...) +\stoptyping + +The given snippets are joined using the \type {/} as this is +rather platform independent. Some checking takes place in order +to make sure that nu funny paths result from this. There is +also \type {collapsepath} that does some cleanup on a path +with relative components, like \type {..}. + +\ShowLuaExampleSix {file} {splitname} {"a:/b/c/d.e"} +\ShowLuaExampleThree {file} {join} {"a","b","c.d"} +\ShowLuaExampleThree {file} {collapsepath} {"a/b/../c.d"} +\ShowLuaExampleThree {file} {collapsepath} {"a/b/../c.d",true} + +\stopsummary + +\startsummary[title={splitpath joinpath}] + +By default splitting a execution path specification is done using the operating +system dependant separator, but you can force one as well: + +\starttyping +file.splitpath(str,separator) +\stoptyping + +The reverse operation is done with: + +\starttyping +file.joinpath(tab,separator) +\stoptyping + +Beware: in the following examples the separator is system dependent so +the outcome depends on the platform you run on. + +\ShowLuaExampleTwo {file} {splitpath} {"a:b:c"} +\ShowLuaExampleTwo {file} {splitpath} {"a;b;c"} +\ShowLuaExampleThree {file} {joinpath} {{"a","b","c"}} + +\stopsummary + +\startsummary[title={robustname}] + +In workflows filenames with special characters can be a pain so the following +function replaces characters other than letters, digits, periods, slashes and +hyphens by hyphens. + +\starttyping +file.robustname(str,strict) +\stoptyping + +\ShowLuaExampleThree {file} {robustname} {"We don't like this!"} +\ShowLuaExampleThree {file} {robustname} {"We don't like this!",true} + +\stopsummary + +\startsummary[title={readdata writedata}] + +These two functions are duplicates of functions with the +same name in the \type {io} library. + +\stopsummary + +\startsummary[title={copy}] + +There is not much to comment on this one: + +\starttyping +file.copy(oldname,newname) +\stoptyping + +\stopsummary + +\startsummary[title={is_qualified_path is_rootbased_path}] + +A qualified path has at least one directory component while a rootbased path is +anchored to the root of a filesystem or drive. + +\starttyping +file.is_qualified_path(filename) +file.is_rootbased_path(filename) +\stoptyping + +\ShowLuaExampleThree {file} {is_qualified_path} {"a"} +\ShowLuaExampleThree {file} {is_qualified_path} {"a/b"} +\ShowLuaExampleThree {file} {is_rootbased_path} {"a/b"} +\ShowLuaExampleThree {file} {is_rootbased_path} {"/a/b"} + +\stopsummary + +\stopsection + +\startsection[title=Dir] + +The \type {dir} library uses functions of the \type {lfs} library that is linked +into \LUATEX. + +\startsummary[title={current}] + +This returns the current directory: + +\starttyping +dir.current() +\stoptyping + +\stopsummary + +\startsummary[title={glob globpattern globfiles}] + +% not yet documented: dir.collectpattern(path,patt,recurse,result) -- collects tree + +The \type {glob} function collects files with names that match a given pattern. +The pattern can have wildcards: \type {*} (oen of more characters), \type {?} +(one character) or \type {**} (one or more directories). You can pass the +function a string or a table with strings. Optionally a second argument can be +passed, a table that the results are appended to. + +\starttyping +local files = dir.glob(pattern,target) +local files = dir.glob({pattern,...},target) +\stoptyping + +The target is optional and often you end up with simple calls like: + +\starttyping +local files = dir.glob("*.tex") +\stoptyping + +There is a more extensive version where you start at a path, and applies an +action to each file that matches the pattern. You can either or not force +recursion. + +\starttyping +dir.globpattern(path,patt,recurse,action) +\stoptyping + +The \type {globfiles} function collects matches in a table that is returned at +the end. You can pass an existing table as last argument. The first argument is +the starting path, the second arguments controls analyzing directories and the +third argument has to be a function that gets a name passed and is supposed to +return \type {true} or \type {false}. This function determines what gets +collected. + +\starttyping +dir.globfiles(path,recurse,func,files) +\stoptyping + +\stopsummary + +\startsummary[title={makedirs}] + +With \type {makedirs} you can create the given directory. If more than one +name is given they are concatinated. + +\starttyping +dir.makedirs(name,...) +\stoptyping + +\stopsummary + +\startsummary[title={expandname}] + +This function tries to resolve the given path, including relative paths. + +\starttyping +dir.expandname(str) +\stoptyping + +\ShowLuaExampleThree {dir} {expandname} {"."} + +\stopsummary + +\stopsection + +\startsection[title=URL] + +\startsummary[title={split hashed construct}] + +This is a specialized library. You can split an \type {url} into its components. +An \URL\ is constructed like this: + +\starttyping +foo://example.com:2010/alpha/beta?gamma=delta#epsilon +\stoptyping + +\starttabulate[|T|T|] +\NC scheme \NC foo:// \NC \NR +\NC authority \NC example.com:2010 \NC \NR +\NC path \NC /alpha/beta \NC \NR +\NC query \NC gamma=delta \NC \NR +\NC fragment \NC epsilon \NC \NR +\stoptabulate + +A string is split into a hash table with these keys using the following function: + +\starttyping +url.hashed(str) +\stoptyping + +or in strings with: + +\starttyping +url.split(str) +\stoptyping + +The hash variant is more tolerant than the split. In the hash +there is also a key \type {original} that holds the original \URL\ +and and the boolean \type {noscheme} indicates if there is a +scheme at all. + +The reverse operation is done with: + +\starttyping +url.construct(hash) +\stoptyping + +\startasciimode +\ShowLuaExampleTwo {url} {hashed} {"foo://example.com:2010/alpha/beta?gamma=delta#epsilon"} +\ShowLuaExampleTwo {url} {hashed} {"alpha/beta"} +\ShowLuaExampleTwo {url} {split} {"foo://example.com:2010/alpha/beta?gamma=delta#epsilon"} +\ShowLuaExampleTwo {url} {split} {"alpha/beta"} +\stopasciimode + +\startsummary[title={hasscheme addscheme filename query}] + +There are a couple of helpers and their names speaks for themselves: + +\starttyping +url.hasscheme(str) +url.addscheme(str,scheme) +url.filename(filename) +url.query(str) +\stoptyping + +\ShowLuaExampleThree {url} {hasscheme} {"http://www.pragma-ade.com/cow.png"} +\ShowLuaExampleThree {url} {hasscheme} {"www.pragma-ade.com/cow.png"} +\ShowLuaExampleThree {url} {addscheme} {"www.pragma-ade.com/cow.png","http://"} +\ShowLuaExampleThree {url} {addscheme} {"www.pragma-ade.com/cow.png"} +\ShowLuaExampleThree {url} {filename} {"http://www.pragma-ade.com/cow.png"} +\ShowLuaExampleTwo {url} {query} {"a=b&c=d"} + +\stopsection + +\startsection[title=OS] + +\startsummary[title={[lua luatex] env setenv getenv}] + +In \CONTEXT\ normally you will use the resolver functions to deal with the +environment and files. However, a more low level interface is still available. +You can query and set environment variables with two functions. In addition there +is the \type {env} table as interface to the environment. This threesome replaces +the built in functions. + +\starttyping +os.setenv(key,value) +os.getenv(key) +os.env[key] +\stoptyping + +\stopsummary + +\startsummary[title={[lua] execute}] + +There are several functions for running programs. One comes directly from \LUA, +the otheres come with \LUATEX. All of them are are overloaded in \CONTEXT\ in +order to get more control. + +\starttyping +os.execute(...) +\stoptyping + +\stopsummary + +\startsummary[title={[luatex] spawn exec}] + +Two other runners are: + +\starttyping +os.spawn(...) +os.exec (...) +\stoptyping + +The \type {exec} variant will transfer control from the current process to the +new one and not return to the current job. There is a more detailed explanation +in the \LUATEX\ manual. + +\stopsummary + +\startsummary[title={resultof launch}] + +The following function runs the command and returns the result as string. +Multiple lines are combined. + +\starttyping +os.resultof(command) +\stoptyping + +The next one launches a file assuming that the operating system knows +what application to use. + +\starttyping +os.launch(str) +\stoptyping + +\stopsummary + +\startsummary[title={type name platform libsuffix binsuffix}] + +There are a couple of strings that reflect the current machinery: \type {type} +returns either \type {windows} or \type {unix}. The variable \type {name} is more +detailed: \type {windows}, \type {msdos}, \type {linux}, \type {macosx}, etc. If +you also want the architecture you can consult \type {platform}. + +\starttyping +local t = os.type +local n = os.name +local p = os.platform +\stoptyping + +These three variables as well as the next two are used internally and normally +they are not needed in your applications as most functions that matter are aware +of what platform specific things they have to deal with. + +\starttyping +local s = os.libsuffix +local b = os.binsuffix +\stoptyping + +These are string, not functions. + +\ShowLuaExampleFive {os} {type} +\ShowLuaExampleFive {os} {name} +\ShowLuaExampleFive {os} {platform} +\ShowLuaExampleFive {os} {libsuffix} +\ShowLuaExampleFive {os} {binsuffix} + +\stopsummary + +\startsummary[title={[lua] time}] + +The built in time function returns a number. The accuracy is +implementation dependent and not that large. + +\ShowLuaExampleThree {os} {time} {} + +\stopsummary + +\startsummary[title={[luatex] times gettimeofday}] + +Although \LUA\ has a built in type {os.time} function, we normally will use the +one provided by \LUATEX\ as it is more precise: + +\starttyping +os.gettimeofday() +\stoptyping + +There is also a more extensive variant: + +\starttyping +os.times() +\stoptyping + +This one is platform dependent and returns a table with \type {utime} (use time), +\type {stime} (system time), \type {cutime} (children user time), and \type +{cstime} (children system time). + +\stopsummary + +\ShowLuaExampleThree {os} {gettimeofday} {} +\ShowLuaExampleTwo {os} {times} {} + +\startsummary[title={runtime}] + +More interesting is: + +\starttyping +os.runtime() +\stoptyping + +which returns the time spent in the application so far. + +\ShowLuaExampleThree {os} {runtime} {} + +Sometimes you need to add the timezone to a verbose time and the following +function does that for you. + +\starttyping +os.timezone(delta) +\stoptyping + +\ShowLuaExampleThree {os} {timezone} {} +\ShowLuaExampleThree {os} {timezone} {1} +\ShowLuaExampleThree {os} {timezone} {-1} + +\stopsummary + +\startsummary[title={uuid}] + +A version 4 UUID can be generated with: + +\starttyping +os.uuid() +\stoptyping + +The generator is good enough for our purpose. + +\ShowLuaExampleThree {os} {uuid} {} + +\stopsummary + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-macros.tex b/doc/context/sources/general/manuals/cld/cld-macros.tex new file mode 100644 index 000000000..a177db9f8 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-macros.tex @@ -0,0 +1,224 @@ +% language=uk + +\startcomponent cld-macros + +\environment cld-environment + +\startchapter[title=Macros] + +\startsection[title={Introduction}] + +You can skip this chapter if you're not interested in defining macros or are +quite content with defining them in \TEX. It's just an example of possible future +interface definitions and it's not the fastest mechanism around. + +\stopsection + +\startsection[title={Parameters}] + +Right from the start \CONTEXT\ came with several user interfaces. As a +consequence you need to take this into account when you write code that is +supposed to work with interfaces other than the English one. The \TEX\ command: + +\starttyping +\setupsomething[key=value] +\stoptyping + +and the \LUA\ call: + +\starttyping +context.setupsomething { key = value } +\stoptyping + +are equivalent. However, all keys at the \TEX\ end eventually become English, but +the values are unchanged. This means that when you code in \LUA\ you should use +English keys and when dealing with assigned values later on, you need to +translate them of compare with translations (which is easier). This is why in the +\CONTEXT\ code you will see: + +\starttyping +if somevalue == interfaces.variables.yes then + ... +end +\stoptyping + +instead of: + +\starttyping +if somevalue == "yes" then + ... +end +\stoptyping + +\stopsection + +\startsection[title={User interfacing}] + +Unless this is somehow inhibited, users can write their own macros and this is +done in the \TEX\ language. Passing data to macros is possible and looks like +this: + +\starttyping +\def\test#1#2{.. #1 .. #2 .. } \test{a}{b} +\def\test[#1]#2{.. #1 .. #2 .. } \test[a]{b} +\stoptyping + +Here \type {#1} and \type {#2} represent an argument and there can be at most 9 +of them. The \type{[]} are delimiters and you can delimit in many ways so the +following is also right: + +\starttyping +\def\test(#1><#2){.. #1 .. #2 .. } \test(a><b) +\stoptyping + +Macro packages might provide helper macros that for instance take care of +optional arguments, so that we can use calls like: + +\starttyping +\test[1,2,3][a=1,b=2,c=3]{whatever} +\stoptyping + +and alike. If you are familiar with the \CONTEXT\ syntax you know that we use +this syntax all over the place. + +If you want to write a macro that calls out to \LUA\ and handles things at that +end, you might want to avoid defining the macro itself and this is possible. + +\startbuffer +\startluacode +function test(opt_1, opt_2, arg_1) + context.startnarrower() + context("options 1: %s",interfaces.tolist(opt_1)) + context.par() + context("options 2: %s",interfaces.tolist(opt_2)) + context.par() + context("argument 1: %s",arg_1) + context.stopnarrower() +end + +interfaces.definecommand { + name = "test", + arguments = { + { "option", "list" }, + { "option", "hash" }, + { "content", "string" }, + }, + macro = test, +} +\stopluacode + +test: \test[1][a=3]{whatever} +\stopbuffer + +An example of a definition and usage at the \LUA\ end is: + +\typebuffer + +The call gives: + +\startpacked +\getbuffer +\stoppacked + +\startbuffer +\startluacode +local function startmore(opt_1) + context.startnarrower() + context("start more, options: %s",interfaces.tolist(opt_1)) + context.startnarrower() +end + +local function stopmore(opt_1) + context.stopnarrower() + context("stop more, options: %s",interfaces.tolist(opt_1)) + context.stopnarrower() +end + +interfaces.definecommand ( "more", { + environment = true, + arguments = { + { "option", "list" }, + }, + starter = startmore, + stopper = stopmore, +} ) +\stopluacode + +more: \startmore[1] one \startmore[2] two \stopmore one \stopmore +\stopbuffer + +If you want to to define an environment (i.e.\ a \type {start}||\type {stop} +pair, it looks as follows: + +\typebuffer + +This gives: + +\startpacked +\getbuffer +\stoppacked + +The arguments are know in both \type {startmore} and \type {stopmore} and nesting +is handled automatically. + +\stopsection + +\startsection[title=Looking inside] + +If needed you can access the body of a macro. Take for instance: + +\startbuffer +\def\TestA{A} +\def\TestB{\def\TestC{c}} +\def\TestC{C} +\stopbuffer + +\typebuffer \getbuffer + +The following example demonsttaies how we can look inside these macros. You need +to be aware of the fact that the whole blob of \LUA\ codes is finished before we +return to \TEX, so when we pipe the meaning of \type {TestB} back to \TEX\ it +only gets expanded afterwards. We can use a function to get back to \LUA. It's +only then that the meaning of \type {testC} is changed by the (piped) expansion +of \type {TestB}. + +\startbuffer +\startluacode +context(tokens.getters.macro("TestA")) +context(tokens.getters.macro("TestB")) +context(tokens.getters.macro("TestC")) +tokens.setters.macro("TestA","a") +context(tokens.getters.macro("TestA")) +context(function() + context(tokens.getters.macro("TestA")) + context(tokens.getters.macro("TestB")) + context(tokens.getters.macro("TestC")) +end) +\stopluacode +\stopbuffer + +\typebuffer \getbuffer + +Here is another example: + +\startbuffer +\startluacode +if tokens.getters.macro("fontstyle") == "rm" then + context("serif") +else + context("unknown") +end +\stopluacode +\stopbuffer + +\typebuffer + +Of course this assumes that you have some knowledge of the \CONTEXT\ internals. + +\getbuffer + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-mkiv.tex b/doc/context/sources/general/manuals/cld/cld-mkiv.tex new file mode 100644 index 000000000..5c35fa4e7 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-mkiv.tex @@ -0,0 +1,91 @@ +% language=uk + +% author : Hans Hagen +% copyright : PRAGMA ADE & ConTeXt Development Team +% license : Creative Commons Attribution ShareAlike 4.0 International +% reference : pragma-ade.nl | contextgarden.net | texlive (related) distributions +% origin : the ConTeXt distribution +% +% comment : Because this manual is distributed with TeX distributions it comes with a rather +% liberal license. We try to adapt these documents to upgrades in the (sub)systems +% that they describe. Using parts of the content otherwise can therefore conflict +% with existing functionality and we cannot be held responsible for that. Many of +% the manuals contain characteristic graphics and personal notes or examples that +% make no sense when used out-of-context. +% +% comment : Some chapters might have been published in TugBoat, the NTG Maps, the ConTeXt +% Group journal or otherwise. Thanks to the editors for corrections. Also thanks +% to users for testing, feedback and corrections. + +% \disabledirectives[paragraphs.normalize.global] +% \disabledirectives[paragraphs.normalize.local] + +% \usemodule[s-lan-03] +% \ctxlua{languages.words.tracers.showwords()} % no run check yet +% \page +% \setupspellchecking[state=start,method=2] + +% \enabletrackers[structures.export.spaces] +% \enabletrackers[structures.export.showtree] + +% \enabletrackers[export.trace] +% \enabletrackers[export.trace.spacing] +% \enabletrackers[export.lessstate] +% \enabletrackers[export.comment] + +% \enablemode[export] + +\startmode[export] + + \setupbackend + [export=yes] + + \setupexport + [hyphen=yes, + width=60em] + +\stopmode + +\setupinteraction + [title=Context Lua Documents, + %subtitle=preliminary version, + author=Hans Hagen] + +\startproduct cld-mkiv + +\environment cld-environment + +\component cld-titlepage + +\startfrontmatter + \component cld-contents + \component cld-introduction +\stopfrontmatter + +\startbodymatter + \component cld-abitoflua + \component cld-gettingstarted + \component cld-moreonfunctions + \component cld-afewdetails + \component cld-somemoreexamples + \component cld-graphics + \component cld-macros + \component cld-verbatim + \component cld-logging + \component cld-luafunctions + \component cld-ctxfunctions + \component cld-callbacks + \component cld-backendcode + \component cld-goodies + \component cld-nicetoknow + %\component cld-xml + \component cld-summary + \component cld-specialcommands + \component cld-files +\stopbodymatter + +% \startbackmatter +% \component cld-index +% \stopbackmatter + +\stopproduct diff --git a/doc/context/sources/general/manuals/cld/cld-moreonfunctions.tex b/doc/context/sources/general/manuals/cld/cld-moreonfunctions.tex new file mode 100644 index 000000000..fab22515e --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-moreonfunctions.tex @@ -0,0 +1,354 @@ +% language=uk + +\startcomponent cld-moreonfunctions + +\environment cld-environment + +\startchapter[title=More on functions] + +\startsection[title=Why we need them] + +\index{functions} + +In a previous chapter we introduced functions as arguments. At first sight this +feature looks strange but you need to keep in mind that a call to a \type +{context} function has no direct consequences. It generates \TEX\ code that is +executed after the current \LUA\ chunk ends and control is passed back to \TEX. +Take the following code: + +\startbuffer +context.framed( { + frame = "on", + offset = "5mm", + align = "middle" + }, + context.input("knuth") +) +\stopbuffer + +\typebuffer + +We call the function \type {framed} but before the function body is executed, the +arguments get evaluated. This means that \type {input} gets processed before +\type {framed} gets done. As a result there is no second argument to \type +{framed} and no content gets passed: an error is reported. This is why we need +the indirect call: + +\startbuffer +context.framed( { + frame = "on", + align = "middle" + }, + function() context.input("knuth") end +) +\stopbuffer + +\typebuffer + +This way we get what we want: + +\startlinecorrection +\ctxluabuffer +\stoplinecorrection + +The function is delayed till the \type {framed} command is executed. If your +applications use such calls a lot, you can of course encapsulate this ugliness: + +\starttyping +mycommands = mycommands or { } + +function mycommands.framed_input(filename) + context.framed( { + frame = "on", + align = "middle" + }, + function() context.input(filename) end +end + +mycommands.framed_input("knuth") +\stoptyping + +Of course you can nest function calls: + +\starttyping +context.placefigure( + "caption", + function() + context.framed( { + frame = "on", + align = "middle" + }, + function() context.input("knuth") end + ) + end +) +\stoptyping + +Or you can use a more indirect method: + +\starttyping +function text() + context.framed( { + frame = "on", + align = "middle" + }, + function() context.input("knuth") end + ) +end + +context.placefigure( + "none", + function() text() end +) +\stoptyping + +You can develop your own style and libraries just like you do with regular \LUA\ +code. Browsing the already written code can give you some ideas. + +\stopsection + +\startsection[title=How we can avoid them] + +\index{delaying} +\index{nesting} + +As many nested functions can obscure the code rather quickly, there is an +alternative. In the following examples we use \type {test}: + +\startbuffer +\def\test#1{[#1]} +\stopbuffer + +\typebuffer \getbuffer + +\startbuffer +context.test("test 1 ",context("test 2a")," test 3") +\stopbuffer + +\typebuffer + +This gives: \ctxluabuffer. As you can see, the second argument is executed before +the encapsulating call to \type {test}. So, we should have packed it into a +function but here is an alternative: + +\startbuffer +context.test("test 1 ",context.delayed("test 2a")," test 3") +\stopbuffer + +\typebuffer + +Now we get: \ctxluabuffer. We can also delay functions themselves, +look at this: + +\startbuffer +context.test("test 1 ",context.delayed.test("test 2b")," test 3") +\stopbuffer + +\typebuffer + +The result is: \ctxluabuffer. This feature also conveniently permits the use of +temporary variables, as in: + +\starttyping +local f = context.delayed.test("test 2c") +context("before ",f," after") +\stoptyping + +Of course you can limit the amount of keystrokes even more by +creating a shortcut: + +\starttyping +local delayed = context.delayed + +context.test("test 1 ",delayed.test("test 2")," test 3") +context.test("test 4 ",delayed.test("test 5")," test 6") +\stoptyping + +So, if you want you can produce rather readable code and readability of code is +one of the reasons why \LUA\ was chosen in the first place. This is a good +example of why coding in \TEX\ makes sense as it looks more intuitive: + +\starttyping +\test{test 1 \test{test 2} test 3} +\test{test 4 \test{test 5} test 6} +\stoptyping + +There is also another mechanism available. In the next example the second +argument is actually a string. + +\starttyping +local nested = context.nested + +context.test("test 8",nested.test("test 9"),"test 10") +\stoptyping + +There is a pitfall here: a nested context command needs to be flushed explicitly, +so in the case of: + +\starttyping +context.nested.test("test 9") +\stoptyping + +a string is created but nothing ends up at the \TEX\ end. Flushing is up to you. +Beware: \type {nested} only works with the regular \CONTEXT\ catcode regime. + +\stopsection + +\startsection[title=Trial typesetting] + +\index {prerolls} +\index {trial typesetting} + +Some typesetting mechanisms demand a preroll. For instance, when determining the +most optimal way to analyse and therefore typeset a table, it is necessary to +typeset the content of cells first. Inside \CONTEXT\ there is a state tagged +\quote {trial typesetting} which signals other mechanisms that for instance +counters should not be incremented more than once. + +Normally you don't need to worry about these issues, but when writing the code +that implements the \LUA\ interface to \CONTEXT, it definitely had to be taken +into account as we either or not can free cached (nested) functions. + +You can influence this caching to some extend. If you say + +\starttyping +function() + context("whatever") +end +\stoptyping + +the function will be removed from the cache when \CONTEXT\ is not in the trial +typesetting state. You can prevent removal of a function by returning \type +{true}, as in: + +\starttyping +function() + context("whatever") + return true +end +\stoptyping + +Whenever you run into a situation that you don't get the outcome that you expect, +you can consider returning \type {true}. However, keep in mind that it will take +more memory, something that only matters on big runs. You can force flushing the +whole cache by: + +\starttyping +context.restart() +\stoptyping + +An example of an occasion where you need to keep the function available is in +repeated content, for instance in headers and footers. + +\starttyping +context.setupheadertexts { + function() + context.pagenumber() + return true + end +} +\stoptyping + +Of course it is not needed when you use the following method: + +\starttyping +context.pagenumber("pagenumber") +\stoptyping + +Because here \CONTEXT\ itself deals with the content driven by the keyword \type +{pagenumber}. + +\stopsection + +\startsection[title=Steppers] + +The \type {context} commands are accumulated within a \type {\ctxlua} call and +only after the call is finished, control is back at the \TEX\ end. Sometimes you +want (in your \LUA\ code) to go on and pretend that you jump out to \TEX\ for a +moment, but come back to where you left. The stepper mechanism permits this. + +A not so practical but nevertheless illustrative example is the following: + +\startbuffer +\startluacode + context.stepwise (function() + context.startitemize() + context.startitem() + context.step("BEFORE 1") + context.stopitem() + context.step("\\setbox0\\hbox{!!!!}") + context.startitem() + context.step("%p",tex.getbox(0).width) + context.stopitem() + context.startitem() + context.step("BEFORE 2") + context.stopitem() + context.step("\\setbox2\\hbox{????}") + context.startitem() + context.step("%p",tex.getbox(2).width) + context.startitem() + context.step("BEFORE 3") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2") + context.stopitem() + context.startitem() + context.step("BEFORE 4") + context.startitemize() + context.stepwise (function() + context.step("\\bgroup") + context.step("\\setbox0\\hbox{>>>>}") + context.startitem() + context.step("%p",tex.getbox(0).width) + context.stopitem() + context.step("\\setbox2\\hbox{<<<<}") + context.startitem() + context.step("%p",tex.getbox(2).width) + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2") + context.stopitem() + context.step("\\egroup") + end) + context.stopitemize() + context.stopitem() + context.startitem() + context.step("AFTER 1\\par") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2\\par") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2\\par") + context.stopitem() + context.startitem() + context.step("AFTER 2\\par") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2\\par") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2\\par") + context.stopitem() + context.stopitemize() +end) +\stopluacode +\stopbuffer + +\typebuffer + +This gives an (ugly) itemize with a nested one: + +\getbuffer + +As you can see in the code, the \type {step} call accepts multiple arguments, but +when more than one argument is given the first one is treated as a formatter. + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-nicetoknow.tex b/doc/context/sources/general/manuals/cld/cld-nicetoknow.tex new file mode 100644 index 000000000..fcc0aa26b --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-nicetoknow.tex @@ -0,0 +1,163 @@ +% language=uk + +\startcomponent cld-nicetoknow + +\environment cld-environment + +\startchapter[title=Nice to know] + +\startsection[title=Introduction] + +As we like to abstract interfaces it is no surprise that \CONTEXT\ and +therefore it's \LUA\ libraries come with all kind of helpers. In this +chapter I will explain a few of them. Feel free to remind of adding more +here. + +\stopsection + +\startsection[title=Templates] + +{\em Eventually we will move this to the utilities section.} + +When dealing with data from tables or when order matters it can be handy +to abstract the actual data from the way it is dealt with. For this we +provide a template mechanism. The following example demonstrate its use. + +\startbuffer +require("util-ran") -- needed for this example + +local preamble = [[|l|l|c|]] +local template = [[\NC %initials% \NC %surname% \NC %length% \NC \NR]] + +context.starttabulate { preamble } + for i=1,10 do + local row = utilities.templates.replace(template, { + surname = utilities.randomizers.surname(5,10), + initials = utilities.randomizers.initials(1,3), + length = string.format("%0.2f",math.random(140,195)), + }) + context(row) + end +context.stoptabulate() +\stopbuffer + +\typebuffer + +This renders a table with random entries: + +\ctxluabuffer + +The nice thing is that when we change the order of the columns, we don't need to +change the table builder. + +\starttyping +local preamble = [[|c|l|l|]] +local template = [[\NC %length% \NC %initials% \NC %surname% \NC \NR]] +\stoptyping + +The \type {replace} function takes a few more arguments. There are also a some +more replacement options. + +\starttyping +replace("test '%[x]%' test",{ x = [[a 'x' a]] })) +replace("test '%[x]%' test",{ x = true })) +replace("test '%[x]%' test",{ x = [[a 'x' a]], y = "oeps" },'sql')) +replace("test '%[x]%' test",{ x = [[a '%y%' a]], y = "oeps" },'sql',true)) +replace([[test %[x]% test]],{ x = [[a "x" a]]})) +replace([[test %(x)% test]],{ x = [[a "x" a]]})) +\stoptyping + +The first argument is the template and the second one a table with mappings from +keys to values. The third argument can be used to inform the replace mechanism +what string escaping has to happen. The last argument triggers recursive +replacement. The above calls result in the following strings: + +\starttyping +test 'a 'x' \127 a' test +test 'true' test +test 'a ''x'' a' test +test 'a ''oeps'' a' test +test a \"x\" \127 a test +test "a \"x\" \127 a" test +\stoptyping + +These examples demonstrate that by adding a pair of square brackets we get +escaped strings. When using parenthesis the quotes get added automatically. This +is somewhat faster in case when \LUA\ is the target, but in practice it is not +that noticeable. + +% replace(str,mapping,how,recurse) + +\stopsection + +\startsection [title=Extending] + +Instead of extending tex endlessly we can also define our own extensions. Here +is an example. When you want to manipulate a box at the \LUA\ end you have the +problem that the following will not always work out well: + +\starttyping +local b = tex.getbox(0) +-- mess around with b +tex.setbox(0,b) +\stoptyping + +So we end up with: + +\starttyping +local b = node.copy_list(tex.getbox(0)) +-- mess around with b +tex.setbox(0,b) +\stoptyping + +The reason is that at the \TEX\ end grouping plays a role which means that values +are saved and restored. However, there is a save way out by defining a function +that cheats a bit: + +\starttyping +function tex.takebox(id) + local box = tex.getbox(id) + if box then + local copy = node.copy(box) + local list = box.list + copy.list = list + box.list = nil + tex.setbox(id,nil) + return copy + end +end +\stoptyping + +Now we can say: + +\starttyping +local b = tex.takebox(0) +-- mess around with b +tex.setbox(b) +\stoptyping + +In this case we first get the box content and then let \TEX\ do some housekeeping. +But, because we only keep the list node (which we copied) in the register the +overhead of copying a whole list is gone. + +\stopsection + +% require("util-sto") require("char-def") require("char-ini") + +% local myformatter = utilities.strings.formatters.new() + +% string.addformatter("upper", [[upper (%s)]],[[local upper = characters.upper ]]) -- maybe handy +% string.addformatter("lower", [[lower (%s)]],[[local lower = characters.lower ]]) -- maybe handy +% string.addformatter("shaped", [[shaped(%s)]],[[local shaped = characters.shaped]]) -- maybe handy + +% utilities.strings.formatters.add("upper", [[lpegmatch(p_toupper,%s)]],[[local p_toupper = lpeg.patterns.toupper]]) -- maybe handy +% utilities.strings.formatters.add("lower", [[lpegmatch(p_tolower,%s)]],[[local p_tolower = lpeg.patterns.tolower]]) -- maybe handy +% utilities.strings.formatters.add("shaped", [[lpegmatch(p_toshape,%s)]],[[local p_toshape = lpeg.patterns.toshape]]) -- maybe handy + +% print("\n>>>",string.formatters["Is this %s handy or not?"](characters.upper("ÀÁÂÃÄÅàáâãäå"))) +% print("\n>>>",string.formatters["Is this %!upper! handy or not?"]("ÀÁÂÃÄÅàáâãäå")) +% print("\n>>>",string.formatters["Is this %!shaped! handy or not?"]("ÀÁÂÃÄÅàáâãäå")) + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-somemoreexamples.tex b/doc/context/sources/general/manuals/cld/cld-somemoreexamples.tex new file mode 100644 index 000000000..a282be4e9 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-somemoreexamples.tex @@ -0,0 +1,753 @@ +% language=uk + +\startcomponent cld-somemoreexamples + +\environment cld-environment + +\usemodule[morse] + +\startchapter[title=Some more examples] + +\startsection[title=Appetizer] + +Before we give some more examples, we will have a look at the way the title page +is made. This way you get an idea what more is coming. + +\typefile {cld-mkiv-simple-titlepage.cld} + +This does not look that bad, does it? Of course in pure \TEX\ code it looks +mostly the same but loops and calculations feel a bit more natural in \LUA\ then +in \TEX. The result is shown in \in {figure} [fig:cover]. The actual cover page +was derived from this. + +\startplacefigure[location=here,reference=fig:cover,title={The simplified cover page.}] + \doiffileexistselse {cld-mkiv-simple-titlepage.pdf} { + \externalfigure + [cld-mkiv-simple-titlepage.pdf] + [height=.5\textheight] + } { + \scale + [height=.5\textheight] + {\cldprocessfile{cld-mkiv-simple-titlepage.cld}} + } +\stopplacefigure + +\stopsection + +\startsection[title=A few examples] + +As it makes most sense to use the \LUA\ interface for generated text, here is +another example with a loop: + +\startbuffer +context.startitemize { "a", "packed", "two" } + for i=1,10 do + context.startitem() + context("this is item %i",i) + context.stopitem() + end +context.stopitemize() +\stopbuffer + +\typebuffer + +\ctxluabuffer + +Just as you can mix \TEX\ with \XML\ and \METAPOST, you can define bits and +pieces of a document in \LUA. Tables are good candidates: + +\startbuffer +local one = { + align = "middle", + style = "type", +} +local two = { + align = "middle", + style = "type", + background = "color", + backgroundcolor = "darkblue", + foregroundcolor = "white", +} +local random = math.random +context.bTABLE { framecolor = "darkblue" } + for i=1,10 do + context.bTR() + for i=1,20 do + local r = random(99) + context.bTD(r < 50 and one or two) + context("%2i",r) + context.eTD() + end + context.eTR() + end +context.eTABLE() +\stopbuffer + +\typebuffer + +\placetable[top][tab:random]{A table generated by \LUA.}{\ctxluabuffer} + +Here we see a function call to \type {context} in the most indented line. The +first argument is a format that makes sure that we get two digits and the random +number is substituted into this format. The result is shown in +\in{table}[tab:random]. The line correction is ignored when we use this table as +a float, otherwise it assures proper vertical spacing around the table. Watch how +we define the tables \type {one} and \type {two} beforehand. This saves 198 +redundant table constructions. + +Not all code will look as simple as this. Consider the following: + +\starttyping +context.placefigure( + "caption", + function() context.externalfigure( { "cow.pdf" } ) end +) +\stoptyping + +Here we pass an argument wrapped in a function. If we would not do that, the +external figure would end up wrong, as arguments to functions are evaluated +before the function that gets them (we already showed some alternative approaches +in previous chapters). A function argument is treated as special and in this case +the external figure ends up right. Here is another example: + +\startbuffer +context.placefigure("Two cows!",function() + context.bTABLE() + context.bTR() + context.bTD() + context.externalfigure( + { "cow.pdf" }, + { width = "3cm", height = "3cm" } + ) + context.eTD() + context.bTD { align = "{lohi,middle}" } + context("and") + context.eTD() + context.bTD() + context.externalfigure( + { "cow.pdf" }, + { width = "4cm", height = "3cm" } + ) + context.eTD() + context.eTR() + context.eTABLE() +end) +\stopbuffer + +\typebuffer + +In this case the figure is not an argument so it gets flushed sequentially +with the rest. + +\ctxluabuffer + +\stopsection + +\startsection[title=Styles] + +Say that you want to typeset a word in a bold font. You can do +that this way: + +\starttyping +context("This is ") +context.bold("important") +context("!") +\stoptyping + +Now imagine that you want this important word to be in red too. As we have +a nested command, we end up with a nested call: + +\starttyping +context("This is ") +context.bold(function() context.color( { "red" }, "important") end) +context("!") +\stoptyping + +or + +\starttyping +context("This is ") +context.bold(context.delayed.color( { "red" }, "important")) +context("!") +\stoptyping + +In that case it's good to know that there is a command that combines both +features: + +\starttyping +context("This is ") +context.style( { style = "bold", color = "red" }, "important") +context("!") +\stoptyping + +But that is still not convenient when we have to do that often. So, you can wrap +the style switch in a function. + +\starttyping +local function mycommands.important(str) + context.style( { style = "bold", color = "red" }, str ) +end + +context("This is ") +mycommands.important( "important") +context(", and ") +mycommands.important( "this") +context(" too !") +\stoptyping + +Or you can setup a named style: + +\starttyping +context.setupstyle( { "important" }, { style = "bold", color = "red" } ) + +context("This is ") +context.style( { "important" }, "important") +context(", and ") +context.style( { "important" }, "this") +context(" too !") +\stoptyping + +Or even define one: + +\starttyping +context.definestyle( { "important" }, { style = "bold", color = "red" } ) + +context("This is ") +context.important("important") +context(", and ") +context.important("this") +context(" too !") +\stoptyping + +This last solution is especially handy for more complex cases: + +\startbuffer +context.definestyle( { "important" }, { style = "bold", color = "red" } ) + +context("This is ") +context.startimportant() +context.inframed("important") +context.stopimportant() +context(", and ") +context.important("this") +context(" too !") +\stopbuffer + +\typebuffer + +\ctxluabuffer + +\stopsection + +\startsection[title=A complete example] + +One day my 6 year old niece Lorien was at the office and wanted to know what I +was doing. As I knew she was practicing arithmetic at school I wrote a quick and +dirty script to generate sheets with exercises. The most impressive part was that +the answers were included. It was a rather braindead bit of \LUA, written in a +few minutes, but the weeks after I ended up running it a few more times, for her +and her friends, every time a bit more difficult and also using different +arithmetic. It was that script that made me decide to extend the basic cld manual +into this more extensive document. + +We generate three columns of exercises. Each exercise is a row in a table. The +last argument to the function determines if answers are shown. + +\starttyping +local random = math.random + +local function ForLorien(n,maxa,maxb,answers) + context.startcolumns { n = 3 } + context.starttabulate { "|r|c|r|c|r|" } + for i=1,n do + local sign = random(0,1) > 0.5 + local a, b = random(1,maxa or 99), random(1,max or maxb or 99) + if b > a and not sign then a, b = b, a end + context.NC() + context(a) + context.NC() + context.mathematics(sign and "+" or "-") + context.NC() + context(b) + context.NC() + context("=") + context.NC() + context(answers and (sign and a+b or a-b)) + context.NC() + context.NR() + end + context.stoptabulate() + context.stopcolumns() + context.page() +end +\stoptyping + +This is a typical example of where it's more convenient to write the code in +\LUA\ that in \TEX's macro language. As a consequence setting up the page also +happens in \LUA: + +\starttyping +context.setupbodyfont { + "palatino", + "14pt" +} + +context.setuplayout { + backspace = "2cm", + topspace = "2cm", + header = "1cm", + footer = "0cm", + height = "middle", + width = "middle", +} +\stoptyping + +This leave us to generate the document. There is a pitfall here: we need to use +the same random number for the exercises and the answers, so we freeze and +defrost it. Functions in the \type {commands} namespace implement functionality +that is used at the \TEX\ end but better can be done in \LUA\ than in \TEX\ macro +code. Of course these functions can also be used at the \LUA\ end. + +\starttyping +context.starttext() + + local n = 120 + + commands.freezerandomseed() + + ForLorien(n,10,10) + ForLorien(n,20,20) + ForLorien(n,30,30) + ForLorien(n,40,40) + ForLorien(n,50,50) + + commands.defrostrandomseed() + + ForLorien(n,10,10,true) + ForLorien(n,20,20,true) + ForLorien(n,30,30,true) + ForLorien(n,40,40,true) + ForLorien(n,50,50,true) + +context.stoptext() +\stoptyping + +\placefigure + [here] + [fig:lorien] + {Lorien's challenge.} + {\startcombination + {\externalfigure[cld-005.pdf][page=1,width=.45\textwidth,frame=on]} {exercises} + {\externalfigure[cld-005.pdf][page=6,width=.45\textwidth,frame=on]} {answers} + \stopcombination} + +A few pages of the result are shown in \in {figure} [fig:lorien]. In the +\CONTEXT\ distribution a more advanced version can be found in \type +{s-edu-01.cld} as I was also asked to generate multiplication and table +exercises. In the process I had to make sure that there were no duplicates on a +page as she complained that was not good. There a set of sheets is generated +with: + +\starttyping +moduledata.educational.arithematic.generate { + name = "Bram Otten", + fontsize = "12pt", + columns = 2, + run = { + { method = "bin_add_and_subtract", maxa = 8, maxb = 8 }, + { method = "bin_add_and_subtract", maxa = 16, maxb = 16 }, + { method = "bin_add_and_subtract", maxa = 32, maxb = 32 }, + { method = "bin_add_and_subtract", maxa = 64, maxb = 64 }, + { method = "bin_add_and_subtract", maxa = 128, maxb = 128 }, + }, +} +\stoptyping + +\stopsection + +\startsection[title=Interfacing] + +The fact that we can define functionality using \LUA\ code does not mean that we +should abandon the \TEX\ interface. As an example of this we use a relatively +simple module for typesetting morse code.\footnote {The real module is a bit +larger and can format verbose morse.} First we create a proper namespace: + +\starttyping + +moduledata.morse = moduledata.morse or { } +local morse = moduledata.morse +\stoptyping + +We will use a few helpers and create shortcuts for them. The first helper loops +over each \UTF\ character in a string. The other two helpers map a character onto +an uppercase (because morse only deals with uppercase) or onto an similar shaped +character (because morse only has a limited character set). + +\starttyping +local utfcharacters = string.utfcharacters +local ucchars, shchars = characters.ucchars, characters.shchars +\stoptyping + +The morse codes are stored in a table. + +\starttyping +local codes = { + + ["A"] = "·—", ["B"] = "—···", + ["C"] = "—·—·", ["D"] = "—··", + ["E"] = "·", ["F"] = "··—·", + ["G"] = "——·", ["H"] = "····", + ["I"] = "··", ["J"] = "·———", + ["K"] = "—·—", ["L"] = "·—··", + ["M"] = "——", ["N"] = "—·", + ["O"] = "———", ["P"] = "·——·", + ["Q"] = "——·—", ["R"] = "·—·", + ["S"] = "···", ["T"] = "—", + ["U"] = "··—", ["V"] = "···—", + ["W"] = "·——", ["X"] = "—··—", + ["Y"] = "—·——", ["Z"] = "——··", + + ["0"] = "—————", ["1"] = "·————", + ["2"] = "··———", ["3"] = "···——", + ["4"] = "····—", ["5"] = "·····", + ["6"] = "—····", ["7"] = "——···", + ["8"] = "———··", ["9"] = "————·", + + ["."] = "·—·—·—", [","] = "——··——", + [":"] = "———···", [";"] = "—·—·—", + ["?"] = "··——··", ["!"] = "—·—·——", + ["-"] = "—····—", ["/"] = "—··—· ", + ["("] = "—·——·", [")"] = "—·——·—", + ["="] = "—···—", ["@"] = "·——·—·", + ["'"] = "·————·", ['"'] = "·—··—·", + + ["À"] = "·——·—", + ["Å"] = "·——·—", + ["Ä"] = "·—·—", + ["Æ"] = "·—·—", + ["Ç"] = "—·—··", + ["É"] = "··—··", + ["È"] = "·—··—", + ["Ñ"] = "——·——", + ["Ö"] = "———·", + ["Ø"] = "———·", + ["Ü"] = "··——", + ["ß"] = "··· ···", + +} + +morse.codes = codes +\stoptyping + +As you can see, there are a few non \ASCII\ characters supported as well. There +will never be full \UNICODE\ support simply because morse is sort of obsolete. +Also, in order to support \UNICODE\ one could as well use the bits of \UTF\ +characters, although \unknown\ memorizing the whole \UNICODE\ table is not much +fun. + +We associate a metatable index function with this mapping. That way we can not +only conveniently deal with the casing, but also provide a fallback based on the +shape. Once found, we store the representation so that only one lookup is needed +per character. + +\starttyping +local function resolvemorse(t,k) + if k then + local u = ucchars[k] + local v = rawget(t,u) or rawget(t,shchars[u]) or false + t[k] = v + return v + else + return false + end +end + +setmetatable(codes, { __index = resolvemorse }) +\stoptyping + +Next comes some rendering code. As we can best do rendering at the \TEX\ end we +just use macros. + +\starttyping +local MorseBetweenWords = context.MorseBetweenWords +local MorseBetweenCharacters = context.MorseBetweenCharacters +local MorseLong = context.MorseLong +local MorseShort = context.MorseShort +local MorseSpace = context.MorseSpace +local MorseUnknown = context.MorseUnknown +\stoptyping + +The main function is not that complex. We need to keep track of spaces and +newlines. We have a nested loop because a fallback to shape can result in +multiple characters. + +\starttyping +function morse.tomorse(str) + local inmorse = false + for s in utfcharacters(str) do + local m = codes[s] + if m then + if inmorse then + MorseBetweenWords() + else + inmorse = true + end + local done = false + for m in utfcharacters(m) do + if done then + MorseBetweenCharacters() + else + done = true + end + if m == "·" then + MorseShort() + elseif m == "—" then + MorseLong() + elseif m == " " then + MorseBetweenCharacters() + end + end + inmorse = true + elseif s == "\n" or s == " " then + MorseSpace() + inmorse = false + else + if inmorse then + MorseBetweenWords() + else + inmorse = true + end + MorseUnknown(s) + end + end +end +\stoptyping + +We use this function in two additional functions. One typesets a file, the other +a table of available codes. + +\starttyping +function morse.filetomorse(name,verbose) + morse.tomorse(resolvers.loadtexfile(name),verbose) +end + +function morse.showtable() + context.starttabulate { "|l|l|" } + for k, v in table.sortedpairs(codes) do + context.NC() context(k) + context.NC() morse.tomorse(v,true) + context.NC() context.NR() + end + context.stoptabulate() +end +\stoptyping + +We're done with the \LUA\ code that we can either put in an external file or put +in the module file. The \TEX\ file has two parts. The typesetting macros that we +use at the \LUA\ end are defined first. These can be overloaded. + +\starttyping +\def\MorseShort + {\dontleavehmode + \vrule + width \MorseWidth + height \MorseHeight + depth \zeropoint + \relax} + +\def\MorseLong + {\dontleavehmode + \vrule + width 3\dimexpr\MorseWidth + height \MorseHeight + depth \zeropoint + \relax} + +\def\MorseBetweenCharacters + {\kern\MorseWidth} + +\def\MorseBetweenWords + {\hskip3\dimexpr\MorseWidth\relax} + +\def\MorseSpace + {\hskip7\dimexpr\MorseWidth\relax} + +\def\MorseUnknown#1 + {[\detokenize{#1}]} +\stoptyping + +The dimensions are stored in macros as well. Of course we could provide a proper +setup command, but it hardly makes sense. + +\starttyping +\def\MorseWidth {0.4em} +\def\MorseHeight{0.2em} +\stoptyping + +Finally we have arrived at the macros that interface to the \LUA\ functions. + +\starttyping +\def\MorseString#1{\ctxlua{moduledata.morse.tomorse(\!!bs#1\!!es)}} +\def\MorseFile #1{\ctxlua{moduledata.morse.filetomorse("#1")}} +\def\MorseTable {\ctxlua{moduledata.morse.showtable()}} +\stoptyping + +\startbuffer +\Morse{A more advanced solution would be to convert a node list. That +way we can deal with weird input.} +\stopbuffer + +A string is converted to morse with the first command. + +\typebuffer + +This shows up as: + +\startalignment[flushleft,tolerant]\getbuffer\stopalignment + +Reduction and uppercasing is demonstrated in the next example: + +\startbuffer +\MorseString{ÀÁÂÃÄÅàáâãäå} +\stopbuffer + +\typebuffer + +This gives: + +\startalignment[flushleft,tolerant]\getbuffer\stopalignment + +\stopsection + +\startsection[title=Using helpers] + +The next example shows a bit of \LPEG. On top of the standard functionality +a few additional functions are provided. Let's start with a pure \TEX\ +example: + +\startbuffer +\defineframed + [colored] + [foregroundcolor=red, + foregroundstyle=\underbar, + offset=.1ex, + location=low] +\stopbuffer + +\typebuffer \getbuffer + +\startbuffer +\processisolatedwords {\input ward \relax} \colored +\stopbuffer + +\typebuffer \blank \getbuffer \blank + +Because this processor macro operates at the \TEX\ end it has some limitations. +The content is collected in a very narrow box and from that a regular paragraph +is constructed. It is for this reason that no color is applied: the snippets that +end up in the box are already typeset. + +An alternative is to delegate the task to \LUA: + +\startbuffer +\startluacode +local function process(data) + + local words = lpeg.split(lpeg.patterns.spacer,data or "") + + for i=1,#words do + if i == 1 then + context.dontleavehmode() + else + context.space() + end + context.colored(words[i]) + end + +end + +process(io.loaddata(resolvers.findfile("ward.tex"))) +\stopluacode +\stopbuffer + +\typebuffer \blank \getbuffer \blank + +The function splits the loaded data into a table with individual words. We use a +splitter that splits on spacing tokens. The special case for \type {i = 1} makes +sure that we end up in horizontal mode (read: properly start a paragraph). This +time we do get color because the typesetting is done directly. Here is an +alternative implementation: + +\starttyping +local done = false + +local function reset() + done = false + return true +end + +local function apply(s) + if done then + context.space() + else + done = true + context.dontleavehmode() + end + context.colored(s) +end + +local splitter = lpeg.P(reset) + * lpeg.splitter(lpeg.patterns.spacer,apply) + +local function process(data) + lpeg.match(splitter,data) +end +\stoptyping + +This version is more efficient as it does not create an intermediate table. The +next one is comaprable: + +\starttyping +local function apply(s) + context.colored("%s ",s) +end + +local splitter lpeg.splitter(lpeg.patterns.spacer,apply) + +local function process(data) + context.dontleavevmode() + lpeg.match(splitter,data) + context.removeunwantedspaces() +end +\stoptyping + +\stopsection + +\startsection[title=Formatters] + +Sometimes can save a bit of work by using formatters. By default, the \type {context} +command, when called directly, applies a given formatter. But when called as table +this feature is lost because then we want to process non|-|strings as well. The next +example shows a way out: + +\startbuffer +context("the current emwidth is %p",\number\emwidth) +context.par() +context.formatted("the current emwidth is %p",\number\emwidth) +context.par() +context.bold(string.formatters["the current emwidth is %p"](\number\emwidth)) +context.par() +context.formatted.bold("the current emwidth is %p",\number\emwidth) +\stopbuffer + +The last one is the most interesting one here: in the subnamespace \type +{formatted} (watch the \type {d}) a format specification with extra arguments is +expected. + +\ctxluabuffer + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-specialcommands.tex b/doc/context/sources/general/manuals/cld/cld-specialcommands.tex new file mode 100644 index 000000000..37ecf45c6 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-specialcommands.tex @@ -0,0 +1,257 @@ +% language=uk + +\startcomponent cld-specialcommands + +\environment cld-environment + +\startchapter[title=Special commands] + +\index{tracing} + +\startsection[title=Tracing] + +There are a few functions in the \type {context} namespace that are no +macros at the \TEX\ end. + +\starttyping +context.runfile("somefile.cld") +\stoptyping + +Another useful command is: + +\starttyping +context.settracing(true) +\stoptyping + +There are a few tracing options that you can set at the \TEX\ end: + +\starttyping +\enabletrackers[context.files] +\enabletrackers[context.trace] +\stoptyping + +\stopsection + +\startsection[title=Overloads] + +A few macros have special functions (overloads) at the \LUA\ end. One of them is +\type {\char}. The function makes sure that the characters ends up right. The +same is true for \type {\chardef}. So, you don't need to mess around with \type +{\relax} or trailing spaces as you would do at the \TEX\ end in order to tell the +scanner to stop looking ahead. + +\starttyping +context.char(123) +\stoptyping + +Other examples of macros that have optimized functions are \type {\par}, +\type{\bgroup} and \type {\egroup}. Or take this: + +\startbuffer +1: \ctxlua{commands.doif(true)}{one} +2: \cldcommand{doif("a","a","two")} +3: \ctxcommand{doif(true)}{three} +\stopbuffer + +\typebuffer + +\startlines +\getbuffer +\stoplines + +\stopsection + +\startsection[title=Steps] + +% added and extended in sync with an article about a generic 'execute' +% feature + +We already mentioned the stepper as a very special trick so let's give +some more explanation here. When you run the following code: + +\setbox0\emptybox + +\starttyping +\startluacode + context.startitemize() + context.startitem() + context("BEFORE 1") + context.stopitem() + context("\\setbox0\\hbox{!!!!}") + context.startitem() + context("%p",tex.getbox(0).width) + context.stopitem() + context.stopitemize() +\stopluacode +\stoptyping + +You get a message like: + +\starttyping +[ctxlua]:8: attempt to index a nil value +... +10 context("\\setbox0\\hbox{!!!!}") +11 context.startitem() +12 >> context("%p",tex.getbox(0).width) +... +\stoptyping + +due to the fact that the box is still void. All that the \CONTEXT\ commands feed +into \TEX\ happens when the code snippet has finished. You can however run a +snippet of code the following way: + +\startbuffer +\startluacode + context.stepwise (function() + context.startitemize() + context.startitem() + context.step("BEFORE 1") + context.stopitem() + context.step("\\setbox0\\hbox{!!!!}") + context.startitem() + context.step("%p",tex.getbox(0).width) + context.stopitem() + context.stopitemize() + end) +\stopluacode +\stopbuffer + +\typebuffer + +and get: + +\getbuffer + +A more extensive example is: + +\startbuffer +\startluacode + context.stepwise (function() + context.startitemize() + context.startitem() + context.step("BEFORE 1") + context.stopitem() + context.step("\\setbox0\\hbox{!!!!}") + context.startitem() + context.step("%p",tex.getbox(0).width) + context.stopitem() + context.startitem() + context.step("BEFORE 2") + context.stopitem() + context.step("\\setbox2\\hbox{????}") + context.startitem() + context.step("%p",tex.getbox(2).width) + context.startitem() + context.step("BEFORE 3") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2") + context.stopitem() + context.startitem() + context.step("BEFORE 4") + context.startitemize() + context.stepwise (function() + context.step("\\bgroup") + context.step("\\setbox0\\hbox{>>>>}") + context.startitem() + context.step("%p",tex.getbox(0).width) + context.stopitem() + context.step("\\setbox2\\hbox{<<<<}") + context.startitem() + context.step("%p",tex.getbox(2).width) + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2") + context.stopitem() + context.step("\\egroup") + end) + context.stopitemize() + context.stopitem() + context.startitem() + context.step("AFTER 1\\par") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2\\par") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2\\par") + context.stopitem() + context.startitem() + context.step("AFTER 2\\par") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2\\par") + context.stopitem() + context.startitem() + context.step("\\copy0\\copy2\\par") + context.stopitem() + context.stopitemize() + end) +\stopluacode +\stopbuffer + +\typebuffer + +which gives: + +\getbuffer + +A step returns control to \TEX\ immediately and after the \TEX\ code that it +feeds back is expanded, returns to \LUA. There are some limitations due to the +input stack but normally that is no real issue. + +You can run the following code: + +\starttyping +\definenumber[LineCounter][way=bypage] +\starttext +\startluacode +for i=1,2000 do + context.incrementnumber { "LineCounter" } + context.getnumber { "LineCounter" } + context.par() +end +\stopluacode +\stoptext +\stoptyping + +You will notice however that the number is not right on each page. This is +because \TEX\ doesn't know yet that there is no room on the page. The next will +work better: + +\starttyping +\definenumber[LineCounter][way=bypage] +\starttext +\startluacode +context.stepwise(function() + for i=1,2000 do + context.testpage { 0 } + context.incrementnumber { "LineCounter" } + context.getnumber { "LineCounter" } + context.par() + context.step() + end +end) +\stopluacode +\stoptext +\starttyping + +Instead of the \type {testpage} function you can also play directly with +registers, like: + +\starttyping +if tex.pagegtotal + tex.count.lineheight > tex.pagetotal then +\starttyping + +but often an already defined helper does a better job. Of course you will +probably never need this kind of hacks anyway, if only because much more is going +on and there are better ways then. + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-summary.tex b/doc/context/sources/general/manuals/cld/cld-summary.tex new file mode 100644 index 000000000..2b62597fa --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-summary.tex @@ -0,0 +1,841 @@ +% language=uk + +\startcomponent cld-summary + +\environment cld-environment + +\usemodule[s][characters-properties] + +\startchapter[title=A sort of summary] + +In this chapter we summarize the functionality provided by the \type {context} +namespace. We repeat some of what has been explained in other chapter so that in +fact you can start with this summary. + +If you have read this manual (or seen code) you know that you can access all the +core commands using this namespace: + +\starttyping +context.somecommand("some argument") +context["somecommand"]("some argument") +\stoptyping + +These calls will eventually expand \type {\somecommand} with the given argument. +This interface has been around form the start and has proven to be quite flexible +and robust. In spite of what you might think, the \type {somecommand} is not +really defined in the \type {context} namespace, but in its own one called \type +{core}, accessible via \type {context.core}. + +Next we describe the commands that are naturally defined in the \type {context} +namespace. Some have counterparts at the macro level (like \type {bgroup}) but +many haven't (for instance \type {rule}). We tried not to polute the \type +{context} namespace too much but although we could have put the helpers in a +separate namespace it would make usage a bit more unnatural. + +\startsection[title=Access to commands] + +\startsummary[title={context(".. some text ..")}] + +The string is flushed as|-|is: + +\starttyping +.. some text .. +\stoptyping + +\stopsummary + +\startsummary[title={context("format",...)}] + +The first string is a format specification according that is passed to the \LUA\ +function \type {format} in the \type {string} namespace. Following arguments are +passed too. + +\stopsummary + +\startsummary[title={context(123,...)}] + +The numbers (and following numbers or strings) are flushed without any +formatting. + +\starttyping +123... (concatenated) +\stoptyping + +\stopsummary + +\startsummary[title={context(true)}] + +An explicit \type {endlinechar} is inserted, in \TEX\ speak: + +\starttyping +^^M +\stoptyping + +\stopsummary + +\startsummary[title={context(false,...)}] + +Strings and numbers are flushed surrounded by curly braces, an indexed table is +flushed as option list, and a hashed table is flushed as parameter set. + +\starttyping +multiple {...} or [...] etc +\stoptyping + +\stopsummary + +\startsummary[title={context(node)}] + +The node (or list of nodes) is injected at the spot. Keep in mind that you need +to do the proper memory management yourself. + +\stopsummary + +\startsummary[title={context["command"] context.core["command"]}] + +The function that implements \type {\command}. The \type{core} table is where +these functions realy live. + +\stopsummary + +\startsummary[title={context["command"](value,...)}] + +The value (string or number) is flushed as a curly braced (regular) argument. + +\starttyping +\command {value}... +\stoptyping + +\stopsummary + +\startsummary[title={context["command"]({ value },...)}] + +The table is flushed as value set. This can be an identifier, +a list of options, or a directive. + +\starttyping +\command [value]... +\stoptyping + +\stopsummary + +\startsummary[title={context["command"]({ key = val },...)}] + +The table is flushed as key|/|value set. + +\starttyping +\command [key={value}]... +\stoptyping + +\stopsummary + +\startsummary[title={context["command"](true)}] + +An explicit \type {endlinechar} is inserted. + +\starttyping +\command ^^M +\stoptyping + +\stopsummary + +\startsummary[title={context["command"](node)}] + +The node(list) is injected at the spot. Keep in mind that you need to do the +proper memory management yourself. + +\starttyping +\command {node(list)} +\stoptyping + +\stopsummary + +\startsummary[title={context["command"](false,value)}] + +The value is flushed without encapsulating tokens. + +\starttyping +\command value +\stoptyping + +\stopsummary + +\startsummary[title={context["command"]({ value }, { key = val }, val, false, val)}] + +The arguments are flushed accordingly their nature and the order can be any. + +\starttyping +\command [value][key={value}]{value}value +\stoptyping + +\stopsummary + +\startsummary[title={context.direct(...)}] + +The arguments are interpreted the same as if \type {direct} was a command, but no +\type {\direct} is injected in front. Braces are added: + +\startbuffer +regular \expandafter \bold \ctxlua{context.direct("bold")} regular +black \expandafter \color \ctxlua{context.direct({"red"})}{red} black +black \expandafter \color \ctxlua{context.direct({"green"},"green")} black +\stopbuffer + +\typebuffer + +The \type {\expandafter} makes sure that the \type {\bold} and \type {\color} +macros see the following \type{{bold}}, \type {[red]}, and \type {[green]{green}} +arguments. + +\startlines\getbuffer\stoplines + +\stopsummary + +\startsummary[title={context.delayed(...)}] + +The arguments are interpreted the same as in a \type {context} call, but instead +of a direct flush, the arguments will be flushed in a next cycle. + +\stopsummary + +\startsummary[title={context.delayed["command"](...)}] + +The arguments are interpreted the same as in a \type {command} call, but instead +of a direct flush, the command and arguments will be flushed in a next cycle. + +\stopsummary + +\startsummary[title={context.nested["command"]}] + +This command returns the command, including given arguments as a string. No +flushing takes place. + +\stopsummary + +\startsummary[title={context.nested}] + +This command returns the arguments as a string and treats them the same as a +regular \type {context} call. + +\stopsummary + +\startsummary[title={context.formatted["command"]([<regime>,]<format>,<arguments>)}] + +This command returns the command that will pass it's arguments to the string +formatter. When the first argument is a number, then it is interpreted as a +catcode regime. + +\stopsummary + +\startsummary[title={context.formatted([<regime>,]<format>,<arguments>)}] + +This command passes it's arguments to the string formatter. When the first +argument is a number, then it is interpreted as a catcode regime. + +\stopsummary + +\stopsection + +\startsection[title=\METAFUN] + +\startsummary[title={context.metafun.start()}] + +This starts a \METAFUN\ (or \METAPOST) graphic. + +\stopsummary + +\startsummary[title={context.metafun.stop()}] + +This finishes and flushes a \METAFUN\ (or \METAPOST) graphic. + +\stopsummary + +\startsummary[title={context.metafun("format",...)}] + +The argument is appended to the current graphic data but the string formatter is +used on following arguments. + +\stopsummary + +\startsummary[title={context.metafun.delayed}] + +This namespace does the same as \type {context.delayed}: it wraps the code in such +a way that it can be used in a function call. + +\stopsummary + +\startsection[title=Building blocks] + +\startsummary[title={context.bgroup() context.egroup()}] + +These are just \type {\bgroup} and \type {\egroup} equivalents and as these are +in fact shortcuts to the curly braced we output these instead. + +\stopsummary + +\startsummary[title={context.space()}] + +This one directly maps onto \type {\space}. + +\stopsummary + +\startsummary[title={context.par()}] + +This one directly maps onto \type {\par}. + +\stopsummary + +\stopsection + +\startsection[title=Basic Helpers] + +\startsummary[title={context.rule(wd,ht,dp,direction) context.rule(specification)}] + +A rule node is injected with the given properties. A specification is just a +table with the four fields. The rule gets the current attributes. + +\stopsummary + +\startsummary[title={context.glyph(fontid,n) context.glyph(n)}] + +A glyph node is injected with the given font id. When no id is given, the current font +is used. The glyph gets the current attributes. + +\stopsummary + +\startsummary[title={context.char(n) context.char(str) context.char(tab)}] + +This will inject one or more copies of \type {\char} calls. You can pass a +number, a string representing a number, or a table with numbers. + +\stopsummary + +\startsummary[title={context.utfchar(n) context.utfchar(str)}] + +This injects is \UTF\ character (one or more bytes). You can pass a number +or a string representing a numbers. You need to be aware of special +characters in \TEX, like \type {#}. + +\stopsummary + +\stopsection + +\startsection[title={Registers}] + +This is a table that hosts a couple of functions. The following \type {new} +ones are available: + +\starttyping +local n = newdimen (name) +local n = newskip (name) +local n = newcount (name) +local n = newmuskip(name) +local n = newtoks (name) +local n = newbox (name) +\stoptyping + +These define a register with name \type {name} at the \LUA\ end and \type {\name} +at the \TEX\ end. The registers' number is returned. The next function is like +\type {\chardef}: it defines \type {\name} with value \type {n}. + +\starttyping +local n = newchar(name,n) +\stoptyping + +It's not likely that you will use any of these commands, if only because when +you're operating from the \LUA\ end using \LUA\ variables is more convenient. + +\stopsection + +\startsection[title=Catcodes] + +Normally we operate under the so called \type {context} catcode regime. This +means that content gets piped to \TEX\ using the same meanings for characters as +you normally use in \CONTEXT. So, a \type {$} starts math. In \in {table} +[tab:catcodes] we show the catcode regimes. + +\startplacetable[location=page,title={Catcode regimes},reference=tab:catcodes] + \showcharactercatcodes +\stopplacetable + +\startsummary[title={context.catcodes}] + +The \type {context.catcodes} tables contains the internal numbers of the +catcode tables used. The next table shows the names that can be used. + +\starttabulate[|lT|cT|lT|] +\BC name \BC mnemonic \BC \TEX\ command \NC \NR +\NC context \NC ctx \NC ctxcatcodes \NC \NR +\NC protect \NC prt \NC prtcatcodes \NC \NR +\NC plain \NC tex \NC texcatcodes \NC \NR +\NC text \NC txt \NC txtcatcodes \NC \NR +\NC verbatim \NC vrb \NC vrbcatcodes \NC \NR +\stoptabulate + +\stopsummary + +\startsummary[title={context.newindexer(catcodeindex)}] + +This function defines a new indexer. You can think of the context command itself +as an indexer. There are two (extra) predefined indexers: + +\starttyping +context.verbatim = context.newindexer(context.catcodes.verbatim) +context.puretext = context.newindexer(context.catcodes.text) +\stoptyping + +\stopsummary + +\startsummary[title={context.pushcatcodes(n) context.popcatcodes()}] + +These commands switch to another catcode regime and back. They have to be used +in pairs. Only the regimes atthe \LUA\ end are set. + +\stopsummary + +\startsummary[title={context.unprotect() context.protect()}] + +These commands switch to the protected regime and back. They have to be used in +pairs. Beware: contrary to what its name suggests, the \type {unprotect} enables +the protected regime. These functions also issue an \type {\unprotect} and \type +{\protect} equivalent at the \TEX\ end. + +\stopsummary + +\startsummary[title={context.verbatim context.puretext}] + +The differences between these are subtle: + +\startbuffer +\startluacode + context.verbatim.bold("Why do we use $ for math?") context.par() + context.verbatim.bold("Why do we use { as start?") context.par() + context.verbatim.bold("Why do we use } as end?") context.par() + context.puretext.bold("Why do we use {\\bi $} at all?") +\stopluacode +\stopbuffer + +\typebuffer + +Verbatim makes all characters letters while pure text leaves the backslash and +curly braces special. + +\startpacked \getbuffer \stoppacked + +\stopsummary + +\startsummary[title={context.protected}] + +The protected namespace is only used for commands that are in the \CONTEXT\ +private namespace. + +\stopsummary + +\startsummary[title={context.escaped(str) context.escape(str)}] + +The first command pipes the escaped string to \TEX, while the second one just +returns an unescaped string. The characters \typ {# $ % \ \ { }} are escaped. + +\stopsummary + +\startsummary[title={context.startcollecting() context.stopcollecting()}] + +These two commands will turn flushing to \TEX\ into collecting. This can be handy +when you want to interface commands that grab arguments using delimiters and as +such they are used deep down in some table related interfacing. You probably +don't need them. + +\stopsummary + +\stopsection + +\startsection[title={Templates}] + +In addition to the regular template mechanism (part of the utilities) there is a +dedicated template feature in the \type {context} namespace. An example demonstrates +its working: + +\startbuffer +\startluacode + local MyTable = [[ + \bTABLE + \bTR + \bTD \bf %one_first% \eTD + \bTD %[one_second]% \eTD + \eTR + \bTR + \bTD \bf %two_first% \eTD + \bTD %[two_second]% \eTD + \eTR + \eTABLE + ]] + + context.templates[MyTable] { + one_first = "one", + two_first = "two", + one_second = "just one $", + two_second = "just two $", + } +\stopluacode +\stopbuffer + +\typebuffer + +This renders: + +\startlinecorrection + \getbuffer +\stoplinecorrection + +You can also use more complex tables. Watch the space before and after the keys: + +\startbuffer +\startluacode + local MyOtherTable = [[ + \bTABLE + \bTR + \bTD \bf % ['one']['first'] % \eTD + \bTD %[ ['one']['second'] ]% \eTD + \eTR + \bTR + \bTD \bf % ['two']['first'] % \eTD + \bTD %[ ['two']['second'] ]% \eTD + \eTR + \eTABLE + ]] + + local data = { + one = { first = "one", second = "only 1$" }, + two = { first = "two", second = "only 2$" }, + } + + context.templates[MyOtherTable](data) + + context.templates(MyOtherTable,data) +\stopluacode +\stopbuffer + +\typebuffer + +We get: + +\startlinecorrection + \getbuffer +\stoplinecorrection + +\stopsection + +\startsection[title={Management}] + +\startsummary[title={context.functions}] + +This is private table that hosts managament of functions. You'd better leave this +one alone! + +\stopsummary + +\startsummary[title={context.nodes}] + +Normally you will just use \type {context(<somenode>)} to flush a node and this +private table is more for internal use. + +\stopsummary + +\stopsection + +\startsection[title=String handlers] + +These two functions implement handlers that split a given string into lines and +do something with it. We stick to showing their call. They are used for special +purpose flushing, like flushing content to \TEX\ in commands discussed here. The +\XML\ subsystem also used a couple of dedicated handlers. + +\starttyping +local foo = newtexthandler { + content = function(s) ... end, + endofline = function(s) ... end, + emptyline = function(s) ... end, + simpleline = function(s) ... end, +} + +local foo = newverbosehandler { + line = function(s) ... end, + space = function(s) ... end, + content = function(s) ... end, + before = function() ... end, + after = function() ... end, +} +\stoptyping + +\startsummary[title={context.printlines(str)}] + +The low level \type {tex.print} function pipes its content to \TEX\ and thereby +terminates at at \type {\r} (cariage return, \ASCII\ 13), although it depends on +the way catcodes and line endings are set up. In fact, a line ending in \TEX\ is +not really one, as it gets replaced by a space. Only several lines in succession +indicate a new paragraph. + +\startbuffer +\startluacode + tex.print("line 1\n line 2\r line 3") +\stopluacode +\stopbuffer + +\typebuffer + +This renders only two lines: + +\getbuffer + +\startbuffer +\startluacode + context("line 1\n line 2\r line 3") +\stopluacode +\stopbuffer + +However, the \type {context} command gives all three lines: + +\typebuffer + +Like: + +\getbuffer + +The \type {context.printlines} command is a direct way to print a string in a way +similar to reading from a file. So, + +\starttyping +tex.print(io.loaddata(resolvers.findfile("tufte"))) +\stoptyping + +Gives one line, while: + +\starttyping +context.printlines(io.loaddata(resolvers.findfile("tufte"))) +\stoptyping + +gives them all, as does: + +\starttyping +context(io.loaddata(resolvers.findfile("tufte"))) +\stoptyping + +as does a na\"ive: + +\starttyping +tex.print((string.gsub(io.loaddata(resolvers.findfile("tufte")),"\r","\n"))) +\stoptyping + +But, because successive lines need to become paragraph separators as bit more +work is needed and that is what \type {printlines} and \type {context} do for +you. However, a more convenient alternative is presented next. + +\stopsummary + +\startsummary[title={context.loadfile(name)}] + +This function locates and loads the file with the given name. The leading and +trailing spaces are stripped. + +\stopsummary + +\startsummary[title={context.runfile(name)}] + +This function locates and processes the file with the given name. The assumption +is that it is a valid \LUA\ file! When no suffix is given, the suffix \type {cld} +(\CONTEXT\ \LUA\ document) is used. + +\stopsummary + +\startsummary[title={context.viafile(data[,tag])}] + +The \type {data} is saved to a (pseudo) file with the optional name \type {tag} +and read in again from that file. This is a robust way to make sure that the data +gets processed like any other data read from file. It permits all kind of +juggling with catcodes, verbatim and alike. + +\stopsummary + +\stopsection + +\startsection[title={Helpers}] + +\startsummary[title={context.tocontext(variable)}] + +For documentation or tracing it can be handy to serialize a variable. The \type +{tocontext} function does this: + +\starttyping +context.tocontext(true) +context.tocontext(123) +context.tocontext("foo") +context.tocontext(tonumber) +context.tocontext(nil) +context.tocontext({ "foo", "bar" },true) +context.tocontext({ this = { foo , "bar" } },true) +\stoptyping + +Beware, \type {tocontext} is also a table that you can assign to, but that might +spoil serialization. This property makes it possible to extend the serializer. + +\stopsummary + +\startsummary[title={context.tobuffer(name,str[,catcodes])}] + +With this function you can put content in a buffer, optionally under a catcode +regime. + +\stopsummary + +\startsummary[title={context.tolines(str[,true])}] + +This function splits the string in lines and flushes them one by one. When the +second argument is \type {true} leading and trailing spaces are stripped. Each +flushed line always gets one space appended. + +\stopsummary + +\startsummary[title={context.fprint([regime,]fmt,...),tex.fprint([regime,]fmt,...)}] + +The \type {tex.fprint} is just there to complement the other flushers in the +\type {tex} namespace and therefore we also have it in the \type {context} +namespace. + +\stopsummary + +\stopsection + +\startsection[title=Tracing] + +\startsummary[title={context.settracing(true or false))}] + +You can trace the \TEX\ code that is generated at the \TEX\ end with: + +\starttyping +\enabletrackers[context.trace] +\stoptyping + +The \LUA\ function sets the tracing from the \LUA\ end. As the \type {context} +command is used a lot in the core, you can expect some more tracing that the code +that you're currently checking. + +\stopsummary + +\startsummary[title={context.pushlogger(fnc) context.poplogger() context.getlogger()}] + +You can provide your own logger if needed. The pushed function receives one string +argument. The getter returns three functions: + +\starttyping +local flush, writer, flushdirect = context.getlogger() +\stoptyping + +The \type{flush} function is similar to \type {tex.sprint} and appends its +arguments, while \type {flushdirect} treats each argument as a line and behaves +like \type {tex.print}. The \type {flush} function adds braces and paranthesis +around its arguments, apartt from the first one, which is considered to be a +command. Examples are: + +\starttyping +flush("one",2,"three") -- catcode, strings|numbers +writer("\\color",{"red"},"this is red") +\stoptyping + +and: + +\starttyping +flush(context.catcodes.verbatim,"one",2,"three") +writer(context.catcodes.verbatim,"\\color",{"red"},"this is red") +\stoptyping + +\stopsummary + +\stopsection + +\startsection[title=States] + +There are several ways to implement alternative code paths in \CONTEXT\ but modes +and conditionals are used mostly. There area few helpers for that. + +\startsummary[title={context.conditionals context.setconditional(name,value)}] + +Conditionals are used to keep a state. You can set their value using the setter, +but their effect is not immediate but part of the current sequence of commands +which is delegated to \TEX. However, you can easily keep track of your state +at the \LUA\ end with an extra boolean. So, after + +\starttyping +if context.conditionals.whatever then + context.setconditional("dothis",false) +else + context.setconditional("dothat",true) +end +\stoptyping + +the value of \type {dothis} and \type {dothat} conditions are not yet set in +\LUA. + +\stopsummary + +\startsummary[title={context.modes context.setmode(name,value)}] + +As with conditionals, you can (re)set the modes in \LUA\ but their values +get changes as part of the command sequence which is delayed till after the +\LUA\ call. + +\stopsummary + +\startsummary[title={context.systemmodes context.setsystemmode(name,value)}] + +The same applies as for regular modes. + +\stopsummary + +\startsummary[title={context.trialtypesetting()}] + +This function returns \type {true} if we're in trial typesetting mode (used when +for instance prerolling a table). + +\stopsummary + +\stopsection + +\startsection[title=Steps] + +The stepper permits stepwise processing of \CONTEXT\ code: after a step contyrol +gets delegated to \CONTEXT\ and afterwards back to \LUA. There main limitation of +this mechanism is that it cannot exceed the number of input levels. + +\startsummary[title={context.stepwise() context.step([str])}] + +Usage is as follows: + +\starttyping +context.stepwise (function() + ... + context.step(...) + ... + context.step(...) + ... + context.stepwise (function() + ... + context.step(...) + ... + context.step(...) + ... + end) + ... + context.step(...) + ... + context.step(...) + ... +end) +\stoptyping + +\stopsummary + +\stopsection + +\stopchapter + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-titlepage.tex b/doc/context/sources/general/manuals/cld/cld-titlepage.tex new file mode 100644 index 000000000..926a98952 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-titlepage.tex @@ -0,0 +1,14 @@ +\startcomponent cld-titlepage + +\environment cld-environment + +% \cldprocessfile{cld-mkiv-titlepage.cld} + +\startTEXpage + \externalfigure[cld-mkiv-titlepage.pdf]% +\stopTEXpage % faster during writing + +\startstandardmakeup[doublesided=no,page=no] +\stopstandardmakeup + +\stopcomponent diff --git a/doc/context/sources/general/manuals/cld/cld-verbatim.tex b/doc/context/sources/general/manuals/cld/cld-verbatim.tex new file mode 100644 index 000000000..2007f7d73 --- /dev/null +++ b/doc/context/sources/general/manuals/cld/cld-verbatim.tex @@ -0,0 +1,470 @@ +% language=uk + +\startcomponent cld-verbatim + +\environment cld-environment + +\startchapter[title=Verbatim] + +\startsection[title=Introduction] + +\index{verbatim} + +If you are familiar with traditional \TEX, you know that some characters have +special meanings. For instance a \type {$} starts and ends inline math mode: + +\starttyping +$e=mc^2$ +\stoptyping + +If we want to typeset math from the \LUA\ end, we can say: + +\starttyping +context.mathematics("e=mc^2") +\stoptyping + +This is in fact: + +\starttyping +\mathematics{e=mc^2} +\stoptyping + +However, if we want to typeset a dollar and use the \type {ctxcatcodes} regime, +we need to explicitly access that character using \type {\char} or use a command +that expands into the character with catcode other. + +One step further is that we typeset all characters as they are and this is called +verbatim. In that mode all characters are tokens without any special meaning. + +\stopsection + +\startsection[title=Special treatment] + +The formula in the introduction can be typeset verbatim as follows: + +\startbuffer +context.verbatim("$e=mc^2$") +\stopbuffer + +\typebuffer + +This gives: + +\ctxluabuffer + +You can also do things like this: + +\startbuffer +context.verbatim.bold("$e=mc^2$") +\stopbuffer + +\typebuffer + +Which gives: + +\ctxluabuffer + +So, within the \type {verbatim} namespace, each command gets its arguments +verbatim. + +\startbuffer +context.verbatim.inframed({ offset = "0pt" }, "$e=mc^2$") +\stopbuffer + +\typebuffer + +Here we get: \ctxluabuffer. So, settings and alike are processed as if the user +had used a regular \type {context.inframed} but the content comes out verbose. + +If you wonder why verbatim is needed as we also have the \type {type} function +(macro) the answer is that it is faster, easier to key in, and sometimes the only +way to get the desired result. + +\stopsection + +\startsection[title=Multiple lines] + +Currently we have to deal with linebreaks in a special way. This is due to the +way \TEX\ deals with linebreaks. In fact, when we print something to \TEX, the +text after a \type {\n} is simply ignored. + +For this reason we have a few helpers. If you want to put something in a buffer, +you cannot use the regular buffer functions unless you make sure that they are +not overwritten while you're still at the \LUA\ end. + +\starttyping +context.tobuffer("temp",str) +context.getbuffer("temp") +\stoptyping + +Another helper is the following. It splits the string into lines and feeds them +piecewise using the \type {context} function and in the process adds a space at +the end of the line (as this is what \TEX\ normally does. + +\starttyping +context.tolines(str) +\stoptyping + +Catcodes can get in the way when you pipe something to \TEX\ that itself changes +the catcodes. This happens for instance when you write buffers that themselves +have buffers or have code that changes the line endings as with \type +{startlines}. In that case you need to feed back the content as if it were a +file. This is done with: + +\starttyping +context.viafile(str) +\stoptyping + +The string can contain newlines. The string is written to a virtual file that is +input. Currently names looks like \type {virtual://virtualfile.1} but future +versions might have a different name part, so best use the variable instead. +After all, you don't know the current number in advance anyway. + +\stopsection + +\startsection[title=Pretty printing] + +In \CONTEXT\ \MKII\ there have always been pretty printing options. We needed it +for manuals and it was also handy to print sources in the same colors as the +editor uses. Most of those pretty printers work in a line|-|by|-|line basis, but +some are more complex, especially when comments or strings can span multiple +lines. + +When the first versions of \LUATEX\ showed up, rewriting the \MKII\ code to use +\LUA\ was a nice exercise and the code was not that bad, but when \LPEG\ showed +up, I put it on the agenda to reimplement them again. + +We only ship a few pretty printers. Users normally have their own preferences and +it's not easy to make general purpose pretty printers. This is why the new +framework is a bit more flexible and permits users to kick in their own code. + +Pretty printing involves more than coloring some characters or words: + +\startitemize[packed] +\startitem spaces should honoured and can be visualized \stopitem +\startitem newlines and empty lins need to be honoured as well \stopitem +\startitem optionally lines have to be numbered but \stopitem +\startitem wrapped around lines should not be numbered \stopitem +\stopitemize + +It's not much fun to deal with these matters each time that you write a pretty +printer. This is why we can start with an existing one like the default pretty +printer. We show several variants of doing the same. We start with a simple clone +of the default parser. \footnote {In the meantime the lexer of the \SCITE\ editor +that I used also provides a mechanism for using \LPEG\ based lexers. Although in +the pretty printing code we need a more liberal one I might backport the lexers I +wrote for editing \TEX, \METAPOST, \LUA, \CLD, \XML\ and \PDF\ as a variant for +the ones we use in \MKIV\ now. That way we get similar colorschemes which might +be handy sometimes.} + +\startbuffer +local P, V = lpeg.P, lpeg.V + +local grammar = visualizers.newgrammar("default", { + pattern = V("default:pattern"), + visualizer = V("pattern")^1 +} ) + +local parser = P(grammar) + +visualizers.register("test-0", { parser = parser }) +\stopbuffer + +\typebuffer \ctxluabuffer + +We distinguish between grammars (tables with rules), parsers (a grammar turned +into an \LPEG\ expression), and handlers (collections of functions that can be +applied. All three are registered under a name and the verbatim commands can +refer to that name. + +\startbuffer +\starttyping[option=test-0,color=] +Test 123, +test 456 and +test 789! +\stoptyping +\stopbuffer + +\typebuffer + +Nothing special happens here. We just get straightforward verbatim. + +\getbuffer + +Next we are going to color digits. We collect as many as possible in a row, so +that we minimize the calls to the colorizer. + +\startbuffer +local patterns, P, V = lpeg.patterns, lpeg.P, lpeg.V + +local function colorize(s) + context.color{"darkred"} + visualizers.writeargument(s) +end + +local grammar = visualizers.newgrammar("default", { + digit = patterns.digit^1 / colorize, + pattern = V("digit") + V("default:pattern"), + visualizer = V("pattern")^1 +} ) + +local parser = P(grammar) + +visualizers.register("test-1", { parser = parser }) +\stopbuffer + +\typebuffer \ctxluabuffer + +Watch how we define a new rule for the digits and overload the pattern rule. We +can refer to the default rule by using a prefix. This is needed when we define a +rule with the same name. + +\startbuffer +\starttyping[option=test-1,color=] +Test 123, +test 456 and +test 789! +\stoptyping +\stopbuffer + +\typebuffer + +This time the digits get colored. + +\getbuffer + +In a similar way we can colorize letters. As with the previous example, we use +\CONTEXT\ commands at the \LUA\ end. + +\startluacode +local patterns, P, V = lpeg.patterns, lpeg.P, lpeg.V + +local function colorize_lowercase(s) + context.color{"darkgreen"} + visualizers.writeargument(s) +end +local function colorize_uppercase(s) + context.color{"darkblue"} + visualizers.writeargument(s) +end + +local grammar = visualizers.newgrammar("default", { + + lowercase = patterns.lowercase^1 / colorize_lowercase, + uppercase = patterns.uppercase^1 / colorize_uppercase, + + pattern = + V("lowercase") + + V("uppercase") + + V("default:pattern"), + + visualizer = V("pattern")^1 + +} ) + +local parser = P(grammar) + +visualizers.register("test-2", { parser = parser }) +\stopluacode + +\startbuffer +\starttyping[option=test-2,color=] +Test 123, +test 456 and +test 789! +\stoptyping +\stopbuffer + +\typebuffer + +Again we get some coloring. + +\getbuffer + +It will be clear that the amount of rules and functions is larger when we use a +more complex parser. It is for this reason that we can group functions in +handlers. We can also make a pretty printer configurable by defining handlers at +the \TEX\ end. + +\startbuffer +\definestartstop + [MyDigit] + [style=bold,color=darkred] + +\definestartstop + [MyLowercase] + [style=bold,color=darkgreen] + +\definestartstop + [MyUppercase] + [style=bold,color=darkblue] +\stopbuffer + +\typebuffer \getbuffer + +The \LUA\ code now looks different. Watch out: we need an indirect call to for +instance \type {MyDigit} because a second argument can be passed: the settings +for this environment and you don't want that get passed to \type {MyDigit} and +friends. + +\startluacode +local patterns, P, V = lpeg.patterns, lpeg.P, lpeg.V +local pattern = visualizers.pattern +local verbatim = context.verbatim + +local MyDigit = verbatim.MyDigit +local MyLowercase = verbatim.MyLowercase +local MyUppercase = verbatim.MyUppercase + +-- local handler = newhandler("default, { +-- digit = function(s) MyDigit (s) end, +-- lowercase = function(s) MyLowercase(s) end, +-- uppercase = function(s) MyUppercase(s) end, +-- } ) + +local handler = { + digit = function(s) MyDigit (s) end, + lowercase = function(s) MyLowercase(s) end, + uppercase = function(s) MyUppercase(s) end, +} + +local grammar = visualizers.newgrammar("default", { + + digit = pattern(handler,"digit", patterns.digit ^1), + lowercase = pattern(handler,"lowercase", patterns.lowercase^1), + uppercase = pattern(handler,"uppercase", patterns.uppercase^1), + + pattern = + V("lowercase") + + V("uppercase") + + V("digit") + + V("default:pattern"), + + visualizer = V("pattern")^1 + +} ) + +local parser = P(grammar) + +visualizers.register("test-3", { parser = parser, handler = handler }) +\stopluacode + +\startbuffer +\starttyping[option=test-3,color=] +Test 123, +test 456 and +test 789! +\stoptyping +\stopbuffer + +\typebuffer + +We get digits, upper- and lowercase characters colored: + +\getbuffer + +You can also use parsers that don't use \LPEG: + +\startbuffer +local function parser(s) + visualizers.write("["..s.."]") +end + +visualizers.register("test-4", { parser = parser }) +\stopbuffer + +\typebuffer \ctxluabuffer + +\startbuffer +\starttyping[option=test-4,space=on,color=darkred] +Test 123, +test 456 and +test 789! +\stoptyping +\stopbuffer + +\typebuffer + +The function \type {visualizer.write} takes care of spaces and newlines. + +\getbuffer + +We have a few more helpers: + +\starttabulate[|||] +\NC \type{visualizers.write} \NC interprets the argument and applies methods \NC \NR +\NC \type{visualizers.writenewline} \NC goes to the next line (similar to \type {\par} \NC \NR +\NC \type{visualizers.writeemptyline} \NC inserts an empty line (similer to \type {\blank} \NC \NR +\NC \type{visualizers.writespace} \NC inserts a (visible) space \NC \NR +\NC \type{visualizers.writedefault} \NC writes the argument verbatim without interpretation \NC \NR +\stoptabulate + +These mechanism have quite some overhead in terms of function calls. In the worst +case each token needs a (nested) call. However, doing all this at the \TEX\ end +also comes at a price. So, in practice this approach is more flexible but without +too large a penalty. + +In all these examples we typeset the text verbose: what is keyed in normally +comes out (either or not with colors), so spaces stay spaces and linebreaks are +kept. + +\startbuffer +local function parser(s) + local s = string.gsub(s,"show","demonstrate") + local s = string.gsub(s,"'re"," are") + context(s) +end + +visualizers.register("test-5", { parser = parser }) +\stopbuffer + +\typebuffer \ctxluabuffer + +\startbuffer +\starttyping[option=test-5,color=darkred,style=] +This is just some text to show what we can do with this mechanism. In +spite of what you might think we're not bound to verbose text. +\stoptyping +\stopbuffer + +We can apply this visualizer as follows: + +\typebuffer + +This time the text gets properly aligned: + +\getbuffer + +It often makes sense to use a buffer: + +\startbuffer +\startbuffer[demo] +This is just some text to show what we can do with this mechanism. In +spite of what you might think we're not bound to verbose text. +\stopbuffer +\stopbuffer + +\typebuffer \getbuffer + +Instead of processing the buffer in verbatim mode you can then +process it directly: + +\startbuffer +\setuptyping[file][option=test-5,color=darkred,style=] +\ctxluabuffer[demo] +\stopbuffer + +\typebuffer + +Which gives: + +\start \getbuffer \stop + +In this case, the space is a normal space and not the fixed verbatim space, which +looks better. + +\stopsection + +\stopchapter + +\stopcomponent |