% language=uk % author : Hans Hagen, PRAGMA ADE, NL % license : Creative Commons, Attribution-NonCommercial-ShareAlike 3.0 Unported \usemodule[abr-02] \setuplayout [width=middle, height=middle, backspace=2cm, topspace=1cm, footer=0pt, bottomspace=2cm] \definecolor [DocumentColor] [r=.5] \setuptype [color=DocumentColor] \setuptyping [color=DocumentColor] \usetypescript [iwona] \setupbodyfont [iwona] \setuphead [chapter] [style=\bfc, color=DocumentColor] \setuphead [section] [style=\bfb, color=DocumentColor] \setupinteraction [hidden] \setupwhitespace [big] \setupheadertexts [] \setupheadertexts [] [{\DocumentColor \type {luatools mtxrun context}\quad\pagenumber}] \usesymbols[cc] \def\sTEXMFSTART{\type{texmfstart}} \def\sLUATOOLS {\type{luatools}} \def\sMTXRUN {\type{mtxrun}} \def\sCONTEXT {\type{context}} \def\sKPSEWHICH {\type{kpsewhich}} \def\sMKTEXLSR {\type{mktexlsr}} \def\sXSLTPROC {\type{xsltproc}} \usemodule[narrowtt] \startdocument [metadata:author=Hans Hagen, metadata:title={Tools: luatools, mtxrun, context}, author=Hans Hagen, affiliation=PRAGMA ADE, location=Hasselt NL, title=Tools, extra-1=luatools, extra-2=mtxrun, extra-3=context, support=www.contextgarden.net, website=www.pragma-ade.nl] \startMPpage StartPage ; picture p ; p := image ( for i=1 upto 21 : for j=1 upto 30 : drawarrow (fullcircle rotated uniformdeviate 360) scaled 10 shifted (i*15,j*15) ; endfor ; endfor ; ) ; p := p ysized (bbheight(Page)-4mm) ; fill Page enlarged 2mm withcolor \MPcolor{DocumentColor} ; draw p shifted (center Page - center p) withpen pencircle scaled 2 withcolor .5white ; numeric dx ; dx := bbwidth(Page)/21 ; numeric dy ; dy := bbheight(Page)/30 ; p := textext("\tt\bf\white\getvariable{document}{extra-1}") xsized(14*dx) ; p := p shifted (-lrcorner p) shifted lrcorner Page shifted (-1dx,8dy) ; draw p ; p := textext("\tt\bf\white\getvariable{document}{extra-2}") xsized(14*dx) ; p := p shifted (-lrcorner p) shifted lrcorner Page shifted (-1dx,5dy) ; draw p ; p := textext("\tt\bf\white\getvariable{document}{extra-3}") xsized(14*dx) ; p := p shifted (-lrcorner p) shifted lrcorner Page shifted (-1dx,2dy) ; draw p ; setbounds currentpicture to Page ; StopPage \stopMPpage \startsubject[title=Contents] \placelist[section][alternative=a] \stopsubject \startsection[title={Remark}] This manual is work in progress. Feel free to submit additions or corrections. Before you start reading, it is good to know that in order to get starting with \CONTEXT, the easiest way to do that is to download the standalone distribution from \type {contextgarden.net}. After that you only need to make sure that \type {luatex} is in your path. The main command you use is then \type {context} and normally it does all the magic it needs itself. \stopsection \startsection[title={Introduction}] Right from the start \CONTEXT\ came with programs that managed the process of \TEX-ing. Although you can perfectly well run \TEX\ directly, it is a fact that often multiple runs are needed as well as that registers need to be sorted. Therefore managing a job makes sense. First we had \TEXEXEC\ and \TEXUTIL, and both were written in \MODULA, and as this language was not supported on all platforms the programs were rewritten in \PERL. Following that a few more tools were shipped with \CONTEXT. When we moved on to \RUBY\ all the \PERL\ scripts were rewritten and when \CONTEXT\ \MKIV\ showed up, \LUA\ replaced \RUBY. As we use \LUATEX\ this means that currently the tools and the main program share the same language. For \MKII\ scripts like \TEXEXEC\ will stay around but the idea is that there will be \LUA\ alternatives for them as well. Because we shipped many scripts, and because the de facto standard \TEX\ directory structure expects scripts to be in certain locations we not only ship tools but also some more generic scripts that locate and run these tools. \stopsection \startsection[title={The location}] Normally you don't need to know so many details about where the scripts are located but here they are: \starttyping /scripts/context/perl /scripts/context/ruby /scripts/context/lua /scripts/context/stubs \stoptyping This hierarchy was actually introduced because \CONTEXT\ was shipped with a bunch of tools. As mentioned, we nowadays focus on \LUA\ but we keep a few of the older scripts around in the \PERL\ and \RUBY\ paths.Now, if you're only using \CONTEXT\ \MKIV, and this is highly recommended, you can forget about all but the \LUA\ scripts. \stopsection \startsection[title={The traditional finder}] When you run scripts multiple times, and in the case of \CONTEXT\ they are even run inside other scripts, you want to minimize the startup time. Unfortunately the traditional way to locate a script, using \sKPSEWHICH, is not that fast, especially in a setup with many large trees Also, because not all tasks can be done with the traditional scripts (take format generation) we provided a runner that could deal with this: \sTEXMFSTART. As this script was also used in more complex workflows, it had several tasks: \startitemize[packed] \item locate scripts in the distribution and run them using the right interpreter \item do this selectively, for instance identify the need for a run using checksums for potentially changed files (handy for image conversion) \item pass information to child processes so that lookups are avoided \item choose a distribution among several installed versions (set the root of the \TEX\ tree) \item change the working directory before running the script \item resolve paths and names on demand and launch programs with arguments where names are expanded controlled by prefixes (handy for \TEX-unware programs) \item locate and open documentation, mostly as part the help systems in editors, but also handy for seeing what configuration file is used \item act as a \KPSEWHICH\ server cq.\ client (only used in special cases, and using its own database) \stopitemize Of course there were the usual more obscure and undocumented features as well. The idea was to use this runner as follows: \starttyping texmfstart texexec texmfstart --tree texexec \stoptyping These are just two ways of calling this program. As \sTEXMFSTART\ can initialize the environment as well, it is basically the only script that has to be present in the binary path. This is quite comfortable as this avoids conflicts in names between the called scripts and other installed programs. Of course calls like above can be wrapped in a shell script or batch file without penalty as long as \sTEXMFSTART\ itself is not wrapped in a caller script that applies other inefficient lookups. If you use the \CONTEXT\ minimals you can be sure that the most efficient method is chosen, but we've seen quite inefficient call chains elsewhere. In the \CONTEXT\ minimals this script has been replaced by the one we will discuss in the next section: \sMTXRUN\ but a stub is still provided. \stopsection \startsection[title={The current finder}] In \MKIV\ we went a step further and completely abandoned the traditional lookup methods and do everything in \LUA. As we want a clear separation between functionality we have two main controlling scripts: \sMTXRUN\ and \sLUATOOLS. The last name may look somewhat confusing but the name is just one on in a series. \footnote {We have \type {ctxtools}, \type {exatools}, \type {mpstools}, \type {mtxtools}, \type {pdftools}, \type {rlxtools}, \type {runtools}, \type {textools}, \type {tmftools} and \type {xmltools}. Most if their funtionality is already reimplemented.} In \MKIV\ the \sLUATOOLS\ program is nowadays seldom used. It's just a drop in for \sKPSEWHICH\ plus a bit more. In that respect it's rather dumb in that it does not use the database, but clever at the same time because it can make one based on the little information available when it runs. It can also be used to generate format files either or not using \LUA\ stubs but in practice this is not needed at all. For \CONTEXT\ users, the main invocation of this tool is when the \TEX\ tree is updated. For instance, after adding a font to the tree or after updating \CONTEXT, you need to run: \starttyping mtxrun --generate \stoptyping After that all tools will know where to find stuff and how to behave well within the tree. This is because they share the same code, mostly because they are started using \sMTXRUN. For instance, you process a file with: \starttyping mtxrun --script context \stoptyping Because this happens often, there's also a shortcut: \starttyping context \stoptyping But this does use \sMTXRUN\ as well. The help information of \sMTXRUN\ is rather minimalistic and if you have no clue what an option does, you probably never needed it anyway. Here we discuss a few options. We already saw that we can explicitly ask for a script: \starttyping mtxrun --script context \stoptyping but \starttyping mtxrun context \stoptyping also works. However, by using \type {--script} you limit te lookup to the valid \CONTEXT\ \MKIV\ scripts. In the \TEX\ tree these have names prefixed by \type {mtx-} and a lookup look for a plural as well. So, the next two lookups are equivalent: \starttyping mtxrun --script font mtxrun --script fonts \stoptyping Both will run \type {mtx-fonts.lua}. Actually, this is one of the scripts that you might need when your font database is somehow outdated and not updated automatically: \starttyping mtxrun --script fonts --reload --force \stoptyping Normally \sMTXRUN\ is all you need in order to run a script. However, there are a few more options: \ctxlua{os.execute("mtxrun > tools-mkiv-help.tmp")} \typefile[ntyping]{tools-mkiv-help.tmp} Don't worry,you only need those obscure features when you integrate \CONTEXT\ in for instance a web service or when you run large projects where runs and paths take special care. \stopsection \startsection[title={Updating}] There are two ways to update \CONTEXT\ \MKIV. When you manage your trees yourself or when you use for instance \TEXLIVE, you act as follows: \startitemize[packed] \item download the file cont-tmf.zip from \type {www.pragma-ade.com} or elsewhere \item unzip this file in a subtree, for instance \type {tex/texmf-local} \item run \type {mtxrun --generate} \item run \type {mtxrun --script font --reload} \item run \type {mtxrun --script context --make} \stopitemize Or shorter: \startitemize[packed] \item run \type {mtxrun --generate} \item run \type {mtxrun font --reload} \item run \type {context --make} \stopitemize Normally these commands are not even needed, but they are a nice test if your tree is still okay. To some extend \sCONTEXT\ is clever enough to decide if the databases need to be regenerated and|/|or a format needs to be remade and|/|or if a new font database is needed. Now, if you also want to run \MKII, you need to add: \startitemize[packed] \item run \type {mktexlsr} \item run \type {texexec --make} \stopitemize The question is, how to act when \sLUATOOLS\ and \sMTXRUN\ have been updated themselves? In that case, after unzipping the archive, you need to do the following: \startitemize[packed] \item run \type {luatools --selfupdate} \item run \type {mtxrun --selfupdate} \stopitemize For quite a while we shipped so called \CONTEXT\ minimals. These zip files contained only the resources and programs that made sense for running \CONTEXT. Nowadays the minimals are installed and synchronized via internet. \footnote {This project was triggered by Mojca Miklavec who is also charge of this bit of the \CONTEXT\ infrastructure. More information can be found at \type {contextgarden.net}.} You can just run the installer again and no additional commands are needed. In the console you will see several calls to \sMTXRUN\ and \sLUATOOLS\ fly by. \stopsection \startsection[title={The tools}] We only mention the tools here. The most important ones are \sCONTEXT\ and \type {fonts}. You can ask for a list of installed scripts with: \starttyping mtxrun --script \stoptyping On my machine this gives: \ctxlua{os.execute("mtxrun --script > tools-mkiv-help.tmp")} \typefile[ntyping]{tools-mkiv-help.tmp} The most important scripts are \type {mtx-fonts} and \type {mtx-context}. By default fonts are looked up by filename (the \type {file:} prefix before font names in \CONTEXT\ is default). But you can also lookup fonts by name (\type {name:}) or by specification (\type {spec:}). If you want to use these two methods, you need to generate a font database as mentioned in the previous section. You can also use the font tool to get information about the fonts installed on your system. \stopsection \startsection[title={Running \CONTEXT}] The \sCONTEXT\ tool is what you will use most as it manages your run. \ctxlua{os.execute("context > tools-mkiv-help.tmp")} \typefile[ntyping]{tools-mkiv-help.tmp} There are few exert options too: \ctxlua{os.execute("context --expert > tools-mkiv-help.tmp")} \typefile[ntyping]{tools-mkiv-help.tmp} You might as well forget about these unless you are one of the \CONTEXT\ developers. \stopsection \startsection[title={Prefixes}] A handy feature of \sMTXRUN\ (and as most features an inheritance of \sTEXMFSTART) is that it will resolve prefixed arguments. This can be of help when you run programs that are unaware of the \TEX\ tree but nevertheless need to locate files in it. \ctxlua{os.execute("mtxrun --prefixes > tools-mkiv-help.tmp")} \typefile[ntyping]{tools-mkiv-help.tmp} An example is: \starttyping mtxrun --execute xsltproc file:whatever.xsl file:whatever.xml \stoptyping The call to \sXSLTPROC\ will get two arguments, being the complete path to the files (given that it can be resolved). This permits you to organize the files in a similar was as \TEX\ files. \stopsection \startsection[title={Stubs}] As the tools are written in the \LUA\ language we need a \LUA\ interpreter and or course we use \LUATEX\ itself. On \UNIX\ we can copy \sLUATOOLS\ and \sMTXRUN\ to files in the binary path with the same name but without suffix. Starting them in another way is a waste of time, especially when \sKPSEWHICH\ is used to find then, something which is useless in \MKIV\ anyway. Just use these scripts directly as they are self contained. For \sCONTEXT\ and other scripts that we want convenient access to, stubs are needed, like: \starttyping #!/bin/sh mtxrun --script context "$@" \stoptyping This is also quite efficient as the \sCONTEXT\ script \type {mtx-context} is loaded in \sMTXRUN\ and uses the same database. On \WINDOWS\ you can copy the scripts as|-|is and associate the suffix with \LUATEX\ (or more precisely: \type {texlua}) but then all \LUA\ script will be run that way which is not what you might want. In \TEXLIVE\ stubs for starting scripts were introduced by Fabrice Popineau. Such a stub would start for instance \sTEXMFSTART, that is: it located the script (\PERL\ or \RUBY) in the \TEX\ tree and launched it with the right interpreter. Later we shipped pseudo binaries of \sTEXMFSTART: a \RUBY\ interpreter plus scripts wrapped into a self contained binary. For \MKIV\ we don't need such methods and started with simple batch files, similar to the \UNIX\ startup scripts. However, these have the disadvantage that they cannot be used in other batch files without using the \type {start} command. In \TEXLIVE\ this is taken care of by a small binary written bij T.M.\ Trzeciak so on \TEXLIVE\ 2009 we saw a call chain from \type {exe} to \type {cmd} to \type {lua} which is somewhat messy. This is why we now use an adapted and stripped down version of that program that is tuned for \sMTXRUN, \sLUATOOLS\ and \sCONTEXT. So, we moved from the original \type {cmd} based approach to an \type {exe} one. \starttyping mtxrun.dll mtxrun.exe \stoptyping You can copy \type {mtxrun.exe} to for instance \type {context.exe} and it will still use \sMTXRUN\ for locating the right script. It also takes care of mapping \sTEXMFSTART\ to \sMTXRUN. So we've removed the intermediate \type {cmd} step, can not run the script as any program, and most of all, we're as efficient as can be. Of course this program is only meaningful for the \CONTEXT\ approach to tools. It may all sound more complex than it is but once it works users will not notice those details. Als, in practice not that much has changed in running the tools between \MKII\ and \MKIV\ as we've seen no reason to change the methods. \stopsection \startsubject[title={Colofon}] \starttabulate[|B|p|] \NC author \NC \documentvariable{author}, \documentvariable{affiliation}, \documentvariable{location} \NC \NR \NC version \NC \currentdate \NC \NR \NC website \NC \documentvariable{website} \endash\ \documentvariable{support} \NC \NR \NC copyright \NC \symbol[cc][cc-by-sa-nc] \NC \NR \stoptabulate \stopsubject \stopdocument