manual, license, readme

3 files changed, 605 insertions, 0 deletions

diff --git a/doc/documentation.rst b/doc/documentation.rst
new file mode 100644
index 0000000..acb8b18
--- /dev/null
+++ b/doc/documentation.rst
@@ -0,0 +1,509 @@
+========================
+Features Not Implemented
+========================
+Nesting
+*******
+Proper nesting. So far only lists support real nested structures.
+There's no way you could have real paragraphs or bulleted lists
+inside table cells. The problem is that with true nesting some
+jobs like the dissection of tables would have to be moved from
+the formatter to the parser. If you feel you need thoroughly
+nested structures -- e.g. grid tables in footnotes or bullet lists
+inside simple tables inside enumerations inside quotations inside
+footnotes -- you should consider including |CONTEXT| code as
+substitution directives. (OTOH docutils' new and old LaTeX
+formatter seems to have problems with tables in footnotes as
+well. Not to mention its preference for enclosing random nested
+structures in ``quote``-environments.)
+
+Should you find yourself in desparate need of tables or whatever
+structures inside footnotes then I might agree to find a solution
+if you ask.
+
+Tabs
+****
+The |rst| specification requests that tabs (ASCII no 9) be
+treated as spaces_. Although the matching patterns should be
+neutral with respect to tabs, I never tested them, neither do I
+guarantee that they will work anywhere. Converting your tabs to
+spaces might be a good preparation for an |rstcontext| run.
+
+.. _spaces: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#whitespace
+
+Hyperlinks
+**********
+The hyperlink implementation should be fine in general use if you
+avoid certain situations.
+
+- Never ever call your hyperlink targets ``anon_#``, where ``#``
+  stands for  any integer. Just don't do it, OK? Great.
+
+- Referencing a structure element like a section heading by means
+  of an *empty link block* does work. However, if the element in
+  question requests a page break (e.g. the vanilla
+  ``\chapter{#1}`` command), the reference will link to the
+  previous page instead and become useless. You can avoid this
+  behaviour by referencing the section directly or by targetting
+  the first paragraph in the section instead.
+
+- Link chaining does not work with internal references. This is
+  considered a low-priority bug and will be addressed during the
+  next big hyperlink overhaul.
+
+Module
+******
+
+A module for MkIV is planned.  Actually, the converter was
+thought of as a module for direct rendering of |rst| input
+initially but certain objections diverted me from this path.
+
+-   *Typography*. It’s all about the details. No matter how good your
+    converter is, it still won’t reach |TEX|’s omnipotence and
+    flexibility. |rstcontext| is a tool to generate raw material
+    for your typesetting job, not a typesetting system in itself.
+
+-   *Testing*. Never underestimate the insights gained from reading
+    the resulting |CONTEXT| file. Quite some effort has been
+    undertaken to make it human-readable, especially the setups.
+
+-   *MkII*. I’m not an MkII user at all save for rapid testing and
+    the occasional check for the sanity of |CONTEXT|’s behaviour.
+    Slow hardware forces me to rund |PDFTEX| instead of |LUATEX|
+    whenvever I need some result as quick as possible, so I wanted
+    to keep the code MkII clean. Do not expect Unicode (as  in
+    this document) to work.
+
+During the development readability of the generated code was
+alway one of the main goals of |rstcontext|. Quite some computing
+effort is made to reflow even simple things as paragraphs into
+a shape understandable by more than only the |TEX| machine.
+If you should at one point decide that your project is
+ripe for the typographical finish and you want to add local
+changes in form of |TEX| code only, you should be able to use the
+output of |rstcontext| as starting point.
+
+=====
+Usage
+=====
+At the moment, |rstcontext| needs to be called as a separate
+program. It is written for use with the Lua interpreter of
+|LUATEX|, ``texlua``, whose libraries and extended capabilities
+it uses. Therefore, |rstcontext| might not run at all on other
+Lua installations, at least not without modification of the
+source. Fortunately, every |CONTEXT| user is equipped with
+|LUATEX| nowadays so this dependency should be trivial.
+
+To generate |CONTEXT| code from a |rst| document named
+``infile.rst``, call ``rst_parser.lua`` through ``texlua``: ::
+
+     $texlua rst_parser.lua infile.rst outfile.tex
+
+You should now have a file ``outfile.tex`` that is ready to be
+run by |CONTEXT|. With some exceptions the generated code is
+downward compatible with MkII, thus it does not matter for a
+start whether you decide to test it with ``texexec`` or
+``context``.
+
+The resulting |TEX| file has rather a basic layout, if at all.
+This is intentional as you are expected to include it in a
+document after your own setups.
+An example for prepended setups can be found in the environment
+for this manual (``doc/manual.tex``).
+
+.. caution::
+    The output of |rstcontext| automatically inserts necessary
+    setups for the components found in the input. Therefore, the
+    ``\starttext`` and ``\stoptext`` commands are part of the
+    output and may not be specified in your setups file.
+    For now you have to use the |CONTEXT| command 
+    ``\appendtoks <token> \to \starttext`` to add content like
+    title pages and indices to the result. This mechanism works
+    reliable as long as you have an eye on the order in which the
+    tokens are given. Again, have a look at ``manual.tex`` to get
+    an impression how useful this can be. User hooks for these
+    and other common constructs are thought of but have yet to be
+    implemented.
+
+To build the documentation, first ``cd`` to the root directory of the
+repository. Now run |rstcontext| as follows: ::
+
+    $texlua rst_parser.lua doc/documentation.rst doc/doc.tex 
+
+Then change to the ``doc`` directory and run |CONTEXT| on
+``manual.tex``. Voilà, you have successfully built ``manual.pdf``.
+
+=========
+Examples
+=========
+
+|rstcontext| was developed for the largest part by going through
+the |rst| specification_ step by step and tested against the
+examples given both in the spec and in the `quick reference`_.
+Therefore you should refer to those examples first (and drop me a
+note immediately if any of them stopped working).
+All kinds of text blocks and inline markup have been implemented
+with the exception of anything mentioned in the section on
+`Features Not Implemented`_.
+Some of them that I have not found a real-world usage for (such
+as *definition lists*) do not yet have a presentable output --
+there is room for improvements that should be supplied by
+somebody who actually uses those features.
+
+.. _specification: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
+.. _quick reference: http://docutils.sourceforge.net/docs/user/rst/quickref.html
+
+Block Quotes
+************
+
+The *block quote* syntax is fully supported, including
+attributions. For instance, the next snippet: ::
+
+    Some people have made the mistake of seeing Shunt’s work as a
+    load of rubbish about railway timetables, but clever people
+    like me, who talk loudly in restaurants, see this as a
+    deliberate ambiguity, a plea for understanding in a
+    mechanized world.
+
+    --- Gavin Millarrrrrrrrrr on Neville Shunt
+
+gets you a neatly indented quotation, typeset in a slightly
+smaller font magnitude.
+
+    Some people have made the mistake of seeing Shunt’s work as a
+    load of rubbish about railway timetables, but clever people
+    like me, who talk loudly in restaurants, see this as a
+    deliberate ambiguity, a plea for understanding in a
+    mechanized world.
+
+    --- Gavin Millarrrrrrrrrr on Neville Shunt
+
+Don’t forget proper indentation.
+
+Numbered List
+*************
+
+Save for nesting lists are fully implemented in |rstcontext|.
+The following code typesets a triple-nested list with different
+kinds of bulleting / numbering: ::
+
+    i   First order list, first entry.
+
+    ii  First order list, second entry.
+
+    iii First order list, third entry.
+
+        -   Second order list, first entry.
+
+            #   Third order list, first entry.
+
+            #   Third order list, second entry.
+
+            #   Third order list, third entry.
+                Real nesting rules!
+
+        -   Second order list, second entry.
+
+    iv  First order list, fourth entry.
+
+    v   First order list, fifth entry.
+
+The result looks like this:
+
+i   First order list, first entry.
+
+ii  First order list, second entry.
+
+iii First order list, third entry.
+
+    -   Second order list, first entry.
+
+        #   Third order list, first entry.
+        #   Third order list, second entry.
+        #   Third order list, third entry.
+            Real nesting rules!
+
+    -   Second order list, second entry.
+
+iv  First order list, fourth entry.
+
+v   First order list, fifth entry.
+
+.. caution:: 
+    Don’t forget the blank lines between list items.
+
+==========
+Directives
+==========
+Admonitions
+************
+The following admonition directives have been implemented:
+
+Caution
+-------
+The *caution* directive results in the text being prefixed by one
+“dangerous bend” symbol in order to resemble the “wizards only”
+passages of the TeXbook.
+For example, the directive: ::
+
+    .. caution:: White mice do worse in experiments than grey mice.
+
+will result in the following:
+
+.. caution:: White mice do worse in experiments than grey mice.
+
+Danger
+------
+Similar to the *caution* directive, the *danger* directive
+prefixes the given text with two “dangerous bends” giving it the
+look of Knuths’s “esoteric” annotations.
+
+.. danger:: Be nice to the parser: 
+    Don’t forget to align paragraphs that end a literal
+    block!
+
+
+Images
+******
+Including pictures is easy using the *image* directive: simply
+supply it the name of the image file as in ``.. image:: cow``.
+If the format is supported by |CONTEXT| the suffix can be
+neglected.
+
+The placement of images can be controlled via a set of optional
+arguments, each of which has to be specified on single line in
+``key: value`` style: ::
+
+    .. image:: cow
+        width: hsize
+        caption: A generic Dutch cow.
+
+This will place your image somewhere close to the spot where you
+defined it. (The placement parameter to ``placefigure`` will be
+set to ``here`` by default.)
+
+.. image:: cow
+    width: hsize
+    alt: A generic Dutch cow (*bos primigenius taurus*).
+
+The supported parameters are ``width``
+(alias: ``size``), ``caption`` and ``scale``.
+The *width* parameter accepts the values ``hsize`` 
+(alias: ``fit``, ``broad``) or ``normal``.
+Alternatively, the *scale* parameter allows for arbitrary
+manipulation of the desired magnification; it defaults to ``1``
+(unscaled).
+The value passed as *caption* parameter will be used in as the
+caption text of the image.
+
+.. |CONTEXT| ctx:: \CONTEXT
+.. |TEX| ctx:: \TeX
+.. |PDFTEX| ctx:: \PDFTEX
+.. |LUATEX| ctx:: \LUATEX
+.. |rstcontext| ctx:: {\em rst}\kern.5pt\CONTEXT
+.. |rst| ctx:: {\rm re}{\ss Structured}{\rm Text}
+.. |LATEX| ctx:: \LATEX
+
+.. _outline: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
+.. _docutils: http://docutils.sourceforge.net/
+.. _Pandoc: http://johnmacfarlane.net/pandoc/
+
+=======================
+Substitution Directives
+=======================
+
+There are substitution directives for simple *replacing* and
+for insertion of |LUATEX|’s three languages: |mp|, Lua and,
+of course, |TEX|.
+
+.. |mp| replace:: \METAPOST
+
+Ordinary text replacement is done via the ``replace``
+substitution directive. E.g. in the main text you consistently
+use ``|replaceme|`` and have all its occurences substituted by
+``I wasn’t in the mood to write out this long sentence.``
+like in the next snippet:
+
+::
+
+    .. |replaceme| replace::
+        I wasn’t in the mood to write out this long sentence.
+
+The code insertions work similarly. You have to specify some
+phrase that gets substituted by the code you supply.
+E.g. this document accesses the fancy logos predefined in the
+|CONTEXT| core via substitutions: ::
+    
+    .. |CONTEXT| ctx:: \CONTEXT
+    .. |LUATEX| ctx:: \LUATEX
+
+Etc. pp. The respective directive names are ``ctx``, ``mp`` and
+``lua``. In order to get a |circle| drawn on spot, you would
+define a Metapost substitution:
+
+::
+
+    .. |circle| mp::
+        fill fullcircle scaled(8) withcolor blue;
+
+================
+Special Features
+================
+Text Roles
+**********
+
+The default *role* for interpreted text is *emphasis*.
+
+The role marker provides explicit access to formatting commands.
+The formatting routine for inline literals can be called with the
+role marker :literal:`literal`, strong emphasis likewise is
+achieved via the role marker :literal:`strong_emphasis`.
+
+Other roles that lack an equivalent among inline markup are
+``bold``, :ss:`ss` (alias :literal:`sans_serif`),
+``uppercase``, ``lowercase`` and colors.
+Color roles begin with the string ``color_`` (the underscore is
+compulsive), followed by either the string ``rgb_`` or a 
+`valid color name`__.
+An rgb vector is specified in decimal.
+Its values can be separated by either dashes or underscores.
+Thus, ``color_rgb_.3_.5_.8`` is a valid rgb expression, as is
+``color_rgb_0-1-0``.
+Unforturnately, the colon character ``:`` has to be escaped in
+color expressions, e.g. ``color_gray\:5``.
+
+__ http://wiki.contextgarden.net/Colors#Using_predefined_colors:_.5Csetupcolor
+
+For example, to give Mr. Neville Shunt’s work an apt
+typographic representation you can use these roles instead of
+the standard inline markup: ::
+
+    :color_rgb_.9_.2_.7:`Chuff`, chuff, :literal:`chuffwoooooch`,
+    woooooch! Sssssssss, sssssssss!  :uppercase:`Diddledum`,
+    `diddledum`, diddlealum.  :literal:`Toot`, toot. The train
+    :bold:`now` standing :color_gray\:5:`at` platform :ss:`eight,
+    tch`, tch, :color_rgb_0-1-0:`tch`,
+    :color_rgb_.5-.6-.2:`diddledum`, diddledum.
+    :lowercase:`Chuffff`, :strong_emphasis:`chuffffiTff`
+    eeeeeeeeeaaaaaaaaa :color_red:`Vooooommmmm`.
+
+which yields when passed through |rstcontext|:
+
+:color_rgb_.9_.2_.7:`Chuff`, chuff, :literal:`chuffwoooooch`,
+woooooch! Sssssssss, sssssssss!  :uppercase:`Diddledum`,
+`diddledum`, diddlealum.  :literal:`Toot`, toot. The train
+:bold:`now` standing :color_gray\:5:`at` platform :ss:`eight,
+tch`, tch, :color_rgb_0-1-0:`tch`,
+:color_rgb_.5-.6-.2:`diddledum`, diddledum. :lowercase:`Chuffff`,
+:strong_emphasis:`chuffffiTff` eeeeeeeeeaaaaaaaaa
+:color_red:`Vooooommmmm`.
+
+**************************
+Bibliography and Citations
+**************************
+
+.. caution::
+    Not much for now concerning the usage of Taco’s bib system.
+    It’s just that I use my own bibliography system and never
+    became sufficiently familiar with the standard |CONTEXT|
+    approach.  *If you feel that the current support should be
+    improved then feel free to contact me!* I will need somebody
+    for testing.
+
+When |rstcontext| first encounters a citation (``[texbook]_``) it
+automatically looks up a bibliography in the working directory by
+the name of ``\jobname``. E.g. with a main file ``manual.tex``
+bibtex will use the database called ``manual.bib``.  Symlinking
+your bibliography file in the local tree should suffice and you
+can keep whatever directory structure you prefer.  (Speaking for
+myself, bib data usually resides in its own subdirectory, so I’d
+use symlinks, too.)
+
+===================
+About this software
+===================
+
+The docutils_ provide means to translate the extra-convenient
+markup language |rst| into various formats like PDF, HTML and
+|LATEX|, unfortunately omitting the One True Macro System:
+|CONTEXT|.
+
+As far as I am aware of it, there is some support for |rst| in
+Pandoc_ but as it relies on a rather large set of dependencies it
+proved very difficult (too difficult for me) to install on my
+favourite distribution.
+From the `interactive demo`__ I gather that support for |rst|’s
+language features is not very extensive and the result did not
+even come with proper setups.
+Additionally, it’s written in a language I am not familiar with
+and that does not make use of one the most awesome features of
+all the the extended capabilities |LUATEX| provides: the Lua
+interpreter.
+
+For quite some time I was thinking about how to implement an
+|rst| parser in |LUATEX|, until some discussion__ emerged on the
+|CONTEXT| mailing list that indicates a broader interest in
+convenient markup languages across the community.
+As the alternatives mentioned above don’t meet the expectations
+of a normal |CONTEXT| user, the initial step to write
+|rstcontext| was done.
+Handling most of the corner cases and usability features of |rst|
+proved in the end not nearly as easy as I imagined.
+
+__ http://johnmacfarlane.net/pandoc/try
+__ http://archive.contextgarden.net/message/20100814.051917.28caafcd.en.html
+
+.. caution::
+    |rstcontext| is experimental software and neither feature
+    complete nor thoroughly commented. Keep this in mind before you
+    start using it. Anything might still be subject to change, so
+    expect breakage *in case you start relying on exceptional
+    behaviour* (read: bugs) that does not conform to the |rst|
+    specification. Consider filing a bug report instead and wait for
+    me (the maintainer) to fix it, because regardless of how much
+    testing I do myself I alway run into the weirdest issues only 
+    during the actual deployment of the software. Thus, if you notice
+    that |rstcontext| does not adhere to the outline_ of |rst|
+    according to the Docutils documentation, very likely you have
+    discovered a corner case I was not aware of.
+
+.. |circle| mp::
+    fill fullcircle scaled(8) withcolor blue;
+
+
+=======
+License
+=======
+
+::
+
+    Copyright 2010 Philipp Gesang. All rights reserved.
+
+    Redistribution and use in source and binary forms, with or
+    without modification, are permitted provided that the
+    following conditions are met:
+
+        1. Redistributions of source code must retain the above
+           copyright notice, this list of conditions and the
+           following disclaimer.
+
+        2. Redistributions in binary form must reproduce the
+           above copyright notice, this list of conditions and
+           the following disclaimer in the documentation and/or
+           other materials provided with the distribution.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER ``AS IS''
+    AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
+    FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
+    SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
+    DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+    CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+    PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
+    AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+    LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+    ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+    ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+
+.. vim:tw=65
diff --git a/doc/manual.bib b/doc/manual.bib
new file mode 100644
index 0000000..638ed80
--- /dev/null
+++ b/doc/manual.bib
@@ -0,0 +1,10 @@
+@BOOK{texbook,
+    author      = {Donald E. Knuth},
+    title       = {The \TEX book},
+    year        = {1984},
+    address     = {Reading, Mass. et. al.},
+    location    = {Reading, Mass. et. al.},
+    series      = {Computers and typesetting vol. 1},
+    keywords    = {all}
+}
+
diff --git a/doc/manual.tex b/doc/manual.tex
new file mode 100644
index 0000000..c95e5ca
--- /dev/null
+++ b/doc/manual.tex
@@ -0,0 +1,86 @@
+\usemodule[bib]
+
+\setuppapersize[A5][A5]
+
+\setupcombinedlist[content][interaction=all,focus=standard]
+
+\setupindenting[yes,next,medium] % -> lead to the glue node error in mkiv
+
+\setuphead[chapter]   [style={\rm\sc},before={\blank[line,force]},after={\blank[2*line,force]}]
+\setuphead[section]   [style={\rm\it},before={\blank[line]}      ,after={\blank[line]}]
+\setuphead[subsection][style={\rm},before={\blank[line]}         ,after={\blank[line]}]
+
+\setupheads[indentnext=yes]
+\setupfloats[indentnext=yes]
+
+\setupbodyfont[10pt]
+
+% title page
+\startbuffer[frontpage]
+\startstandardmakeup
+\raggedcenter
+\vfill
+{\setupbodyfont[,19pt]
+{\em Typesetting}
+\blank [2*big]
+{\tfc {\em re}{\ss Structured}{\em \kern-6ptText}}
+\blank [2*big]
+{\em with \CONTEXT}
+\blank [5*big]
+{\tfa A Manual for {\em rst}{\kern1pt\CONTEXT}}
+}
+\vfill
+\stopstandardmakeup
+\stopbuffer
+
+% additional information
+\startbuffer[author]
+\startstandardmakeup
+\vfill
+\framed [align=right,frame=off,topframe=on] {%
+\tfxx\ss\setupinterlinespace[small]
+Copyright 2010 by Philipp Gesang, Dossenheim.\par
+Mail any patches or suggestions to\par
+\type{string.format("%s@%s.com", "megas.kapaneus", "gmail")}\par
+or pay a visit to \goto{my BitBucket home}[url(http://bitbucket.org/phg/)].\par
+}
+\stopstandardmakeup
+\stopbuffer
+
+
+% table of contents
+\startbuffer[toc]
+\setuppagenumbering[state=start,alternative=doublesided,location=]
+\setupheadertexts
+  [{\tfx\sc\getmarking[chapter]}]   [{\tfx\bf \pagenumber}]
+  [{\tfx\bf \pagenumber}]           [{\tfx{\em rst}{\kern.5pt\CONTEXT}}]
+\completecontent
+\stopbuffer
+
+% something radically changed in mkiv
+\startluacode
+jobvariables = jobvariables or {}
+jobvariables.tobesaved = jobvariables.tobesaved or {}
+\stopluacode
+
+% list of publications
+\startbuffer[pubs]
+\setuplayout[grid=no]
+\setuptolerance[verytolerant]
+\setuptolerance[vertical,verytolerant]
+
+\completepublications
+\stopbuffer
+
+\appendtoks  \getbuffer[frontpage] \to \everystarttext
+\appendtoks  \getbuffer[author]    \to \everystarttext
+\appendtoks  \getbuffer[toc]       \to \everystarttext
+
+%\prependtoks \getbuffer[pubs]      \to \everystoptext
+
+\setupwhitespace[none]
+\setuplayout[grid=verystrict]
+\setuptolerance[tolerant]
+\language[en]
+\setuppagenumbering[state=stop,location=]
+\input doc.tex