% language=uk engine=luajittex % criterium: all + sorttype=cite => citex before rest % criterium: all + sorttype=database => database order % criterium: used % % numbering: label, short, indexinlist, indexused % % maybeyear % % \cite[data][whatever] % \showframe \usemodule[abr-02] \usemodule[set-11] \loadsetups[publications-en.xml] \enablemode[interface:setup:defaults] \setupbackend [export=yes, xhtml=yes, css=export-example.css] \setupexport [hyphen=yes, width=60em] % \input publ-tmp.mkiv \setupbodyfont [dejavu,10pt] \setuphead [chapter] [header=high, style=\bfc, color=darkmagenta] \setuplayout [topspace=2cm, bottomspace=1cm, header=0cm, width=middle, height=middle] \setupwhitespace [big] \setuptyping [color=darkmagenta] \setuptyping [keeptogether=yes] \setuptype [color=darkcyan] \setupfootertexts [pagenumber] \setupMPgraphics [mpy=\jobname.mpy] \setupinteraction [state=start, color=darkcyan, contrastcolor=darkyellow] \starttext \startMPpage StartPage ; % input "mkiv-publications.mpy" ; picture pic ; pic := image ( path pth ; pth := ((0,0) for i=1 step 2 until 20 : -- (i,1) -- (i+1,0) endfor) ; for i=0 upto 9 : draw pth shifted (0,2*i) ; endfor ; ) ; % picture btx ; btx := textext("\ssbf BIBTEX") ; % picture ctx ; ctx := textext("\ssbf THE CONTEXT WAY") ; picture btx ; btx := image(graphictext("\ssbf BIBTEX") withfillcolor white) ; picture ctx ; ctx := image(graphictext("\ssbf THE CONTEXT WAY") withfillcolor white) ; pic := pic shifted - llcorner pic ; btx := btx shifted - llcorner btx ; ctx := ctx shifted - llcorner ctx ; pic := pic xysized (PaperWidth,PaperHeight) ; btx := btx xsized (2PaperWidth/3) shifted (.25PaperWidth,.15PaperHeight) ; ctx := ctx xsized (2PaperWidth/3) shifted (.25PaperWidth,.075PaperHeight) ; fill Page withcolor \MPcolor{darkcyan} ; draw pic withcolor \MPcolor{darkmagenta} ; draw btx withcolor \MPcolor{lightgray} ; draw ctx withcolor \MPcolor{lightgray} ; % draw boundingbox btx ; % draw boundingbox ctx ; StopPage ; \stopMPpage \startfrontmatter \starttitle[title=Contents] \placelist[chapter,section][color=black] \stoptitle \startchapter[title=Introduction] This manual describes how \MKIV\ handles bibliographies. Support in \CONTEXT\ started in \MKII for \BIBTEX, using a module written by Taco Hoekwater. Later his code was adapted to \MKIV, but because users demanded more, I decided that reimplementing made more sense than patching. In particular, through the use of \LUA, the \BIBTEX\ data files can be easily directly parsed, thus liberating \CONTEXT\ from the dependency on an external \BIBTEX\ executable. The CritEd project (by Thomas Schmitz, Alan Braslau, Luigi Scarso and myself) was a good reason to undertake this rewrite. As part that project users were invited to come up with ideas about extensions. Not all of them are (yet) honored, but the rewrite makes more functionality possible. This manual is dedicated to Taco Hoekwater who in a previous century implemented the first \BIBTEX\ module and saw it morf into a \TEX||\LUA\ hybrid in this century. The fact that there was support for bibliographies made it possible for users to use \CONTEXT\ in an academic environment, dominated by bibliographic databases encoded in the \BIBTEX\ format. \startlines Hans Hagen PRAGMA ADE Hasselt NL \stoplines \stopchapter \stopfrontmatter \startbodymatter \startchapter[title=The database] The \BIBTEX\ format is rather popular in the \TEX\ community and even with its shortcomings it will stay around for a while. Many publication websites can export and many tools are available to work with this database format. It is rather simple and looks a bit like \LUA\ tables. Unfortunately the content can be polluted with non|-|standardized \TEX\ commands which complicates pre- or postprocessing outside \TEX. In that sense a \BIBTEX\ database is often not coded neutrally. Some limitations, like the use of commands to encode accented characters root in the \ASCII\ world and can be bypassed by using \UTF\ instead (as handled somewhat in \LATEX\ through extensions such as \type {bibtex8}). The normal way to deal with a bibliography is to refer to entries using a unique tag or key. When a list of entries is typeset, this reference can be used for linking purposes. The typeset list can be processed and sorted using the \type {bibtex} program that converts the database into something more \TEX\ friendly (a \type {.bbl} file). I never used the program myself (nor bibliographies) so I will not go into too much detail here, if only because all I say can be wrong. In \CONTEXT\ we no longer use the \type {bibtex} program: we just use database files and deal with the necessary manipulations directly in \CONTEXT. One or more such databases can be used and combined with additional entries defined within the document. We can have several such datasets active at the same time. A \BIBTEX\ file looks like this: \starttyping @Article{sometag, author = "An Author and Another One", title = "A hopefully meaningful title", journal = maps, volume = "25", number = "2", pages = "5--9", month = mar, year = "2013", ISSN = "1234-5678", } \stoptyping Normally a value is given between quotes (or curly brackets) but single words are also OK (there is no real benefit in not using quotes, so we advise to always use them). There can be many more fields and instead of strings one can use predefined shortcuts. The title for example quite often contains \TEX\ macros. Some fields, like \type {pages} have funny characters such as the endash (typically as \type {--}) so we have a mixture of data and typesetting directives. If you are covering non||english references, you often need characters that are not in the \ASCII\ subset but \CONTEXT\ is quite happy with \UTF. If your database file uses old|-|fashioned \TEX\ accent commands then these will be internally converted automatically to \UTF. Commands (macros) are converted to an indirect call, which is quite robust. The \BIBTEX\ files are loaded in memory as \LUA\ table but can be converted to \XML\ so that we can access them in a more flexible way, but that is a subject for specialists. In the old \MKII\ setup we have two kinds of entries: the ones that come from the \BIBTEX\ run and user supplied ones. We no longer rely on \BIBTEX\ output but we do still support the user supplied definitions. These were in fact prepared in a way that suits the processing of \BIBTEX\ generated entries. The next variant reflects the \CONTEXT\ recoding of the old \BIBTEX\ output. \starttyping \startpublication[k=Hagen:Second,t=article,a={Hans Hagen},y=2013,s=HH01] \artauthor[]{Hans}[H.]{}{Hagen} \arttitle{Who knows more?} \journal{MyJournal} \pubyear{2013} \month{8} \volume{1} \issue{3} \issn{1234-5678} \pages{123--126} \stoppublication \stoptyping The split \type {\artauthor} fields are collapsed into a single \type {author} field as we deal with the splitting later when it gets parsed in \LUA. The \type {\artauthor} syntax is only kept around for backward compatibility with the previous use of \BIBTEX. In the new setup we support these variants as well: \starttyping \startpublication[k=Hagen:Third,t=article] \author{Hans Hagen} \title{Who knows who?} ... \stoppublication \stoptyping and \starttyping \startpublication[tag=Hagen:Third,category=article] \author{Hans Hagen} \title{Who knows who?} ... \stoppublication \stoptyping and \starttyping \startpublication \tag{Hagen:Third} \category{article} \author{Hans Hagen} \title{Who knows who?} ... \stoppublication \stoptyping Because internally the entries are \LUA\ tables, we also support loading of \LUA\ based definitions: \starttyping return { ["Hagen:First"] = { author = "Hans Hagen", category = "article", issn = "1234-5678", issue = "3", journal = "MyJournal", month = "8", pages = "123--126", tag = "Hagen:First", title = "Who knows nothing?", volume = "1", year = "2013", }, } \stoptyping Files set up like this can be loaded too. The following \XML\ input is rather close to this, and is also accepted as input. \starttyping Hans Hagen article 1234-5678 3 MyJournal 8 123--126 Hagen:First Who knows nothing? 1 2013 \stoptyping {\em Todo: Add some remarks about loading EndNote and RIS formats, but first we need to complete the tag mapping (on Alan's plate).} So the user has a rather wide choice of formatting style for bibliography database files. \stopchapter You can load more data than you actually need. Only entries that are referred to explicitly through the \type {\cite} and \type {\nocite} commands will be shown in lists. We will cover these details later. \startchapter[title=Commands in entries] One unfortunate aspect commonly found in \BIBTEX\ files is that they often contain \TEX\ commands. Even worse is that there is no standard on what these commands can be and what they mean, at least not formally, as \BIBTEX\ is a program intended to be used with many variants of \TEX\ style: plain, \LATEX, and others. This means that we need to define our use of these typesetting commands. However, in most cases, they are just abbreviations or font switches and these are often known. Therefore, \CONTEXT\ will try to resolve them before reporting an issue. In the log file there is a list of commands that has been seen in the loaded databases. For instance, loading \type {tugboat.bib} gives a long list of commands of which we show a small set here: \starttyping publications > start used btx commands publications > standard CONTEXT 1 known publications > standard ConTeXt 4 known publications > standard TeXLive 3 KNOWN publications > standard eTeX 1 known publications > standard hbox 6 known publications > standard sltt 1 unknown publications > stop used btxcommands \stoptyping You can define unknown commands, or overload existing definitions in the following way: \starttyping \definebtxcommand\TUB {TUGboat} \definebtxcommand\sltt{\tt} \definebtxcommand\<#1>{\type{#1}} \stoptyping Unknown commands do not stall processing, but their names are then typeset in a mono|-|spaced font so they probably stand out for proofreading. You can access the commands with \type {\btxcommand {...}}, as in: \startbuffer commands like \btxcommand{MySpecialCommand} are handled in an indirect way \stopbuffer \typebuffer As this is an undefined command we get: \quotation {\inlinebuffer}. ?? \stopchapter \startchapter[title=Datasets] Normally in a document you will use only one bibliographic database, whether or not distributed over multiple files. Nevertheless we support multiple databases as well which is why we talk of datasets instead. A dataset is loaded with the \type {\usebtxdataset} command. Although currently it is not necessary to define a (default) dataset you can best do this because in the future we might provide more options. Here are some examples: \starttyping \definebtxdataset[standard] \usebtxdataset[standard][tugboat.bib] \usebtxdataset[standard][mtx-bibtex-output.xml] \usebtxdataset[standard][test-001-btx-standard.lua] \stoptyping These three suffixes are understood by the loader. Here the dataset has the name \type {standard} and the three database files are merged, where later entries having the same tag overload previous ones. Definitions in the document source (coded in \TEX\ speak) are also added, and they are saved for successive runs. This means that if you load and define entries, they will be known at a next run beforehand, so that references to them are independent of when loading and definitions take place. \showsetup{setupbtxdataset} \showsetup{definebtxdataset} \showsetup{usebtxdataset} In this document we use some example databases, so let's load one of them now: \startbuffer \definebtxdataset[example] \usebtxdataset[example][mkiv-publications.bib] \stopbuffer \typebuffer \getbuffer You can ask for an overview of entries in a dataset with: \startbuffer \showbtxdatasetfields[example] \stopbuffer \typebuffer this gives: \getbuffer You can set the current active dataset with \starttyping \setbtxdataset[standard] \stoptyping but most publication|-|related commands accept optional arguments that denote the dataset and references to entries can be prefixed with a dataset identifier.. More about that later. \stopchapter \startchapter[title=Renderings] A list of publications can be rendered at any place in the document. A database can be much larger than needed for a document. The same is true for the fields that make up an entry. Here is the list of fields that are currently handled, but of course there can be additional ones: \startalignment[flushleft,verytolerant,nothyphenated] \startluacode local fields = publications.tracers.fields for i=1,#fields do if i > 1 then context(", ") end context.type(fields[i]) end \stopluacode \stopalignment If you want to see what publications are in the database, the easiest way is to ask for a complete list: \startbuffer \definebtxrendering [example] [dataset=example, method=local, alternative=apa] \placelistofpublications % \placebtxrendering [example] [criterium=all] \stopbuffer \typebuffer This gives: \getbuffer The rendering itself is somewhat complex to set up because we have not only many different standards but also many fields that can be set up. This means that there are several commands involved. Often there is a prescribed style to render bibliographic descriptions, for example \type {apa}. A rendering is setup and defined with: \showsetup[setupbtxrendering] %showrootvalues[btxrendering] \showsetup[definebtxrendering] And a list of such descriptions is generated with: \showsetup[placebtxrendering] A dataset can have all kind of entries: \startalignment[flushleft,verytolerant,nothyphenated] \startluacode local categories = publications.tracers.categories for i=1,#categories do if i > 1 then context(", ") end context.type(categories[i]) end \stopluacode \stopalignment Each has its own rendering variant. To keep things simple we have their settings separated. However, these settings are shared for all rendering alternatives. In practice this is seldom a problem in a publication as only one rendering alternative will be active. If this be not sufficient, you can always group local settings in a setup and hook that into the specific rendering. \showsetup[setupbtxlistvariant] %showrootvalues[btxlistvariant] \showsetup[definebtxlistvariant] Examples of list variants are: \startluacode local variants = publications.tracers.listvariants for i=1,#variants do context.showinstancevalues( { "btxlistvariant" }, { variants[i] }) end \stopluacode The exact rendering of list entries is determined by the \type {alternative} key and defaults to \type {apa} which uses definitions from \type {publ-imp-apa.mkiv}. If you look at that file you will see that each category has its own setup. You may also notice that additional tests are needed to make sure that empty fields don't trigger separators and such. % \showsetup{setuplists} There are a couple of accessors and helpers to get the job done. When you want to fetch a field from the current entry you use \type {\btxfield}. In most cases you want to make sure this field has a value, for instance because you don't want fences or punctuation that belongs to a field. \starttyping \btxdoif {title} { \bold{\btxfield{title}}, } \stoptyping There are three test macros: \starttyping \btxdoifelse{fieldname}{action when found}{action when not found} \btxdoif {fieldname}{action when found} \btxdoifnot {fieldname} {action when not found} \stoptyping An extra conditional is available for testing interactivity: \starttyping \btxdoifelseinteraction{action when true}{action when false} \stoptyping In addition there is also a conditional \type {\btxinteractive} which is more efficient, although in practice efficiency is not so important here. There are three commands to flush data: \starttabulate[|||] % Funny usage here! Could not tabulate work without % even specifying the number of columns? \NC \type {\btxfield} \NC fetch a explicit field (e.g. \type {year}) \NC \NR \NC \type {\btxdetail} \NC fetch a derived field (e.g. \type {short}) \NC \NR \NC \type {\btxflush} \NC fetch a derived or explicit field \NC \NR \stoptabulate Normally you can use \type {\btxfield} or \type {\btxflush} as derived fields just like analyzed author fields are flushed in a special way. You can improve readability by using setups, for instance: \starttyping \btxdoifelse {author} { \btxsetup{btx:apa:author:yes} } { \btxsetup{btx:apa:author:nop} } \stoptyping Keep in mind that normally you don't need to mess with definitions like this because standard rendering styles are provided. These styles use a few helpers that inject symbols but also take care of leading and trailing spaces: \starttabulate[|||] \NC \type {\btxspace } \NC before \btxspace after \NC \NR \NC \type {\btxperiod } \NC before \btxperiod after \NC \NR \NC \type {\btxcomma } \NC before \btxcomma after \NC \NR \NC \type {\btxlparent } \NC before \btxlparent after \NC \NR \NC \type {\btxrparent } \NC before \btxrparent after \NC \NR \NC \type {\btxlbracket} \NC before \btxlbracket after \NC \NR \NC \type {\btxrbracket} \NC before \btxrbracket after \NC \NR \stoptabulate So, the previous example setup can be rewritten as: \starttyping \btxdoif {title} { \bold{\btxfield{title}} \btxcomma } \stoptyping There is a special command for rendering a (combination) of authors: \starttyping \btxflushauthor{author} \btxflushauthor{editor} \btxflushauthor[inverted]{editor} \stoptyping Instead of the last one you can also use: \starttyping \btxflushauthorinverted{editor} \stoptyping You can use a (configurable) default or pass directives: Valid directives are \starttabulate \NC \bf conversion \NC \bf rendering \NC \NR \HL \NC \type{inverted} \NC the Frog jr, Kermit \NC \NR \NC \type{invertedshort} \NC the Frog jr, K \NC \NR \NC \type{normal} \NC Kermit, the Frog, jr \NC \NR \NC \type{normalshort} \NC K, the Frog, jr \NC \NR \stoptabulate \stopchapter \startchapter[title=Citations] Citations are references to bibliographic entries that normally show up in lists someplace in the document: at the end of a chapter, in an appendix, at the end of an article, etc. We discussed the rendering of these lists in the previous chapter. A citation is normally pretty short as its main purpose is to refer uniquely to a more detailed description. But, there are several ways to refer, which is why the citation subsystem is configurable and extensible. Just look at the following commands: \startbuffer \cite[author][example::demo-003] \cite[authoryear][example::demo-003] \cite[authoryears][example::demo-003] \cite[author][example::demo-003,demo-004] \cite[authoryear][example::demo-003,demo-004] \cite[authoryears][example::demo-003,demo-004] \cite[author][example::demo-004,demo-003] \cite[authoryear][example::demo-004,demo-003] \cite[authoryears][example::demo-004,demo-003] \stopbuffer \typebuffer \startlines \getbuffer \stoplines The first argument is optional. % What is the default? How can one set this up? \showsetup[cite] You can tune the way a citation shows up: \startbuffer \setupbtxcitevariant[author] [sorttype=author,color=darkyellow] \setupbtxcitevariant[authoryear] [sorttype=author,color=darkyellow] \setupbtxcitevariant[authoryears][sorttype=author,color=darkyellow] \cite[author][example::demo-004,demo-003] \cite[authoryear][example::demo-004,demo-003] \cite[authoryears][example::demo-004,demo-003] \stopbuffer \typebuffer Here we sort the authors and color the citation: \startlines \getbuffer \stoplines For reasons of backward compatibility the \type {\cite} command is a bit picky about spaces between the two arguments, of which the first is optional. This is a consequence of allowing its use with the key specified between curly brackets as is the traditional practice. (We do encourage users to adopt the more coherent \CONTEXT\ syntax by using square brackets for keywords and reserving curly brackets to regroup text to be typeset.) % Just how is it picky? The \type {\citation} command is synonymous but is more flexible with respect to spacing of its arguments: \starttyping \citation[author] [example::demo-004,demo-003] \citation[authoryear] [example::demo-004,demo-003] \citation[authoryears][example::demo-004,demo-003] \stoptyping % The first argument of cite and citation is optional. What is the default and how does one set it? There is a whole bunch of cite options and more can be easily defined. \startluacode local variants = publications.tracers.citevariants context.starttabulate { "|l|p|" } context.NC() context.bold("key") context.NC() context.bold("rendering") context.NC() context.NR() context.FL() for i=1,#variants do local variant = variants[i] context.NC() context.type(variant) context.NC() context.citation( { variant }, { "example::demo-005" }) context.NC() context.NR() end context.stoptabulate() \stopluacode Because we are dealing with database input and because we generally need to manipulate entries, much of the work is delegated to \LUA. This makes it easier to maintain and extend the code. Of course \TEX\ still does the rendering. The typographic details are controlled by parameters but not all are used in all variants. As with most \CONTEXT\ commands, it starts out with a general setup command: \showsetup[setupbtxcitevariant] On top of that we can define instances that inherit either from a given parent or from the topmost setup. \showsetup[definebtxcitevariant] % The default values are: % \showrootvalues[btxcitevariant] But, specific variants can have them overloaded: % \showinstancevalues[setupbtxcitevariant][author] % \showinstancevalues[setupbtxcitevariant][authornum] \startluacode local variants = publications.tracers.citevariants for i=1,#variants do context.showinstancevalues( { "btxcitevariant" }, { variants[i] }) end \stopluacode A citation variant is defined in several steps and if you really want to know the dirty details, you should look into the \type {publ-imp-*.mkiv} files. Here we stick to the concept. \starttyping \startsetups btx:cite:author \btxcitevariant{author} \stopsetups \stoptyping You can overload such setups if needed, but that only makes sense when you cannot configure the rendering with parameters. The \type {\btxcitevariant} command is one of the build in accessors and it calls out to \LUA\ where more complex manipulation takes place if needed. If no manipulation is known, the field with the same name (if found) will be flushed. A command like \type {\btxcitevariant} assumes that a dataset and specific tag has been set. This is normally done in the wrapper macros, like \type {\cite}. For special purposes you can use these commands \starttyping \setbtxdataset[example] \setbtxentry[hh2013] \stoptyping But don't expect too much support for such low level rendering control. Unless you use \type {criterium=all} only publications that are cited will end up in the lists. You can force a citation into a list using \type {\usecitation}, for example: \starttyping \usecitation[example::demo-004,demo-003] \stoptyping This command has two synonyms: \type {\nocite} and \type {\nocitation} so you can choose whatever fits you best. \showsetup[nocite] \stopchapter \startchapter[title=The \LUA\ view] Because we manage data at the \LUA\ end it is tempting to access it there for other purposes. This is fine as long as you keep in mind that aspects of the implementation may change over time, although this is unlikely once the modules become stable. The entries are collected in datasets and each set has a unique name. In this document we have the set named \type {example}. A dataset table has several fields, and probably the one of most interest is the \type {luadata} field. Each entry in this table describes a publication: \startluacode context.tocontext(publications.datasets.example.luadata["demo-001"]) \stopluacode This is \type {publications.datasets.example.luadata["demo-001"]}. There can be a companion entry in the parallel \type {details} table. \startluacode context.tocontext(publications.datasets.example.details["demo-001"]) \stopluacode These details are accessed as \type {publications.datasets.example.details["demo-001"]} and by using a separate table we can overload fields in the original entry without losing the original. You can loop over the entries using regular \LUA\ code combined with \MKIV\ helpers: \startbuffer local dataset = publications.datasets.example context.starttabulate { "|l|l|l|" } for tag, entry in table.sortedhash(dataset.luadata) do local detail = dataset.details[tag] or { } context.NC() context.type(tag) context.NC() context(detail.short) context.NC() context(entry.title) context.NC() context.NR() end context.stoptabulate() \stopbuffer \typebuffer This results in: \ctxluabuffer You can manipulate a dataset after loading. Of course this assumes that you know what kind of content you have and what you need for rendering. As example we load a small dataset. \startbuffer \definebtxdataset[drumming] \usebtxdataset[drumming][mkiv-publications.lua] \stopbuffer \typebuffer \getbuffer Because we're going to do some \LUA, we could also have loaded the dataset with: \starttyping publications.load("drumming","mkiv-publications.lua","lua") \stoptyping The dataset has three entries: \typefile{mkiv-publications.lua} As you can see, we can have a subtitle. We will combine the title and subtitle into one: \startbuffer \startluacode for tag, entry in next, publications.datasets.drumming.luadata do if entry.subtitle then if entry.title then entry.title = entry.title .. ", " .. entry.subtitle else entry.title = entry.subtitle end entry.subtitle = nil logs.report("btx","combining title and subtitle of entry tagged %a",tag) end end \stopluacode \stopbuffer \typebuffer \getbuffer We can now typeset the entries with: \startbuffer \definebtxrendering[drumming][dataset=drumming,method=dataset] \placebtxrendering[drumming] \stopbuffer \typebuffer Because we just want to show the entries, and have no citations that force them to be shown, we have to the \type {method} to \type {dataset}. \footnote {Gavin Harrison is in my opinion one of the most creative, diverse and interesting drummers of our time. It's also fascinating to watch him play and a welcome distraction from writing code and manuals.} \blank \getbuffer \blank \stopchapter \startchapter[title=The \XML\ view] The \type {luadata} table can be converted into an \XML\ representation. This is a follow up on earlier experiments with an \XML|-|only approach. I decided in the end to stick to a \LUA\ approach and provide some simple \XML\ support in addition. Once a dataset is accessible as \XML\ tree, you can use the regular \type {\xml...} commands. We start with loading a dataset, in this case from just one file. \startbuffer \usebtxdataset[tugboat][tugboat.bib] \stopbuffer \typebuffer \getbuffer The dataset has to be converted to \XML: \startbuffer \convertbtxdatasettoxml[tugboat] \stopbuffer \typebuffer \getbuffer The tree is now accessible by its root reference \type {btx:tugboat}. If we want simple field access we can use a few setups: \startbuffer \startxmlsetups btx:initialize \xmlsetsetup{#1}{bibtex|entry|field}{btx:*} \xmlmain{#1} \stopxmlsetups \startxmlsetups btx:field \xmlflushcontext{#1} \stopxmlsetups \xmlsetup{btx:tugboat}{btx:initialize} \stopbuffer \typebuffer \getbuffer The two setups are predefined in the core already, but you might want to change them. They are applied in for instance: \startbuffer \starttabulate[|||] \NC \type {tag} \NC \xmlfirst {btx:tugboat} {/bibtex/entry[string.find(@tag,'Hagen')]/attribute('tag')} \NC \NR \NC \type {title} \NC \xmlfirst {btx:tugboat} {/bibtex/entry[string.find(@tag,'Hagen')]/field[@name='title']} \NC \NR \stoptabulate \stopbuffer \typebuffer \getbuffer \startbuffer \startxmlsetups btx:demo \xmlcommand {#1} {/bibtex/entry[string.find(@tag,'Hagen')][1]}{btx:table} \stopxmlsetups \startxmlsetups btx:table \starttabulate[|||] \NC \type {tag} \NC \xmlatt{#1}{tag} \NC \NR \NC \type {title} \NC \xmlfirst{#1}{/field[@name='title']} \NC \NR \stoptabulate \stopxmlsetups \xmlsetup{btx:tugboat}{btx:demo} \stopbuffer \typebuffer \getbuffer Here is another example: \startbuffer \startxmlsetups btx:row \NC \xmlatt{#1}{tag} \NC \xmlfirst{#1}{/field[@name='title']} \NC \NR \stopxmlsetups \startxmlsetups btx:demo \xmlfilter {#1} { /bibtex /entry[@category='article'] /field[@name='author' and (find(text(),'Knuth') or find(text(),'DEK'))] /../command(btx:row) } \stopxmlsetups \starttabulate[|||] \xmlsetup{btx:tugboat}{btx:demo} \stoptabulate \stopbuffer \typebuffer \getbuffer A more extensive example is the following. Of course this assumes that you know what \XML\ support mechanisms and macros are available. \startbuffer \startxmlsetups btx:getkeys \xmladdsortentry{btx}{#1}{\xmlfilter{#1}{/field[@name='author']/text()}} \xmladdsortentry{btx}{#1}{\xmlfilter{#1}{/field[@name='year' ]/text()}} \xmladdsortentry{btx}{#1}{\xmlatt{#1}{tag}} \stopxmlsetups \startxmlsetups btx:sorter \xmlresetsorter{btx} % \xmlfilter{#1}{entry/command(btx:getkeys)} \xmlfilter{#1}{ /bibtex /entry[@category='article'] /field[@name='author' and find(text(),'Knuth')] /../command(btx:getkeys)} \xmlsortentries{btx} \starttabulate[||||] \xmlflushsorter{btx}{btx:entry:flush} \stoptabulate \stopxmlsetups \startxmlsetups btx:entry:flush \NC \xmlfilter{#1}{/field[@name='year' ]/context()} \NC \xmlatt{#1}{tag} \NC \xmlfilter{#1}{/field[@name='author']/context()} \NC \NR \stopxmlsetups \xmlsetup{btx:tugboat}{btx:sorter} \stopbuffer \typebuffer \getbuffer The original data is stored in a \LUA\ table, hashed by tag. Starting with \LUA\ 5.2 each run of \LUA\ gets a different ordering of such a hash. In older versions, when you looped over a hash, the order was undefined, but the same as long as you used the same binary. This had the advantage that successive runs, something we often have in document processing gave consistent results. In today's \LUA\ we need to do much more sorting of hashes before we loop, especially when we save multi||pass data. It is for this reason that the \XML\ tree is sorted by hash key by default. That way lookups (especially the first of a set) give consistent outcomes. \stopchapter \startchapter[title=Standards] The rendering of bibliographic entries is often standardized and prescribed by the publisher. If you submit an article to a journal, normally it will be reformatted (or even re|-|keyed) and the rendering will happen at the publishers end. In that case it may not matter how entries were rendered when writing the publication, because the publisher will do it his or her way. This means that most users probably will stick to the standard \APA\ rules and for them we provide some configuration. Because we use setups it is easy to overload specifics. If you really want to tweak, best look in the files that deal with it. Many standards exist and support for other renderings may be added to the core. Interested users are invited to develop and to test alternate standard renderings according to their needs. Todo: maybe a list of categories and fields. \stopchapter \startchapter[title=Cleaning up] Although the \BIBTEX\ format is reasonably well defined, in practice there are many ways to organize the data. For instance, one can use predefined string constants that get used (either or not combined with other strings) later on. A string can be enclosed in curly braces or double quotes. The strings can contain \TEX\ commands but these are not standardized. The databases often have somewhat complex ways to deal with special characters and the use of braces in their definition is also not normalized. The most complex to deal with are the fields that contain names of people. At some point it might be needed to split a combination of names into individual ones that then get split into title, first name, optional inbetweens, surname(s) and additional: \type {Prof. Dr. Alfred B. C. von Kwik Kwak Jr. II and P. Q. Olet} is just one example of this. The convention seems to be not to use commas but \type {and} to separate names (often each name will be specified as lastname, firstname). We don't see it as challenge nor as a duty to support all kinds of messy definitions. Of course we try to be somewhat tolerant, but you will be sure to get better results if you use nicely setup, consistent databases. Todo: maybe some examples of bad. \stopchapter \startchapter[title=Transition] In the original bibliography support module usage was as follows (example taken from the contextgarden wiki): \starttyping % engine=pdftex \usemodule[bib] \usemodule[bibltx] \setupbibtex [database=xampl] \setuppublications [numbering=yes] \starttext As \cite [article-full] already indicated, bibtex is a \LATEX||centric program. \completepublications \stoptext \stoptyping For \MKIV\ the modules were partly rewritten and ended up in the core so the two {\usemodule} commands were no longer needed. The overhead associated with the automatic loading of the bibliography macros can be neglected these days, so standardized modules such as \type {bib} are all being moved to the core and do not need to be explicitly loaded. The first \type {\setupbibtex} command in this example is needed to bootstrap the process: it tells what database has to be processed by \BIBTEX\ between runs. The second \type {\setuppublications} command is optional. Each citation (tagged with \type {\cite}) ends up in the list of publications. In the new approach we no longer use \BIBTEX so we don't need to setup \BIBTEX. Instead we define dataset(s). We also no longer set up publications with one command, but have split that up in rendering-, list-, and cite|-|variants. The basic \type {\cite} command remains. The above example becomes: \starttyping \definebtxdataset [document] \usebtxdataset [document] [mybibfile.bib] \definebtxrendering [document] \setupbtxrendering [document] [numbering=yes] \starttext As \cite [article-full] already indicated, bibtex is a \LATEX||centric program. \completebtxrendering[document] \stoptext \stoptyping So, we have a few more commands to set up things. If you intend to use just a single dataset and rendering, the above preamble can be simplified to: \starttyping \usebtxdataset [mybibfile.bib] \setupbtxrendering [numbering=yes] \stoptyping But keep in mind that compared to the old \MKII\ derived method we have moved some of the options to the rendering, list and cite setup variants. Another difference is now the use of lists. When you define a rendering, you also define a list. However, all entries are collected in a common list tagged \type {btx}. Although you will normally configure a rendering you can still set some properties of lists, but in that case you need to prefix the list identifier. In the case of the above example this is \type {btx:document}. \stopchapter \startchapter[title=\MLBIBTEX] Todo: how to plug in \MLBIBTEX\ for sorting and other advanced operations. \stopchapter \startchapter[title=Extensions] As \TEX\ and \LUA\ are both open and accessible in \CONTEXT\ it is possible to extend the functionality of the bibliography related code. For instance, you can add extra loaders. \starttyping function publications.loaders.myformat(dataset,filename) local t = { } -- Load data from 'filename' and convert it to a Lua table 't' with -- the key as hash entry and fields conforming the luadata table -- format. loaders.lua(dataset,t) end \stoptyping This then permits loading a database (into a dataset) with the command: \starttyping \usebtxdataset[standard][myfile.myformat] \stoptyping The \type {myformat} suffix is recognized automatically. If you want to use another suffix, you can do this: \starttyping \usebtxdataset[standard][myformat::myfile.txt] \stoptyping \stopchapter \startchapter[title=Notes] The move from external \BIBTEX\ processing to internal processing has the advantage that we stay within the same run. In the traditional approach we had roughly the following steps: \startitemize[packed] \startitem the first run information is collected and written to file \stopitem \startitem after that run the \BIBTEX\ program converts that file to another one \stopitem \startitem successive runs use that data for references and producing lists \stopitem \stopitemize In the \MKIV\ approach the bibliographic database is loaded in memory each run and processing also happens each run. On paper this looks less efficient but as \LUA\ is quite fast, in practice performance is much better. Probably most demanding is the treatment of authors as we have to analyze names, split multiple authors and reassemble firstnames, vons, surnames and juniors. When we sort by author sorting vectors have to be made which also has a penalty. However, in practice the user will not notice a performance degradation. We did some tests with a list of 500.000 authors, sorted them and typeset them as list (producing some 5400 dense pages in a small font and with small margins). This is typical one of these cases where using \LUAJITTEX\ saves quite time. On my machine it took just over 100 seconds to get this done. Unfortunately not all operating systems performed equally well: 32 bit versions worked fine, but 64 bit \LINUX\ either crashed (stalled) the machine or ran out of memory rather fast, while \OSX\ and \WINDOWS\ performed fine. In practice you will never run into this, unless you produce massive amounts of bibliographic entries. \stopchapter \stopbodymatter \stoptext Todo: \setuplabeltext[en][reprint=reprint] \setuplabeltext[de][reprint=Nachdruck] note = {\labeltext{reprint} 2004}