diff options
Diffstat (limited to 'doc/context/sources/general/manuals/mk/mk-memory.tex')
-rw-r--r-- | doc/context/sources/general/manuals/mk/mk-memory.tex | 142 |
1 files changed, 142 insertions, 0 deletions
diff --git a/doc/context/sources/general/manuals/mk/mk-memory.tex b/doc/context/sources/general/manuals/mk/mk-memory.tex new file mode 100644 index 000000000..4c16e613e --- /dev/null +++ b/doc/context/sources/general/manuals/mk/mk-memory.tex @@ -0,0 +1,142 @@ +% language=uk + +\environment mk-environment + +\usemodule[timing] + +\startcomponent mk-memory + +\chapter{Collecting garbage} + +We use the \type {mk.tex} document for testing and because it keeps track of +how \LUATEX\ evolves. As a result it has some uncommon characteristics. For +instance, you can see increments in memory usage at points where we +load fonts: the chapters on Zapfino, Arabic and CJK (unfinished). This memory +is not freed because the font memory is used permanently. In the following +graphic, the red line is the memory consumption of \LUATEX\ for the current +version of \type {mk.tex}. The blue line is the runtime per page. + +\ShowNamedUsage{mk-luatex-progress}{luastate_bytes}{elapsed_time} + +At the moment of writing this Taco has optimized the \LUATEX\ code base and +I have added dynamic feature support to the \MKIV\ and optimized much of the +critical \LUA\ code. At the time of writing this (December 23, 2007), +\type {mk.tex} counted 142 pages. Our rather aggressive optimizations brought +down runtime from about 29 seconds to under 16 seconds. By sharing as much font +data as possible at the \LUA\ end (at the cost of a more complex implementation) +the memory consumption of huge fonts was brought down to a level where +a somewhat \quote {older} computer with 512 MB memory could also cope with +\MKIV. Keep in mind that some fonts are just real big. Eventually we may decide +to use a more compact table model for passing \OPENTYPE\ fonts to \LUA, but this +will not happen in 2007. + +The following tests show when \LUA's garbage collector becomes active. The blue +spike shows that some extra time is spent on this initially. After that garbage +more garbage is collected, which makes the time spent per page slightly higher. + +\starttyping +\usemodule[timing] \starttext \dorecurse{2000}{ + \input tufte \par \input tufte \par \input tufte \page +} \stoptext +\stoptyping + +\ShowNamedUsage{mk-timing-1-luatex-progress}{luastate_bytes}{elapsed_time} + +The maximum memory footprint is somewhat misleading because \LUA\ reserves more +than needed. As discussed in an earlier chapter, it is possible to tweak to control +memory management somewhat, but eventually we decided that it does not make much sense +to divert from the default settings. + +\starttyping +\usemodule[timing] \starttext \dorecurse{2000}{ + \input tufte \par \input tufte \par \input tufte \par +} \stoptext +\stoptyping + +\ShowNamedUsage{mk-timing-2-luatex-progress}{luastate_bytes}{elapsed_time} + +The last example of this set does not load files, but stores the text in +a macro. This is faster, although not that mich because the operating system +caches the file and there is not \UTF\ collapsing needed for this file. + +\starttyping +\usemodule[timing] \starttext \dorecurse{2000}{ + \tufte \par \tufte \par \tufte \par +} \stoptext +\stoptyping + +\ShowNamedUsage{mk-timing-3-luatex-progress}{luastate_bytes}{elapsed_time} + +There are subtle differences in memory usage between the examples and eventually +test like these will permit us to optimize the code even further. For the record: +the first test runs in 39.5 seconds, the second on in 36.5 seconds and the last one +only takes 31.5 seconds (all in batch mode). + +Keep in mind that these quotes in \type {tufte.tex} are just test samples, and +not that realistic in everyday documents. On the other hand, these tests involve +the usual font loading, node processing, attribute handling etc. They provide a +decent baseline. + +Another document that we use for testing functionality and performance is the +reference manual. The preliminary beta~2 version gives the following statistics. + +\ShowNamedUsage{luatexref-t-luatex-progress-runtime}{luastate_bytes}{elapsed_time} + +The previous graphic shows the statistics of a run with runtime \METAPOST\ graphics +enabled. This means that, because each pagenumber comes with a graphic, for each +page \METAPOST\ is called. The speed of this call is heavily influenced by the +\METAPOST\ startup time, which in turn (in a windows platform) is influences by the +initialization time of the \KPSE\ library. Technically the call time can near zero +but this demands sharing libraries and databases. Anyhow, we're moving towards an +embedded \METAPOST\ library anyway, and the next graphic shows what will happen then. +Here we run \CONTEXT\ in delayed \METAPOST\ mode: graphics are collected and processed +between runs. Where the runtime variant takes some 45 seconds processing time, the +intermediate versions takes 15. + +\ShowNamedUsage{luatexref-t-luatex-progress-intermediate}{luastate_bytes}{elapsed_time} + +In the \type {mk.tex} document we use \TYPEONE\ fonts for the main body of the text and +load some (huge) \OPENTYPE\ fonts later on. Here we use \OPENTYPE\ fonts exclusively and +since \CONTEXT\ loads fonts only when needed, you see several spikes in the time per page +bars and memory consumption quickly becomes stable. Interesting is that contrary to the +\type {tufte.tex} samples, memory usage is quite stable. Here we don't have a memory +sawtooth and no garbage collection spikes. + +The previous graphics combine \LUA\ memory consumption with time spent per page. The +following graphics show variants of this. The graphics concern this document +(\type{mk.tex}). Again, the blue lines represent the runtime per page. + +\ShowMemoryUsage{mk-luatex-progress} + +In \LUATEX\ node memory management is rewritten. Contrary to what you may expect, node memory +consumption is not that large. Pages seldom contain more than 5000 nodes, although +extensive use of attributes can easily duplicate this. Node usage in this documents is +as follows. + +\ShowNodeUsage{mk-luatex-progress} + +If node memory usage stays high, i.e.\ is not reclaimed, this can be an indication of +a memory leak. In the December 2007 beta version there is such a leak in math subformulas, +something that will be resolved when math node processing is opened up. The current \MKIV\ +code cleans up most of its temporary data. We do so, because it permits us to keep an eye +on unwanted memory leaks. When writing this chapter, some of the peaks in the graphics +coincided with peaks in the runtime per page, which is no surprise. + +If you want to run such tests yourself, you need to load a module at startup: + +\starttyping +\usemodule[timing] +\stoptyping + +The graphics can be generated with: + +\starttyping +\def\ShowUsage {optional filename} +\def\ShowNamedUsage {optional filename}{red graphic}{blue graphic} +\def\ShowMemoryUsage{optional filename} +\def\ShowNodeUsage {optional filename} +\stoptyping + +(This interface may change.) + +\stopcomponent |