SourceForge.net Logo Home Page Project Page Download CVS repository

Managing large projects

XInclude

Instructions for large fonts and font families may be split into parts so as to speed compilation for testing purposes, distribute work among members of a team, or create libraries of shared functions. Xgridfit's mechanism for splitting projects is XInclude, the W3C "XML Inclusions" specification. XInclude is extremely flexible and simple to use: it allows you to structure a complex project in a number of ways.

To use XInclude in an Xgridfit file requires just two steps. First, include the XInclude namespace declaration in the <xgridfit> element:

    <xgridfit xmlns:xi="http://www.w3.org/2001/XInclude">

Next, to merge part of another file into the file you're editing, add an <xi:include> element:

    <xi:include href="Junicode-Regular-Basic.xgf#xpointer(//glyph)"/>

Here the portion of the href attribute before the "#" is a URI (usually just the name of a file in the present directory), and the portion after the "#" is an XPointer (usually a simple XPath expression). In the case above, xpointer(//glyph) pulls in all the <glyph> elements from the file Junicode-Regular-Basic.xgf. An alternative syntax is to use an xpointer attribute on the <xi:include> element:

    <xi:include href="Junicode-Regular-Basic.xgf" xpointer="//glyph"/>

In addition, several elements can take an xml:id attribute, and can be included by referencing that id: <xgridfit>, <pre-program>, <function>, <macro>, <glyph>: To fetch this <function> element

    <function name="myfunc" xml:id="jun-reg-myfunc">
       . . .
    </function>

use one of these <xi:include> elements:

    <xi:include href="Junicode-Regular-common.xgf" xpointer="jun-reg-myfunc"/>

    <xi:include href="Junicode-Regular-common.xgf#jun-reg-myfunc"/>

Some XSLT engines can handle XInclude and others cannot. The XInclude capability in xsltproc (the engine invoked by the xgridfit and xgridfit-ttx shell scripts) is turned on by simply including the parameter --xinclude on the command line (this is done for you in the shell scripts). For some other engines, turning on XInclude capability is an arcane and difficult matter: you must check the documentation for your preferred engine.

Example of a large project

XInclude is flexible enough to permit you to structure your project in many different ways (you must just remember to avoid recursive includes). Here is a simple example involving the Junicode font (complete source code available at the download site).

Instructions for Junicode-Regular are compiled by running xgridfit against a master file, Junicode-Regular.xgf, which contains nothing but <xi:include> elements:

    <?xml version="1.0"?>
    <xgridfit xmlns:xi="http://www.w3.org/2001/XInclude">
      <xi:include href="Junicode-Regular-common.xgf#xpointer(/xgridfit/infile)"/>
      <xi:include href="Junicode-Regular-common.xgf#xpointer(/xgridfit/outfile)"/>
      <xi:include href="Junicode-Regular-common.xgf#xpointer(/xgridfit/constant)"/>
      <xi:include href="Junicode-Regular-common.xgf#xpointer(//control-value)"/>
      <xi:include href="Junicode-Regular-common.xgf#xpointer(//function)"/>
      <xi:include href="Junicode-Regular-common.xgf#xpointer(//macro)"/>
      <xi:include href="Junicode-Regular-common.xgf" xpointer="jun-reg-prep"/>
      <xi:include href="Junicode-Regular-Basic.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-Latin1.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-LatExtA.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-LatExtB.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-IPA.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-SpacMod.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-CombDiac.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-LatExtAdd.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-GenPunct.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-PUA.xgf#xpointer(//glyph)"/>
      <xi:include href="Junicode-Regular-NoEncode.xgf#xpointer(//glyph)"/>
    </xgridfit>

The file Junicode-Regular-common.xgf contains no <glyph> elements, but only global elements: <infile>, <outfile>, <constant>, <control-value>, <function>, <macro>, <pre-program>. These are included with the first seven xi:include elements (the <pre-program> is included by xml:id) Eleven more files contain only <glyph> elements organized by Unicode range. Each of the twelve files validates against the Relax NG schema xgridfit-strict.rnc.

When working with any particular Unicode range, most or all of the other <xi:include> elements can be commented out in the master file, speeding compilation. If more than one developer were working on this font, the file Junicode-Regular-common.xgf might be controlled by the lead developer so as to avoid conflicts introduced by individual contributors, who when necessary would submit patches on that file for approval.

Shared functions and libraries

You very likely have some favorite functions that you insert in all your fonts. In addition, similar members of a font family (such as Bold and Bold Italic) are likely to share functions. For a library of shared functions, simply create an Xgridfit program file in which the only children of the <xgridfit> element are <function> elements. An Xgridfit program file can very easily merge the functions from this library anywhere in the sequence of your own functions:

    <function name="local-function">
      . . .
    </function>

    <xi:include href="My-Library.xgf#xpointer(//function)"/>

    <function name="another-local-function">
      . . .
    </function>

Alternatively, you can merge a single function from the library if you have supplied that function with an xml:id attribute:

    <function name="local-function">
      . . .
    </function>

    <xi:include href="My-Library.xgf#set-left-sidebearing"/>

    <function name="another-local-function">
      . . .
    </function>

Note that the included function must be called by its required name attribute and not by the xml:id attribute, which is needed only when a function is merged, before it gets compiled. The name and xml:id attributes should probably be the same when both are present.

GNU Make

Xgridfit can easily be invoked from a Makefile. But to take advantage of the power and convenience of GNU Make, one ought to be able to split a large Xgridfit program into smaller parts that are compiled only when necessary. This is possible and very practical with Xgridfit.

Here is a possible layout of files for an Xgridfit project:

Our plan will be to compile each of these files separately; then we can use the Unix/Linux cat command to assemble them into a complete script to run in FontForge.

Before we do that, though, we need to solve several problems: the files that contain only glyph programs need to refer to control values, functions, macros, and various globally visible items (constants, variables, custom round states), but these are in another file, MyFont-common.xgf. They may also need to refer to glyph programs in other files. The obvious solution is to import these items into our glyph files via XInclude. But the default behavior of Xgridfit is to compile any control values and functions that it can see, and that can lead to duplicate compilations of many items--not harmful in itself, but certainly inefficient and undesirable.

Xgridfit provides two methods for hiding imported elements from the compiler: one for glyph programs and the other for everything else. Glyph programs can be imported via <xi:include> elements nested inside a <no-compile> element. For example, the following is a child element of <xgridfit> in MyFont-Latin1.xgf:

  <no-compile>
    <xi:include href="MyFont-Basic.xgf#xpointer(//glyph)"/>
    <xi:include href="MyFont-SpacMod.xgf#circumflex"/>
    <xi:include href="MyFont-SpacMod.xgf#tilde"/>
    <xi:include href="MyFont-LatExtA-dotlessi.xgf#dotlessi"/>
  </no-compile>

Many items are imported here: all the <glyph> elements from the Basic Latin range (we could specify the glyphs we need individually, but we need so many from this file that it is less labor to import them all), two from Spacing Modifier Letters and the dotlessi from Latin Extended A. Now a glyph program like this one in our Latin1 range will compile correctly:

  <glyph ps-name="icircumflex">
    <constant name="circumflex-contour" value="0"/>
    <call-function name="ensure-diacritic-gap">
      <with-param name="char-top" value="dotlessi/top + circumflex/total"/>
      <with-param name="diacritic-bottom" value="circumflex/bottom"/>
      <with-param name="diacritic-contour" value="circumflex-contour"/>
    </call-function>
  </glyph>

Imported <glyph> elements enclosed in <no-compile> will also be visible to <call-glyph> elements. And yet the dotlessi and circumflex glyphs will not be compiled while Xgridfit is processing MyFont-Latin1.xgf.

To gain access to the global items in MyFont-common.xgf, we import them with XInclude, but we add a <default> element that tells Xgridfit not to compile them:

  <default type="compile-globals" value="no"/>

  <xi:include href="MyFont-common.xgf#xpointer(/xgridfit/constant)"/>
  <xi:include href="MyFont-common.xgf#xpointer(//control-value)"/>
  <xi:include href="MyFont-common.xgf#xpointer(//function)"/>
  <xi:include href="MyFont-common.xgf#xpointer(//pre-program)"/>
  <xi:include href="MyFont-common.xgf#xpointer(//macro)"/>

(It is not strictly speaking necessary to import the <pre-program>, but bear with me for a moment.) Notice that the work of the <default> element can also be done with the -c command-line option:

    xgridfit -c no MyFont-Latin1.xgf

These imported elements work rather like header files in a C program, supplying information, macro definitions and templates for function calls, but not code to be compiled. They permit the <glyph> programs in MyFont-Latin1.xgf to be compiled as a fragment of the larger FontForge script which the Makefile will assemble.

An important caution is in order. Because of the way Xgridfit numbers control values and functions, it is necessary to import all control values and functions for the font into every file, and always in precisely the same order. If you import only the ones needed in the present file, or if you get them out of order, the compiled code will be seriously broken. For this reason it is best to import control values and functions from a common file with a single <xi:include> element, as above. If a font's functions come from several sources, the safest procedure is to import them into the common file, and then import them from that common file into all other files with a single <xi:include> element.

Here is another caution. When you are making heavy use of XInclude, as in the example above, you may find that includes are being made recursively. This is an error. For example, Several composite glyphs in MyFont-Latin1.xgf refer to dotlessi, a glyph in the Latin Extended A range. But when the XSLT engine tries to include MyFont-LatExtA.xgf, it finds that that file contains XInclude references to MyFont-Latin1.xgf. If the XSLT engine were not clever enough to detect the error, the result would be an endless loop. The solution adopted here is to place the glyph program for dotlessi in a file by itself.

Finally, why did we import the font's <pre-program> into MyFont-Latin1.xgf? It's so we can turn that file, a script fragment, into a freestanding script for testing purposes. Here's the trick: On the command line, turn on the compile-globals option (-c yes) and supply names of input files and output files (-i MyFont.sfd -o MyFont.ttf. Also on the command line, supply the names of the particular glyphs you want to compile. The -g (glyph-select) option overrides the <no-compile> element so that any glyph selected in that way is <always> compiled. If you are working on the composite glyph aacute, for example, this command line will produce a little script for testing purposes:

    xgridfit -c yes -g a+acute+aacute -i MyFont.sfd -o MyFont.ttf MyFont-Latin1.xgf

The a and acute glyphs are imported into MyFont-Latin1.xgf in a <no-compile> element, but they are compiled when named with the -g option.

Here are the Makefile commands for the files discussed above:

SCRIPTS = MyFont-common.pe MyFont-Basic.pe MyFont-LatExtA-dotlessi.pe \
	MyFont-LatExtA.pe MyFont-Latin1.pe MyFont-SpacMod.pe MyFont-out.pe

MyFont.pe : $(RSCRIPTS)
	cat $(RSCRIPTS) > MyFont.pe

MyFont-common.pe : MyFont-common.xgf
	xgridfit -q MyFont-common.xgf

MyFont-Basic.pe : MyFont-Basic.xgf MyFont-common.xgf
	xgridfit -q MyFont-Basic.xgf

MyFont-LatExtA-dotlessi.pe : MyFont-LatExtA-dotlessi.xgf \
	MyFont-common.xgf
	xgridfit -q MyFont-LatExtA-dotlessi.xgf

MyFont-LatExtA.pe : MyFont-LatExtA.xgf MyFont-common.xgf \
	MyFont-Basic.xgf MyFont-Latin1.xgf \
	MyFont-LatExtA-dotlessi.xgf MyFont-SpacMod.xgf
	xgridfit -q MyFont-LatExtA.xgf

MyFont-Latin1.pe : MyFont-Latin1.xgf MyFont-common.xgf \
	MyFont-Basic.xgf MyFont-NoEncode.xgf \
	MyFont-SpacMod.xgf MyFont-LatExtA-dotlessi.xgf
	xgridfit -q MyFont-Latin1.xgf

MyFont-SpacMod.pe : MyFont-SpacMod.xgf MyFont-common.xgf
	xgridfit -q MyFont-SpacMod.xgf

MyFont-out.pe : MyFont-out.xgf
	xgridfit -q MyFont-out.xgf