toward unified, structured documentation

[Douglas Gregor]

On Nov 13, 2007, at 10:03 AM, Benjamin Kosnik wrote:
> I plan on unifying this into something coherent, and more structured
> along the lines of a proper manual.

Great!

> For a GNU project, this usually
> means something like texinfo and docbook. Things that are off the
> table: writing our own freaking documentation tool.

Good. Keep it off the table, because that idea has the nasty habit of  
crawling back on top...

> 1) renee rivera / doug gregor work on boost book with doxygen xml  
> output

For reference, I'm not actually involved in BoostBook at this point.  
Matias Capeletto is leading a renewed BoostBook effort, and has  
gathered a number of volunteers to improve BoostBook and convert more  
Boost documentation to the BoostBook format. BoostBook is getting  
more features, more style, and they've been doing some work to make  
the toolchain more robust. I'll talk a bit more about BoostBook  
below; for more information about the "Improving Boost Docs" (IBD)  
project, which Matias has started, please see:

	http://svn.boost.org/trac/boost/wiki/ImprovingBoostDocs

> 3) docbook tools

The DocBook toolchain gets pretty long, especially when you're  
starting with BoostBook and Doxygen.

> 7) man pages

If you use DocBook (or BoostBook, which transforms to DocBook), you  
get these for free.


So, BoostBook...

BoostBook is the name of a suite of tools and data formats (some new,  
some existing) that I developed a few years ago to provide a unified  
look and feel for Boost documentation, including tutorials and  
reference documentation. Boost has a lot of authors, with a lot of  
different styles, so the grand idea was to provide a way to express  
the interfaces to C++ libraries in a presentation-agnostic way, so  
that reference documentation especially could be presented and cross- 
indexed in one consistent format (that happens to look a lot like the  
C++ Standard). This project has grown well beyond me, but I still  
keep track of what's going on.

BoostBook is currently used by more than 20 Boost libraries (as of  
Boost 1.34.1). Most new libraries are written using the BoostBook  
toolchain, and the aforementioned IBD project has several volunteers  
porting documentation for existing Boost libraries over to BoostBook.  
The point here, is that BoostBook is used by quite a few people in  
the Boost community, Boost depends on it greatly, and it's not going  
away any time soon. To see what kind of documentation BoostBook  
produces, check out:

	http://www.boost.org/doc/html/index.html

The IBD project is making much of this documentation more  
aesthetically appealing (and useful), so things are getting better  
incrementally.

BoostBook's greatest weakness is the length of its toolchain. These  
are the top-level tools and formats in the BoostBook toolchain:

	-  BoostBook XML: this is an XML format (a superset of DocBook XML)  
used for all of the internals of the BoostBook toolchain. Few people  
actually write in BoostBook XML directly, because it's XML, and  
therefore painful. Hence, we have:

	- Doxygen: parses C++ source files to generate XML documentation  
that eventually becomes BoostBook XML. We use the standard Doxygen  
distribution, of course.

	- Quickbook: a Wiki- or ReStructuredText-like format for writing  
tutorial/introductory documentation. The quickbook program (part of  
Boost) translates Quickbook into BoostBook XML.

Most users of BoostBook nowadays write in Doxygen+Quickbook; Doxygen  
is wonderful for reference documentation (since most of it is  
automatic), while Quickbook makes it easy to write tutorial/ 
introductory documentation and gives some simple ways to link those  
tutorials into the reference documentation. Everything gets  
translated into BoostBook XML, then lowered to DocBook XML (via the  
BoostBook XSL style sheets). From DocBook XML, one can go to HTML,  
FO, or man pages (via the DocBook XSL stylesheets). FO can be  
translated into PDF or PostScript (we've used XEP; the open source  
translator, FOP, is horrid).

If I haven't scared you yet, I'll put it in a diagram:

C++ sources -> Doxygen XML -> BoostBook XML -\
                                               -> Merged BoostBook XML
Quickbook sources -> BoostBook XML ----------/

Then, for the output:

Merged                      /--> HTML
BoostBook --> DocBook XML ------> FO --------> PDF
    XML                       \--> Man pages

Look complex? It is. To get BoostBook working, one needs doxygen,  
quickbook (from Boost), and XSLT processor (xsltproc), BoostBook XSL,  
DocBook DTD, DocBook XSL, Granted, we have scripts that automate this  
setup, which mitigate the problem for most people.

I truly don't know whether BoostBook is the right answer for libstdc+ 
+ documentation. It works well for Boost, and its getting better due  
to the efforts of several volunteers (myself not currently included).  
It produces what I think is excellent output, with relatively minimal  
inputs---both Doxygen and Quickbook are very easy to work with---and  
it's really designed for C++ libraries like Boost or libstdc++. The  
toolchain will give us some headaches, certainly, but it might be  
worth the effort.

I don't now of any other good documentation tools out there for C++  
code. Raw Doxygen (with lots of styling) might be a suitable  
alternative with a much smaller toolchain. Synopsis could probably be  
used in the same way as Doxygen in this regard, but I know less about  
it.

	- Doug