The Tirex tile rendering system can use several rendering backends for creating the tiles. The Mapnik backend uses the Mapnik renderer to create tiles, it will probably be used in most cases.
In order to use the Mapnik backend, you have to install Mapnik first. See Mapnik and www.mapnik.org for more information on that. The Mapnik backend is written in C++ to integrate well with Mapnik. Mapnik is in active development and changes a lot, your Mapnik backend needs to fit to the Mapnik libraries you have installed. Currently you'll have to compile it yourself, see Tirex/Building and Installing.
Its recommended you install the Mapnik example map for starters. Its in the Debian/Ubuntu package tirex-example-map or can be installed with make install-example-map. It contains everything necessary to create a very simple map (just rough continent outlines) with Mapnik. This way you can verify that your Tirex and Mapnik installation works without the need to install a database or load it with OSM data.