Serf Tools

Introduction to SERF Code Generator
Purpose
The purpose of the SERF Code Generator is to provide an all purpose, easy-to-learn, small and flexible code generator based on XML. I want to build it in Java and use Xerces and Xalan for XML/XSL processing and hopefully also provide a Win32 executable to make it easier for non-programmers to use it.

Introduction
The SERF Code Generator was originally written in Perl and developed because I needed such a tool in my work as a Technical Project Manager at Satama Interactive. It was built for use in web development (specifically with HTML, JavaScript, CSS and ASP in mind) but there is no limitation to what type of code that can be produced.

SERF uses an XML based, template driven approach to code generation. SERF is based on ideas from, in particular, two articles. For more information see Code Generation Templates Using XML and XSL [01] and Code Generation using XML based Document Transformation [02]

Why build SERF?
After having tried to use the XML approach (as described in [01] and [02]), I soon found a few problems (see Problems of the XML approach) and instead downloaded information about Jostraca [03] for evaluation. I think Jostraca has a very interesting approach and while I
really like the idea about mimicking JSP syntax I also found it difficult to produce JSP documents since I needed to escape the <% and %> sequences. I also felt that it was hard to see (since all is written in Java) what parts of the code were JSP code and template script instructions.

I wanted a tool that could be used to produce web source code and that would use a template script language that was compact and that wouldn't be mistaken for source code (template plain text). It was also important for the template script language to be easy enough to
learn, so that web designers could use them without consulting a programmer.

Why use XML?
The choice of using XML and XSLT was easy because of the many tools available and for the wide acceptance of XML as a standard format for storing information. And by using an open source XSLT component a large and complex part of the code generator was taken care of.
Also, most programmers already know or can easily learn the basics of XSL programming (and the SERF template script language) which allows them to produce templates. There are several books and articles written about producing PDF documents, WAP or HTML pages with XML and XSLT components and they all use the same approach as used for code generation and it is therefore easy to transfer that knowledge into knowledge about writing templates for SERF.

An outline of the XML approach
The XML approach to template driven code generation uses two types of documents: object (domain) models and templates. The object model is represented as an XML file and the template is an XSL style sheet containing references to the object model. The template
always consist of two different types of information: plain text and instructions. The template plain text is the source code (with comments) and the template instructions are XSL statements. An XSL Transformer object is used to produce the resulting source code by
combining the template with the object model.

Problems of the XML approach
In [01] and [02] the programmer enters XSL instructions directly into the template. This is a very straight forward, simple and direct approach but it has a few problems.

1) Entity references.
Most CG tools (and all XSLT objects) wants valid XML as input. Since SERF is meant to be used for web development, that would mean encoding some, but not all, of the <, >, &, " and ' characters in the template. SERF transforms the (valid) template input into a valid XSL document as the first step of the process. The user doesn't have to think about what should and what should not be encoded in the template.

2) Readability
XSL statements are rather large and it can be difficult to read the code in the template because it's mixed up and broken apart by XSL elements. SERF executes a series of simple substitutions to transform template script statements into valid XSL elements. The template
script statements are written as XML Processing Instructions (PI) in two parts: the instruction and the XPath argument. For example <xsl:value-of select="@name"/> is written as <?= @name?>. This makes the template smaller and easier to read and understand because it focuses on the XPath instruction which is the most important part of the statement.

Because the template script statements are written as PI's XML editors can set them apart graphically from other parts of the template.

3) Reusing the object model with templates of different languages
Often the same data type (in the object model) have different representations in different languages. Consider the following object model element:

01: <property name="name" type="String"/>

This (generic) data type description cannot be used to create the following SQL code:

01: CREATE PROCEDURE getUserByName @name varchar(255)
02: SELECT id, name FROM users WHERE name= @name;
03: GO

it can however be used directly with a Java template or any other template for a language where String is a valid data type. There are several solutions to this, SERF introduces a post processor that translates generic data types to language specific ones based on a
translation table.

4) XSL solutions are complex
Consider the following template fragment:

01: Dim <xsl:for-each select="properties/property"><xsl:value-of
: select="@name"/>, </xsl:for-each>

It will produce the following VBScript source code:

01: Dim name, address, phone, e-mail, age,

The problem here is the two last characters (, ). We don't want the comma after the last property name, we can fix this in several ways. The best would probably be something like this:

01: <xsl:for-each select="properties/property">Dim <xsl:value-of
: select="@name"/>
02: </xsl:for-each>

which doesn't use the comma at all. But, if we want to use the comma, we can do it like this:

01: Dim <xsl:for-each select="properties/property"><xsl:value-of
: select="@name"/><xsl:if test="position()!=last()">
: , </xsl:if></xsl:for-each>

but now we introduced yet another XSL element and made the code even harder to read than before. SERF can trim trailing characters by adding a post processor statement which is smaller and doesn't break the flow of the source code.

Yet another problem that often is left unhandled is that sometimes you will need to change letter casing. This problem arises for example when declaring Java methods. Consider the following object model element:

01: <property name="age" type="int"/>

and this template fragment:

01: /* get/set methods for each of the properties */
02:
03: <xsl:for-each select="properties/property">
04: public void set<xsl:value-of select="@name"/>(
: <xsl:value-of select="@type"/> <xsl:value-of
: select="@name"/>) {
05: this.<xsl:value-of select="@name"/>= <xsl:value-of
: select="@name"/>;
06: }
07:
08: public <xsl:value-of select="@type"/> get
: <xsl:value-of select="@name"/>() {
09: return <xsl:value-of select="@name"/>;
10: }
11: </xsl:for-each>

together they produce the following Java source code:

01: /* get/set methods for each of the properties */
02:
03: public void setage(int age) {
04: this.age= age;
05: }
06:
07: public int getage() {
08: return age;
09: }

The method signatures does not read getName and setName as is preferred in Java. By changing the object model element you can fix the method signature but then you will have destroyed other parts of the code. The better solution is to define a template function in
XSL which translates the first character of the selected value into upper case. This would add another xsl element around the value-of statement. SERF handles this particular case with a post processor statement similar to the Perl commands (\u, \U, \l and \L) for changing letter cases.

Current position and future plans
I'm sure there are other problems that I haven't yet bumped into. The above mentioned was enough for me to develop SERF and have so far been sufficient for developing a web based Content Management System. The object model together with a number of templates for
generating SQL (create table and create procedure statements) and ASP pages (for adding, editing and deleting content) produces all of the source code except for the CSS for presentation.

The next step is to port SERF to Java (and start using JAXP). I'm planning a GUI for running the code generation tasks instead of using a command line interface. There are also plans to implement a few enhanced template instructions, such as a switch statement.

References
[01] Cristian Georgescu, Code Generation Templates Using XML and XSL
C/C++ Users Journal, January 2002
[http://www.cuj.com/]

[02] Soumen Sarkar and Craig Cleveland, Code Generation using XML based Document Transformation
The Server Side, Your J2EE Community
[http://www.serverside.com/]

[03] Richard J.Rodger, Jostraca: a Template Engine for Generative Programming
[http://www.jostraca.org/]