Introduction
XCLE was originally started as OKit, a basic object manipulation toolkit for
use in Genetic Programming projects, that soon evolved toward a full-fledged
language and compiler.
While most of the code and object-handling algorithms have been changed with
respect to the OKit libraries, the goal remains essentially the same: to
provide a two-level executable code manipulation system, by building the code
programmatically, or by compiling human-readable and -writable sources.
Anything that XCLE can manipulate is an object, with a complete set of
methods to create, edit and delete internal data. And most of these can also
be specified inline, in a character stream, and parsed to produce binary
object. This second method relies on "Primitives Libraries", or definitions
of a set of low-level instructions, called thereon "primitives". These
primitives associate a name with informations related to execution (think
"C-prototype") and machine-code that realize the actual operation.
We will attempt in this tutorial to show how these objects interact, how
they are produced and used, from a language-user (as opposed to developer)
perspective.
Syntax of XCLE programs
Types
XCLE knows how to handle a few predefined types: integers, floats, strings, lists,
and primitives. These are objects, sharing a common "generic" ancestor. Integers
are 32bit signed integers, floats are double precision floating-point numbers,
lists are arbitrarily long vectors of generic objects, and primitives are the base
of the 'executable' feature of these programs. All these objects are memory-managed,
which means that, except when programming new primitives, very little is to be done
to ensure proper memory allocation and waste-collection.
XCLE Class Diagram
Most, if not all, data types can be implemented in terms of combinations of these
structures. And objects imported from C can be integrated in XCLE using a numeric
pointer value as an integer object in XCLE.
If having a dedicated type for a given data structure is an issue (e.g. to ensure
that the data type is accessed only by the proper handlers), this can be obtained
using a dedicated primitive, in a procedure more extensively detailed in a further
document (see the Extending XCLE tutorial).
Syntax
Syntaxically, XCLE programs are blank-separated successions of strings, lists,
primitives and barewords (i.e. blocks of alphanumeric characters without delimiters).
Integers are all-numeric barewords (containing only the characters 0123456789+-).
Floats are barewords containing numeric characters, or the characters '.', the
floationg point marker, and 'e' or 'E', the exponent marker.
Strings are double-quote delimited successions of characters, allowing for
escaped special characters (\a,\f,\r,\n,\t,\") and octal escapes (of the form
\xxx).
Lists are simply an XCLE-parsable succession of characters, leading to a valid
list of objects, encased in square brackets ( '[' and ']' ).
Primitives take the form '<name>', or '<name:data>', where 'name' is a group
of characters without whitespace nor XCLE delimiters (i.e. <>[]:"), and
'data' is a --single-- valid object. In this form, they need not be defined,
that is, have defined prototype and machine-code properties.
Unaccounted-for barewords (those that are not integers or floats) are first
parsed as primitives (like <bareword>, but only if this exists as a defined
primitive), and failing that as strings.
An example
Here is an instance of a syntaxically valid XCLE program, in its string
representation:
[ "three: " .4e+1 -1 <+> <dupN:2> <tostr> <strcat> ]
Executing XCLE programs
Execution Model
XCLE makes no distinction between executable code and data. Programs are lists,
containing objects of the predefined types. Upon execution of such a list/program,
each object in turn is deposited on a stack, except for primitives.
Primitives are checked against the current state of the stack (as defined by
the objects presents before execution, and those deposited during it). If the
test fails, an exception is raised. Otherwise, the machine-code section of the
primitive gets executed, and the remaining part of the list is examined.
This step-by-step processing, reminding of the RPL syntax (operators are cited
after their arguments), will be called hereafter the "execution stream".
At each step, the contents of the stack are critical to the execution of the
remains of the stream, both in the necessary sense (i.e. the arguments needed
to successfully complete the execution) and the sufficent sense (i.e., the
end of the execution stream behaves only according to the arguments supplied,
whatever the means -- the first part of the program, or objets put on the stack
by hand).
A concrete example
Assuming primitives called '+', 'dupN', 'tostr' and 'strcat' have been defined,
and behave like their name would lead us to expect (respectiveley, numeric addition,
replicating a given number of arguments on the stack, stringification and string
concatenation) the above program would thus be executed in the following way:
- the string "three: " is pushed on the stack
- the float 4.0e+00 is pushed on the stack
- the integer -1 is pushed on the stack
- the '+' primitive is executed, taking 4.0e+00 and -1 from the stack and
leaving the float 3.0e+00
- the 'dupN' primitive is executed, with parameter 2, replicating the
two first levels of the stack
- the 'tostr' primitive is executed, the first stack level 3.0e+00 becoming
the string "3.0e+00"
- the 'strcat' primitive is executed, making "three: 3.0e+00" out of
"three: " and "3.0e+00"
- the execution terminates successfully
Parsing and execution contexts
Primitives definitions
XCLE by itself defines no primitive. The syntax allows you to define named primitives
using the <name> construct, created, by default, empty. Only already defined
primitives will have actual content, meaning their execution will result in anything
else than keeping the stack unchanged.
As we see, defining primitives is thus necessary for XCLE programs to do anything useful.
This can be done in two ways. The first gives you complete control on the primitives
definitions, using API calls to build a primitive template, and register it with the
parsing tools. This process is described in depth in the second tutorial,
Using the XCLE API.
However, the easiest method is to load a predefined module (that is, a file
containing binary definitions of several primitives). This can be done via API
calls, or by using command-line options with the XCLE compiler
cxcl,
A standard primitives module for XCLE,
XCLstd, defines
a wide set of basic operations on predefined objects, like stack operations,
string and list toolkits, standard mathematic functions.
If a particular set of operations is needed, or the standard set is too large, custom
modules can be defined. This process is explained in the Extending XCLE
part of this tutorial.
Practical use of modules
Let us assume we have defined the '+', 'dupN', 'tostr' and 'strcat'
primitives we needed for our above examples, in a module named "MyTest". The effective
file name will more probably be "MyTest.so" (or "MyTest.dll" on Windows), since the
modules are actually shared libraries. We will assume this file is in the current
working directory, or in the library search path. We will use cxcl to execute
our program, through the command line:
cxcl -L -l "MyTest.so" '[ "three: " .4e+1 -1 <+> <dupN:2> <tostr> <strcat> ]'
The -L switch tells cxcl not to load the default module, XCLstd. We then specify
the module we want loaded, with the -l switch. The first non-switch argument
is our program, that will be parsed using the module's definitions, then executed.
The output shows the execution status of the program and the resulting stack contents:
Evaluated [ "three: " 4.0e+00 -1 <+> <dupN:2> <tostr> <strcat> ] ; OK
.= STACK ======================================================================.
| 00008 : |
| 00007 : |
| 00006 : |
| 00005 : |
| 00004 : |
| 00003 : "three: " |
| 00002 : 3.0e+00 |
| 00001 : "three: 3.0e+00" |
*==============================================================================*
