Home

Composition of data structures.

Composition in this sense is building a complex data structure via the
operations of cartesian products and unions. Basically, Deft allows
the (almost) trivial creation of complex data structures, and the
manipulation of the structure (and data, of course). Deft's internal
data structure is represented by a single instance of a first normal
table. That Deft is able to do composition is a hopefully obvious
consequence of relational operations and normal form.

Deft currently relies on SQL for data storage, however, SQL has
limited abilities which prevent SQL from supplanting Deft as far as
data structure. Due to the limits of SQL, creating and managing a
Deft-like unitary table is quite difficult. If you consider basic
(standard) SQL, then the act of creating a unified table would require
that all the SQL queries necessary to build the table would be in a
single query (as subqueries).

Deft has scalar variables in the (more or less) normal sense. Deft
does not have (nor needs) arrays, lists, hashes, user created
"structs/classes", or combinations of these structures. Deft has a
single, implicit, first normal table. We distinguish it with the
special name "the Stream" when speaking about it. The Stream is never
referred to since (like mass or gravity) it is fundamental (and
automatic) to the fabric of Deft. It is well established that any
arbitrary data structure can be represented in first normal form. In
order to understand Deft, you should easily understand why (if not,
please contact the authors of Deft so we can write applicable
tutorials). What is not so obvious is that describing the complex
structure only takes a small number of functions (dcc(), desc(),
duc()). The Stream (being a first normal table) is powerful enough for
most uses without additional structure imputed on it. When extra
complexity is necessary (e.g. when you need a tree), you only need a
single, fairly simple function call. Additionally, several different
tree structures are just as easily imputed onto the Stream.


Scalar variables and the execution model.

Each line of Deft is run in a declarative manner. Each line of code
will be applied to each row in the Stream. Scalar variables are
columns from the Stream. In order to illuminate the execution model
you should know about two internal functions: unwind() and
rewind(). Unwind reads a record from the Stream. Rewind writes a
record back into the Stream. (The actual implementation involves a
streaming queue, and is akin to a pop/unshift.) The execution model
for a single line of scalar Deft code:

  $my_counter++;

is 

  while(unwind())
  {
    $my_counter++;
    rewind();
  }

In other words, unwind a row, increment the value in the column
$my_counter, and then rewind the row. Some of you may recognize that
many optimizations of unwind/rewind are possible. 

The way Deft implements composition of data structures allows (almost)
trivial creation of structure. A column containing the structure
information is a column of data like any other column (strucutre data
is first class data). dcc() creates a structure column. A new column
is created that contains an ordering, with a "where" clause. Existing
columns aren't changed.  dcc() happens to be used almost exclusively
for rendering web pages. To give a concrete example, a template with a
loop would require a list of hashes with HTML::Template. The same
template in Deft requires only a single dcc() call (Deft has a
powerful built-in template engine). Unlike HTML::Template where the
programmer is required to build the list of hashes, in Deft the "list
of hash" equivalent structure is merely imputed onto the Stream. The
more complex the problem, the greater the advantage of composing
structure. Any HTML::Template with a doubly nested loop requires "list
of hashes of lists of hashes". Parts of the template outside the loops
will require a separate hash. Traditional programming quickly acquires
many data structures, all with their attendant management code. Our
example of a template with some "flat" values, values in the outer
loop and values in the inner loop requires 2 data structures. Deft
requires two dcc() calls. Deft uses two lines of code, and the data
needs no prior preparation.

Those of you who have been required to create and manage nested lists
of hashes will understand that these structures are a headache. 

The traditional "list of hashes of lists of hashes" must be built to
purpose, and that structure isn't really useful for much else. If the
programmer with an n-way structure needs the same data flat, there
will be two (or more) sets of data to manage. Deft will always only
have one set of data. The traditional subroutines and associated data
structures can't easily be repurposed. Deft subroutines work on data
you care about, and ignore columns you don't care about (including
columns with imputed structure).