I hope you enjoy the ride.

Recently I’ve become fascinated with the concept of functional programming. For several years, I’ve had friends jump from language to language, but they all seem to settle on the functional side of the fence. “Why?” was the inevitable conundrum. For the longest time I couldn’t understand what it had to offer that you couldn’t do in other languages. Like most programming phenomena, it was an epiphanic moment that changed the way I looked at programming for the better forever.

Code is Data; Data is Code

Ok, omgwthbbq does this mean? How is code parseable data? How is data trustable code? For this, I had to rethink my stance on what makes a program an executable, not where the program comes from; it’s a cart-before-the-horse thing. So what makes a program? A program is executed lines of human-readable text compiled or interpreted but just the same converted into machine code before the processor reads the 1′s and 0′s from one stream and pushes a different set of 1′s and 0′s into another stream. Who cares how those lines got there. Could be from hand-entry, or it could be from a variable.
Epiphany the First: Who cares where we get code as long as it’s the right code we need?
PHP has this: variable function names, eval(), call_user_func(). After you start coding using these patterns, it becomes nearly impossible to think otherwise. All of a sudden, “Gee, it would be so much easier if we could just tell functionY about functionX without having to teach classB about classA.” We stop thinking in terms of step-by-grudging-step and into segmented solutions to problems. When we can just pass a function into functionY of classB, this problem disappears. classB doesn’t have to give one darn about classA, just that when it needs to, it can use functionX and the world will keep on spinning.
Epiphany the Second: What data can I trust?
Well, you can trust mine, silly! After all, I wrote it. Ah, so data doesn’t have to come from the user, it’s just a way of saying the A’s and underscores I put into the program can be used to reference other parts of the program without explicitly teaching them about each other. Data can come from anywhere, even code; and code can be executed from anything (with the right compiler/interpreter) including data.

Hopefully, this will spawn my first Git repository. Stay tuned for the linky!

[Edit July 7] Instead of a full-fledged repo, I just threw the PHP code up on gist.github.

What I didn’t know, is that this could get us into a lot of trouble without even expecting it. The problem occurs whenever you try to obtain a reference to an element in a PHP array. Case in point:

$a=Array("one"=>1, "two"=>2,"three"=>null);
$b = $a["four"];
$c =& $a["five"];

What do we get if we print the value of $a variables? $a is now

$a = Array(
  ["a"] => 1,
  ["b"] => 2,
  ["d"] => null,
  ["c"] => &null
)

WHAT?! What happened to $b? What in the world is a pointer to null? PHP actually created the “c” key within my Array, just by referencing it in a read context. Lesson learned: always use isset($a["key"]) before attempting to draw from an Array by reference!

I am going to argue against myself here. I realize that in a prior post I praised PHP dereferencing, longing for the direct pointer manipulation of Perl, by using $value=true; $name='value'; ${$name}==true. However, for the next developer, he or she must understand prior to inheriting the code, how PHP handles inline eval() statements.

In situations where the use of these is unavoidable – usually in dynamically-created variables, LISP anyone? – a detailed comment is assuredly in order. Even if it is simple, an example, the next programmer to stumble across your code should be able to reasonably expect an outcome from the procedure.

In JavaScript I’ve been writing lately, I’ve become very cautious about the nature of functions and closures. Frequently, I’ve adopted the practice of namespacing everything I write. Because of that, grouping functions down to the program area can be tedious. No programmer wants to see com.acme.financial.payment.forms.construct(). With JavaScript, we can fake a with() statement. var pay_form = com.acme.financial.payment.forms; additionally com.acme.financial[params['object_type']].forms[action](params_hash) can save a lot of code later in the process.

As an engineer, we should always be looking for ways to perfect our trade. Often this means coding cleanly, and using conventions. Sometimes this means using quirks to their biggest advantage: shrinking the codebase. Remember, one bug for every ten lines.

It’s not much different than Design By Contract, (DBC) – a coding technique promoted in The Pragmatic Programmer, another book I’m reading for work. The thought is that every method has a contract to which it and other methods – and in OOP, objects/classes – abide by. Everyone fulfills their contract, everyone benefits, and the programmer can go home early that night.

The downside, as T.P.P. describes, is that OOP does not lend itself nicely to automatic DBC programming: namely that method inheritance doesn’t imply contract inheritance. This seems like a relatively small gripe to be made about OOP, in my opinion, especially when a simple coding standard can surmount this problem.

In what I’ve learned from OOP, good style is to use both public and private methods, and methodologies, to their advantage. This, thankfully, lends itself well to implementing DBC.

As with most of my posts, this one includes an example!

/**
 * Basic object class, all other classes should extend this
 */
class BaseObject {
    /**
     * Public accessor to the toString() concept
     */
    public function toString() {
        // contract invariants
        $invariants = Array('this'=>$this);
        // contract presumptions
        assert(is_object($this), get_type($this)." is not an Object");

        // inherited method
        $r_val = $this->_toString();

        // contract post assertions
        assert(equals($invariants['this'], $this), get_class($this)."::toString() modified the Object in a read-only context");
        assert(is_string($r_val), get_class($this)."::toString() did not return a String.");

        return $r_val;
    }

    /**
     * Private implementation of the toString() concept
     */
    private function _toSring() {
        $r_val = Array();
        $r_val[] = $this->some_val;

        return implode("\n", $r_val);
    }
}

/**
 * A class which does what it's supposed to do
 */
class GoodObject extends BaseObject {
    /**
     * This function abides by the BaseObject::toString() contract
     */
    public function _toString() {
        $r_val = Array();
        $r_val[] = $this->some_val;
        $r_val[] = $this->other_val;

        return implode("\n", $r_val);
    }
}

/**
 * A mischevious class
 */
class BadObject extends BaseObject {
    /**
     * This function does not abide by the BaseObject::toString() contract
     */
    public function _toString() {
        $r_val = Array();
        $r_val[] = $this->yet_another_val;
        $r_val[] = $this->some_val;

        return $r_val;
    }
}

So we can see that it’s not too hard to notice that the assertions set in place in the BaseObject class will catch whatever the contract for toString() has been set up. Better even than that, the contract can be amended in one place and it will filter down throughout all subclasses. It’s OOP-DBC; and it could save your life – or at least your program.

Our RDB now contains two paths from Baby (B) to Deceased (D), one including the Person (P) entity. We also presume, though our system does not outright, that there are roughly 10:1 paths from B->P->D than B->D. We’ve been trying to solve one of the old problems of a User-generated SQL query: power and risk versus ease and safety. These are not necessarily exclusive, and I have argued why. We have the technology, now lets put it to good use!

The semantics, as stated, is the easy part: We will store them in our foreign key properties in the code. When the user requests a path be made, we’ll retrieve and display the semantics that relate to the links we’ve chosen.

Now comes the tricky part: choosing which links are best. By default, “average” queries follow a consistent path. Trying to determine if Alice and Bob work for Acme generally means we’re looking for the same link. As is the case if we’re looking for anyone who works for Acme and Borax. In our second query, we’d presumably cross the same link twice. This isn’t a coincidence, it’s a natural state of the human expectation.

Let’s put this into practice. Suppose we cache a rough estimate as to how many unique foreign keys are in one table for its neighbors. (This can be achieved through indexing and is a very fast query in modern RDBMS’s.) Traditionally, when attempting to discern a path from one entity to another, shortest-distance was applied. However, given our RDB situation above, this is probably flawed: What we really wanted was from B->P->D, two hops instead of one. Given that we have a fast lookup that informs us we have an alternate path out of B, let’s try that.

Graph theorists recognize this as a max-flow/min-cut problem. We are looking to maximize our probability that the path chosen is correct; likely the path with the most foreign keys. If I had the time and energy, (maybe I will later), I’d make up a diagram or code snippet to illustrate; but you get my point. Some times the shortest path isn’t always the right one, and with the proper tool set in place we can offer a much more elegant – and noticeably more user-friendly – interface.

In the next part, (which I don’t expect to have finished anytime soon), I’ll be addressing some caveats I think would hinder our progress – most notably worst-case scenarios and impossible Source-Sink choices. It’s the conclusion to a thought constructed more than a month ago, and possibly one of the funnest thought experiments I’ve had in quite some time. I hope you enjoy thinking outside the box!

Using our previous example, we can already see how rapidly a RDb can gain entities and relationships as correctly storing information can require many additional entities. For our “intelligent SQL constructor,” the system will need to weed through the myriad of tables to get to find the fastest path from start to finish.

Now suppose we expand our example yet again and include a second path between Baby and Deceased: “A Baby can be born and soon after become Deceased.” This new relationship throws a monkey wrench in some interfaces, as the most direct path from Baby to Deceased is now a single hop. For some users, that we may abstract the fact a Person will become Deceased can lead to erroneous expectations either from the developer or the user; in either event, this is a discrepancy which will cost patrons and development costs in the end.

So how can we expose this fact? My personal inclination is to be explicit is possible. By describing to users what you, as the developer, were intending, harmful reactions should realistically be cut to a minimum. To do this, the system will need to be mathematically aware of how the RDb is structured and what the relationships imply. As proposed in the exposition, knowledge of existing information, its meaning, and a relation to other algorithms should help us greatly.

The first tool we’ll need is a mathematical expectation of how to go from Baby to Deceased. Because of the wonders of modern medicine, users expect that life for a Baby does not cease until having consumed a significant time on Earth. From the data-modeling standpoint, this is to trace the relationship from Baby to Deceased through the Person entity. However, some UIs may not account for this and choose the direct, 1-hop, relationship.

From a user’s standpoint, this is misleading. “I’m calculating social security checks for the elderly; why do the results include a dozen records which are extraneous?” The system’s presumption that it must follow the most direct path from Baby to Deceased seems flawed, though – for the criteria – its data may be perfectly correct. Shouldn’t we instead initially propose to follow the path with the highest probability of success?

In such an example, based on our knowledge of the use of the system, we estimate that there are ten times the number of records for paths between Baby to Deceased using the Person table as without. Taking this into account, we almost assuredly are retrieving the right result set for social security calculations. Then, for the remaining ten percent, we offer the option to “trim” the Person entity from our query.

Additionally, what the system hasn’t exposed here is the semantic relationship the surmised from the entities provided. Perhaps the results were appropriate for what was provided, just not what was expected. This is the second tool we must produce: a means of exposing to the user our presumption of the semantic meaning of their query. This is, admittedly, less for our system to act upon than to influence what the user will input next, or perhaps withdraw and try again.

By now it might sound familiar to problems faced in graph theory. At the moment, there are two that I can think of which aptly fit into this scenario: maximal flow and minimal cut. Both are incredibly annoying problems with surprisingly low-tech solutions. They also both make use of our mathematical model tool we prescribed earlier. An improvement to this theory which deserves its own article will be proposed at the end of the article.

Luckily, there are means to do all of these things when given a well thought-out framework. Semantic ties can be stored within the framework and the mathematical model can be derived quickly from the data via a few SQL queries. (I already anticipate the cries of outrage over additional SQL queries, but those drawbacks will be addressed in the conclusion; so please, save your flames until the end of the series.)

Typically for relational database schemas, (RDb’s), this results in a tree-generating interface which abstracts the actual syntax away from the user. In theory, it’s an excellent way to afford the power user the ability to generate near-infinite custom reporting tools and also help bug-fixing by relying on a few, axiomatic functions.

This doesn’t come without cost, however. Since a relational database is, by definition, segregated into distinct entities, it can be difficult for a program to discern which relationship between two entities to choose. This decision can be critical and time-sensitive for a user. As mentioned in the previous post, if a Person can choose a Profession to be a Painter or a Plumber but not both, then software which attempts to offer criteria based on these relationships should be contextually aware of what the user is asking for.

Most often, problems like these are kept to a minimum by not offering choices too complex for the system to evaluate in a reasonable time. For instance, let’s continue our previous example and introduce two new entities/tables called Baby and Ghost. These have the relationships “A Baby grows up to be a Person” and “A Person can die and become a Ghost.” For our system, we would like to hide the fact that between the Person and Ghost steps, the person who dies must be listed as Deceased. This is much the same as we do not store the Person’s profession as a line-item, but rather as a separate listing – perhaps with dates they started and ended a particular job. To the user, the details should be available but not mandatory, and requiring the Deceased entry could be an undue hastle and cost precious processing time.

Thusly, our system has to be quite intelligent. It must realize that the Baby becomes a Person, which can obtain none or many Professions, will become Deceased, and could be believed to return as a Ghost. To keep the system under control, an interface is often limited to entities only two relationships, (“two-hops”), away from those already specified. Keeping up with our requirement for interface ease-of-use, the next logical step is to define a way to recursively discern these two-hop relationships and offer the user a means to resolve conflicts or ambiguous decisions. In so doing, we could offer the Baby and Ghost entities, (a four-hop relationship), from the start.

The question then becomes, “What tools will we need to perform such an operation?” Handily enough, I believe we have the capability, even on the Web, with JavaScript and proper algorithm implementation.

The problem with making anything a solitary factor more expandable is that it increases the complexity required to interface with that feature by at least twice that factor — twice in the sense that time and complexity, remember them?, are lower-bounded to being capable of handling the expansion.

Case in point: The difference between a pre-populated list and one whose values are variant depending on which portions of a codebase are considered “active.” This can pose quite the problem depending on how mission-critical this list is. (After all, this is a science; and all science should be repeatable ad infinity.) But how to retain this ability to accept a near-infinite range of values and manage them in some human-readable way?

Here we find the strengths and weaknesses of a language. In C, everything you touch is memory whether you realize it or not. From byte-pointers to malloc-ed blocks of hard disk, C keeps you up to your elbows in references. Perl scales the complexity back by a good factor, but grabbing references is still do-able: $ref = \$scalar is a direct accessor to the memory allocated by $scalar; and conversely, $val = $$ref gets the value of whatever variable resides at the memory location $ref. Pretty handy if you have an unknown-length set of variables with similar names. Now we have a common variable name we can manipulate without having a half-dozen Eval() functions in our code. (Aside: Eval() is nice, but sneakily evil; it’s slow, offers arbitrary code execution if use improperly, and just plain ugly.) Our syntax highlighters can show us exactly what we meant, and in six months when we’ll use this as the basis for the program’s next feature, we’ll have a much clearer idea of what the hell we wanted to get out of it.

But this is Perl. How do we do this in a language which masks references like PHP? Unfortunately, we can’t go flying around the memory stack as easily as Perl does, but we can get pretty close with a single Eval(). (Aside 2: I know, I just got done bashing Eval(); I haven’t forgotten, but we’re in control of the string this code evaluates, so take a breath.)

PHP offers an arguably more elegant Eval() than most languages, the inline eval. So let’s reassume our situation: an unknown number of similarly-named variables is in our module’s scope and we want an elegant, memorable way to access their values for modulation.
In Perl:

$value_1 = "some important value";
for($i=1; $i < $some_known_upper_bound; $i++) {
    $value_variable_name = "value_" . $i;
    $value = $$value_variable_name;
    # some modulator code
}

In PHP:

$value_1 = "some important value";
for($i = 1; $i < $some_known_upper_bound; $i++) {
    $value_variable_name = "value_" . $i;
    # the magic...
    $value = ${$value_variable_name};
    # some modulator code
}

And there you have it. Future proof code that modifies a like-named set of variables with a single, easily understood and aptly named variable.