Using our previous example, we can already see how rapidly a RDb can gain entities and relationships as correctly storing information can require many additional entities. For our “intelligent SQL constructor,” the system will need to weed through the myriad of tables to get to find the fastest path from start to finish.

Now suppose we expand our example yet again and include a second path between Baby and Deceased: “A Baby can be born and soon after become Deceased.” This new relationship throws a monkey wrench in some interfaces, as the most direct path from Baby to Deceased is now a single hop. For some users, that we may abstract the fact a Person will become Deceased can lead to erroneous expectations either from the developer or the user; in either event, this is a discrepancy which will cost patrons and development costs in the end.

So how can we expose this fact? My personal inclination is to be explicit is possible. By describing to users what you, as the developer, were intending, harmful reactions should realistically be cut to a minimum. To do this, the system will need to be mathematically aware of how the RDb is structured and what the relationships imply. As proposed in the exposition, knowledge of existing information, its meaning, and a relation to other algorithms should help us greatly.

The first tool we’ll need is a mathematical expectation of how to go from Baby to Deceased. Because of the wonders of modern medicine, users expect that life for a Baby does not cease until having consumed a significant time on Earth. From the data-modeling standpoint, this is to trace the relationship from Baby to Deceased through the Person entity. However, some UIs may not account for this and choose the direct, 1-hop, relationship.

From a user’s standpoint, this is misleading. “I’m calculating social security checks for the elderly; why do the results include a dozen records which are extraneous?” The system’s presumption that it must follow the most direct path from Baby to Deceased seems flawed, though – for the criteria – its data may be perfectly correct. Shouldn’t we instead initially propose to follow the path with the highest probability of success?

In such an example, based on our knowledge of the use of the system, we estimate that there are ten times the number of records for paths between Baby to Deceased using the Person table as without. Taking this into account, we almost assuredly are retrieving the right result set for social security calculations. Then, for the remaining ten percent, we offer the option to “trim” the Person entity from our query.

Additionally, what the system hasn’t exposed here is the semantic relationship the surmised from the entities provided. Perhaps the results were appropriate for what was provided, just not what was expected. This is the second tool we must produce: a means of exposing to the user our presumption of the semantic meaning of their query. This is, admittedly, less for our system to act upon than to influence what the user will input next, or perhaps withdraw and try again.

By now it might sound familiar to problems faced in graph theory. At the moment, there are two that I can think of which aptly fit into this scenario: maximal flow and minimal cut. Both are incredibly annoying problems with surprisingly low-tech solutions. They also both make use of our mathematical model tool we prescribed earlier. An improvement to this theory which deserves its own article will be proposed at the end of the article.

Luckily, there are means to do all of these things when given a well thought-out framework. Semantic ties can be stored within the framework and the mathematical model can be derived quickly from the data via a few SQL queries. (I already anticipate the cries of outrage over additional SQL queries, but those drawbacks will be addressed in the conclusion; so please, save your flames until the end of the series.)

Typically for relational database schemas, (RDb’s), this results in a tree-generating interface which abstracts the actual syntax away from the user. In theory, it’s an excellent way to afford the power user the ability to generate near-infinite custom reporting tools and also help bug-fixing by relying on a few, axiomatic functions.

This doesn’t come without cost, however. Since a relational database is, by definition, segregated into distinct entities, it can be difficult for a program to discern which relationship between two entities to choose. This decision can be critical and time-sensitive for a user. As mentioned in the previous post, if a Person can choose a Profession to be a Painter or a Plumber but not both, then software which attempts to offer criteria based on these relationships should be contextually aware of what the user is asking for.

Most often, problems like these are kept to a minimum by not offering choices too complex for the system to evaluate in a reasonable time. For instance, let’s continue our previous example and introduce two new entities/tables called Baby and Ghost. These have the relationships “A Baby grows up to be a Person” and “A Person can die and become a Ghost.” For our system, we would like to hide the fact that between the Person and Ghost steps, the person who dies must be listed as Deceased. This is much the same as we do not store the Person’s profession as a line-item, but rather as a separate listing – perhaps with dates they started and ended a particular job. To the user, the details should be available but not mandatory, and requiring the Deceased entry could be an undue hastle and cost precious processing time.

Thusly, our system has to be quite intelligent. It must realize that the Baby becomes a Person, which can obtain none or many Professions, will become Deceased, and could be believed to return as a Ghost. To keep the system under control, an interface is often limited to entities only two relationships, (“two-hops”), away from those already specified. Keeping up with our requirement for interface ease-of-use, the next logical step is to define a way to recursively discern these two-hop relationships and offer the user a means to resolve conflicts or ambiguous decisions. In so doing, we could offer the Baby and Ghost entities, (a four-hop relationship), from the start.

The question then becomes, “What tools will we need to perform such an operation?” Handily enough, I believe we have the capability, even on the Web, with JavaScript and proper algorithm implementation.

But, no matter how dumb you play, something seeps in. That’s how you weed good teachers from the mediocre ones: they can disseminate knowledge without you even knowing it. They’re kind of like God, in a way; sometimes the vague answers and delayed responses aren’t intended as insults or shirking, it’s a device to make you think and learn for yourself. Sometimes, when you think you need a helicopter to save you, a properly thrown life vest will do just fine.

While at Pittsburgh, I had a tough-as-nails professor for my algorithms class. I spent so many nights compiling Java code until 3AM when my throat was sore from too much tea and my eyes bled zeros and ones, but I got the programs working to spec and in on time. It was, for all intents and purposes, a trial by fire¹. At the end of it, I thought I would forget everything and never use an iota of what it tought me for anything practical in my career. This view was incredibly, humerously, short-minded.

For a while, it was correct. Still, something lingered, and I should be glad it did. What remained was a desire to apply what I’d learned to what already existed. Solving old problems with more efficient solutions is always a challenge. It’s why neural networks and packet routing have evolved. But with these challenges comes personal and professional rewards, not the least of which is the ability to re-use the mental framework for the problem solved.

One of my least-favorite problems was graph search. (There’s a reason no one but mathematicians specialize in developing, proving, or otherwise entrenching themselves in graph theory.) Yet, graph searching is one of the most widely-applicable skills a software architect can develop. Case in point: dynamic SQL construction.

As a mental exercise in theory and practice, there are several parts to this post, interleaved with time for mental stretching, retractions, and analyses & comments. (This is also no coincidence, as for those literary-educated readers the scaffolding is of the classical drama; enjoy.) This is Act I: Exposition.

In my experiment, there are three factors:

• tree complexity,
• semantic-path evaluation, and
• reasonable application to graph theory.

Tree complexity is the time and effort required to maneuver ‘blindly’ from one entity to another, 2-hop-away, entity in a database. Semantic-path evaluation is the real-world implication of the statement’s conjoined entities. For instance, using the tables Painter, Plumber, and Person, a Person can become a Painter or a Plumber but each has a specific role and though both may use water they may not be what we intended to achieve. Without predefining rules, the system will have to build an assumption based on the entities provided. Lastly, while possibly obvious, the application to graph theory may need to be spelled out; this will probably be spent the least time developing.

If you can think of anything I’ve missed or misunderstood, constructive criticizm is encouraged! Act II: WYSIWYG will hopefully follow soon.

¹Taking the class alone isn’t the Pitt CS Trial by Fire. Taking 1520 and 1550 in the same semester is. To exit both classes with B’s was a personal accomplishment, as well as bragging rights.

This is not to say, however, that they are insurmountable. Often, in fact, when we stop trying to over-engineer our system and evaluate the situation as it is, we can make optimizations beyond our expectations. As a simple “for instance,” presuming we have a determinant for a conditional that is initially a free-text entry. According to this pretense, Boolean evaluation can take as little as the first byte in the string or as much as the full text of the string; thus, for sake of computation time, memory, and most of all maintainability, it should be avoided at high cost. Compare this with replacing this free-text with a numerically-indexed list of choices and the Boolean evaluation drops to a static value. (Even better, most implementations of bit-wise integer comparison are executed as close to machine code as possible.)

But, inevitably, there are cases which solutions such as these are not so prevalent or obvious. For instance, when implementing join-context SQL conditionals as PHP, it can take a tremendous amount of derived and out of band knowledge to perform the translation. Combine this with already complex logic, (nested XORs, negated string comparisons, and cross-table lookups), and the code can get slow and ugly very fast. So what do you do as an engineer with an email, the SQL, and a pat on the back? Get a game plan.

It’s more important to pencil in a strategy for defining your criteria than going off of pre-existing assumptions. For one measure, repeated calculations in the original may be consolidated as a single conditional whose value is stored and later addressed; the processing efficiency alone is worth the change when scaling up to enterprise-level datasets. Additionally, as I stated above, changing data types of pre-existing conditionals can be a great boon as well, and performing integer arithmetic is always faster than strcmp.

Once the parameters are defined, we zoom out to observe the algorithm. Keeping in mind the practice degrades maintainability, nesting some repeated conditionals can save processing power down the line. But don’t get nesting-happy; (especially with code you’re simply translating and didn’t have the mental inception for) you’ll forget why two conditionals had been nested or why their order mattered so much. This is where your formal logic class enters the picture. Class, what is {A, B, C, D} & {A, B, C}? Of course it’s {A, B, C}; it’s also ¬{D}. The optimization, however slight, of changing

if($var == A OR $var == B OR $var == C) { something_true(); }

to

if($var != D) { something_true(); }

may not increase performance by leaps and bounds, but if you’re deep in the code anyway why wouldn’t you shave off the time?

Now, after the code is written and tested, there is much rejoicing. The only thing we now have to contend with is if the translation needs to change. And while we may not, (and hopefully will not), go back to square-one with our evaluation, we should at least take the time to consider where we may have gone wrong in our previous implementations and where we can continue to go right.