Last Modified 2008-05-27 by Chuck Hagenbuch

Rdo - Rampage Data Objects

Rdo is a DataMapper implementation for Horde, using domain/entity objects combined with Mapper objects that handle the actual data manipulation.



ChuckHagenbuch is the primary author of Rdo.


Notes on database abstraction

MDB2_Schema xslt:

regarding charset support, here's a topic summary:


In MDB2, most drivers basically resort to a "SET NAMES" query.
The MySQL driver initially used the "SET character_set_client"
statement, but I changed it after some complaints, for it
was not effective for most setups.

Another thing that bothers me, is the fact that
there isn't a single charset naming convention.


For instance, Traditional Chinese is labelled as "BIG_5",
"BIG5" or "ZHT16BIG5", Russian as "KOI8", "KOI8R" or "CL8KOI8R",
Western European as "latin1", "ISO8859_1" or "WE8ISO8859P1"...
In the end, a common setCharset() method is kind of pointless
without a standard charset naming convention...
Should the DBAL define a set of constants for the most
common charsets, and leave the mapping to the drivers?

"LIKE" case sensitivity:

I was not thinking of SQLite. I was thinking that MySQL LIKE operator is
case insensitive (which is what most people want) but in other databases
LIKE is case sensitive by default.

Not sure how you did in Metabase, but in MDB2 I have the following code for this:
switch ($operator) {
case insensitive
case 'ILIKE':
$match = $field.'LIKE ';
case sensitive
case 'LIKE':
$match = $field.'LIKE BINARY ';

Not sure if I would use "ILIKE" again, probably I would use like with an optional
attribute to flag case sensitivity.

Logging what Rdo does

See "ror_logs.png" attachment.

Database field sizing

When possible, make varchar fields 2**n - 1 (255, etc). Internally (MySQL), one extra byte is needed to store the length of the string. So when you add the extra byte, the length is then an even power of two. This is desirable for technical reasons. In particular, it ensures that the field will be on an even word boundary given that eight byte words are used.

Obviously, an exception is when the data in a field is inherently of a particular length. For example, NYAuthUser.UserPass contains an MD5 hash of the actual password. md5() always returns exactly 32 characters. So the field should indeed be exactly 32 characters long - and thus a CHAR instead of a VARCHAR. CHAR fields don't need the extra byte.


You hardly need a library for figuring out pagination. A very simple formula tells you how many "pages" you have.

$itemsPerPage = 10;
$totalRecords = 52; //Result of SELECT FOUND_ROWS()
$pages = ceil($totalRecords/$itemsPerPage);

Based on those numbers, you know you have 6 pages of data. You can then create a loop to generate the links with the proper parameter(s), however you want to format them. For example:


Would make the following query:

SELECT * FROM table LIMIT ($_GET['page']-1)*10,10

I may be off by 1 on that. Hope that helps.

On 10/18/07, Brent Baisley <brenttech@gmail.com> wrote:

You don't need to do an extra count query. If you are using MySQL, just add SQL_CALC_FOUND_ROWS to your select query. SELECT SQL_CALC_FOUND_ROWS * FROM ...

You can then run SELECT FOUND_ROWS() to get the total rows without any limits. It's still 2 queries, but the second one is essentially free.

Adding backup database servers

I'm kind of of the opinion that this sort of thing (failover) belongs on the database server, not in the app.

In our environment, the apps that we use all utilize this sort of thing for a rudimentary failover behaviour (postfix, amavisd, etc). We modified courier-authlib and dovecot to include this type of behaviour. We can handle a complete crash of our primary r/w db server and still keep our imap/http/pop3/smtp services operational.

We have two database servers that are circular redundant -> mysql-01 and mysql-02. mysql-01 is the primary r/w server. If an app cannot connecto mysql-01, then I would like to to connec to to mysql-02 and process as normal. When mysql-01 comes back online, it reads the updates from -02 and continues normal operation. We also have a handful of r/o slave servers for the mx servers to query for user metadata.

How would you suggest the database server handle this instead of the app? Obviously if the db server is down, it won't be able to process a message to redirect the host to the next server...

Horde is the only app that we have that doesn't include some sort of failover (on connect). I quick whipped up a patch if you think it might be usefull, otherwise we will just use it here. I have tested it and it works sufficent for our needs. We have had to modify the framework, horde, jonah, kronolith, and turba. The modifications are minor to each file and I have attached them. All changes are off of CVS HEAD.

See "Re_ _horde_ adding backup servers to mysql db driver.zip" attachment.



Back to the Project List