6.0.0-git
2018-12-13
Last Modified 2014-11-27 by Jan Schneider

Rdo - Rampage Data Objects

Rdo is a DataMapper implementation for Horde, using domain/entity objects combined with Mapper objects that handle the actual data manipulation.

Bugs

Rdo currently gives unexpected results for to-one relations where no or a wrong relation key exists. [#10880]

Ideas for H5

* __toString() for Horde_Rdo_Query objects. Basically a convenience wrapper for ->getQuery().
* returning exceptions instead of null for wrong relation keys or null relation keys. Facilitates chaining relations like $baby->mother->granny->name. Currently if granny isn't present, we run into a php fatal (method on non-object). This is a BC break.

People

ChuckHagenbuch is the primary author of Rdo.

Description

Notes on database abstraction

MDB2_Schema xslt:
http://cvs.php.net/viewvc.cgi/pear/MDB2_Schema/docs/

regarding charset support, here's a topic summary:

http://www.alberton.info/dbms_charset_settings_explained.html
http://oss.backendmedia.com/MDB2/CharacterSet
http://pear.php.net/bugs/4666

In MDB2, most drivers basically resort to a "SET NAMES" query.
The MySQL driver initially used the "SET character_set_client"
statement, but I changed it after some complaints, for it
was not effective for most setups.

Another thing that bothers me, is the fact that
there isn't a single charset naming convention.

http://www.postgresql.org/docs/8.2/interactive/multibyte.html#MULTIBYTE-CHARSET-SUPPORTED
http://dev.mysql.com/doc/refman/5.0/en/charset-charsets.html
http://www.destructor.de/firebird/charsets.htm
http://www.csee.umbc.edu/help/oracle8/server.815/a67789/appa.htm#956722

For instance, Traditional Chinese is labelled as "BIG_5",
"BIG5" or "ZHT16BIG5", Russian as "KOI8", "KOI8R" or "CL8KOI8R",
Western European as "latin1", "ISO8859_1" or "WE8ISO8859P1"...
In the end, a common setCharset() method is kind of pointless
without a standard charset naming convention...
Should the DBAL define a set of constants for the most
common charsets, and leave the mapping to the drivers?

"LIKE" case sensitivity:

I was not thinking of SQLite. I was thinking that MySQL LIKE operator is
case insensitive (which is what most people want) but in other databases
LIKE is case sensitive by default.

Not sure how you did in Metabase, but in MDB2 I have the following code for this:
switch ($operator) {
case insensitive
case 'ILIKE':
$match = $field.'LIKE ';
break;
case sensitive
case 'LIKE':
$match = $field.'LIKE BINARY ';
break;

Not sure if I would use "ILIKE" again, probably I would use like with an optional
attribute to flag case sensitivity.

Logging what Rdo does

See "ror_logs.png" attachment.

Database field sizing

When possible, make varchar fields 2**n - 1 (255, etc). Internally (MySQL), one extra byte is needed to store the length of the string. So when you add the extra byte, the length is then an even power of two. This is desirable for technical reasons. In particular, it ensures that the field will be on an even word boundary given that eight byte words are used.

Obviously, an exception is when the data in a field is inherently of a particular length. For example, NYAuthUser.UserPass contains an MD5 hash of the actual password. md5() always returns exactly 32 characters. So the field should indeed be exactly 32 characters long - and thus a CHAR instead of a VARCHAR. CHAR fields don't need the extra byte.

Paging

You hardly need a library for figuring out pagination. A very simple formula tells you how many "pages" you have.

$itemsPerPage = 10;
$totalRecords = 52; //Result of SELECT FOUND_ROWS()
$pages = ceil($totalRecords/$itemsPerPage);

Based on those numbers, you know you have 6 pages of data. You can then create a loop to generate the links with the proper parameter(s), however you want to format them. For example:

link.php?page=3&itemsperpage=10

Would make the following query:

SELECT * FROM table LIMIT ($_GET['page']-1)*10,10

I may be off by 1 on that. Hope that helps.

On 10/18/07, Brent Baisley <brenttech@gmail.com> wrote:

You don't need to do an extra count query. If you are using MySQL, just add SQL_CALC_FOUND_ROWS to your select query. SELECT SQL_CALC_FOUND_ROWS * FROM ...

You can then run SELECT FOUND_ROWS() to get the total rows without any limits. It's still 2 queries, but the second one is essentially free.

Adding backup database servers

I'm kind of of the opinion that this sort of thing (failover) belongs on the database server, not in the app.

In our environment, the apps that we use all utilize this sort of thing for a rudimentary failover behaviour (postfix, amavisd, etc). We modified courier-authlib and dovecot to include this type of behaviour. We can handle a complete crash of our primary r/w db server and still keep our imap/http/pop3/smtp services operational.

We have two database servers that are circular redundant -> mysql-01 and mysql-02. mysql-01 is the primary r/w server. If an app cannot connecto mysql-01, then I would like to to connec to to mysql-02 and process as normal. When mysql-01 comes back online, it reads the updates from -02 and continues normal operation. We also have a handful of r/o slave servers for the mx servers to query for user metadata.

How would you suggest the database server handle this instead of the app? Obviously if the db server is down, it won't be able to process a message to redirect the host to the next server...

Horde is the only app that we have that doesn't include some sort of failover (on connect). I quick whipped up a patch if you think it might be usefull, otherwise we will just use it here. I have tested it and it works sufficent for our needs. We have had to modify the framework, horde, jonah, kronolith, and turba. The modifications are minor to each file and I have attached them. All changes are off of CVS HEAD.

See "Re_ _horde_ adding backup servers to mysql db driver.zip" attachment.

Resources

http://ajaxian.com/archives/ext-scaffold-generator-plugin-for-rails
http://ajaxian.com/archives/lipsiadmin-rails-20-ext-admin
http://blog.stuartherbert.com/php/2008/05/08/what-should-an-orm-offer/
http://code.nytimes.com/projects/dbslayer
http://www.phpdoctrine.org/index.php/documentation/manual?chapter=migration
http://michaelkimsal.com/blog/gorm-goodness-in-php/
http://grails.org/GORM
http://jan.kneschke.de/2007/2/19/typesafe-objects-in-php
http://jeremy.zawodny.com/mysql/mysql-optimization.html
http://kore-nordmann.de/blog/why_active_record_sucks.html
http://code.google.com/p/pdorm/
http://phing.info/docs/guide/current/chapters/appendixes/AppendixC-OptionalTasks.html#DbDeployTask
http://pooteeweet.org/blog/0/918#m918
http://pooteeweet.org/blog/0/1083#m1083
http://sebastian-bergmann.de/archives/751-Testing-with-SQLite-In-Memory-Databases.html
http://www.blobstreaming.org/
http://www.ddataobjects.com/current/prog/
http://www.phpdoctrine.org/documentation/manual?chapter=migration
http://framework.zend.com/manual/en/zend.db.profiler.html


Back to the Project List