Last Modified 2013-11-11 by Jan Schneider

The following is an archive copy from http://fourmont.org/drupal/node/2 which seems to have gone, at least temporarily.

New SyncML version 0.7.0

Sun, 01/14/2007 - 18:02 — karsten

SyncML (Synchronization Markup Language) is a protocol standard defined by the Open Mobile Alliance for platform-independent information synchronization.

SyncML is mostly used as a method to synchronize contact and calendar information between some type of handheld device and a computer (personal, or network-based service), such as between a mobile phone and a Horde installation.

For some time now I've been working on a PHP PEAR package to provide a SyncML implementation. This package is part of the Horde groupware application, but is intended to be useful independent from Horde as well.

The new version 0.7.0 is a major step towards separating the SyncML part and the backend (Horde) part. There's now an example SQL backend that works with plain sql tables and does not require Horde at all.

The aim of the SyncML package is to provide a robust implementation of the SyncML protocol to allow groupware applications (especially but not limited to Horde) providing the SyncML functionality of replication their data with PDAs, phones and Outlook.
To allow integration with different groupware apps, the SyncML protocol part needs to be separated from the part communicating with the actual groupware application.
This is done by means of a Backend class that contains all the backend application (horde) dependent code.

Until now, this has been more a vague idea than concrete code but 0.7.0 does change that:
there's now an (abstract) SyncML_Backend class in Backend.php, that defines and documents all the functions a Backend must provide. For some functionality like session handling and logging the Backend class does also provide a default implementation. Plese read the documentation at the beginning of Backend.php.

Individual backend implementation are derived from SyncML_Backend and reside in the Backend/ directory. Currently there are two such backends:

Horde backend

This backend allows replication with the Horde groupware suite, see http://www.horde.org. Development of the SyncML package is done as part of the Horde development, so the Horde backend is the one best tested and most heavily used.

SQL backend

This is new for 0.7.0 and more or less all changes in this release were done to allow creation of this class. The SQL backend is some kind of reference implementation of a backend that uses only plain database tables. The database is directly accessed using the PEAR MDB2 package, successor to the pear DB package. The SQL backend does not require a Horde installation at all. Developers who want to create their own backend can have a look at the SQL backend and take it as a starting point for their own implementation. It provides almost all the functionality of a backend server: the only thing it lacks is conversion of different content types: for example phone1 may store an address book entry with content type text/vcard (vcard3.0, as of rfc2426) while phone2 then needs this entry presented as text/x-vcard (vcard2.1, as of http://www.imc.org). The SQL backend can't do such a conversion. However actual backends (like Horde) normally provide it as they have to "understand" the data's content rather than just store it.

Structural Changes

The backend is one end point of the SyncML package. The other end is the place where the HTTP request gets into the System. Prior to 0.7.0 this has been a bit confusing as some of the logic was done in the Horde RPC package, from there it went to SyncML.php and in there to two other handlers. This has been greatly simplied. As of 0.7.0 there is one central function in SyncML.php:

SyncML_HandleRequest($request, $contentType, $backend = 'Horde',$backendparms = array())

This one you provide with a HTTP request and its content type: application/vnd.syncml+xml or application/vnd.syncml+wbxml as provided by the client.
The function does all the processing, WBXML en- and decoding if necessary and returns the XML or WBXML data the can be passed on to the client.

The SyncML package now only requires the XML_WBXML package, all dependencies to other Horde packages have been removed. Of course individual backends may require additional packages: MDB2 for the SQL backend and a complete Horde installation for the Horde backend.

There's one caveat at the moment: if you plan using one of the Funambol clients for syncing with Outlook, Blackberry or PocketPC, you also need the Horde pear packages iCalendar, String and NLS.

How things work

When the SyncML_HandleRequest in SyncML.php is invoked, it creates a backend with the given parameters and an instance of class SyncML_ContentHandler. The process() method of this class does the processing: it creates either an XML parser or an WBXML decoder which then parse the (WB)XML and call the startElement(), endElement() and characters() callback functions of SyncML_ContentHandler. The startElement() function creates appropriate subclasses of SyncML_Command like SyncML_Command_Alert when an <Alert> is received. These have their own startElement(), endElement() and characters() which are called and collect the individual XML data inside the commands. (The SyncHdr is considered a command here, too). The endElement() of SycML_HandleRequest then invokes handleHeader(), handleCommand() or handleEnd(), to deal with the header, individual commands or the end of the SyncML request.

To create the (WB)XML output, there's a SyncML_XMLOutput singleton. This class creates all the response elements like the response <SyncHdr> by calling outputHeader(). To produce XML, it uses the XML_WBXML_ContentHandler class which, despite its name, has nothing to do with WBXML but creates plain XML output. If the desired output is WBXML, XML_WBXML_Encoder is used. It has the same interface as the XML_WBXML_ContentHandler but produces WBXML.

Most of the business logic is inside SyncML_Sync class. During one sync session more than one database can be synced, like calendar and address book. The SyncML_Sync class handles the Sync of exactly one database. It is setup by the Alert command and then does most of the processing when responding to the Sync command.

For more details about the SyncML protocol see the SyncML spec at http://www.openmobilealliance.org/tech/affiliates/syncml/syncmlindex.html.
The document SyncML Sync Protocol, v1.1 (pdf) is the most relevant one: it describes how a sync session works.

The test framework

The test framework consists of a test script and individual test cases. A test case for SyncML is basically a recording of a successful sync session. The testsync.php script can then replay such a recording by sending the recorded client requests to the SyncML package and check if the response matches the one in the original recording. Actually the test script has to do quite a bit of magic to make this work: it has to simulate entries in the backend so the server returns them on the next sync and it has to adapt between newly created IDs and IDs used in the recording.

The big advantage of the test framework is that a test case recorded with one backend can be used with any other backend implementation. So when developing a new backend, you can use the test framwork to verify that your implementation is correct.

How to setup a simple SyncML server

Setting up a simple SyncML server using the SyncML package and the Sql backend is as simple as this:

1. Install pear packages for MDB2 and MDB2 database drivers:

pear install MDB2_Driver_mysql MDB2_Driver_mysql

2. Install the SyncML package and the XML_WBXML package from http://pear.horde.org:

pear channel-discover pear.horde.org; pear install --force -c pear.horde.org SyncML XML_WBXML

3. If you want to use the Funambol clients, you also need iCalendar, String and NLS:

pear install --force -c pear.horde.org iCalendar String NLS

4. Create a database in your database engine (like mysql) as described in Backend/Sql.php:

require_once 'SyncML.php';

// Backend Setup:
$backend = 'Sql';
$backend_parms = array('dsn' => 'mysql://syncml:password@localhost/syncml', // adjust as required
                       'debug_dir' => '/tmp/sync', // debug output to this dir, must be writeable be web server
                       'debug_files' => true, // log all (wb)xml packets received or sent to debug_dir:
                       'log_level' => PEAR_LOG_DEBUG); // log everything

// Not all phpinstalls have http_raw_post_data set, ini-config needed.
// Is it also deprecated in later versions? Well, php://input works.
if( isset($GLOBALS['HTTP_RAW_POST_DATA']) ) 
    $input = $GLOBALS['HTTP_RAW_POST_DATA'];
    $raw_post = file_get_contents('php://input');

$response = SyncML_HandleRequest($input, $_SERVER['CONTENT_TYPE'], $backend, $backend_parms);
if (is_a($response, 'PEAR_Error')) {
    header('HTTP/1.0 500 Internal Server Error');

/* Return the response to the client. */
header('Content-Type: ' . $_SERVER['CONTENT_TYPE']);
header('Content-length: ' . strlen($response));
header('Accept-Charset: UTF-8');
echo $response;

Where to get the SyncML package

The SyncML package is part of the Horde framework. At the moment you have to download the framework from htt://www.horde.org to obtain the SyncML and WBXML packages. It'll be included in the pear channel server as soon as possible to allow installation using the PEAR installer.

If you have any questions, send me a message to karsten at horde _dot_ org.