6.0.0-git
2024-03-29
Last Modified 2005-11-08 by Karsten Fourmont

Horde Syncing Roadmap

Things that need to be worked on

Bugs/Limitations

  • History retrieval has to be fixed to narrow results to meaningful entries.
    • sometimes the synthesis clients crashes upon receiving calendar data.
    • The various vcard/vcalendar import/export functions really need some work. Recurring events are not dealt with at all yet.
    • when server has a matching anchor it will always do a delta sync, even when the client requests a slowsync.
    • the setup page should contain buttons to really reset a sync for one database by deleting all mapping entries.

Nokia Issues

  • FIXED, PLEASE TEST: on the second sync (first incremental sync), timestamp negotiation may fail resulting in another complete slow sync.
    • one hour offsets when syncing calendar entries.

Possible Extensions

  • Allow customizations which calendars/addressbooks should be synced. Maybe allow merging of more than one data source (think of private and shared calendar)
    • Proper conflict resolution. current policy is: client wins. (As the client send it changes first.)
    • Aligning Client and Server categories (work started on that. Currently visible only for P800/P900)
    • Fancy features like 'delete old entries from client calendar/todo list' etc. to save space on client.

Code Cleanup

This sections lists stuff that does not directly affects functionality but rather the quality of the code

  • create meaningful phpdoc source code documentation
    • create a testing backend, test cases for sync sessions
    • better error handling (SyncML): currently no real error handling is in place.
    • finish refactoring: all horde specific stuff should go into backend.php. The rest of the package should be Horde independent
    • event handling is a big mess: the current major design flaw is that the classes in SyncML/command are use for parsing the input from the client as well as creating the output for the client: this results in a ulgy double-usage of the output method: the output method of Status.php illustrates these two functions: first it's automatically called by the event handler in SyncML.php to produce the output in response to a Status element sent from the client. This means producing no output is all. And then it's use to create Status responses as result to other operations (Add, Sync etc.). Here the output method of status is called as well, but this time it should do something very much different: produce an actual "Status" output.
    • each syncml command has a (per message) unique commandID. The various output methods take at least the currentCommandID and the ContentHandler as parameters. They return the increased currentCommandID. Non of these three is necessary: they should be moved to global variables or (better) a global object dealing with the state of one message (one script run) in contrast to state.php dealing with the state between different messages.
    • Check where's the best place to draw the border between the rpc and syncml package. Currently syncml.php in RPC does some XML parsing (for body and header). This should be moved to Syncml to leave RPC will all the transfer protocol stuff and Syncml with all the contents.
    • currently the horde api only supports listBy($action,$ts) to retrieve $actions since timestamp $ts. The SyncML protocol requires that the ending timestamp of the sync timeframe to be exchanged _before_ the actual syncing starts. So we need an additional parameter: listby($action, $ts, $ts_end)

Other stuff:

Debugging info

Once the current code makes it to CVS people should give it a try and check if it works. If not, the xml in /tmp/sync/, the horde log and the php error log should be helpful. If you can provide a patch to make it work: great!

Otherwise mail the offending xml and/or logs to the list. I'll have a look at it.

As many phones/pdas/whater should be thrown at the code. I'm especially interested to see the <devinf> device information section that the phones sends in its initial packet. This gets stored in /tmp/sync/syncml_client_0.xml when you start with an empty /tmp/sync/ dir.

Make sure to modify the <cred>...</cred> part in the SyncML header if it exists. It contains your username and password.

I'll try to come up with a brief "how to help with debugging" and add it to the installation docs. Otherwise we might get lots of "just installed the horde rpm and SyncML doesn't work with my XXYY phone! pls hlp!!!! I'm new to linux!!!!!" messages :-)

Data Issues

Another area where I expect problems are the various implementations of the text/x-vcalendar,vcard,... formats flying around. Basically something like


BEGIN:VNOTE

VERSION:1.1

BODY: Here's a quick note

END:VNOTE

SyncML itself doesn't understand these things: it just passes them around between the phone and the external horde api "as they come".

If you have tricky stuff like recurring events, inline photos or other exotic data in the file (or got them "treated" by Outlook), horde or your phone might quickly get a hickup. So we need to test as many of these formats as we can get our hands on.

This is more of a general horde issue than about SyncML. You can test the behaviour by trying to import/export files with this ical/vcard data into kronolith/nag/turba using their respective import/export menu entries. That should use the same codebase.

Right now as I'm writing this, I've noticed Jan creating a framework/iCalendar/tests/charset1.phpt test file. Excellent!

Timestamp and Horde API Issues

a) currently the horde api only supports listBy($action,$ts) to retrieve $actions since timestamp $ts. The SyncML protocol requires that the ending timestamp of the sync timeframe to be exchanged _before_ the actual syncing starts. So we need an additional parameter: listby($action,$ts,$ts_end)

b) the SyncML works as follows:

1) the clients sends its changes to the server

2) the server sends its changes to the client.

so when in step 2), the horde api is called with a request like "give me all changes in horde since the last sync", you get the changes induced by the client in step 1) as well. You have to somehow "tag" them to avoid echoing (and thus duplicatinging) them back to the client. Simply storing the guids in the session is not sufficient: the changes are made _after_ the end timestamp (see a)) of the current sync so you'll dupe them in the next sync. My current implementation deals with this as follows: directly after a client induced change is done in horde, state.php's gettsforAction is called to find out the biggest=latest ts for the given guid. If the horde api would provide me with this info directly (maybe just store it in a static var and provide a getLastTS() method in Driver), that would be great and eliminate costly and redundant DataTree calls. Similar in listBy where you get a list of guids but still need to retrieve the latest History timestamps for these guids manually using the DataTree so you can check out if the changes are a result of a client request or not. If listBy would return an assoc array $guid->ts that would help a lot (and solve a) as well).

Testing Suite

Nobody has access to all the SyncML clients all the time. So there's a always the risk that you supply a fancy enhancement patch that works fine for you but breaks down everything for everyone else. So it would be great to have a "testing suite" (in JUNIT terms) which simulates sync runs for all supported phones. (Maybe done by sending specifically crafted XML files based on the logging data of various phones and checking the results). Before a commit of a new patch a test run is done with this suite to ensure it doesn't break things up. Just an idea (dream) for now, suggestions how this could be done are welcome.