[[toc]]
+ !ActiveSync
Horde 4 has support for allowing Horde to sync with mobile devices using the mobile device's support for Microsoft Exchange/!ActiveSync. The code that Horde's !ActiveSync implementation uses for handling the protocol communication is based in part on the Z-Push library (http://z-push.sourceforge.net).
++ People
[MichaelRubinsky Michael Rubinsky]
++ Description
The Horde !ActiveSync implementation provides over the air synchronization of contacts, calendar, and task data to devices such as the iPhone/iPod Touch, Android and any other system supporting !ActiveSync. This implementation supports Microsoft !ActiveSync protocol versions up to 2.5 - the version implemented by Microsoft Exchange 2003.
++ Setup
!ActiveSync support is stable, but there are still devices on which it has not been tested yet. If you have used a device with Horde's !ActiveSync support that is not listed below, please feel free to let us know how it went.
To activate the server, it needs to be enabled in Horde's configuration, on the !ActiveSync tab. It's already enabled by default. The SQL tables that horde uses are created as usual from the Horde configuration screen.
Next, you will need to configure your webserver to redirect the URL /Microsoft-Server-ActiveSync to your horde/rpc.php file. How you do this depends on your webserver and it's configuration. For Apache, something like:
<code>
Alias /Microsoft-Server-ActiveSync /var/www/horde/rpc.php
</code>
// Note: // It has been reported that when running PHP via mod_fcgid on Apache that the Alias directive will not pass the correct URL to the fcgid-script handler. This can be worked around by using a !RewriteRule instead (adapted from http://maurus.net/weblog/2010/10/26/running-z-push-1-4-2-with-apache-and-fastcgifcgid/):
<code>
RewriteEngine On
RewriteRule ^/Microsoft-Server-ActiveSync /horde/rpc.php [PT,L,QSA]
</code>
There has also been a report from that the Authorization headers are not correctly passed when using mod_php with Apache. These are known issues and are should actually already be taken care of by the Horde_Controller_Request object. However, if you are still having issues with !ActiveSync complaining about no Authorization errors, you can try the following configuration:
<code>
RewriteRule .* - [E=HTTP_MS_ASPROTOCOLVERSION:%{HTTP:Ms-Asprotocolversion}]
RewriteRule .* - [E=HTTP_X_MS_POLICYKEY:%{HTTP:X-Ms-Policykey}]
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
</code>
None of these issues have been reported using lighttpd/fastcgi.
For lighttpd:
<code>
alias.url = ("/Microsoft-Server-ActiveSync" => "/var/www/horde/rpc.php");
</code>
++ What works
Contacts, Calendar and Task syncing are working. Note that not all devices support Tasks. Of the tested devices, iOS (versions < 5.0) and Android are lacking native Task applications. The !TouchDown client and Windows Mobile both support Tasks.
Client searching of an "Exchange Global Address List" (GAL) is supported, provided that Turba (or other application providing the //contacts// api) is configured to provide a GAL. See the Turba configuration for details.
Provisioning along with remote wipe is also working on devices that actually support it.
This code has been reported to work on the following devices:
|| Device || Version(s) || Provisioning || GAL Searching || Notes ||
|| HP !WebOS || 2.1.0 || Yes || ? || Contacts, Calendar, Tasks are working, for SSL with a private certificate you have to trust the certificate in the browser ||
|| HTC Desire Z / HTC Desire HD || 2.2 || yes || yes || Contacts and Calendar via native !ActiveSync, SSL ok ||
|| HTC Desire S || 2.3.3 || ? || ? || Contacts and Calendar via native !ActiveSync, SSL ok ||
|| HTC Magic || Android 2.2.1 unbranded || ? || ? || Contacts, Calendars ||
|| iOS Devices (iPhone, iPad, iPod) || 3.1.3 -> 4.3.5 || Yes, with Bugs. Certain versions of iOS - 4.3(8F190) for one, go into a provisioning loop due to a bug in iOS (it continues to send the OLD X-Ms-Policykey value after it receives a new one). || Yes || Contacts and Calendar ||
|| iOS Devices (iPhone, iPad, iPod) || 5.0 || Yes || Yes || Contacts and Calendar, and basic support for Tasks via the Reminders App. ||
|| Motorola Droid || 2.0.1, 2.1, 2.2, 2.3 || Broken support before 2.2, works with 2.2 and above. || Native support in 2.2 and later, earlier versions can use the //Corporate Directory// app in the Marketplace. || Contacts and Calendar data. On Froyo/2.2 SSL connections will NOT work with a self signed certificate even if the "Accept All Certificates" checkbox is selected. See http://www.google.com/support/forum/p/android/thread?tid=45e6836618212fdf&hl=en (A (Free) Level One certificate from http://www.startssl.com/ seems to work well here).||
|| Motorola Milestone || 2.1, 2.2 || See Motorola Droid above. || See Motorola Droid above. || Contacts and Calendar ||
|| Nokia E5-00 || ? || ? || ? || Contacts & calendar via !RoadSync. Calendar works native client but contacts do not seem to work. ||
|| Nokia E90 || !MfE 3.0 || ? || ? || Contacts verified to work. ||
|| Nokia N900 || Maemo 1.3 || No || ? || Client fails to send standard post variables, but Horde works around this. Contacts verified to work. Earlier versions of firmware are either completely broken, or only support !ActiveSync version 12.1. ||
|| Samsung Galaxy Nexus || 4.0.2 (ICS) || Yes, full support. || Yes, native support via the search functionality. || Initial testing with Calendar and Contacts works. Be sure to ENABLE email syncing (even though Horde doesn't support it. Disabling email syncing - even if enabling calendar and contacts - seems to prevent the initial folder sync required for the account to be initially set up on the phone. ||
|| !TouchDown For Droid || Version 6.5.0002 || Yes || Yes || Contacts, Calendar (recurrence/exceptions mostly work - minor bugs still being worked out), and Tasks. ||
|| Windows Mobile Device || 6.1 || ? || ? || Contacts reported to work. ||
|| Windows Mobile Emulator || 6.5 || Yes || Yes || Contacts, Calendars (with proper support for recurring events and exceptions), and Tasks. ||
|| Windows Phone 7 || 7.0 || ? || ? || Contacts, Calendars ||
++ Setting up the device
It's beyond the scope of this page to go into detail for each individual device. In general, you will need to create a new account on the device. The account type should be something like //Microsoft Exchange// or //!ActiveSync//. Some devices use //Corporate//. You will need to enter your normal Horde username and password in the appropriate fields. In the field for the server address, you should enter the **root** of the webserver or virtual host that hosts Horde. For example, if you host horde at http://host.example.com/horde then you should enter //host.example.com//. You can ignore any reference to a //domain// entry.
A special note for the iPhone/iPod (and possibly others) - if you do not use a SSL enabled site you may receive errors about not being able to find the !ActiveSync server. If this happens, just continue, or save, or whatever your option is to continue. On the iPhone, after everything is completely set up, you must go back into the account settings and disable SSL.
After the connection particulars are entered, you should choose to enable the folders that you want sync'd. Contacts, Calendar and Tasks are supported, but your device also needs to support the requested folders.
++ Provisioning/!RemoteWipe
Provisioning allows devices to be more tightly registered with a particular server. It enables the server to be able to send policy settings to the device. These policy settings include things like requiring a PIN to unlock the device, the complexity of the PIN required, the number of failed login attempts allowed etc... Additionally, it enables devices to be remotely wiped so that if a device is lost or stolen, the user or administrator can request the device to be wiped.
Users can initiate a remote wipe, as well as view/manage their partnered devices in the !ActiveSync user preference.
Clicking //Wipe// in the Horde interfaces for device management flags the server to send the wipe command to the device the next time it synchronizes. The next time the device attempts to request a command other then PING or OPTIONS, it will be wiped. The !ActiveSync preference page shows the status of all the user's devices. If the status is listed as //Pending//, and you wish to cancel the wipe request, you may do this by clicking the //Cancel Wipe// button. You should see the status be reset to //Provisioned//. After it is wiped, the status will be shown as //Wiped//, if you wish to allow the device to connect to your server again, you need to explicitly remove the device as a sync partner by clicking the //Remove// button. If you do not remove this entry, the device will continue to be wiped each time it reconnects to the server.
++ What to do if you have problems (or How to help us help you)
**If you are not even able to get past the initial setup page on your phone**: you should first check to be sure you do not have SSL enabled on the phone when you're server is not serving SSL. The iPhone/iPod will not let you turn this off until after you save the configuration, so you must continue through all the errors and go back into the settings to disable SSL. You should also make sure that you have not enabled Provisioning support if your phone does not support it.
**If the configuration went well, but you are not seeing any contacts/calendar items appear on the device:** Some clients require a manual refresh or folder selection after setup when not using the "Automatic Discovery" facility of Exchange. With !TouchDown, for example, after setup you must select the folders you want sync'd under the Advanced settings tab.
If all else fails and you can't figure out the issue, we will be happy to try to help you work it out, but you should be able to check/provide us with the following:
* Be sure to check the notes in the chart above for your specific device. This might be a known issue, or a workaround may be known.
* Check the web server error logs and see if there are any PHP errors being logged.
* Configure Horde to send !ActiveSync log messages to a separate logfile. This is configured on the !ActiveSync tab of Horde's configuration screen.
* If you are able to, it would also be useful to run a wireshark session to capture the network communication.
* In some cases, it might be useful for us to see the affected device's state records in the database.
++++Using tshark (command line wireshark) to obtain a network capture
If you want to sniff the traffic on your server, and wireshark is not available becuase there is no windowing system, you can use the tshark application instead. The following command will capture http traffic on port 80, and will ignore most requests we are not interested in. It's worth mentioning that for the capture to be useful, you MUST not setup SSL on the device. Depending on your user's rights, you may need to run this as sudo:
<code>
tshark 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)' -w /path/to/capture/file
</code>
++ Horde_ActiveSync vs. Z-Push
Horde_ActiveSync was based on Z-Push. The code that handles the protocol level is essentially the same, though it has been heavily refactored and cleaned.
Z-Push comes out of the box with a number of backends. The only one that is really //fully// functional is the "ICS" backend which connects to a Zarafa server. In addition to the ICS backend, Z-Push also provides a number of other backends - all of which extend what they call the "Diff" backend.
The diff backend is a very inefficient way of determining what needs to be synched. It uses file based storage - depending on the Z-Push version it uses either a single file or a directory of files for each device. These files contain, along with some basic device state information, a list of every UID that is on the device.
To determine what has changed, Z-Push essentially polls whatever storage backend e.g., the IMAP server, every $timeout seconds to get the **full** list of message IDs on the server. It then iterates over all the UIDs that are known to be on the device and stats every single one of these UIDs against the server to get the modification time, flags etc.
Like mentioned above, a number of backends are based on this Diff backend. Out of the box, if you are not using a Zarafa server you have the following options:
* An IMAP backend - obviously only for syncing emails. This uses the PHP IMAP functions for fetching the complete list of message IDs and stating each and every UID - on each and every PING loop - by default something like every 10 seconds or so.
* A vCard backend that syncs contacts against a directory of vCards on the server.
* A Kolab backend - I'm not sure how complete this is, but this may be the backend that's used by Kolab's !ActiveSync support.
* A Maildir backend that - from what I can tell - syncs email by using filesystem based commands instead of going through an IMAP server.
* A !SearchLdap backend that - from what I can tell - is used as a Global Address Book source for searching for individual contacts (not for syncing the entire address book).
As of the time Horde_ActiveSync was written, you could only use //one// backend at a time - so, unless you were using Zarafa (or maybe Kolab) you could sync email //or// contacts. Since then they have started a "combined" backend that is supposed to wrap any number of backends. Last I checked it wasn't complete yet.
The main differences between 1.5.x and 2 are the versions of EAS that are supported. 1.5.x supports only up to Exchange 2003sp1 (same as Horde). Version 2 is supposedly going to support up to EAS 14 (Exchange 2010) - though I believe only up to 12.1 (Exchange 2007) is currently working.
+++The main differences
Some of these are specific to using Horde data as a backend to Horde_ActiveSync:
# Modularity. Separate classes for maintaining device state and for obtaining message diffs. If not using Horde as a backend, all that would theoretically be needed is to write a new class that extends Horde_ActiveSync_Driver_Base. This allows the backend, itself, to determine the best way to calculate diffs...like using Horde's History system. Support is also built in to allow using a different method of determining diffs based on the collection we are syncing. So, for example, if we ever implement email we would need a way of determining diffs that is not based on on the History system. This implementation allows us to easily add a new diff system and use multiple diff systems at once e.g., use History for contacts/calendar/tasks and some other diff system for email. In Z-Push, diff generation and state management are tightly coupled.
# Efficiency. The history based diff engine is orders of magnitude more efficient than using Z-Push's file based diff backend.
# Unless using Zarafa, Z-Push contained no message-specific logic. For example, no code for dealing with appointment related issues such as timezones, recurrence series etc. Horde_ActiveSync contains support for these things out of the box.
# At least at the time of writing, Horde allowed better configuration of things such as heartbeat/timeout intervals. We also actually allow configuring available security policies. Z-Push had basic support for provisioning and for turning on or off the requirement for a device PIN - but contained no facility for configuring any of the other options. Also, if using Horde_ActiveSync as part of a typical Horde install you get all the ease of configuration that our administration interface provides. I think some of this has improved as part of implementing more recent EAS versions in Z-Push.
# I don't have hard data on device compatibility as it compares to Z-Push, but I do know that we have fixed some device specific issues in the past that - at least at the time - were not working with Z-Push. Certain Nokia devices come to mind that use !MfE. Z-Push lists these as "unknown" compatibility but they work, at least for contacts/calendar with Horde.
# Horde_ActiveSync doesn't currently support email sync - where it is possible, albeit inefficient, to sync email using Z-Push.
# Horde_ActiveSync currently only supports up to Exchange 2003sp1 - Z-Push has code in development that works to some extent with 2007 and plans to look at supporting up to 2010.
# Horde_ActiveSync supports multiple users per device - when the device supports it. AFAIK, Z-Push only supports a single user account per device.
++ Todo
* Implement more recent protocol version support - version 12EAS 12.0 (Exchange 2007) or maybeEAS 12.1 (Exchange 2007??) should be fairly non-disruptive. Version2007sp1).
EAS 12 would get us moreadds HTML/RTF email, more atomic policy settings, additional search sources, local wipe rules, as well as the ability to send the policy settings to the client asand WBXML based provisioning instead of the more compact wbxml. 14 (Exchange 2010?) would probably be lots more work as itplaintext xml sent in 2.5.
EAS 12.1 adds binary encoded request variables (reduced bandwidth), additional security policies, and a revamped SYNC command that does away with PING, using SYNCthe need for waiting for changes instead.an additional PING request (reduced bandwidth as well as reduced battery usage).
++ Resources
http://z-push.sourceforge.net
https://zimbrabackend.svn.sourceforge.net/svnroot/zimbrabackend/zimbra.php
Some discussion/issues/info on calendar sycing
http://z-push.sourceforge.net/phpbb/viewtopic.php?f=4&t=119&start=10
Some discussion on remote wipe issues with various devices
http://z-push.sourceforge.net/phpbb/viewtopic.php?f=4&t=802
Useful information/examples:
http://www.scribd.com/doc/6601589/W11-Server-Active-Sync
http://en.wikipedia.org/wiki/Exchange_ActiveSync - Good description of differences between AS versions.
Another AS implementation in PHP - there is some good implementation information there for the taking.
https://svn.tine20.org/svn/trunk/tine20/ActiveSync
Not much here at all yet, but worth keeping an eye on:
http://code.google.com/p/libeas/
Some links to very detailed articles. Mostly more recent EAS versions though.
http://paulrobichaux.wordpress.com/2011/08/09/advice-to-exchange-activesync-developers/
http://blogs.msdn.com/b/exchangedev/
!TouchDown Client:
http://www.nitrodesk.com/dk_touchdownFeatures.aspx
Android SDK:
http://developer.android.com/sdk/index.html
iPhone/iPod Touch EAS:
http://manuals.info.apple.com/en_US/Enterprise_Deployment_Guide.pdf
Windows Mobile Emulators:
The cellular network emulator:
http://www.microsoft.com/downloads/info.aspx?na=47&p=2&SrcDisplayLang=en&SrcCategoryId=&SrcFamilyId=38c46aa8-1dd7-426f-a913-4f370a65a582&u=details.aspx%3ffamilyid%3dA6F6ADAF-12E3-4B2F-A394-356E2C2FB114%26displaylang%3den
Device Images:
http://www.microsoft.com/downloads/details.aspx?familyid=38C46AA8-1DD7-426F-A913-4F370A65A582&displaylang=en
How to set it up:
http://www.devx.com/wireless/Article/40981/1954