SyncML Conformance Test Suite
http://sourceforge.net/projects/oma-scts/
Test Results for SyncML V1.1 Server Tests
| 0101 |
passed |
To check if the Test Object implements 'Server Layer Authentication'. SCTS sends SyncHdr with wrong credentials. |
| 0201 |
passed |
To check if the Test Object implements 'Server Layer Authentication'. SCTS sends SyncHdr with no credentials. |
| 0202 |
passed |
To check if the Test Object accepts the credentials sent and proceeds with the Sync Session. |
| 0203 |
passed |
To check if the Test Object responds with Results for a Get on device information. |
| 0204 |
passed |
To check if the Test Object responds with Alerts for all the databases alerted by SCTS. |
| 0205 |
passed |
To check if the Test Object sends valid Sync Tags. |
| 0204 |
passed |
To check if the Test Object responds with Alerts for multiple databases alerted by SCTS. |
| 0301 |
passed |
To check if the Test Object's and SCTS' Sync Anchors match by sending/receiving Alerts for normal two way sync. |
| 0302 |
passed |
To check if the Test Object handles Adds correctly. |
| 0401 |
passed |
To check if the Test Object sends valid Adds. |
| 0402 |
passed |
To check if the Test Object handles a Replace on a data item added by SCTS. |
| 0403 |
passed |
To check if the Test Object sends a valid Replace on a data item added by SCTS. |
| 0404 |
passed |
To check if the Test Object handles a Delete on a data item added by SCTS. |
| 0405 |
passed |
To check if the Test Object sends a valid Delete on a data item added by SCTS. |
| 0501 |
passed |
To check if the Test Object handles a Replace on a data item added by it. |
| 0502 |
passed |
To check if the Test Object sends a valid Replace on a data item added by it. |
| 0503 |
passed |
To check if the Test Object handles a Delete on a data item added by it. |
| 0504 |
passed |
To check if the Test Object sends a valid Delete on a data item added by it. |
| 0505 |
passed |
To check if the Test Object handles a Replace on a non-existant data item as an Add. |
| 0601 |
passed |
To check if the databases are in Sync by forcing a slow sync (SCTS does not send any modifications). |
| 0701 |
passed |
To check if the Test Object responds with an appropriate status code for a Delete with a nonexistent target/source. |
| 0801 |
passed |
To check if the Test Object handles sync without seperate intialization. |
| 0901 |
passed |
To check if the Test Object can handle an Add with multiple items. |
| 0902 |
passed |
To check if the Test Object can handle a Replace with multiple items. |
| 0903 |
passed |
To check if the Test Object can handle a Delete with multiple items. |
| 0904 |
failed |
To check if the Test Object responds with individual 'Status' for each item in a Replace (One item is non-existant). |
| 0905 |
failed |
To check if the Test Object responds with individual 'Status' for each item in a Delete (One item is non-existant). |
| 1001 |
passed |
To check if the Test Object can handle multiple messages. |
| 1101 |
failed |
To check if the Test Object determines from the DevInfo whether or not NumberOfChanges is supported, and if it sends proper NumberOfChanges information to SCTS. |
| 1201 |
passed |
To check if the Test Object determines from the DevInfo whether or not NumberOfChanges is supported, and if it sends proper NumberOfChanges information to SCTS. |
| 1301 |
failed |
To check if the Test Object determines from the DevInfo whether or not Large Objects are supported, and if it properly declares the MaxObjSize and can properly receive Large Objects. |
| 1401 |
failed |
To check if the Test Object determines from the DevInfo whether or not Large Objects are supported. |
| 1501 |
failed |
To check if the Test Object determines from the DevInfo whether or not Large Objects are supported, and if it properly declares the MaxObjSize and can properly send Large Objects. |
| 1601 |
failed |
To check if the Test Object correctly handles Large Objects with incorrect size. |
| 1602 |
failed |
To check if the Test Object correctly handles large objects that are not completely sent. |
| 1701 |
failed |
To check if the Test Object did not commit the Large Object with incorrect size and the incomplete Large Object sent in the last session. |
Test Results for SyncML V1.2 Server Tests
SyncML V1.2 is still a draft. The tests seem to be quite preliminary and buggy. (as of July 2006)
| 0101 |
passed |
To check if the Test Object implements 'Server Layer Authentication'. SCTS sends SyncHdr with wrong credentials. |
| 0201 |
passed |
To check if the Test Object implements 'Server Layer Authentication'. SCTS sends SyncHdr with no credentials. |
| 0202 |
passed |
To check if the Test Object accepts the credentials sent and proceeds with the Sync Session. |
| 0203 |
passed |
To check if the Test Object responds with Results for a Get on device information. |
| 0204 |
passed |
To check if the Test Object responds with Alerts for all the databases alerted by SCTS. |
| 0205 |
passed |
To check if the Test Object sends valid Sync Tags. |
| 0204 |
passed |
To check if the Test Object responds with Alerts for multiple databases alerted by SCTS. |
| 0301 |
passed |
To check if the Test Object's and SCTS' Sync Anchors match by sending/receiving Alerts for normal two way sync. |
| 0302 |
crash |
To check if the Test Object handles Adds correctly. |
| 0401 |
passed |
To check if the Test Object sends valid Adds. |
| 0402 |
crash |
To check if the Test Object handles a Replace on a data item added by SCTS. |
| 0403 |
crash |
To check if the Test Object sends a valid Replace on a data item added by SCTS. |
| 0404 |
crash |
To check if the Test Object handles a Delete on a data item added by SCTS. |
| 0405 |
crash |
To check if the Test Object sends a valid Delete on a data item added by SCTS. |
| 0501 |
crash |
To check if the Test Object handles a Replace on a data item added by it. |
| 0502 |
loop |
To check if the Test Object sends a valid Replace on a data item added by it. |
| 0503 |
crash |
To check if the Test Object handles a Delete on a data item added by it. |
| 0504 |
crash |
To check if the Test Object sends a valid Delete on a data item added by it. |
| 0505 |
crash |
To check if the Test Object handles a Replace on a non-existant data item as an Add. |
| 0601 |
crash |
To check if the databases are in Sync by forcing a slow sync (SCTS does not send any modifications). |
| 0701 |
crash |
To check if the Test Object responds with an appropriate status code for a Delete with a nonexistent target/source. |
| 0801 |
passed |
To check if the Test Object handles sync without seperate intialization. |
| 0901 |
crash |
To check if the Test Object can handle an Add with multiple items. |
| 0902 |
crash |
To check if the Test Object can handle a Replace with multiple items. |
| 0903 |
crash |
To check if the Test Object can handle a Delete with multiple items. |
| 0904 |
|
To check if the Test Object responds with individual 'Status' for each item in a Replace (One item is non-existant). |
| 0905 |
|
To check if the Test Object responds with individual 'Status' for each item in a Delete (One item is non-existant). |
| 1001 |
|
To check if the Test Object can handle multiple messages. |
| 1101 |
|
To check if the Test Object determines from the DevInfo whether or not NumberOfChanges is supported, and if it sends proper NumberOfChanges information to SCTS. |
| 1201 |
passed |
To check if the Test Object determines from the DevInfo whether or not NumberOfChanges is supported, and if it sends proper NumberOfChanges information to SCTS. |
| 1301 |
|
To check if the Test Object determines from the DevInfo whether or not Large Objects are supported, and if it properly declares the MaxObjSize and can properly receive Large Objects. |
| 1401 |
|
To check if the Test Object determines from the DevInfo whether or not Large Objects are supported. |
| 1501 |
|
To check if the Test Object determines from the DevInfo whether or not Large Objects are supported, and if it properly declares the MaxObjSize and can properly send Large Objects. |
| 1601 |
|
To check if the Test Object correctly handles Large Objects with incorrect size. |
| 1602 |
|
To check if the Test Object correctly handles large objects that are not completely sent. |
| 1701 |
|
To check if the Test Object did not commit the Large Object with incorrect size and the incomplete Large Object sent in the last session. |
| 1801 |
|
To check if the the Server responds correctly when both the client and the server edit the same data item. Server attempts to perform a Replace on client updated item. |
| 1802 |
|
To check if the the Server responds correctly when the client receives invalid data. |
| 1901 |
|
To check if the Test can correctly respond to a suspend request. |
| 1902 |
|
To check if the Test can correctly respond to a resume of a suspend sync request. |
| 2001 |
|
To check if the Test can correctly respond to a one-way Sync request. |
| 2002 |
|
To check if the Test can correctly respond to a refresh Sync request. |
| 2101 |
|
To check the Test Object reply when an Add command for a new item is performed to a folder that doesn't exist on the other side. |
| 2102 |
|
To check the Test Object reply when a Delete command is performed to a folder which still has items stored in it. |
| 2103 |
|
To check the Test Object reply when Delete command is performed to a folder that doesn't exist. |
| 2104 |
|
To check the Test Object reply when Move command is performed to move an existing item into a folder that doesn't exist. |
| 2201 |
|
To test whether the server can return correct status code when field level changes cannot be processed for that object and thus requesting for the full object to be resent |
| 2301 |
|
To check the Test Object ability to handle filtering with "VCHAR = %x20-7E ;Visible latin characters within UTF-8 or SPACE character" filter grammar type. |
| 2302 |
|
To check the Test Object ability to handle filtering with "string-value = 1*VCHAR ;Case sensitive string value" filter grammar type. |
| 2303 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "EQ;" ;Equal To (case sensitive)" filter grammar type. |
| 2304 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "iEQ;" ;Equal To (case insensitive)" filter grammar type. |
| 2305 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "NE;" ;Not Equal To (case sensitive)" filter grammar type. |
| 2306 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "iNE;" ;Not Equal To (case insensitive)" filter grammar type. |
| 2307 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "GT;" ;Greater Than (case sensitive)" filter grammar type. |
| 2308 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "iGT;" ;Greater Than (case insensitive)" filter grammar type. |
| 2309 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "GE;" ;Greater Than Or Equal To (case sensitive)" filter grammar type. |
| 2310 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "iGE;" ;Greater Than Or Equal To (case insensitive)" filter grammar type. |
| 2311 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "LT;" ;Less Than (case sensitive)" filter grammar type. |
| 2312 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "iLT;" ;Less Than (case insensitive)" filter grammar type. |
| 2313 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "LE;" ;Less Than Or Equal To (case sensitive)" filter grammar type. |
| 2314 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "iLE;" ;Less Than Or Equal To (case insensitive)" filter grammar type. |
| 2315 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "CON; ;Contains the value (case sensitive)" filter grammar type. |
| 2316 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "iCON; ;Contains the value (case insensitive)" filter grammar type. |
| 2317 |
|
To check the Test Object ability to handle filtering with "log-equalitycomp = "NCON; ;Does Not Contain the value (case sensitive)" filter grammar type. |
| 2319 |
|
To check the Test Object ability to handle filtering with "log-sep = "OR;";Logical OR" filter grammar type. |
| 2320 |
|
To check the Test Object ability to handle filtering with "log-sep = "AND;";Logical AND " filter grammar type. |
| 2321 |
|
To check the Test Object ability to handle filtering with "luid-expression = "LUID;" log-equalitycomp string-value" filter grammar type. |
| 2322 |
|
To check the Test Object ability to handle filtering with "ct-no-value = "NULL;"; No property value for the item" filter grammar type. |
| 2323 |
|
To check the Test Object ability to handle filtering with "ct-filter-keyword = string-value; Valid content-type specific filter keywords" filter grammar type. |
| 2324 |
|
To check the Test Object ability to handle filtering with "ct-filter-value = string-value ; Valid content-type specific property value" filter grammar type. |
Notes
- crash means the test suite crashed, not the Horde server
- loop means the test suite infinitely sends identical messages.
After an incomplete test you may have to manually remove the session data from your php session directory. Otherwise a session may get reused in the next test: this results in a failed test as the server does not request authorization again.
SyncML/CompatTable