6.0.0-beta13
4/10/26
Project/Hippo | Project/HordeForm
Project/HordeApiRewrite

Project Name

Project to rewrite parts of Horde_Rpc and the Horde Core Registry inter-app API for a new major release

Bugs

List any tickets on http://bugs.horde.org/ that cover this issue or are relevant to it.

People

Ralf Lang

Description

Main Design Goals:

  • Facilitate mapping of free form URIs to API calls (for REST)
  • Facilitate customizing/amending APIs by administrator/downstream
  • Allow passing PHP Objects when using the API internally -> Need a defined way to fold them to array/scalar representations on RPC
  • Allow separate authentication mechanisms for external calls (REST, RPC) like using API keys, even limiting them.
  • Allow unauthenticated usage of select RPC API calls.
  • Ability for administrators to turn off remote access to some APIs
  • Avoid boilerplate inside Api method implementation
  • Little or no setup for default case
  • Allow but don't require versioning
  • keep code upgrade costs from H5 to H6 reasonable

Side Goal:

  • Allow routing API calls to external horde instances

Current H5 API

API routing

An API is a named collection of methods

H5 registry allows to route an API to an app and to override this decision for certain methods
For example, ingo is the default target for "filter" api and the defined target for certain methods of the "mail" api, which defaults to imp

    'provides' =\> array\(
        'filter',
        'mail/blacklistFrom',
        'mail/showBlacklist',
        'mail/whitelistFrom',
        'mail/showWhitelist',
        'mail/applyFilters',
        'mail/canApplyFilters',
        'mail/showFilters',
        'mail/newEmailFilter'
    \),

However, currently an App cannot provide separate implementations for the same method name in different APIs. Calling filter/newEmailFilter would yield the mail/newEmailFilter method due to all methods being part of one collective Api.php

The current API also does not allow named parameters, even if some RPC mechanisms would deal with it

If an API provides an evolving protocol, discerning versions and supporting older input and output formats would be the method implementation's task

The registry allows limited runtime introspection (hasMethod, listMethods, listApis) but no hints on input/output formats (parameter list and type)

Proposed H6 API changes

implementation strategy

A runner checks all known ApiTypes (Jsonrpc, Rest, Xmlrpc, ...) if it can handle the request until it finds a suitable candidate

  • Each ApiType decides by request headers, URI scheme, request method, and request body content to rule out api types as fast/cheap as possible
  • A formally suitable apitype checks its router for known command implementations which match any parameter set (path, method, subdomain...)
  • ApiType unpacks request parameters from apitype specific format
  • Perform any prefilter tasks like authentication, permission checking, if required by router configuration.
  • Delegate execution to the implementation class and method found from the router
  • pack return value into backend specific format (if not directly returned by implementation class)
  • Apply post filters if configured (API accounting, gzip postprocessing,
  • The runner writes the response to the stdout
    OPTIONAL
  • A simplified ApiType for internal/interapp calls allows to skip most detection magic but use the extra features is desired - no runner needed here

misc ideas

  • Split $App_Api class into $App_Api_$Api classes - Apis still need to be registered in registry. (MEDIUM)
    -> Limited support with fallback to $App_Api exists in https://github.com/ralflang/Core/tree/class-per-api
    -> needs cleanup

  • Implement parameter and return hints -- TBD -> only needed for external interfaces? (Needed for REST, maybe SOAP)
    -> Generate openapi data from these hints

  • move DAV browsing/CRUD code to API methods -> Does this also make sense for ActiveSync? (OPTIONAL)

  • Factor method routing out of Horde_Rpc_* into a loadable module to decouple Horde_Rpc from the Horde ecosystem (HIGH)

-> The Rest feature will not use the Horde_Rpc package, at least in its current form. Lessons Learned and helper infrastructure should go into a rewrite for Horde_Rpc backends with some smooth transition path.

-> For any meaningful Rest support, Horde_Controller needs to support raw request body.
https://github.com/horde/Controller/pull/1

A simple boilerplate router should be part of Horde_Rpc to facilitate unit testing
A Horde-specific router Horde_Core_Rpc_Router should be part of Horde_Core and interact with registry

Call internal api $App_Api_$Api to actually run a command
should check what returns for PHP objects and reduce them to arrays (sleep or Serializable)
allow plugging separate authentication, accounting ...
may have to be amended by specific classes or config to support some RPC types (REST)

In Question: Does it make sense to also wrap internal calls into a Runner_Internal ?

$registry->calendar->addEvent($args);
$registry->call('calendar/addEvent', $args);

The Internal runner simply moves setting up the execution environment and calling the actual class and method from the registry class.
As of now, no additional functionality is considered but it may be handy for debugging.

++ Accounting

Accounting is optional and will probably only be implemented if somebody is interested. Accounting defines a set of limits (value per timespan) and stores usage timestamps in a backend (user, timestamp, action). If the number of usages inside a sliding window hits the limit, the request is denied.
Asking for a past window (like first of month, first of next month) allows for billing/reporting, if this is of any interest.
Implementation could use horde_histories, horde_locks (no billing due to cleanup) or a text file backend (probably inefficient).

The accounting lib can probably be reused for other things.

++ CODE UPGRADE
For internal app calls, git mv $app/lib/Api.php to $app/lib/Api/$Api.php and change class name. Move overrides into a separate class. Done.
For most external calls, additionally extend to-be-written Horde_Core_Api_Rpc into $App_Api_$Api_Rpc - probably the defaults will be just fine for upgrade.

++ API Protection
Protection modules may be combined, probably at first we only want Horde Auth (like Login, for ActiveSync, DAV) and Extra Auth (API keys or SSL)

No Auth
Horde Default Auth
Admin Only
Need Permission (implies any auth)
Extra Auth (how to map to Horde Users for permissions?)

Per-User global accounting
Per-User Per-Api accounting
Per-User Per-Method accounting

++ CONFIG:
Horde Level:
Rpc
Decide which RPC Types are generally enabled. Default?
Decide which RPC Types use Horde Auth, Separate Auth or both

Accounting
Set up accounting system at all (yes/no)
Set up global per-user accounting (yes/no)
Set up anonymous accounting (yes/no)

App Level:
Upstream provides defaults on which RPC methods are available and how they are protected (multiple available).
Admin / Config may override, empty config uses defaults

++ API characteristics

LOCAL: INTERNAL

  • use registry to discover API methods and their implementations
  • any api/method combination may be served by any application (registry knows)
  • no real metadata on parameter count and formats available
  • currently no named parameters
  • can directly use objects as parameters or return types (rarely used yet, breaks external apis as of now)

RPC: JSON-RPC, XML-RPC

  • use registry to discover API methods and their implementations
  • wraps INTERNAL API in specific formats (JSON, XML)
  • Currently breaks for INTERNAL apis which use objects
  • Requires authentication, but any authorization/limits beyond that is left to the INTERNAL method implementation
  • Needs custom serialization for API objects which don't implement toString
  • In theory, named parameters are possible, but the Horde infrastructure can't handle them
  • HTTP-wise, it's POST to a well-known endpoint
  • need to parse data to find the appropriate api/method, hence implementation

Webdav, Caldav, Carddav:

  • Fixed set of methods
  • Webdav implemented for multiple resources, Caldav/Carddav backends are really designed to work for one app each
  • No need to discover or route methods, can use fixed class layout for implementation (browse, delete, ...)

REST:

  • Resource-centric
  • Should we assume each registry API is a resource?
  • Any URL formats and lengths possible,
  • methods per resource should be limited to http verbs
  • multiple input and output formats may be governed by content type
  • Supports multiple API versions in parallel (either by URL or by content type)
  • each request must be authenticated, no sessions.

Horde AJAX Framework

  • Not currently part of the RPC family
  • separate endpoint
  • URL exposes application and method
  • Uses cookies/sessions/tokens
  • mostly JSON

SOAP:

  • Unsure if it really still works
  • similar to XML-RPC
  • but SHOULD generate meta information for the client (where? how?)

Common:

  • We can discern each http remote protocol by URL and/or content type. What is not covered, should be assumed REST.
  • Once we know the protocol, we know how to find out API and method (and, via registry, implementation)

Resources

Back to the Project List

Home
Usage
Recent Changes
All Pages