Skip to end of metadata
Go to start of metadata

 

Background

Sorcery is the new data access layer in use in Asterisk 12 and above by the PJSIP architecture. As it's a generic implementation it will be used in the future for additional functionality where it makes sense. Sorcery is built upon several core principles:

  • A user of sorcery should minimize the amount of caching it performs
  • A user of sorcery should query sorcery for objects when they are needed
  • A user of sorcery can not modify a sorcery object, unless it has just created it or copied an existing one
  • Multiple wizards (backends) can exist to service requests for an object type

These core principles provide certain guarantees and a feature set which is quite powerful. Currently there is one area, however, where the implementation of the sorcery data access layer is lacking: caching.

In a moderately loaded system running against a database it can be hazardous to query the database for information constantly. Depending on the underlying implementation of the database connector it can even introduce a substantial bottleneck where queries are serialized. Depending on the database itself it can also take some time to respond. A trade-off of having up to date information is to cache previously retrieved information so it is retrieved from memory instead of querying the database.

How Sorcery Caching Works Now

Sorcery caching is currently possible by explicitly configuring a module and object type with a wizard marked as caching as the first wizard followed by the normal wizard. An example is as follows:

Icon

The order of wizards in sorcery.conf defines the order in which they are used for the object type, top to bottom.

 

The caching wizard will be queried first for the object, but will also be given objects that have not been retrieved from a caching wizard. The interface used for this is the same as normal wizards, allowing normal wizards to be used where it makes sense. The only current caching wizard that makes sense, memory, uses a container in memory for object storage. It does not expose a mechanism to expire objects or to clear itself. This is problematic since in a cache you do want the original source to be queried eventually to retrieve the new object.

On this Page

Knowing When To Expire An Object

Knowing when to expire an object is difficult though. Without querying the database you have to rely on other mechanisms:

External Notification

This allows an external entity to provide notification that a specific object, or all objects, should be expired from the cache.

Object Lifetime

This expires an object after a specific time after it was first cached.

 

Both of these mechanisms still provide a period of time where the cached object may differ from the current object. I think both of these provide a well-rounded solution with enough configuration to meet most use cases. It also allows a deployer to choose the balance they need between database access and caching.

Sorcery Memory Caching Module

To keep the existing memory module as slim as possible and to reduce potential regressions a new module will be created, memory_cache, which has additional functionality and configuration for expiring stored objects. Objects will be stored in a container on a per object type basis. Configuration options, CLI commands, and AMI actions will be exposed to control the caching behavior. Invalid values provided to the configuration options will cause an error to occur, and the cache to not get completed.

Configuration Options

Maximum number of cached objects (maximum_objects)

The ability to set the maximum number of cached objects will be present. This will allow memory usage to be reduced if need be. If not set there will be no limit. It will also allow an optimal bucket size for the container to be determined. Once the cache is full if new objects need to be added to it they will replace objects which are both old and have not been used in some time.

Object maximum lifetime (object_lifetime_maximum)

This option will configure the lifetime of objects that have been added to the cache. If an object has not been updated as a result of going stale before the maximum lifetime has been met the object will be removed from the cache. Any subsequent retrievals after this will query the backing wizard instead of the cache. The value will be set in seconds. If this option is set to '0' there will be no object lifetime and the object will never expire.

Object stale lifetime (object_lifetime_stale)

This option will configure the amount of time before an object is marked as stale. This option will be optional but if set the object will be automatically refreshed in the background if it is used when stale but before the maximum lifetime has been met. This refresh will not block the current request and while an object is marked as stale it will still be returned from the cache. The value will be set in seconds.

Rebalancing Interval (rebalancing_interval)

This option will configure the interval at which rebalancing will occur. If this option is not set then rebalancing will not occur. At the provided interval a new container will be created with the optimal number of buckets based on the current number of cached objects and all current cached objects will be moved to it. This will reduce the time spent doing lookups in the cache. The value will be set in seconds. It may be possible, depending on implementation, to have this automatically occur depending on operations performed on the cache itself.

Pre-fetching (prefetch)

This option will configure caching to prefetch objects upon load and store them according to the configured options. This option will accept 'yes' or 'no' and default to 'no'.

CUD (cud)

This option will configure whether create, update, and delete operations are supported. This means that if the backing wizard performs these operations they will be reflected in the cache sooner than if the object was refreshed while in a stale state or expired. This option will accept 'yes' or 'no' and default to 'no'.

Expire on reload (expire_on_reload)

This option will configure whether the memory cache module will expire all objects in the cache upon a reload occurring. This option will accept 'yes' or 'no' and default to 'no'.

CLI Commands

sorcery memory cache expire object

This CLI command will expire a specific object from a specific memory cache.

sorcery memory cache expire

This CLI command will expire all objects from a specific memory cache.

sorcery memory cache stale object

This CLI command will mark an object in a specific cache as stale.

sorcery memory cache stale

This CLI command will mark all objects in a specific cache as stale.

sorcery memory cache show

This CLI command will show the objects stored in the cache.

AMI Actions

SorceryCacheExpireObject

This AMI action will expire a specific object from a specific memory cache.

SorceryCacheExpire

This AMI action will expire all objects from a specific memory cache.

SorceryCacheStaleObject

This AMI action will mark an object in a specific memory cache as stale.

SorceryCacheStale

This AMI action will mark all objects in a specific memory cache as stale.

Sorcery Operations

For all operations except retrieve the module will add an observer to the sorcery instance for the object type to be notified of the create, update, and delete operations. This functionality will be configurable.

Load

If enabled the memory cache module will perform the prefetch operation when sorcery tells it to load all objects.

Reload

If enabled the memory cache module will expire all objects in the cache when sorcery tells it to reload all objects.

Create

On creation of an object the cache will persist it in the container.

Retrieve

On retrieval of an object the cache will pull it from the container.

Update

On update of an object the cache will update it in the container.

Delete

On delete of an object the cache will delete it from the container.

Required Sorcery Changes

When marking an object as stale an automatic refresh occurs in the background. Currently within sorcery there is no mechanism available to explicitly say that you do not want to query the cache when retrieving an object. Since the cache is the one doing the query and needs to bypass itself there are two options for doing so.

API Addition

The API can have an additional function added which bypasses the cache.

Thread Local Storage

When the cache queries for an updated object thread local storage can be used to set additional information on the thread. In the cache retrieve function this thread local storage can be retrieved and checked. If set then the cache immediately returns no object. This would require no code changes.

 

Example Configurations


A cache of 1000 objects with each object being stored for only 5 minutes

 

An unlimited cache with each object being stored for only 2 minutes

 

A cache of 1000 objects with each object being stored for a maximum of 10 minutes, but being refreshed after 5 minutes if used after then

An unlimited cache with each object being stored for a maximum of 10 minutes, but being refreshed after 5 minutes if used after then

 

Testing

The core functionality provided by sorcery already has unit tests which cover caching wizards. It confirms that the caching wizard is used when expected. What needs to be tested for this implementation is the memory_cache sorcery wizard itself. This can be accomplished using the wizard interface it provides.

Each test can create an instance of a memory cache wizard with the options that are to be tested. Objects can be created in the instance and then after a period of time retrieved. The result can be validated against what is expected depending on the options specified.

Potential Test Cases

  • Configure the caching wizard with no object limits, create an object in the caching wizard, confirm it can be retrieved
  • Configure the caching wizard for 10 objects, create 15 objects in the wizard, confirm only 10 objects were created
  • Configure the caching wizard for 10 objects, create 15 objects in the wizard, confirm additional 5 objects displaced older objects
  • Configure the caching wizard to give objects a lifetime of 30 seconds, create an object in the wizard, wait 15 seconds, confirm object still exists
  • Configure the caching wizard to give objects a lifetime of 5 seconds, create an object in the wizard, wait 15 seconds, confirm object no longer exists
  • Configure the caching wizard to give objects a lifetime of 60 seconds with stale time of 30 seconds, create an object in the wizard, wait 30 seconds, retrieve object, confirm background fetch occurs
  • Configure the caching wizard to prefetch objects, confirm that prefetching occurs
  • Configure the caching wizard to rebalance every 5 seconds, create 8 objects in the wizard, wait 10 seconds, confirm that rebalancing occurred
  • Configure the caching wizard with invalid options, confirm they are rejected and the wizard not created
  • Enable CUD support and confirm create/update/delete operations result in cache changes
  • Disable CUD support and confirm create/update/delete operations do not result in cache changes
  • Configure the caching wizard for 10 objects, create 10 objects in the wizard, expire a specific object using AMI action, confirm it is removed from the cache
  • Configure the caching wizard for 10 objects, create 10 objects in the wizard, expire all objects using AMI action, confirm they are removed from the cache
  • Configure the caching wizard for 10 objects, create 10 objects in the wizard, mark a specific object as stale using AMI action, confirm it is marked as stale
  • Configure the caching wizard for 10 objects, create 10 objects in the wizard, mark all objects as stale using AMI action, confirm they are marked as stale

 

  • No labels