This wiki page describes using parts of the new Configuration Framework introduced in Asterisk 11, and the motivation behind its creation.
- Configuration Loading Overview
- Traditional Configuration Loading in Asterisk
- Introducing: the Configuration Framework
Configuration Loading Overview
Modules in Asterisk - be they applications, functions, channel drivers, supplementary resources, etc. - are responsible for managing their own resources and responding to operations initiated by the Asterisk core. During module load and reload operations, a large part of this responsibility consists of loading and parsing the module's configuration information from an Asterisk compatible configuration file or, optionally, an Asterisk Realtime Architecture (ARA) backend.
The act of loading and parsing configuration information from either source typically involves doing the following operations:
- Load the information from the backing storage into an
- If the configuration supports global values, set each global variable's value by either retrieving individual values one at a time or by browsing all supported values.
- If the configuration supports multiple in-memory objects (users, endpoints, mailboxes, etc.), browse each item that configures the in-memory object and create the object in the module. This may also entail disposing of the current in-memory object and replacing it with the new information.
As we'll see, while performing these operations there are some common pitfalls that many modules in Asterisk fall into.
Traditional Configuration Loading in Asterisk
A Basic Configuration
Let's assume that we want to configure a module
my_module from the configuration file
my_module.conf. We need to populate three variables in
my_module, which are global values in the module:
foobar- a Boolean value, whose default value should be "true".
foo- an integer value ranging between
bar- some string value that we know should not exceed 64 characters.
Our configuration file
my_module.conf may look something like this:
[general] foobar = True foo = 1 bar = Some string value
So that's fairly straight forward. How would we consume it in a module in Asterisk?
my_module Resource Management
my_module that uses these values may look something like the following. We'll start with the basic structure, and then explore the actual loading and parsing of the configuration.
That's fairly simple. So, what do we have?
- Three variables -
foobarthat correspond to the values in our configuration file
- A default value defined for
foobarand range values defined for
- A function
log_module_valuesthat uses the three values by logging them to the Asterisk logging subsystem.
- Handlers for each of the module operations that can be initiated by the Asterisk core,
reloadhandlers defer their work to a function that we haven't defined yet -
So, let's see what
load_configuration might look like.
load_configuration boils down to two operations:
- Load the Asterisk configuration file
my_module.conf. Whether or not we're reloading makes a difference here - if we're reloading, the configuration subsystem will let us know that nothing in the configuration file has changed if the file has not been modified, which means we don't have to go through the mechanics of changing the values in the module. Once we know we have a good configuration object - and that we have to load the values due to a change - we proceed to the next step.
- Load the configuration values into memory. This consists of iterating over each context in the configuration file, and, for each context, the variables in that context. Since we only have one context
[general]in the configuration file, if we encounter a context with any other name, we skip it. Note that the business logic for each of the variables is embedded in this section of code. This includes setting the default value of
foobar, as well as making sure that
foois a valid integer value and within the accepted ranges. If anything would result in an invalid module configuration, we bail by jumping to the cleanup label, disposing of the configuration and returning an error.
While this looks pretty good, there are a few problems with it that a bad configuration can exploit. Worse, even 'normal' operations can cause problems! Let's see how.
A Problem Configuration
First, let's assume that our module is loaded and running with the configuration in the previous section. A system administrator updates the configuration file
my_module.conf with the following values:
[general] foobar = False foo = Oh snap I'm not an integer bar = I'm a new string value
foo is a string and not an integer, the
load_configuration function will fail and cause the module reload operation to be declined. Ideally, if that happens, none of the values in the module should change - we don't want operations currently using the module to start having weird behavior just because an administrator entered some invalid data. What would the actual state of the module be?
Well, since the variables will most likely be parsed in the order that they appear in the
[general] section, the first variable parsed will be
foobar. Its value will be changed from
0. So far so good. Unfortunately, the next variable parsed is
foo, and it most definitely is not an integer. The call to
sscanf will fail, causing the configuration loading to stop. This results in the module having half of the configuration loaded into memory, while the other half has been left untouched:
foobar = False foo = 1 bar = Some string value
There are of course some things we could do to alleviate this.
- We could decide to try and restore the state of the variables in
my_moduleif an error is detected.
- We could parse the configuration values into a temporary object, and only set the values of the module once all of the application level logic has been passed.
Both of these approaches naturally tend to lead towards storing the configuration values in a struct, as opposed to individual variables. Unfortunately, there are some other problems that even this approach won't completely solve.
Another Problem - Threading During Reloads
Remember that both the CLI and AMI can initiate a reload - and those happen on a different thread of execution than the thread executing the dialplan! In fact, there is a very good chance that any reload operation will happen on a different thread than whatever thread executes the logic in your module.
So, what happens if someone calls
log_module_values while a reload is happening? The answer is "that depends" - but you can bet that it won't be pretty. You may get the values before the reload, after the reload, or some combination thereof. Worse, you can run into all sorts of strange (and difficult to debug) problems if the
bar occurs at the same time as
ast_verb, as both items will be attempting to modify/read the contents of the array at the same time.
In more complex modules where the configuration information is stored on the heap, this can lead to all sorts of strange problems - items that get removed from a configuration will no longer exist, pointers will point to NULL (if you're lucky, garbage if you're not), and general mayhem can ensue. Not pretty.
We could, of course, put some locking in to help. What would that look like?
So, that's a little bit better, and now we can guarantee that a reload won't mess with
log_module_values. That's nice, but now we have to put a lock around every access to
foobar. That can get tricky - and expensive - very quickly. And it doesn't solve the previous problem of inconsistent module state when an off nominal reload occurs.
This also ends up being a lot for a developer to keep straight. If only there was an API that helped us with all of this...
Introducing: the Configuration Framework
The examples above illustrate why we looked at writing a new layer of abstraction on top of Asterisk's configuration file loading and parsing. While the existing mechanisms provided a consistent way of loading a configuration file into memory, they didn't provide a way of mapping that configuration information to an in-memory representation of that data suitable for consumption by any module. Worse, rampant concurrency problems and off nominal code path handling have caused headaches for system administrators and developers alike.
So, we set off with the following goals:
- Create a consistent mechanism for parsing configuration information into a module.
- Provide a thread-safe mechanism for loading configuration information into memory.
- Prevent errors in configuration from creating inconsistent state in a module.
- Allow operations currently 'in-flight' to finish with the configuration information they started with.
The Configuration Framework in Asterisk 11 provides meets these goals, although things are going to appear a little different. The module developer has to do a little more work initially in setting up the in-memory objects and providing mappings for those values back to an Asterisk configuration file. The very flexible nature of Asterisk configuration files - and how modules interpret those files - also meant that the Configuration Framework had to be flexible. So we'll take this slow.
my_module using the Configuration Framework
All configuration information is stored in a reference counted object using Asterisk's
astobj2 API. That object can be replaced in a thread-safe manner with a new configuration information object, as we'll see later. Since the object is reference counted, as long as a consumer of the configuration information holds a reference to that object, it will continue to use the configuration information it started with, even if the configuration information is reloaded.
So, that looks different! Let's run down what we have.
- Our global configuration information is now stored in struct
global_options. That struct has
foobarmembers that correspond to the static variables we previously had defined.
- The global configuration information is stored in another struct,
module_config. It has a pointer to an instance of struct
global_options. If we had more information other than global information (such as users or endpoints), we might have more items in this struct.
AO2_GLOBAL_OBJ_STATICis a new macro that creates a special struct that can hold a single item. It also gives us a lock on the object, providing the Configuration Framework a way to safely access, replace, and remove the information stored inside of it.
aco_typeis the first instance of a special type of object that the Configuration Framework provides. It gives us a way to map our general configuration information - stored in
generalpointer - to a context in the Asterisk configuration file. This defines for the Configuration Framework how information is extracted from the configuration file. Looking at each individual field in the struct:
.type = ACO_GLOBAL. Two types are allowed here - either
ACO_GLOBALimplies that global information should be extracted and populated one time out of a configuration file,
ACO_ITEMimplies that the information should be extracted multiple times into multiple items.
item_offset = offsetof(struct module_config, general). This uses the
offsetofkeyword to map the configuration information that will be extracted into the appropriate object - in this case, the object pointed to by the
generalfield in the
category = "^general$". A regular expression matching the category to parse out. In this case,
"^general$"will match the context with a string name of 'general'.
category_match = ACO_WHITELIST. This means that only contexts that match the regular expression specified by the
categorywill be processed. Alternatively, this option can be specified as a blacklist using
So now we have a mapping of our module configuration, and the in-memory representation of the global settings stored in context
[general] to an object that will process that mapping for us. So what's next?
Well, as we mentioned previously, the configuration objects are going to be
ao2 objects, using the
astobj2 API. Let's define the constructor and destructor functions for the
Note that as part of creating the
module_config object, we also create the general settings object. Because we want the lifetime of the general settings to be tied to the lifetime of the
module_config object, we explicitly handle its destruction in
module_config_destructor, rather than pass a destructor function to
ao2_alloc when we create it.
Now, we can associate our general configuration mapping object
general_option with a configuration file that will provide the data.
That isn't a lot of code, but what is there does a lot of powerful stuff. Let's go down the list:
aco_filedefines a mapping of a configuration file to the mapping types that should be applied to that file. The Configuration Framework will consume the
aco_fileobjects and process each file passed to it, populate the in-memory objects based on the mappings that are associated with that file.
- We notify the Configuration Framework of our top level objects using the CONFIG_INFO_STANDARD macro. This:
- Defines a handle
cfg_infoto access the Configuration Framework.
- Associates the special
module_configscontainer with the constructor,
module_config_alloc, that will put the module configuration object into the container.
- Specifies the files to process when the configuration for the module is loaded/reloaded. In this case, this happens to be the file specified by the
- Defines a handle
- Finally, we have
general_options, which is a special array that will be used when we register items to extract. We'll see this in use later.
We're finally ready to start doing some loading! But wait... where's the application level logic for our various variables? And how do we extract them from the Configuration Framework?
Rather than have a separate function that provides the application logic with the parsing, we instead tell the Configuration Framework how to extract each configuration value out of the configuration file, and what logic we want applied to it. We do all of this when we first load the module, as shown below.
foo has to be an integer between
32, and that
foobar should be a boolean value with a default value of
1. Using the Configuration Framework, we've specified how we want those parameters to be extracted in
aco_option_register. Once we've registered the configuration items to be extracted, all of the configuration parsing and loading into the in-memory objects is handled by
aco_process_config is finished, the
ao2 container will have an instance of
module_config inside of it populated with the configuration information from the configuration file
Now how would we use our in-memory object? And what about reloads?
Let's take those in reverse order.
- So, reloads are much easier, and thread-safe to boot. All we have to do is call
aco_process_configand tell it (in the second parameter) that we're performing a reload. This will reload the module's configuration and - if it succeeds - safely swap out the config in the
module_configsobject. If it fails, the previous configuration is left alone so that any operations currently using the module can finish up without having bad data lying around.
- Using the configuration information is a bit trickier, but not by much. We use a new macro
RAII_VARto safely get the
module_configinstance out of the
module_configsobject. Once we have the
module_configobject, we should (because we're practicing defensive coding, and we should always be careful) check whether or not we got valid data. If we did, we're free to use the values in our function. Plus, since
module_configis reference counted, we 'own' that instance for as long as we want it. If a reload happens while we're logging out the current configuration values, its not a problem - our instance won't be changed and won't be disposed of until we leave the scope of
Note that due to some careful use of
C extensions, the
RAII_VAR macro sets up the clean-up of the reference counted objects for us, so that when execution leaves the scope of the
log_module_values function, the object's references are all cleaned up properly! That makes it much easier to simply return out of the function if an off nominal condition is detected, or if the function has multiple exit points.
unload handler is shown below with the complete
my_module source code.
And... that's it! Using the Configuration Framework, we've successfully solved the threading problems that were present in the previous
my_module implementation, as well as prevented errors that can occur when a configuration file has erroneous data. As always, there's no free lunch - there's a bit more work to be done in setting up the mappings between the in-memory objects and their representation in the configuration file - but taken all together, the benefits the Configuration Framework provides are enormous.
There's a lot more to the Configuration Framework than what is described here. The framework lets you:
- Define items using a wide variety of types, including several Asterisk specific types (such as stringfields).
- Define callbacks for individual configuration items that allow you to apply custom application logic when the item is parsed.
- Define callbacks for different stages of configuration parsing.
- Populate multiple types of objects from different sections in a configuration file.
- Handle multiple configuration files for a single module.
And much, much more. The
app_skel application has been rewritten for Asterisk 11 and demonstrates some of the more complex capabilities that the Configuration Framework provides. Some modules were also migrated over to using the new framework, and new modules in Asterisk 11 - such as
chan_motif - also make use of it for their configuration management.
Going forward, we hope that all new modules in Asterisk make use of the new framework to minimize the race conditions and other bugs that can occur when loading configuration information.