Policy Regarding Acceptable Use of SCOUT
PURPOSE:
The purpose of this policy is to provide a clear statement to all users of the Situation Awareness & Collaboration Tool (SCOUT) and the responsibility of those using SCOUT to protect information and use the system properly.
SCOPE & APPLICATION
This policy applies to all those who are granted access to SCOUT including, but not limited to, employees of the Governor’s Office of Emergency Services, employees of the Department of Forestry and Fire Protection (CAL FIRE), participating agencies, and all other authorized users.
RESTRICTIONS ON USE
Use must be legal, ethical, and not detrimental to the goals, mission, objectives, and reputation of Cal OES, CALFIRE, or the State of California. Users may not:
- Use the Portal in violation of applicable local, state, national laws, rules, and regulations.
- Use the Portal to upload, transmit, or otherwise distribute any content that is unlawful or unprofessional.
- Distribute information contained within the Portal without authorization.
- Mine the portal to obtain user data.
- Use any automated device or manual process to monitor or gather any data from the Portal or its users in any manner inconsistent with the agency’s public safety mission.
- Create multiple user accounts or create user accounts by automated means or under false or fraudulent pretenses.
- Permit anyone to access the Portal using the User’s credentials.
- Use the Portal for any fraudulent or inappropriate purpose.
ENFORCEMENT, AUDITING, AND REPORTING
Anyone accessing SCOUT who has reasonable belief that another individual or entity is accessing SCOUT in violation of this policy must report such activity to the Governor’s Office of Emergency Services. Violation of this policy may result in the revocation of access to SCOUT. Violators of this Agreement will be held accountable to the greatest extent permitted by law. State employees who violate this Agreement may be subjected to discipline up to and including termination.
SCOUT USER AGREEMENT
User acknowledges, has read and understands, and hereby agrees to abide by the SCOUT Acceptable Use Agreement set forth above.
User understands that as a SCOUT user, user may have access to confidential and/or protected information controlled by state and federal law. All access to the SCOUT system and any type of protected information obtained through SCOUT is based on a “need to know” and the “right to know.” Misuse of such information may adversely affect an individual’s right to privacy, civil liberties, and civil rights, and violates constitutional and state law and/or policy.
"Need to know” means you have a legitimate business purpose in accessing the information. "Right to know" means you have legal authority pursuant to both law and department policy to access the information. To access information, the "need to know" and the "right to know" must exist at the same time. User understands that the unauthorized or improper use of SCOUT may amount to a violation of California Penal Code section 502, as well as other state and federal laws.
Register


Broadcasting
- Introduction
- Concept Overview
- Defining Broadcast Events
- Authorizing Channels
- Broadcasting Events
- Receiving Broadcasts
- Presence Channels
- Client Events
- Notifications
Introduction
In many modern web applications, WebSockets are used to implement realtime, live-updating user interfaces. When some data is updated on the server, a message is typically sent over a WebSocket connection to be handled by the client. This provides a more robust, efficient alternative to continually polling your application for changes.
To assist you in building these types of applications, Laravel makes it easy to "broadcast" your events over a WebSocket connection. Broadcasting your Laravel events allows you to share the same event names between your server-side code and your client-side JavaScript application.
{tip} Before diving into event broadcasting, make sure you have read all of the documentation regarding Laravel events and listeners.
Configuration
All of your application's event broadcasting configuration is stored in the config/broadcasting.php
configuration file. Laravel supports several broadcast drivers out of the box: Pusher, Redis, and a log
driver for local development and debugging. Additionally, a null
driver is included which allows you to totally disable broadcasting. A configuration example is included for each of these drivers in the config/broadcasting.php
configuration file.
Broadcast Service Provider
Before broadcasting any events, you will first need to register the App\Providers\BroadcastServiceProvider
. In fresh Laravel applications, you only need to uncomment this provider in the providers
array of your config/app.php
configuration file. This provider will allow you to register the broadcast authorization routes and callbacks.
CSRF Token
Laravel Echo will need access to the current session's CSRF token. You should verify that your application's head
HTML element defines a meta
tag containing the CSRF token:
<meta name="csrf-token" content="{{ csrf_token() }}">
Driver Prerequisites
Pusher
If you are broadcasting your events over Pusher, you should install the Pusher PHP SDK using the Composer package manager:
composer require pusher/pusher-php-server "~3.0"
Next, you should configure your Pusher credentials in the config/broadcasting.php
configuration file. An example Pusher configuration is already included in this file, allowing you to quickly specify your Pusher key, secret, and application ID. The config/broadcasting.php
file's pusher
configuration also allows you to specify additional options
that are supported by Pusher, such as the cluster:
'options' => [
'cluster' => 'eu',
'encrypted' => true
],
When using Pusher and Laravel Echo, you should specify pusher
as your desired broadcaster when instantiating the Echo instance in your resources/js/bootstrap.js
file:
import Echo from "laravel-echo";
window.Pusher = require('pusher-js');
window.Echo = new Echo({
broadcaster: 'pusher',
key: 'your-pusher-key'
});
Redis
If you are using the Redis broadcaster, you should install the Predis library:
composer require predis/predis
The Redis broadcaster will broadcast messages using Redis' pub / sub feature; however, you will need to pair this with a WebSocket server that can receive the messages from Redis and broadcast them to your WebSocket channels.
When the Redis broadcaster publishes an event, it will be published on the event's specified channel names and the payload will be a JSON encoded string containing the event name, a data
payload, and the user that generated the event's socket ID (if applicable).
Socket.IO
If you are going to pair the Redis broadcaster with a Socket.IO server, you will need to include the Socket.IO JavaScript client library in your application. You may install it via the NPM package manager:
npm install --save socket.io-client
Next, you will need to instantiate Echo with the socket.io
connector and a host
.
import Echo from "laravel-echo"
window.io = require('socket.io-client');
window.Echo = new Echo({
broadcaster: 'socket.io',
host: window.location.hostname + ':6001'
});
Finally, you will need to run a compatible Socket.IO server. Laravel does not include a Socket.IO server implementation; however, a community driven Socket.IO server is currently maintained at the tlaverdure/laravel-echo-server GitHub repository.
Queue Prerequisites
Before broadcasting events, you will also need to configure and run a queue listener. All event broadcasting is done via queued jobs so that the response time of your application is not seriously affected.
Concept Overview
Laravel's event broadcasting allows you to broadcast your server-side Laravel events to your client-side JavaScript application using a driver-based approach to WebSockets. Currently, Laravel ships with Pusher and Redis drivers. The events may be easily consumed on the client-side using the Laravel Echo Javascript package.
Events are broadcast over "channels", which may be specified as public or private. Any visitor to your application may subscribe to a public channel without any authentication or authorization; however, in order to subscribe to a private channel, a user must be authenticated and authorized to listen on that channel.
Using An Example Application
Before diving into each component of event broadcasting, let's take a high level overview using an e-commerce store as an example. We won't discuss the details of configuring Pusher or Laravel Echo since that will be discussed in detail in other sections of this documentation.
In our application, let's assume we have a page that allows users to view the shipping status for their orders. Let's also assume that a ShippingStatusUpdated
event is fired when a shipping status update is processed by the application:
event(new ShippingStatusUpdated($update));
The ShouldBroadcast
Interface
When a user is viewing one of their orders, we don't want them to have to refresh the page to view status updates. Instead, we want to broadcast the updates to the application as they are created. So, we need to mark the ShippingStatusUpdated
event with the ShouldBroadcast
interface. This will instruct Laravel to broadcast the event when it is fired:
<?php
namespace App\Events;
use Illuminate\Broadcasting\Channel;
use Illuminate\Queue\SerializesModels;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Broadcasting\PresenceChannel;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
class ShippingStatusUpdated implements ShouldBroadcast
{
/**
* Information about the shipping status update.
*
* @var string
*/
public $update;
}
The ShouldBroadcast
interface requires our event to define a broadcastOn
method. This method is responsible for returning the channels that the event should broadcast on. An empty stub of this method is already defined on generated event classes, so we only need to fill in its details. We only want the creator of the order to be able to view status updates, so we will broadcast the event on a private channel that is tied to the order:
/**
* Get the channels the event should broadcast on.
*
* @return \Illuminate\Broadcasting\PrivateChannel
*/
public function broadcastOn()
{
return new PrivateChannel('order.'.$this->update->order_id);
}
Authorizing Channels
Remember, users must be authorized to listen on private channels. We may define our channel authorization rules in the routes/channels.php
file. In this example, we need to verify that any user attempting to listen on the private order.1
channel is actually the creator of the order:
Broadcast::channel('order.{orderId}', function ($user, $orderId) {
return $user->id === Order::findOrNew($orderId)->user_id;
});
The channel
method accepts two arguments: the name of the channel and a callback which returns true
or false
indicating whether the user is authorized to listen on the channel.
All authorization callbacks receive the currently authenticated user as their first argument and any additional wildcard parameters as their subsequent arguments. In this example, we are using the {orderId}
placeholder to indicate that the "ID" portion of the channel name is a wildcard.
Listening For Event Broadcasts
Next, all that remains is to listen for the event in our JavaScript application. We can do this using Laravel Echo. First, we'll use the private
method to subscribe to the private channel. Then, we may use the listen
method to listen for the ShippingStatusUpdated
event. By default, all of the event's public properties will be included on the broadcast event:
Echo.private(`order.${orderId}`)
.listen('ShippingStatusUpdated', (e) => {
console.log(e.update);
});
Defining Broadcast Events
To inform Laravel that a given event should be broadcast, implement the Illuminate\Contracts\Broadcasting\ShouldBroadcast
interface on the event class. This interface is already imported into all event classes generated by the framework so you may easily add it to any of your events.
The ShouldBroadcast
interface requires you to implement a single method: broadcastOn
. The broadcastOn
method should return a channel or array of channels that the event should broadcast on. The channels should be instances of Channel
, PrivateChannel
, or PresenceChannel
. Instances of Channel
represent public channels that any user may subscribe to, while PrivateChannels
and PresenceChannels
represent private channels that require channel authorization:
<?php
namespace App\Events;
use Illuminate\Broadcasting\Channel;
use Illuminate\Queue\SerializesModels;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Broadcasting\PresenceChannel;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
class ServerCreated implements ShouldBroadcast
{
use SerializesModels;
public $user;
/**
* Create a new event instance.
*
* @return void
*/
public function __construct(User $user)
{
$this->user = $user;
}
/**
* Get the channels the event should broadcast on.
*
* @return Channel|array
*/
public function broadcastOn()
{
return new PrivateChannel('user.'.$this->user->id);
}
}
Then, you only need to fire the event as you normally would.html. Once the event has been fired, a queued job will automatically broadcast the event over your specified broadcast driver.
Broadcast Name
By default, Laravel will broadcast the event using the event's class name. However, you may customize the broadcast name by defining a broadcastAs
method on the event:
/**
* The event's broadcast name.
*
* @return string
*/
public function broadcastAs()
{
return 'server.created';
}
If you customize the broadcast name using the broadcastAs
method, you should make sure to register your listener with a leading .
character. This will instruct Echo to not prepend the application's namespace to the event:
.listen('.server.created', function (e) {
....
});
Broadcast Data
When an event is broadcast, all of its public
properties are automatically serialized and broadcast as the event's payload, allowing you to access any of its public data from your JavaScript application. So, for example, if your event has a single public $user
property that contains an Eloquent model, the event's broadcast payload would be:
{
"user": {
"id": 1,
"name": "Patrick Stewart"
...
}
}
However, if you wish to have more fine-grained control over your broadcast payload, you may add a broadcastWith
method to your event. This method should return the array of data that you wish to broadcast as the event payload:
/**
* Get the data to broadcast.
*
* @return array
*/
public function broadcastWith()
{
return ['id' => $this->user->id];
}
Broadcast Queue
By default, each broadcast event is placed on the default queue for the default queue connection specified in your queue.php
configuration file. You may customize the queue used by the broadcaster by defining a broadcastQueue
property on your event class. This property should specify the name of the queue you wish to use when broadcasting:
/**
* The name of the queue on which to place the event.
*
* @var string
*/
public $broadcastQueue = 'your-queue-name';
If you want to broadcast your event using the sync
queue instead of the default queue driver, you can implement the ShouldBroadcastNow
interface instead of ShouldBroadcast
:
<?php
use Illuminate\Contracts\Broadcasting\ShouldBroadcastNow;
class ShippingStatusUpdated implements ShouldBroadcastNow
{
//
}
Broadcast Conditions
Sometimes you want to broadcast your event only if a given condition is true. You may define these conditions by adding a broadcastWhen
method to your event class:
/**
* Determine if this event should broadcast.
*
* @return bool
*/
public function broadcastWhen()
{
return $this->value > 100;
}
Authorizing Channels
Private channels require you to authorize that the currently authenticated user can actually listen on the channel. This is accomplished by making an HTTP request to your Laravel application with the channel name and allowing your application to determine if the user can listen on that channel. When using Laravel Echo, the HTTP request to authorize subscriptions to private channels will be made automatically; however, you do need to define the proper routes to respond to these requests.
Defining Authorization Routes
Thankfully, Laravel makes it easy to define the routes to respond to channel authorization requests. In the BroadcastServiceProvider
included with your Laravel application, you will see a call to the Broadcast::routes
method. This method will register the /broadcasting/auth
route to handle authorization requests:
Broadcast::routes();
The Broadcast::routes
method will automatically place its routes within the web
middleware group; however, you may pass an array of route attributes to the method if you would like to customize the assigned attributes:
Broadcast::routes($attributes);
Customizing The Authorization Endpoint
By default, Echo will use the /broadcasting/auth
endpoint to authorize channel access. However, you may specify your own authorization endpoint by passing the authEndpoint
configuration option to your Echo instance:
window.Echo = new Echo({
broadcaster: 'pusher',
key: 'your-pusher-key',
authEndpoint: '/custom/endpoint/auth'
});
Defining Authorization Callbacks
Next, we need to define the logic that will actually perform the channel authorization. This is done in the routes/channels.php
file that is included with your application. In this file, you may use the Broadcast::channel
method to register channel authorization callbacks:
Broadcast::channel('order.{orderId}', function ($user, $orderId) {
return $user->id === Order::findOrNew($orderId)->user_id;
});
The channel
method accepts two arguments: the name of the channel and a callback which returns true
or false
indicating whether the user is authorized to listen on the channel.
All authorization callbacks receive the currently authenticated user as their first argument and any additional wildcard parameters as their subsequent arguments. In this example, we are using the {orderId}
placeholder to indicate that the "ID" portion of the channel name is a wildcard.
Authorization Callback Model Binding
Just like HTTP routes, channel routes may also take advantage of implicit and explicit route model binding. For example, instead of receiving the string or numeric order ID, you may request an actual Order
model instance:
use App\Order;
Broadcast::channel('order.{order}', function ($user, Order $order) {
return $user->id === $order->user_id;
});
Authorization Callback Authentication
Private and presence broadcast channels authenticate the current user via your application's default authentication guard. If the user is not authenticated, channel authorization is automatically denied and the authorization callback is never executed. However, you may assign multiple, custom guards that should authenticate the incoming request if necessary:
Broadcast::channel('channel', function() {
// ...
}, ['guards' => ['web', 'admin']])
Defining Channel Classes
If your application is consuming many different channels, your routes/channels.php
file could become bulky. So, instead of using Closures to authorize channels, you may use channel classes. To generate a channel class, use the make:channel
Artisan command. This command will place a new channel class in the App/Broadcasting
directory.
php artisan make:channel OrderChannel
Next, register your channel in your routes/channels.php
file:
use App\Broadcasting\OrderChannel;
Broadcast::channel('order.{order}', OrderChannel::class);
Finally, you may place the authorization logic for your channel in the channel class' join
method. This join
method will house the same logic you would have typically placed in your channel authorization Closure. You may also take advantage of channel model binding:
<?php
namespace App\Broadcasting;
use App\User;
use App\Order;
class OrderChannel
{
/**
* Create a new channel instance.
*
* @return void
*/
public function __construct()
{
//
}
/**
* Authenticate the user's access to the channel.
*
* @param \App\User $user
* @param \App\Order $order
* @return array|bool
*/
public function join(User $user, Order $order)
{
return $user->id === $order->user_id;
}
}
{tip} Like many other classes in Laravel, channel classes will automatically be resolved by the service container. So, you may type-hint any dependencies required by your channel in its constructor.
Broadcasting Events
Once you have defined an event and marked it with the ShouldBroadcast
interface, you only need to fire the event using the event
function. The event dispatcher will notice that the event is marked with the ShouldBroadcast
interface and will queue the event for broadcasting:
event(new ShippingStatusUpdated($update));
Only To Others
When building an application that utilizes event broadcasting, you may substitute the event
function with the broadcast
function. Like the event
function, the broadcast
function dispatches the event to your server-side listeners:
broadcast(new ShippingStatusUpdated($update));
However, the broadcast
function also exposes the toOthers
method which allows you to exclude the current user from the broadcast's recipients:
broadcast(new ShippingStatusUpdated($update))->toOthers();
To better understand when you may want to use the toOthers
method, let's imagine a task list application where a user may create a new task by entering a task name. To create a task, your application might make a request to a /task
end-point which broadcasts the task's creation and returns a JSON representation of the new task. When your JavaScript application receives the response from the end-point, it might directly insert the new task into its task list like so:
axios.post('/task', task)
.then((response) => {
this.tasks.push(response.data);
});
However, remember that we also broadcast the task's creation. If your JavaScript application is listening for this event in order to add tasks to the task list, you will have duplicate tasks in your list: one from the end-point and one from the broadcast. You may solve this by using the toOthers
method to instruct the broadcaster to not broadcast the event to the current user.
{note} Your event must use the
Illuminate\Broadcasting\InteractsWithSockets
trait in order to call thetoOthers
method.
Configuration
When you initialize a Laravel Echo instance, a socket ID is assigned to the connection. If you are using Vue and Axios, the socket ID will automatically be attached to every outgoing request as a X-Socket-ID
header. Then, when you call the toOthers
method, Laravel will extract the socket ID from the header and instruct the broadcaster to not broadcast to any connections with that socket ID.
If you are not using Vue and Axios, you will need to manually configure your JavaScript application to send the X-Socket-ID
header. You may retrieve the socket ID using the Echo.socketId
method:
var socketId = Echo.socketId();
Receiving Broadcasts
Installing Laravel Echo
Laravel Echo is a JavaScript library that makes it painless to subscribe to channels and listen for events broadcast by Laravel. You may install Echo via the NPM package manager. In this example, we will also install the pusher-js
package since we will be using the Pusher broadcaster:
npm install --save laravel-echo pusher-js
Once Echo is installed, you are ready to create a fresh Echo instance in your application's JavaScript. A great place to do this is at the bottom of the resources/js/bootstrap.js
file that is included with the Laravel framework:
import Echo from "laravel-echo"
window.Echo = new Echo({
broadcaster: 'pusher',
key: 'your-pusher-key'
});
When creating an Echo instance that uses the pusher
connector, you may also specify a cluster
as well as whether the connection should be encrypted:
window.Echo = new Echo({
broadcaster: 'pusher',
key: 'your-pusher-key',
cluster: 'eu',
encrypted: true
});
Using An Existing Client Instance
If you already have a Pusher or Socket.io client instance that you would like Echo to utilize, you may pass it to Echo via the client
configuration option:
const client = require('pusher-js');
window.Echo = new Echo({
broadcaster: 'pusher',
key: 'your-pusher-key',
client: client
});
Listening For Events
Once you have installed and instantiated Echo, you are ready to start listening for event broadcasts. First, use the channel
method to retrieve an instance of a channel, then call the listen
method to listen for a specified event:
Echo.channel('orders')
.listen('OrderShipped', (e) => {
console.log(e.order.name);
});
If you would like to listen for events on a private channel, use the private
method instead. You may continue to chain calls to the listen
method to listen for multiple events on a single channel:
Echo.private('orders')
.listen(...)
.listen(...)
.listen(...);
Leaving A Channel
To leave a channel, you may call the leaveChannel
method on your Echo instance:
Echo.leaveChannel('orders');
If you would like to leave a channel and also its associated private and presence channels, you may call the leave
method:
Echo.leave('orders');
Namespaces
You may have noticed in the examples above that we did not specify the full namespace for the event classes. This is because Echo will automatically assume the events are located in the App\Events
namespace. However, you may configure the root namespace when you instantiate Echo by passing a namespace
configuration option:
window.Echo = new Echo({
broadcaster: 'pusher',
key: 'your-pusher-key',
namespace: 'App.Other.Namespace'
});
Alternatively, you may prefix event classes with a .
when subscribing to them using Echo. This will allow you to always specify the fully-qualified class name:
Echo.channel('orders')
.listen('.Namespace\\Event\\Class', (e) => {
//
});
Presence Channels
Presence channels build on the security of private channels while exposing the additional feature of awareness of who is subscribed to the channel. This makes it easy to build powerful, collaborative application features such as notifying users when another user is viewing the same page.
Authorizing Presence Channels
All presence channels are also private channels; therefore, users must be authorized to access them. However, when defining authorization callbacks for presence channels, you will not return true
if the user is authorized to join the channel. Instead, you should return an array of data about the user.
The data returned by the authorization callback will be made available to the presence channel event listeners in your JavaScript application. If the user is not authorized to join the presence channel, you should return false
or null
:
Broadcast::channel('chat.{roomId}', function ($user, $roomId) {
if ($user->canJoinRoom($roomId)) {
return ['id' => $user->id, 'name' => $user->name];
}
});
Joining Presence Channels
To join a presence channel, you may use Echo's join
method. The join
method will return a PresenceChannel
implementation which, along with exposing the listen
method, allows you to subscribe to the here
, joining
, and leaving
events.
Echo.join(`chat.${roomId}`)
.here((users) => {
//
})
.joining((user) => {
console.log(user.name);
})
.leaving((user) => {
console.log(user.name);
});
The here
callback will be executed immediately once the channel is joined successfully, and will receive an array containing the user information for all of the other users currently subscribed to the channel. The joining
method will be executed when a new user joins a channel, while the leaving
method will be executed when a user leaves the channel.
Broadcasting To Presence Channels
Presence channels may receive events just like public or private channels. Using the example of a chatroom, we may want to broadcast NewMessage
events to the room's presence channel. To do so, we'll return an instance of PresenceChannel
from the event's broadcastOn
method:
/**
* Get the channels the event should broadcast on.
*
* @return Channel|array
*/
public function broadcastOn()
{
return new PresenceChannel('room.'.$this->message->room_id);
}
Like public or private events, presence channel events may be broadcast using the broadcast
function. As with other events, you may use the toOthers
method to exclude the current user from receiving the broadcast:
broadcast(new NewMessage($message));
broadcast(new NewMessage($message))->toOthers();
You may listen for the join event via Echo's listen
method:
Echo.join(`chat.${roomId}`)
.here(...)
.joining(...)
.leaving(...)
.listen('NewMessage', (e) => {
//
});
Client Events
{tip} When using Pusher, you must enable the "Client Events" option in the "App Settings" section of your application dashboard in order to send client events.
Sometimes you may wish to broadcast an event to other connected clients without hitting your Laravel application at all. This can be particularly useful for things like "typing" notifications, where you want to alert users of your application that another user is typing a message on a given screen.
To broadcast client events, you may use Echo's whisper
method:
Echo.private('chat')
.whisper('typing', {
name: this.user.name
});
To listen for client events, you may use the listenForWhisper
method:
Echo.private('chat')
.listenForWhisper('typing', (e) => {
console.log(e.name);
});
Notifications
By pairing event broadcasting with notifications, your JavaScript application may receive new notifications as they occur without needing to refresh the page.html. First, be sure to read over the documentation on using the broadcast notification channel.
Once you have configured a notification to use the broadcast channel, you may listen for the broadcast events using Echo's notification
method. Remember, the channel name should match the class name of the entity receiving the notifications:
Echo.private(`App.User.${userId}`)
.notification((notification) => {
console.log(notification.type);
});
In this example, all notifications sent to App\User
instances via the broadcast
channel would be received by the callback. A channel authorization callback for the App.User.{id}
channel is included in the default BroadcastServiceProvider
that ships with the Laravel framework.


Esperanto Technologies, Mountaint View, CA. Verification Research Internship. Implementing congestion and fuzzing techniques in the RTL of Esperanto's out-of-order core to increase verification coverage metrics and find more bugs.
Esperanto TechnologiesPerformance Modeling Internship. Maintaining and adding features to a thousand-core cycle-accurate C++ architectural simulator. Studying trade-offs and exploring design space.
MicroArchitecture Santa Cruz (MASC) Group, Santa Cruz, CA, USAResearch Assistant. Currently trying to build a verification infrastructure for RISC-V cores.
Nazarbayev University, Astana, Kazakhstan. Teaching Assitant. Assisting faculty of School of Science and Technology with the following courses: Operating Systems, Computer Systems and Organization, Data Structures. Also taught Programming Fundamentals course.
Alstom Transport, Rochester, NY, USA Junior Quality Engineer. Developed a quality assurance management web-app for “industrial-supplier-quality” team. The app allowed to track and manage defects of the parts from different suppliers.


This section explains how a DefaultConfigurationBuilderobject is setup that provides access to a collection of different configuration sources. DefaultConfigurationBuilder is the option of choice for applications that have to deal with multiple configuration sources. It provides the following features:
This document starts with some explanations of DefaultConfigurationBuilder basics. Then the configuration definition files processed by DefaultConfigurationBuilder are discussed in detail. Finally an advanced example is presented.
In previous chapters we have already seen how specific configuration classes like PropertiesConfiguration or XMLConfiguration can be used to load configuration data from a single source. This may be sufficient for small applications, but if requirements for configuration become more complex, additional support for managing a set of different configuration sources is desired. This is the domain of DefaultConfigurationBuilder which allows combining multiple configuration sources. The properties defined in these sources can then be accessed as if they were defined in a single configuration file. The sources to be loaded have to be defined in a XML document with a specific structure, a so-called configuration definition file. The following listing shows a simple example of such a definition file:
<?xml version="1.0" encoding="ISO-8859-1" ?> <configuration> <properties fileName="usergui.properties"/> </configuration>
A configuration definition file can contain an arbitrary number of elements declaring the configuration sources to load. The <properties> element is one of these; it is used to include properties files. For this example we store the definition file in the same directory as the properties file and call it config.xml. The properties file used in this example is the same as in the section about properties files.
Now we have to create a DefaultConfigurationBuilder object and let it read this definition file. This is quite simple: Just create a new instance and set the name of the definition file (DefaultConfigurationBuilder is derived from XMLConfiguration, so all options for specifying the document to load are available here, too). The combined configuration collecting all sources defined in the configuration definition file can then be obtained by calling the getConfiguration() method:
DefaultConfigurationBuilder builder = new DefaultConfigurationBuilder(); builder.setFile(new File("config.xml")); Configuration config = builder.getConfiguration(true);
Now the config object can be accessed in the usual way to query configuration properties, e.g. by using methods like getString(), or getInt(). We will see in a moment how properties defined in different configuration sources are accessed.
Using DefaultConfigurationBuilder to collect configuration sources does not make much sense if there is only a single source to be loaded. So let's add another one! This time we will embedd a XML file: gui.xml which is shown in the next listing:
<?xml version="1.0" encoding="ISO-8859-1" ?> <gui-definition> <colors> <background>#808080</background> <text>#000000</text> #008000 <link normal="#000080" visited="#800080"/> </colors> <rowsPerPage>15</rowsPerPage> </gui-definition>
To make this XML document part of our global configuration we have to modify our configuration definition file to also include the new file. For XML documents the element <xml> can be used so that we ave now:
<?xml version="1.0" encoding="ISO-8859-1" ?> <configuration> <properties fileName="usergui.properties"/> <xml fileName="gui.xml"/> </configuration>
The code for setting up the DefaultConfigurationBuilder object remains the same. From the Configuration object returned by the factory the new properties can be accessed in the usual way.
There is one problem with this example configuration setup: The color.background property is defined in both the properties and the XML file, and - to make things worse - with different values. Which value will be returned by a call to getString()?
The answer is that the configuration sources are searched in the order they are defined in the configuration definition file. Here the properties file is included first, then comes the XML file. Because the color.background property can be found in the properties file the value specified there will be returned (which happens to be #FFFFFF).
It might not be obvious why it makes sense to define the value of one and the same property in multiple configuration sources. But consider the following scenario: An application comes with a set of default properties and allows the user to override some or all of them. This can now easily be realized by saving the user's settings in one file and the default settings in another. Then in the configuration definition file the file with the user settings is included first and after that the file with the default values. The application code that queries these settings needs no be aware whether a property was overriden by the user. DefaultConfigurationBuilder takes care that properties defined in the first file (the user file) are found; other properties which the user has not changed will still be returned from the second file (the defaults file).
The example above with two configuration sources - one for user settings and one with default values - raises an interesting question: What happens if the user has not defined specific properties yet? Or what if a new user starts our application for the first time and thus no user specific properties exist?
The default behavior of DefaultConfigurationBuilder is to throw a ConfigurationException exception if one of the sources defined in the configuration definition file cannot be loaded. For our example this behavior is not desired: the properties file with specific user settings is not required. If it cannot be loaded, the example application should still work because a complete set of configuration properties is defined in the second file.
DefaultConfigurationBuilder supports such optional configuration sources. For this purpose in the definition of a configuration source the config-optional attribute can be placed. An example of this is shown below:
<?xml version="1.0" encoding="ISO-8859-1" ?> <configuration> <properties fileName="usersettings.properties" config-optional="true"/> <properties fileName="default.properties"/> </configuration>
In this configuration definition file the first properties file with user specific settings is marked as optional. This means that if it cannot be loaded, DefaultConfigurationBuilder will not throw an exception, but only write a warning message to its logger. Note that the config-optional attribute is absent for the second properties file. Thus it is mandatory, and the getConfiguration() method of DefaultConfigurationBuilder would throw an exception if it could not be found.
In an earlier section about the configuration definition file for DefaultConfigurationBuilder it was stated that configuration files included first can override properties in configuraton files included later, and an example use case for this behaviour was given. There may be cases when there are other requirements.
Let's continue the example with the application that somehow process database tables and that reads the definitions of the affected tables from its configuration. This example and the corresponding XML configuration files were introduced in the section about XMLConfiguration. Now consider that this application grows larger and must be maintained by a team of developers. Each developer works on a separated set of tables. In such a scenario it would be problematic if the definitions for all tables would be kept in a single file. It can be expected that this file needs to be changed very often and thus can be a bottleneck for team development when it is nearly steadily checked out. It would be much better if each developer had an associated file with table definitions and all these information could be linked together at the end.
DefaultConfigurationBuilder provides support for such a use case, too. It is possible to specify in the configuration definition file that from a set of configuration sources a logic union configuration is to be constructed. Then all properties defined in the provided sources are collected and can be accessed as if they had been defined in a single source. To demonstrate this feature let us assume that a developer of the database application has defined a specific XML file with a table definition named tasktables.xml:
<?xml version="1.0" encoding="ISO-8859-1" ?> <config>
 <name>tasks</name> <fields> <field> <name>taskid</name> <type>long</type> </field> <field> <name>name</name> <type>java.lang.String</type> </field> <field> <name>description</name> <type>java.lang.String</type> </field> <field> <name>responsibleID</name> <type>long</type> </field> <field> <name>creatorID</name> <type>long</type> </field> <field> <name>startDate</name> <type>java.util.Date</type> </field> <field> <name>endDate</name> <type>java.util.Date</type> </field> </fields>
 </config>
This file defines the structure of an additional table, which should be added to the so far existing table definitions. To achieve this the configuration definition file has to be changed: A new section is added that contains the include elements of all configuration sources which are to be combined.
<?xml version="1.0" encoding="ISO-8859-1" ?>  <configuration> <override> <properties fileName="usergui.properties"/> <xml fileName="gui.xml"/> </override> <additional> <xml fileName="tables.xml"/> <xml fileName="tasktables.xml" config-at="tables"/> </additional> </configuration>
Compared to the older versions of this file some changes have been done. One major difference is that the elements for including configuration sources are no longer direct children of the root element, but are now contained in either an <override> or <additional> section. The names of these sections already imply their purpose.
The override section is not strictly necessary. Elements in this section are treated as if they were children of the root element, i.e. properties in the included configuration sources override properties in sources included later. So the <override> tags could have been ommitted, but for the sake of clearity it is recommended to use them if there is also an <additional> section.
It is the <additonal> section that introduces a new behaviour. All configuration sources listed here are combined to a union configuration. In our example we have put two xml elements in this area that load the available files with database table definitions. The syntax of elements in the additional section is analogous to the syntax described so far. In this example the config-at attribute is introduced. It specifies the position in the logic union configuration where the included properties are to be added. Here it is set for the second element to the value tables. This is because the file starts with a
 element, but to be compatible with the other table definition file it should be accessable under the key tables.table.
After these modifications have been performed, the configuration obtained from DefaultConfigurationBuilder allows access to three database tables. A call of config.getString("tables.table(2).name"); results in a value of tasks. In an analogous way it is possible to retrieve the fields of the third table.
Note that it is also possible to override properties defined in an additonal section. This can be done by placing a configuration source in the override section that defines properties that are also defined in one of the sources listed in the additional section. The example does not make use of that. Note also that the order of the override and additional sections in a configuration definition file does not matter. Sources in an override section are always treated with higher priority (otherwise they could not override the values of other sources).
Configuration definition files are XML documents telling DefaultConfigurationBuilder which configuration sources to load and how to process them in order to create the resulting combined configuration.
Overall structure of a configuration definition file
A configuration definition file for DefaultConfigurationBuilder can contain three sections, all of which are optional. A skeleton looks as follows:
<?xml version="1.0" encoding="ISO-8859-1" ?> <configuration systemProperties="path to property file">    <override>  </override> <additional>  </additional> </configuration>
Declaring configuration sources
The override and additional sections have already been introduced when the basics of DefaultConfigurationBuilder were discussed. They contain declarations for the configuration sources to be embedded. For convenience reasons it is also possible to declare configuration sources outside these sections; they are then treated as if they were placed inside the override section.
Each declaration of a configuration source is represented by an XML element whose name determines the type of the configuration source. Attributes or nested elements can be used to provide additional configuration options for the sources to be included (e.g. a name of a file to be loaded or a reloading strategy). Below is a list of all tags which can be used out of the box:
In the declaration of a configuration source it is possible to set properties on the corresponding configuration objects. Configuration declarations are indeed Bean declarations. That means they can have attributes matching simple properties of the configuration object to create, and sub elements matching complex properties. The following example fragment shows how complex initialization can be performed in a configuration declaration:
<properties fileName="test.properties" throwExceptionOnMissing="true"> <reloadingStrategy refreshDelay="10000" config-class="org.apache.commons.configuration.reloading.FileChangedReloadingStrategy"/> </properties> <xml fileName="test.xml" delimiterParsingDisabled="true"> <expressionEngine config-class="org.apache.commons.configuration.tree.DefaultExpressionEngine" propertyDelimiter="/" indexStart="[" indexEnd="]"/> </xml>
In this example a configuration source for a properties file and one for an XML document are defined. For the properties source the throwExceptionOnMissing property is set to true, which means that it should throw an exception if a requested property is not found. In addition it is assigned a reloading strategy, which is declared and configured in a sub element. The XML configuration source is initialized in a similar way: a simple property is set, and an expression engine is assigned. More information about the format for declaring objects and initializing their properties can be found in the section about bean declarations.
In addition to the attributes that correspond to properties of the configuration object to be created, a configuration declaration can have a set of special attributes that are evaluated by DefaultConfigurationBuilder when it creates the objects. These attributes are listed in the following table:
| Attribute | Meaning |
|---|---|
| config-name | Allows a name to be specified for this configuration. This name can be used to obtain a reference to the configuration from the resulting combined configuration (see below). |
| config-at | With this attribute an optional prefix can be specified for the properties of the corresponding configuration. |
| config-optional | Declares a configuration as optional. This means that errors that occur when creating the configuration are silently ignored. The default behavior when an error occurs is that no configuration is added to the resulting combined configuration. This behavior can be used to find out whether an optional configuration could be successfully created or not. If you specify a name for the optional configuration (using the config-name attribute), you can later check whether the combined configuration contains a configuration with this name. With the config-forceCreate attribute (see below) this default behavior can be changed. |
| config-forceCreate | This boolean attribute is only evaluated for configurations declared as optional. It determines the behavior of the configuration builder when the optional configuration could not be created. If set to true, the builder tries to create an empty, uninitialized configuration of the correct type and add it to the resulting combined configuration. This is especially useful for file-based configurations. Consider a use case where an application wants to store user specific configuration files in the users' home directories. When a user starts this application for the first time, the user configuration does not exist yet. If it is declared as optional and forceCreate, the missing configuration file won't cause an error, but an empty configuration will be created. The application can then obtain this configuration, add properties to it (e.g. user specific settings) and save it. Without the config-forceCreate attribute the application would have to check whether the user configuration exists in the combined configuration and eventually create it manually. Note that not all configuration providers support this mechanism. Sometimes it may not be possible to create an empty configuration if the standard initialization fails. In this case no configuration will be added to the combined configuration (with other words: the config-forceCreate attribute will not have any effect). |
Note: In older versions of Commons Configuration the attributes config-at and config-optional were named at and optional respective. They have been renamed in order to avoid possible name clashes with property names for configuration sources. However, for reasons of backwards compatibility, the old attribute names can still be used.
Another useful feature is the built-in support for interpolation (i.e. variable substitution): You can use variables in your configuration definition file that are defined in declared configuration sources. For instance, if the name of a configuration file to be loaded is defined by the system property CONFIG_FILE, you can do something like this:
<configuration>  <system/>  <properties fileName="${CONFIG_FILE}"/> </configuration>
Note that you can refer only to properties that have already been loaded. If you change the order of the <system> and the <properties> elements in the example above, an error will occur because the ${CONFIG_FILE} variable will then be undefined at the moment it is evaluated.
<configuration systemProperties="systemProperties.xml">  <system/>  <properties fileName="${CONFIG_FILE}"/> </configuration>
This example differs from the previous one by the systemProperties attribute added to the root element. It causes the specified to be read and all properties defined therein to be added to the system properties. So properties like CONFIG_FILE can be defined in a properties file and are then treated as if they were system properties.
The header section
In the header section properties of the resulting combined configuration object can be set. The main part of this section is a bean declaration that is used for creating the resulting configuration object. Other elements can be used for customizing the Node combiners used by the override and the union combined configuration. The following example shows a header section that uses all supported properties:
 <result delimiterParsingDisabled="true" forceReloadCheck="true"> <nodeCombiner config-class="org.apache.commons.configuration.tree.OverrideCombiner"/> <expressionEngine config-class="org.apache.commons.configuration.tree.xpath.XPathExpressionEngine"/> </result> <combiner> <override> <list-nodes> <node>table</node> <node>list</node> </list-nodes> </override> <additional> <list-nodes> <node>table</node> </list-nodes> </additional> </combiner>
The result element points to the bean declaration for the resulting combined configuration. In this example we set some attributes and initialize the node combiner (which is not necessary because the default override combiner is specified), and the expression engine to be used. Note that the config-class attribute makes it possible to inject custom classes for the resulting configuration or the node combiner.
The combiner section allows nodes to be defined as list nodes. This can be necessary for certain node combiner implementations to work correctly. More information can be found in the section about Node combiners.
Note: From time to time the question is raised whether there is a document type definition or a schema defining exactly the structure of a configuration definition file. Frankly, the answer is no. This is due to the fact that the format is extensible. As will be shown below, it is possible to register yout own tags in order to embedd custom configuration sources.
After all that theory let's go through a more complex example! We start with the configuration definition file that looks like the following:
<?xml version="1.0" encoding="ISO-8859-1" ?>  <configuration>  <result delimiterParsingDisabled="true" forceReloadCheck="true"> <expressionEngine config-class="org.apache.commons.configuration.tree.xpath.XPathExpressionEngine"/> </result> <combiner> <additional> <list-nodes> <node>table</node> </list-nodes> </additional> </combiner>  <override> <properties fileName="user.properties" throwExceptionOnMissing="true" config-name="properties" config-optional="true"> <reloadingStrategy refreshDelay="10000" config-class="org.apache.commons.configuration.reloading.FileChangedReloadingStrategy"/> </properties> <xml fileName="settings.xml" config-name="xml"/> </override> <additional> <xml config-name="tab1" fileName="table1.xml" config-at="database.tables"/> <xml config-name="tab2" fileName="table2.xml" config-at="database.tables" validating="true"/> </additional> </configuration>
This configuration definition file includes four configuration sources and sets some properties for the resulting CombinedConfiguration. Of special interest is the forceReloadCheck property, which enables a special check for detecting property changes in the contained configuration sources. If this property is not set, reloading won't work. Because we have configured a reloading strategy for one of the included configuration sources we have to set this flag so that this reloading strategy can function properly. More details about this topic can be found in the Javadocs for CombinedConfiguration. We also set some properties for the configurations to be loaded; for instance we declare that one of the XML configurations should be validated.
With the following code we can create a DefaultConfigurationBuilder and load this file:
DefaultConfigurationBuilder builder = new DefaultConfigurationBuilder(); builder.setFile(new File("configuration.xml")); CombinedConfiguration cc = builder.getConfiguration(true);
It would have been possible to specify the location of the configuration definition file in multiple other ways, e.g. as a URL. The boolean argument in the call to getConfiguration() determines whether the configuration definition file should be loaded. For our simple example we want this to happen, but it would also be possible to load the file manually (by calling the load() method), and after that updating the configuration. (Remember that DefaultConfigurationBuilder is derived from XMLConfiguration, that means you can use all methods provided by this class to alter its data, e.g. to add further configuration sources.) If the configuration's data was manually changed, you should call getConfiguration() with the argument false. XMLConfiguration also provides the registerEntityId() method that can be used to define the location of DTD files (refer to the section Validation of XML configuration files for more details). This method is available for DefaultConfigurationBuilder, too. The entities registered here will be passed to the loaded child XML configurations. So you can register the DTDs of all child XML configurations globally at the configuration builder.
In the header section we have chosen an XPATH expression engine for the resulting configuration. So we can query our properties using the convenient XPATH syntax. By providing the config-name attribute we have given all configuration sources a name. This name can be used to obtain the corresponding sources from the combined configuration. For configurations in the override section this is directly possible:
Configuration propertiesConfig = cc.getConfiguration("properties"); Configuration xmlConfig = cc.getConfiguration("xml");
Configurations in the additional section are treated a bit differently: they are all packed together in another combined configuration and then added to the resulting combined configuration. So in our example the combined configuration cc will contain three configurations: the two configurations from the override section, and the combined configuration with the additional configurations. The latter is stored under a name determined by the ADDITIONAL_NAME constant of DefaultConfigurationBuilder. The following code shows how the configurations of the additional section can be accessed:
CombinedConfiguration ccAdd = (CombinedConfiguration) cc.getConfiguration(DefaultConfigurationBuilder.ADDITIONAL_NAME); Configuration tab1Config = ccAdd.getConfiguration("tab1"); Configuration tab2Config = ccAdd.getConfiguration("tab2");
If you have written a custom configuration class, you might want to declare instances of this class in a configuration definition file, too. With DefaultConfigurationBuilder this is now possible by registering a ConfigurationProvider.
ConfigurationProvider is an inner class defined in DefaultConfigurationBuilder. Its task is to create and initialize a configuration object. Whenever DefaultConfigurationBuilder encounters a tag in the override or the additional section it checks whether for this tag a ConfigurationProvider was registered. If this is the case, the provider is asked to create a new configuration instance; otherwise an exception will be thrown.
So for adding support for a new configuration class you have to create an instance of ConfigurationProvider (or a derived class) and register it at the configuration builder using the addConfigurationProvider() method. This method expects the name of the associated tag and the provider instance as arguments.
If your custom configuration class does not need any special initialization, you can use the ConfigurationProvider class directly. It is able of creating an instance of a specified class (which must be derived from AbstractConfiguration). Let's take a look at an example where we want to add support for a configuration class called MyConfiguration. The corresponding tag in the configuration definition file should have the name myconfig. The code for registering the new provider and loading the configuration definition file looks as follows:
DefaultConfigurationBuilder builder = new DefaultConfigurationBuilder(); DefaultConfigurationBuilder.ConfigurationProvider provider = new DefaultConfigurationBuilder.ConfigurationProvider(MyConfiguration.class); builder.addConfigurationProvider("myconfig", provider); builder.setFileName("configuration.xml"); Configuration config = builder.getConfiguration();
If your configuration provider is registered this way, your configuration definition file can contain the myconfig tag just as any other tag for declaring a configuration source:
<configuration> <additional> <xml fileName="settings.xml"/> <myconfig delimiterParsingDisabled="true"/> </additional> </configuration>
As is demonstrated in this example, it is possible to specify attributes for initializing properties of your configuration object. In this example we set the default delimiterParsingDisabled property inherited from AbstractConfiguration. Of course you can set custom properties of your configuration class, too.
If your custom configuration class is a file-based configuration, you should use the FileConfigurationProvider class instead of ConfigurationProvider. FileConfigurationProvider is another inner class of DefaultConfigurationBuilder that knows how to deal with file-based configurations: it ensures that the correct base path is set and takes care of invoking the load() method.
If your custom configuration class requires special initialization, you need to create your own provider class that extends ConfigurationProvider. Here you will have to override the getConfiguration(ConfigurationDeclaration) method, which is responsible for creating the configuration instance (all information necessary for this purpose can be obtained from the passed in declaration object). It is recommended that you call the inherited method first, which will instantiate and initialize the new configuration object. Then you can perform your specific initialization.