This is the Conf4h configuration library. It is a continuation of the Treecster project, but implemented very differently in Scala with incompatible changes in syntax of the configuration file.
So, why does the world need yet another configuration framework? Well, it does not. Neither does the software. But humans do! All configuration solutions exist to let users to set parameters based on environment without polluting the source code and these settings are devilishly hard to perform, remember, search for, maintain, scan for differences, adapt for different environments, etc., etc., etc. Conf4h intends to remedy that.
On the simplest side of the problem, the configuration can be represented by a set of name-value pairs. Things get difficult when there are more than a pageful of them. Then all kinds of federated name spaces, multiple repositories, and hierarchical structures start to appear. Since configurations are almost always dependent on the environment, the typical configuration files are supplemented by programs: from C-preprocessor to elaborate m4 macros. On the most complicated side, it sometimes easier to write a whole program that will adapt a configuration to the environment, GNU Autoconf is one example.
Conf4h provides easy hierarchical configuration structures. This is done with advanced hierarchy functions and inheritance of properties, implemented in Scala and available via Java and Scala APIs.
Visit our project site for downloads and information on how to contribute.
With Conf4h, you can do away with lengthy configuration files specifying every property for every instance of every application in every environment. Conf4h is designed around the idea of defaults and hierarchy. Unwieldy property files with endless key-value pairs are replaced with single Conf4h file with a syntax similar to HOCON, with defaults specified at each configuration level (i.e. environment, application, instance, etc.) and custom values defined in the sub-levels of the hierarchy. See the example.
You can download the latest version at the Sourceforge download page, or you can get the source code by following this link.
The main bragging points are based on combined experience of our shop's development team and reflects our deeply held beliefs of what configuration should and should not be. For details see the examples section.
Configuration properties are always "un-typed" and can be obtained only as Strings. This is the first fundamental belief and the idea is derived from a self-evident truth that if the configuration is expected to be read by humans, it lives in a text file and, therefore, is text. It obviously can be converted to anything a developer would want, but it's not the job of a configuration system. In other words, Conf4h will provide you with strings and you can do whatever else you want with them.
If a developer asks for a property that cannot be supplied by the configuration, immediate runtime error will result. This is the second fundamental belief and its origin is in many, many hours spent troubleshooting missing property problems. In so many words, the configuration system is assumed to be the only source of truth and if one relies on configuration system to provide that truth, it better be there.
Defaults are prominently featured in Conf4h, but there are rules. We fundamentally believe that defaults must be set explicitly in the configuration file - either on higher levels of the hierarchy or by reference in the macro variables. Macro variable can have their own defaults, but again, they must be present in order to be used.
Interaction with environment is only allowed via configuration construction interface. All others are prohibited, discouraged, and frowned upon. Violators will be prosecuted, chastised, and subjected to public humiliation.
Configuration properties are presented as name-value pairs
organized in trees. The values can be strings, maps, or lists of strings,
maps, lists, etc. Traversing of configuration tree is purposely not
supported by the API, but you can get a HOCON string dump for your
debugging pleasure.
Since properties are organized as branches of a tree, there is a notion of a "level", i.e., a set of properties accessible via common parent key. The fully qualified name of a key is presented as path with elements separated by "." (dot) for convenience.
Properties from parent levels that are not present in the current level are inherited. This is the fundamental feature of the whole Conf4h concept. It is also where the defaulting functionality comes in: all of the properties that are possibly relevant to a particular instance of a particular application are copied over from the upper levels if not present in the current one.
Conf4h interpolates specially noted macro variables. It allows to use frequently used and environment dependent values by reference. Interpolation is only performed when you explicitly ask for a property that has a macro variable reference in it, for the purpose of propagating changes to the referenced variable.
We share the goals of HOCON's creators to provide a configuration file syntax suitable for human use, hence the name Conf4h, as in "Configuration for(4) humans". Here are major requirements as we see them:
Name and value separated by an optional "=" (equal sign) denote a property:
a = b
c = d
Properties must be separated by white space. Values can be strings or nested data structures( lists, maps, and variants).
Strictly speaking, equal signs are not required, but omitting them can make configuration files difficult to read. The above example can be written as
a b c d
The same is true for other data structures that follow.
Comments are either in-line delimited by "//" (double slash) or block style enclosed in "/* */":
a = b // This is an in-line comment
/* And this is the
block-style comment */
c = d
Comments are allowed anywhere where white space is permitted. White space is allowed everywhere except where prohibited.
Maps are enclosed in curly braces and must be named unless in a list:
com = {
a = b
c = {
d = e
}
}
Names of the nested maps create a path. In the above example:
com.c.d = "e"
Equal signs between a key and a map can be omitted for simplicity, thus this is the same as the above:
com {
a = b
c {
d = e
}
}
Lists are enclosed in square braces, member values must be separated by ',' (commas) or any other white space, equal signs can be omitted:
com {
a = [b, c, d]
e = [f
g
h]
i [
{
k = l
}, { m = n }
{ o = p }
]
}
Elements of a list can be of different types, whether it is a good idea is up to you. Elements are addressed by their 0-based index. In the above example:
com.a.1 = "c"
com.i.2.o = "p"
Variant is a special map enclosed in angle brackets, which contains only other maps called variant values:
r {
d <
1 { v=s}
a { v=v}
b { w=x}
> // d
} // r
Depending on what the value of "d" is (we call it a selector), this is equivalent to the following pseudo-code:
if d = "1" -> r.v = "s"
if d = "a" -> r.v = "v"
if d = "b" -> r.v - Error!
otherwise -> Problem!
A common strategy to avoid surprises with variants is to define defaults:
r {
v = ddd
d <
1 { v=s}
a { v=v}
b { w=x}
> // d
} // r
then
if d = "1" -> r.v = "s"
if d = "a" -> r.v = "v"
anything else -> r.v = "ddd"
Each configuration file must contain one or more roots, i.e., maps on the top-most configuration level. For example:
root {
a = b
// ...
}
com {
conf4h {
// ...
}
}
Macro variables can appear anywhere, where values are allowed, and have the special syntax:
${key:default}
For example,
root {
a = say ${b:bar} now
}
will evaluate to root.a="say foo now" if b="foo"
and root.a="say bar now" if b is
undefined. Default value can be omitted, but then the previous example
will generate a runtime error if b is not defined.
If the key is complex, i.e., it contains one or more "." separators, the value of the macro will be resolved from the top of the hierarchy. For simple keys resolution starts at the present level.
Current Conf4h is implemented using a custom combination parser based on FastParse library with implementation details mostly hidden.
Both Java and Scala APIs are provides, the differences are as follows:
| Java | Scala | |
|---|---|---|
| Main class | com.conf4h.Conf4hJ | com.conf4h.Conf4h |
| Maps | java.util.Map<String, String> | scala.collection.Map[String, String] |
| Lists | java.util.List<String> | scala.collection.immutable.List[String] |
The rest of the API calls and structures are exactly the same. The examples below are given in Scala though.
We will use the following example to illustrate the API:
moving {
env = ${env}
speed = ${speed}
plan = "Getting away in ${env} environment with ${speed} speed on ${moving.name} using ${moving.propulsion} for power."
env <
water {
propulsion = enthusiasm
hull = timber
speed <
fast {
name = Cruiser Port Royal
propulsion = turbine
fuel = kerosene
hull = [steel]
} // fast
moderate {
name = Clipper Catty Sark
propulsion = wind
} // moderate
slow {
name = raft
fuel = sashimi
} // slow
> // speed
} // water
air {
propulsion = jet
fuel = kerosene
speed <
fast {
name = fighter jet
} // fast
moderate {
name = airliner
hull = [
aluminum
composite
]
} // moderate
slow {
name = balloon
propulsion = wind
view = [ clouds, houses, {sky=blue, grass=green}, "smoke stacks"]
fuel = propane
} // slow
> // speed
} // air
> // env
} // moving
It describes a moving problem in two media (water and air) with three
possible speeds (fast, moderate, slow). We'll call the file
propulsion.c4h
The configuration can be initialized either from a text file or from a string with optional variant map:
val conf1 = Conf4h.loadString("r {a=b}", Map("c" -> "d"))
val conf2 = Conf4h.loadFile("myconfig.c4h")
For example:
val conf = Conf4h.loadFile("propulsion.c4h", Map("env"->"air", "speed"->"slow"))
Note, however, the environment settings are ignored. It means, if you want to use command line initialization of the variants like
java -Denv=air -Dspeed=slow ... scala.tools.nsc.MainGenericRunner ...
You must explicitly transfer environment variables values to the initial variants map:
val conf = Conf4h.loadFile("propulsion.c4h", Map("env"->sys.env("env"), "speed"->sys.env("speed")))
The interpolation variables use the regular ${var} syntax
where the variable may refer to either another property (key) or variant.
Given two different sets of variants, the following results will be obtained:
| Property | Variants | |
|---|---|---|
| env=air speed=moderate |
env=water speed=slow |
|
| moving.name | airliner | raft |
| moving.plan | Getting away in air environment with moderate speed on airliner using jet for power. |
Getting away in water environment with slow speed on raft using enthusiasm for power. |
| moving.env | air | floating in water |
| moving.speed | moderate | moving very slow |
| moving.hull | [aluminum, composite] | timber |
A simple getter is as simple as it gets:
val x1 = conf.getString("moving.name") // "balloon"
val x2 = conf.getString("moving", "name") // "balloon"
val x3 = conf.getString("moving", "hull") // RuntimeException thrown
These are similarly simple:
val lst = conf.getList("moving", "view") // List("clouds", "houses", "smoke stacks")
val mp = conf.getMap("moving") // Map("name"->"balloon", "fuel"->"propane", "propulsion"->"wind")
Note that "colors" map is not listed in the list and view is not among map's elements - you are getting strings only. If you want to check the "colors", this will get it:
val mp = conf.getMap("moving.view.2") // Map("sky"->"blue", "grass"->"green")
Individual properties and elements of arrays are available via simple getters:
val three = conf.getString("moving", "view", "3") // "smoke stacks"
val grass = conf.getString("moving", "view", "2", "grass") // "green"
Sometimes it's convenient to use Java's own Properties class, especially for third-party components. Conf4h provides a converter method:
val p: java.util.Properties = conf.toProperties("moving");
which will return something like
moving.name=balloon
moving.propulsion=wind
moving.fuel=propane
moving.view.0=clouds
moving.view.1=houses
moving.view.2.sky=blue
moving.view.2.grass=green
moving.view.3=smoke stacks
The configuration will be fully processed with variants and interpolations before conversion. Lists are converted to strings by adding an index to the key, as shown above.
There is one caveat, though. java.util.Properties allows the
same key to have a value and sub-keys at the same time. For example:
log4j.appender.CONSOLE = org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern = %d [%t] %-5p %c %x - %m%n
To accommodate such a case, Conf4h uses a special key named '~' (tilde):
log4j {
appender {
CONSOLE {
layout {
~=org.apache.log4j.PatternLayout
ConversionPattern="%d [%t] %-5p %c %x - %m%n"
}
~=org.apache.log4j.ConsoleAppender
} // CONSOLE
} // appender
} // log4j
When '~' is encountered during properties conversion, Conf4h creates a
key one level up as in the above example,
so log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
can also have sub-keys like log4j.appender.CONSOLE.layout,
which, in turn, also has a value and a sub-key.
Converting to Lightbend's configuration is easy:
val c: com.typesafe.config.Config = conf.toConfig(keys...)
Beware that in the converted configuration all the keys are double-quoted to prevent the Lightbend's library from freaking out on HOCON-improper keys. It does not affect its getter methods though.
It is a good idea to isolate your Akka settings from the rest of it, for example
moving {
// ...
}
my-akka {
akka {
loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = "INFO"
actor {
debug {
autoreceive = on
lifecycle = on
}
provider = "akka.cluster.ClusterActorRefProvider"
} // actor
remote {
log-remote-lifecycle-events = off
netty.tcp {
hostname = "127.0.0.1"
port = 0
}
} // remote
} // akka
cluster-dispatcher {
type = "Dispatcher"
executor = "fork-join-executor"
fork-join-executor {
parallelism-min = 2
parallelism-max = 4
}
} // cluster-dispatcher
} // my-akka
log4j {
// ...
}
Then the value you feed to your actors will be
val c = com.typesafe.config.ConfigFactory.load(con.toConfig().getConfig("my-akka"))
This allows to consolidate all of your reference.conf
and application.conf files into one main configuration file
with application- or environment-specific portions sectioned off.
The Log4j v.1.2 configuration can be specified in a separate file or included with any other configuration. The only requirement for it is to follow Log4j's naming hierarchy and to have all required components like appenders, loggers, etc. See Converter to java.util.Properties section for details.
The following environment variables should be used in order to utilize this functionality, both must be present:
java -Dlog4j.configuration=file:myconfig.c4h -Dlog4j.configuratorClass=com.conf4h.Log4jConfigurator ...
Note that only file URLs are supported for log4j.configuration now.
This is a list of missing features of Conf4h:
null or Nil values.
${variable:its_default_value_if_not_resolved}.