ConfMe is a simple to use, production ready application configuration management library 😍

iwanbolzern, updated πŸ•₯ 2022-10-26 05:52:13

ConfMe: Configuration Made Easy πŸ’–

image image image image

ConfMe is a simple to use, production ready application configuration management library, which takes into consideration the following three thoughts: 1. Access to configuration values must be safe at runtime. No myconfig['value1']['subvalue'] anymore! 2. The configuration must be checked for consistency at startup e.g. type check, range check, ... 3. Secrets shall be injectable from environment variables

ConfMe makes all these features possible with just a few type annotations on plain Python objects.


ConfMe can be installed from the official python package repository pypi

pip install confme

Or, if you're using pipenv:

pipenv install confme

Or, if you're using poetry:

poetry add confme

Basic Usage of confme

Define your config structure as plain python objects with type annotations:

```python from confme import BaseConfig

class DatabaseConfig(BaseConfig): host: str port: int user: str

class MyConfig(BaseConfig): name: str database: DatabaseConfig ```

Create a configuration yaml file with the same structure as your configuration classes have:

yaml name: "Database Application" database: host: "localhost" port: 5000 user: "any-db-user"

Load the yaml file into your Python object structure and access it in a secure manner:

```python my_config = MyConfig.load('config.yaml')

print(f'Using database connection {} ' f'on port {my_config.database.port}') ```

In the background the yaml file is parsed and mapped to the defined object structure. While mapping the values to object properties, type checks are performed. If a value is not available or is not of the correct type, an error is generated already when the configuration is loaded.

Supported Annotations

ConfMe is based on pydantic and supports all annotations provided by pydantic. The most important annotations are listed and explain bellow. For the whole list, please checkout Field Types: - str - int - float - bool - typing.List[x] - typing.Optional[x] - Secret - Range - Enum


With the Secret annotation you can inject secrets from environment variables directly into your configuration structure. This is especially handy when you're deploying applications by using docker. Therefore, let's extend the previous example with a Secret annotation:

```python from confme import BaseConfig from confme.annotation import Secret

class DatabaseConfig(BaseConfig): ... password: str = Secret('highSecurePassword') ```

Now set the password to the defined environment variable:

bash export highSecurePassword="This is my password"

Load your config and check for the injected password.

my_config = MyConfig.load('config.yaml') print(f'My password is: {my_config.database.password}')


ConfME supports OpenRange, ClosedRange and MixedRange values. The terms open and close are similar to open and closed intervals in mathematics. This means, if you want to include the lower and upper range use ClosedRange otherwise OpenRange: * ClosedRange(2, 3) will include 2 and 3 * OpenRange(2, 3) will not include 2 and 3

If you want to have a mixture of both, e.g. include 2 but exclude 3 use MixedRange: * MixedRange(ge=2, lt=3) will include 2 but exclude 3

```python from confme import BaseConfig from confme.annotation import ClosedRange

class DatabaseConfig(BaseConfig): ... password: int = ClosedRange(2, 3) ```


If a Python Enum is set as type annotation, ConfMe expect to find the enum value in the configuration file.

```python from confme import BaseConfig from enum import Enum

class DatabaseConnection(Enum): TCP = 'tcp' UDP = 'udp'

class DatabaseConfig(BaseConfig): ... connection_type: DatabaseConnection ```

Switching configuration based on Environment

A very common situation is that configurations must be changed based on the execution environment (dev, test, prod). This can be accomplished by registering a folder with one .yaml file per environment and seting the ENV environment variable to the value you need. An example could look like this:

Let's assume we have three environments (dev, test, prod) and one configuration file per environment in the following folder structure: project β”‚ └───config β”‚ β”‚ my_prod_config.yaml β”‚ β”‚ my_test_config.yaml β”‚ β”‚ my_dev_config.yaml β”‚ └───src β”‚ β”‚ β”‚ β”‚ The definition of is equivalent to the one used in the basic introduction section and uses our configuration the following way: ```python

we register the folder where ConfME can find the configuration files

MyConfig.register_folder(Path(file).parent / '../config') ...

we access the instance of the corresponding configuration file anywhere in our project.

my_config = MyConfig.get() print(f'Using database connection {} ' f'on port {my_config.database.port}') `` If now one of the following environment variables (precedence in descending order):['env', 'environment', 'environ', 'stage']is set e.g.export ENV=prodit will load the configuration file withprod` in its name.

Parameter overwrite

In addition to loading configuration parameters from the configuration file, they can be passed/overwritten from the command line or environment variables. Thereby, the following precedences apply (lower number means higher precedence): 1. Command Line Arguments: Check if parameter is set as command line argument. If not go one line done... 2. Environment Variables: Check if parameter is set as environment variable. If not go one line done... 3. Configuration File: If value was not found in one of the previous sources, it will check in the configuration file.

Overwrite Parameters from Command Line

Especially in the Data Science and Machine Learning area it is useful to pass certain parameters for experimental purposes as command line arguments. Therefore, all properties defined in the configuration classes are automatically offered as command line arguments in the following format:

```python from confme import BaseConfig

class DatabaseConfig(BaseConfig): host: str port: int user: str

class MyConfig(BaseConfig): name: int database: DatabaseConfig

config = MyConfig.load('test.yaml') ```

When you now start your program from the command line with the ++help argument, you get the full list of all configuration options. CAVEAT! In order to not interfere with other cli tools, the prefix - was changed to +: ```shell $ python --help usage: [+h] [++name NAME] [ DATABASE.HOST] [++database.port DATABASE.PORT] [++database.user DATABASE.USER]

optional arguments: +h, ++help show this help message and exit

Configuration Parameters: With the parameters specified bellow, the configuration values from the config file can be overwritten.

++name NAME DATABASE.HOST ++database.port DATABASE.PORT ++database.user DATABASE.USER ```

Overwrite Parameters with Environment Variables

Likewise to overwriting parameters from the commandline you can also overwrite by passing environment variables. Therefore, simply set the environment variable in the same format as it would be passed as command line arguments and run your application: shell $ export $ python


ConfMe is released under the MIT license.


Feautre/generate example config

opened on 2022-10-25 15:36:03 by iwanbolzern

Closes #5

Add support for configuration update

opened on 2020-08-12 12:11:34 by iwanbolzern

It shall be possible to update a nested configuration structure by passing in a dot (.) separated string.

Add possibility for referencing other yaml files like imports

opened on 2020-08-10 14:14:57 by iwanbolzern None

Add possibility to generate example yml file based on class definition

opened on 2020-03-03 14:59:27 by iwanbolzern None

Add support for complex post init validation

opened on 2020-03-03 14:48:48 by iwanbolzern

It must be possible to add custom, complex post init validation steps.

Therefore, add something like: ``` @configclass class RootConfig: rootValue: int childNode: ChildNode

def __post_init__(self):



v1.3.0 support for default environment 2022-01-21 15:00:57

  • Added support for default environment in register_folder(...)

v1.2.1 fixed incompatibility issue with pydantic 2022-01-18 15:20:51

Fixed incompatibility issue with pydantic 1.9

v1.2.0 added load from dict option 2022-01-18 11:55:24

  • Load from dict option added to base config

v1.1.0 changed cli prefix from - to + 2022-01-13 15:54:46

Feature: changed CLI prefix from - to + e.g. ++help

v1.0.4 Fixed test runner 2022-01-13 14:51:18

  • Bugfix: test runner definition changed

v1.0.2 Updated cli help command 2022-01-13 14:42:11

  • changed --help to --help-config to not interfere with cli tools
Iwan Bolzern
GitHub Repository