Structured Configs¶

Structured configs are used to create OmegaConf configuration object with runtime type safety. In addition, they can be used with tools like Pyrefly or your IDE for static type checking.

Two types of structures classes are supported: dataclasses and attrs classes (that offers slightly cleaner syntax in some cases but depends on the attrs pip package).

This documentation will use dataclasses, but you can use the annotation @attr.s(auto_attribs=True) from attrs instead of @dataclass.

Basic usage involves passing in a structured config class or instance to OmegaConf.structured(), which will return an OmegaConf config that matches the values and types specified in the input. At runtime, OmegaConf will validate modifications to the created config object against the schema specified in the input class.

Currently, type hints supported in OmegaConf’s structured configs include:

primitive types (int, float, bool, str, bytes, Path) and enum types (user-defined subclasses of enum.Enum). See the Simple types section below.
unions of primitive/enum types, e.g. Union[float, bool, MyEnum], typed container types, e.g. Union[List[int], Dict[str, int]], and structured config types, e.g. Union[CatConfig, DogConfig]. See Unions below.
literal types, e.g. Literal["train", "eval"]. See Literal types below.
structured config fields (i.e. MyConfig.x can have type hint MySubConfig). See the Nesting structured configs section below.
dict, list, and tuple types: typing.Dict[K, V], typing.List[V], or typing.Tuple[...]. Dictionary keys K are primitive or enum values, and container values may use any supported type, including nested containers. See the Lists, Tuples (experimental), and Dictionaries sections below.
optional types (any of the above can be wrapped in a typing.Optional[...] annotation). See Other special features below.

Simple types ¶

Simple types include

int: numeric integers
float: numeric floating point values
bool: boolean values (True, False, On, Off etc)
str: any string
bytes: an immutable sequence of numbers in [0, 255]
pathlib.Path: filesystem paths as represented by python’s standard library pathlib
Enums: User defined enums

The following class defines fields with all simple types:

>>> class Height(Enum):
...     SHORT = 0
...     TALL = 1

>>> @dataclass
... class SimpleTypes:
...     num: int = 10
...     pi: float = 3.1415
...     is_awesome: bool = True
...     height: Height = Height.SHORT
...     description: str = "text"
...     data: bytes = b"bin_data"
...     path: pathlib.Path = pathlib.Path("hello.txt")

You can create a config based on the SimpleTypes class itself or an instance of it. Those would be equivalent by default, but the Object variant allows you to set the values of specific fields during construction.

>>> conf1 = OmegaConf.structured(SimpleTypes)
>>> conf2 = OmegaConf.structured(SimpleTypes())
>>> # The two configs are identical in this case
>>> assert conf1 == conf2
>>> # But the second form allow for easy customization of the values:
>>> conf3 = OmegaConf.structured(
...   SimpleTypes(num=20,
...   height=Height.TALL))
>>> print(OmegaConf.to_yaml(conf3))
num: 20
pi: 3.1415
is_awesome: true
height: TALL
description: text
data: !!binary |
  YmluX2RhdGE=
path: !!python/object/apply:pathlib.PosixPath
- hello.txt

The resulting object is a regular OmegaConf DictConfig, except that it will utilize the type information in the input class/object and will validate the data at runtime. The resulting object and will also rejects attempts to access or set fields that are not already defined (similarly to configs with their to Struct flag set, but not recursive).

>>> conf = OmegaConf.structured(SimpleTypes)
>>> with raises(AttributeError):
...    conf.does_not_exist

Static type checker support ¶

Python type annotation can be used by static type checkers like Mypy/Pyre or by IDEs like PyCharm.

>>> conf: SimpleTypes = OmegaConf.structured(SimpleTypes)
>>> # Passes static type checking
>>> conf.description = "text"
>>> # Fails static type checking (but will also raise a Validation error)
>>> with raises(ValidationError):
...     conf.num = "foo"

This is duck-typing; the actual object type of conf is DictConfig. You can access the underlying type using OmegaConf.get_type():

>>> type(conf).__name__
'DictConfig'

>>> OmegaConf.get_type(conf).__name__
'SimpleTypes'

Runtime type validation and conversion ¶

OmegaConf supports merging configs together, as well as overriding from the command line. This means some mistakes can not be identified by static type checkers, and runtime validation is required.

>>> # This is okay, the string "100" can be converted to an int
>>> # Note that static type checkers will not like it and you should
>>> # avoid such explicit mistyped assignments.
>>> conf.num = "100"
>>> assert conf.num == 100

>>> with raises(ValidationError):
...     # This will fail at runtime because num is an int
...     # and foo cannot be converted to an int
...     # Note that the static type checker can't help here.
...     conf.merge_with_dotlist(["num=foo"])

Runtime validation and conversion works for all supported types, including Enums:

>>> conf.height = Height.TALL
>>> assert conf.height == Height.TALL

>>> # The name of Height.TALL is TALL
>>> conf.height = "TALL"
>>> assert conf.height == Height.TALL

>>> # This works too
>>> conf.height = "Height.TALL"
>>> assert conf.height == Height.TALL

>>> # The ordinal of Height.TALL is 1
>>> conf.height = 1
>>> assert conf.height == Height.TALL

For enums with string values, assignments can use either the enum member name or the enum value:

>>> class HttpStatus(str, Enum):
...     OK = "ok-status"
...     ERROR = "error-status"
...
>>> @dataclass
... class StringEnumConfig:
...     status: HttpStatus = HttpStatus.OK
...
>>> conf = OmegaConf.structured(StringEnumConfig)
>>> conf.status = "ERROR"
>>> assert conf.status == HttpStatus.ERROR
>>> conf.status = "error-status"
>>> assert conf.status == HttpStatus.ERROR

Nesting structured configs ¶

Structured configs can be nested.

>>> @dataclass
... class User:
...     # A simple user class with two missing fields
...     name: str = MISSING
...     height: Height = MISSING
>>>
>>> @dataclass
... class DuperUser(User):
...     duper: bool = True
...
>>> # Group class contains two instances of User.
>>> @dataclass
... class Group:
...     name: str = MISSING
...     # data classes can be nested
...     admin: User = field(default_factory=User)
...
...     # You can also specify different defaults for nested classes
...     manager: User = field(default_factory=lambda: User(name="manager", height=Height.TALL))

>>> conf: Group = OmegaConf.structured(Group)
>>> print(OmegaConf.to_yaml(conf))
name: ???
admin:
  name: ???
  height: ???
manager:
  name: manager
  height: TALL

OmegaConf will validate that assignment of nested objects is of the correct type:

>>> with raises(ValidationError):
...     conf.manager = 10

You can assign subclasses:

>>> conf.manager = DuperUser()
>>> assert conf.manager.duper == True

Literal types ¶

typing.Literal can be used when a field must be exactly one of a fixed set of values. Literal annotations are also supported inside containers and unions.

>>> @dataclass
... class HasLiteral:
...     mode: Literal["train", "eval"] = "train"
...     stages: List[Literal["train", "eval"]] = field(default_factory=lambda: ["train"])

>>> cfg = OmegaConf.structured(HasLiteral)
>>> cfg.mode = "eval"
>>> cfg.stages.append("eval")
>>> cfg.mode = "debug"
Traceback (most recent call last):
...
omegaconf.errors.ValidationError: Invalid value 'debug', expected one of ['train', 'eval']
    full_key: mode
    object_type=HasLiteral

Lists ¶

Structured Config fields annotated with typing.List can hold any type supported by OmegaConf (int, float. bool, str, bytes, pathlib.Path, Enum or Structured configs).

>>> from dataclasses import dataclass, field
>>> from typing import List
>>> @dataclass
... class User:
...     name: str = MISSING

>>> @dataclass
... class ListsExample:
...     # Typed list can hold Any, int, float, bool, str,
...     # bytes, pathlib.Path and Enums as well as arbitrary Structured configs.
...     ints: List[int] = field(default_factory=lambda: [10, 20, 30])
...     users: List[User] = field(default_factory=lambda: [User(name="omry")])

OmegaConf verifies at runtime that your Lists contains only values of the correct type. In the example below, the OmegaConf object conf (which is actually an instance of DictConfig) is duck-typed as ListExample.

>>> conf: ListsExample = OmegaConf.structured(ListsExample)

>>> # Okay, 10 is an int
>>> conf.ints.append(10)
>>> # Okay, "20" can be converted to an int
>>> conf.ints.append("20")

>>> conf.users.append(User(name="Joe"))
>>> # Not okay, 10 cannot be converted to a User
>>> with raises(ValidationError):
...     conf.users.append(10)

Tuples (experimental)¶

Tuple support is experimental in OmegaConf 2.4, and its semantics may evolve based on user feedback. Tuple fields use TupleConfig rather than ListConfig. They preserve tuple identity, support fixed heterogeneous and homogeneous variadic annotations, and are structurally immutable. See Migrating tuple usage in OmegaConf 2.4 for the breaking change from earlier OmegaConf versions and guidance for choosing between tuple and list annotations.

>>> from typing import Tuple
>>> @dataclass
... class TupleExample:
...     fixed: Tuple[int, str] = (10, "name")
...     variadic: Tuple[int, ...] = (1, 2, 3)

>>> conf = OmegaConf.structured(TupleExample)
>>> isinstance(conf.fixed, TupleConfig)
True
>>> conf.fixed == (10, "name")
True
>>> conf.fixed = [20, 30]
>>> conf.fixed == (20, "30")
True
>>> with raises(TypeError):
...     conf.fixed[0] = 30

Nested mutable containers remain mutable, as they do inside a native Python tuple. Complete tuple replacement is allowed through a mutable parent, but fixed annotations enforce arity and positional types. OmegaConf.typed_tuple creates an explicitly typed tuple and requires its complete content up front.

>>> value = OmegaConf.typed_tuple([1, "x"], Tuple[int, str])
>>> value == (1, "x")
True

Dictionaries ¶

Dictionaries are supported via annotation of structured config fields with typing.Dict. Keys must be typed as one of str, int, Enum, float, bytes, or bool. Values can be any of the types supported by OmegaConf (Any, int, float, bool, bytes, pathlib.Path, str and Enum as well as arbitrary Structured configs)

>>> from dataclasses import dataclass, field
>>> from typing import Dict
>>> @dataclass
... class DictExample:
...     ints: Dict[str, int] = field(default_factory=lambda: {"a": 10, "b": 20, "c": 30})
...     bools: Dict[str, bool] = field(default_factory=lambda: {"Uno": True, "Zoro": False})
...     users: Dict[str, User] = field(default_factory=lambda: {"omry": User(name="omry")})

Like with Lists, the types of values contained in Dicts are verified at runtime.

>>> conf: DictExample = OmegaConf.structured(DictExample)

>>> # Okay, correct type is assigned
>>> conf.ints["d"] = 10
>>> conf.bools["Dos"] = True
>>> conf.users["James"] = User(name="Bond")

>>> # Not okay, 10 cannot be assigned to a User
>>> with raises(ValidationError):
...     conf.users["Joe"] = 10

Nested dict and list annotations ¶

Dict and List annotations can be nested flexibly:

>>> @dataclass
... class NestedContainers:
...     dict_of_dict: Dict[str, Dict[str, int]]
...     list_of_list: List[List[int]] = field(default_factory=lambda: [[123]])
...     dict_of_list: Dict[str, List[int]] = MISSING
...     list_of_dict: List[Dict[str, int]] = MISSING
...
...
>>> cfg = OmegaConf.structured(NestedContainers(dict_of_dict={"foo": {"bar": 123}}))
>>> print(OmegaConf.to_yaml(cfg))
dict_of_dict:
  foo:
    bar: 123
list_of_list:
- - 123
dict_of_list: ???
list_of_dict: ???

>>> with raises(ValidationError):
...     cfg.list_of_dict = [["whoops"]]  # not a list of dicts

Unions ¶

You can use typing.Union to combine supported simple types, Literal annotations, typed container types, and structured config types.

A union containing Any is normalized to Any because every value already matches that member. PEP 695 type aliases are transparent on Python 3.12 and newer: an alias can be used wherever its expanded annotation is supported, including aliases with type parameters.

>>> from typing import Union
>>>
>>> @dataclass
... class HasUnion:
...     u: Union[float, bool] = 10.1
...
>>> cfg = OmegaConf.structured(HasUnion)
>>> assert cfg.u == 10.1
>>> cfg.u = True  # ok
>>> cfg.u = b"binary"  # bytes not compatible with union
Traceback (most recent call last):
...
omegaconf.errors.ValidationError: Cannot assign 'b'binary'' of type 'bytes' to Union[float, bool]
    full_key: u
    object_type=HasUnion
>>> OmegaConf.structured(HasUnion("abc"))  # str not compatible
Traceback (most recent call last):
...
omegaconf.errors.ValidationError: Cannot assign 'abc' of type 'str' to Union[float, bool]
    full_key: u
    object_type=None

For unions of structured config types, typed values select the most specific declared branch according to their runtime type’s method resolution order. Plain mappings do not select between multiple structured branches by matching their fields:

>>> @dataclass
... class Dog:
...     name: str = MISSING
...     breed: str = MISSING
...
>>> @dataclass
... class Cat:
...     name: str = MISSING
...     indoor: bool = MISSING
...
>>> @dataclass
... class PetOwner:
...     pet: Union[Dog, Cat] = MISSING
...
>>> cfg = OmegaConf.structured(PetOwner)
>>> cfg.pet = Dog(name="Rex", breed="Lab")
>>> assert OmegaConf.get_type(cfg, "pet") is Dog
>>> cfg.pet |= {"name": "Fido"}  # merge into the selected branch
>>> assert cfg.pet == {"name": "Fido", "breed": "Lab"}
>>> override = OmegaConf.structured(PetOwner)
>>> override.pet = Dog(name="Buddy")
>>> merged = OmegaConf.merge(cfg, override)  # same branch: recursive merge
>>> assert merged.pet == {"name": "Buddy", "breed": "Lab"}
>>> cfg.pet = {"name": "Spot"}  # assignment replaces the selected branch
>>> assert cfg.pet.name == "Spot"
>>> assert OmegaConf.is_missing(cfg.pet, "breed")
>>> cfg.pet.breed = 123  # selected branches retain normal field conversion
>>> assert cfg.pet.breed == "123"

A mapping can initialize a structured branch when it is the only mapping-compatible member of the union. If multiple structured or dictionary branches are possible and none is selected, OmegaConf raises ValidationError and requires a typed value. A typed structured value may switch the selected branch. Branch selection itself is strict, but after selection the structured branch retains normal structured-config validation and conversion behavior, including support for Any fields and open_dict.

Merge operations distinguish branch identity from assignment. A typed source that selects the same declared structured branch is merged recursively, so MISSING source fields do not replace existing values. A typed source that selects a different declared branch replaces the destination branch. An untyped mapping merges into an already selected structured branch.

OmegaConf does not inject a discriminator or other type tag when serializing a structured union. Consequently, to_yaml followed by load does not retain the selected branch’s structured type. If multiple structured branches are possible, callers must restore that information with a typed value rather than expecting OmegaConf to select a branch from mapping keys.

If any argument of a Union type hint is Optional, the whole union is considered optional. For example, OmegaConf treats all four of the following type hints as equivalent:

Optional[Union[int, str]]
Union[Optional[int], str]
Union[int, str, None]
Union[int, str, type(None)]

Ordinarily, assignment to a structured config field results in coercion of the assigned value to the field’s type. For example, assigning an integer to a field typed as str results in the integer being coverted to a string:

>>> @dataclass
... class HasStr:
...     s: str
...
>>> cfg = OmegaConf.structured(HasStr)
>>> cfg.s = 10.1
>>> assert cfg.s == "10.1"  # The assigned value has been converted to a string

When selecting among members of a Union, conversion is disabled so as to avoid ambiguity. Values assigned directly to a union-typed field of a structured config must precisely match one of the types in the Union annotation:

>>> @dataclass
... class StrOrInt:
...     u: Union[str, float]
...
>>> cfg = OmegaConf.structured(StrOrInt)
>>> cfg.u = 10.1
>>> assert cfg.u == 10.1  # The assigned value remains a `float`.
>>> cfg.u = "10.1"
>>> assert cfg.u == "10.1"  # The assigned value remains a `str`.
>>> cfg.u = 123  # Conversion from `int` to `float` does not occur.
Traceback (most recent call last):
...
omegaconf.errors.ValidationError: Value '123' of type 'int' is incompatible with type hint 'Union[str, float]'
    full_key: u
    object_type=StrOrInt

Once a structured member is selected, assignments to its fields use the normal structured-config conversion rules described above.

Unions of container types ¶

Union members may also be typed List[...] or Dict[...] containers. OmegaConf selects the matching branch at assignment time by validating the assigned value against each candidate:

>>> from dataclasses import field
>>> from typing import Dict, List, Union
>>>
>>> @dataclass
... class HasContainerUnion:
...     value: Union[List[int], Dict[str, int]] = field(default_factory=lambda: [1, 2])
...
>>> cfg = OmegaConf.structured(HasContainerUnion)
>>> assert cfg.value == [1, 2]        # default selects List[int]
>>> cfg.value = {"x": 1}             # selects Dict[str, int]
>>> assert cfg.value == {"x": 1}
>>> cfg.value = [3, 4]               # back to List[int]
>>> assert cfg.value == [3, 4]

Once a branch is selected, the field is fully typed and rejects values that violate the selected container’s element type:

>>> cfg.value = [1, 2]
>>> cfg.value.append(1)              # ok — List[int]
>>> cfg.value.append("x")           # not ok — str is not int
Traceback (most recent call last):
...
omegaconf.errors.ValidationError: Value 'x' of type 'str' is incompatible with type hint 'int'
    full_key: value.[3]
    reference_type=List[int]
    object_type=list

Ambiguous assignments. When a value is valid for more than one union member — most commonly an empty container — OmegaConf raises a ValidationError rather than silently picking the first branch:

>>> @dataclass
... class ListUnion:
...     value: Union[List[int], List[str]] = field(default_factory=lambda: [1])
...
>>> cfg = OmegaConf.structured(ListUnion)
>>> cfg.value = []
Traceback (most recent call last):
...
omegaconf.errors.ValidationError: Ambiguous assignment to Union[List[int], List[str]]. Value '[]' matches multiple union members: List[int], List[str]. Use an explicitly typed container to disambiguate.
    full_key: value
    object_type=ListUnion

Use OmegaConf.typed_list() or OmegaConf.typed_dict() to create an explicitly typed container and resolve the ambiguity:

>>> cfg.value = OmegaConf.typed_list([], element_type=str)
>>> assert cfg.value == []
>>> cfg.value.append("hello")
>>> assert cfg.value == ["hello"]

Both methods accept an optional initial content argument:

>>> lst = OmegaConf.typed_list([1, 2, 3], element_type=int)
>>> d = OmegaConf.typed_dict({"x": 1}, key_type=str, element_type=int)

Other special features ¶

OmegaConf supports field modifiers such as MISSING and Optional.

>>> from typing import Optional
>>> from omegaconf import MISSING

>>> @dataclass
... class Modifiers:
...     num: int = 10
...     optional_num: Optional[int] = 10
...     another_num: int = MISSING
...     optional_dict: Optional[Dict[str, int]] = None
...     list_optional: List[Optional[int]] = field(default_factory=lambda: [10, MISSING, None])

>>> conf: Modifiers = OmegaConf.structured(Modifiers)

Mandatory missing values ¶

Fields assigned the constant MISSING do not have a value and the value must be set prior to accessing the field. Otherwise a MissingMandatoryValue exception is raised.

>>> with raises(MissingMandatoryValue):
...     x = conf.another_num
>>> conf.another_num = 20
>>> assert conf.another_num == 20

Optional fields ¶

>>> with raises(ValidationError):
...     # regular fields cannot be assigned None
...     conf.num = None

>>> conf.optional_num = None
>>> assert conf.optional_num is None
>>> assert conf.list_optional[2] is None

Interpolations ¶

Variable interpolation works normally with Structured configs, but static type checkers may object to you assigning a string to another type. To work around this, use the special functions omegaconf.SI and omegaconf.II described below.

>>> from omegaconf import SI, II
>>> @dataclass
... class Interpolation:
...     val: int = 100
...     # This will work, but static type checkers will complain
...     a: int = "${val}"
...     # This is equivalent to the above, but static type checkers
...     # will not complain
...     b: int = SI("${val}")
...     # This is syntactic sugar; the input string is
...     # wrapped with ${} automatically.
...     c: int = II("val")

>>> conf: Interpolation = OmegaConf.structured(Interpolation)
>>> assert conf.a == 100
>>> assert conf.b == 100
>>> assert conf.c == 100

Interpolated values are validated, and converted when possible, to the annotated type when the interpolation is accessed, e.g:

>>> from omegaconf import II
>>> @dataclass
... class Interpolation:
...     str_key: str = "string"
...     int_key: int = II("str_key")

>>> cfg = OmegaConf.structured(Interpolation)
>>> cfg.int_key  # fails due to type mismatch
Traceback (most recent call last):
  ...
omegaconf.errors.InterpolationValidationError: Value 'string' could not be converted to Integer
    full_key: int_key
    object_type=Interpolation
>>> cfg.str_key = "1234"  # string value
>>> assert cfg.int_key == 1234  # automatically convert str to int

Note however that this validation step is currently skipped for container node interpolations:

>>> @dataclass
... class NotValidated:
...     some_int: int = 0
...     some_dict: Dict[str, str] = II("some_int")

>>> cfg = OmegaConf.structured(NotValidated)
>>> assert cfg.some_dict == 0  # type mismatch, but no error

Frozen classes ¶

Frozen dataclasses and attr classes are supported via OmegaConf Read-only flag, which makes the entire config node and all if it’s child nodes read-only.

>>> from dataclasses import dataclass, field
>>> from typing import List
>>> @dataclass(frozen=True)
... class FrozenClass:
...     x: int = 10
...     list: List = field(default_factory=lambda: [1, 2, 3])

>>> conf = OmegaConf.structured(FrozenClass)
>>> with raises(ReadonlyConfigError):
...    conf.x = 20

The read-only flag is recursive:

>>> with raises(ReadonlyConfigError):
...    conf.list[0] = 20

Merging with other configs ¶

Once an OmegaConf object is created, it can be merged with others regardless of its source. OmegaConf configs created from Structured configs contains type information that is enforced at runtime. This can be used to validate config files based on a schema specified in a structured config class

example.yaml file:

server:
  port: 80
log:
  file: ???
  rotation: 3600
users:
  - user1
  - user2

A Schema for the above config can be defined like this.

>>> @dataclass
... class Server:
...     port: int = MISSING

>>> @dataclass
... class Log:
...     file: str = MISSING
...     rotation: int = MISSING

>>> @dataclass
... class MyConfig:
...     server: Server = field(default_factory=Server)
...     log: Log = field(default_factory=Log)
...     users: List[int] = field(default_factory=list)

I intentionally made an error in the type of the users list (List[int] should be List[str]). This will cause a validation error when merging the config from the file with that from the scheme.

>>> schema = OmegaConf.structured(MyConfig)
>>> conf = OmegaConf.load("source/example.yaml")
>>> with raises(ValidationError):
...     OmegaConf.merge(schema, conf)

Using Metadata to Ignore Fields ¶

OmegaConf inspects the metadata of dataclasss / attr class fields, ignoring any fields where metadata["omegaconf_ignore"] is True. When defining a dataclass or attr class, fields can be given metadata by passing the metadata keyword argument to the dataclasses.field function or the attrs.field function:

>>> @dataclass
... class HasIgnoreMetadata:
...     normal_field: int = 1
...     field_ignored: int = field(default=2, metadata={"omegaconf_ignore": True})
...     field_not_ignored: int = field(default=3, metadata={"omegaconf_ignore": False})
...
>>> cfg = OmegaConf.create(HasIgnoreMetadata)
>>> cfg
{'normal_field': 1, 'field_not_ignored': 3}

In the above example, field_ignored is ignored by OmegaConf.