Bro/Zeek Logging Framework for Python¶
Table of Contents¶
- Dumpers
- Loaders
- Data Model
- Data Types
- Typing Annotations
- Data Classes
- Exceptions & Warnings
- Internal Auxiliary Functions
- Enum Namespace
- Module Contents
- Namespaces
BrokerNamespaceClusterNamespaceDCE_RPCNamespaceHTTPNamespaceInputNamespaceIntelNamespaceJSONNamespaceKnownNamespaceLoadBalancingNamespaceLogNamespaceMOUNT3NamespaceMQTTNamespaceNFS3NamespaceNetControlNamespaceNoticeNamespaceOpenFlowNamespaceProtocolDetectorNamespaceReporterNamespaceSMBNamespaceSOCKSNamespaceSSLNamespaceSignaturesNamespaceSoftwareNamespaceSumStatsNamespaceTunnelNamespaceWeirdNamespaceZeekygenExampleNamespacezeekNamespace
Module Contents¶
Bro/Zeek logging framework.
-
zlogging.write(data, filename, format, *args, **kwargs)[source] Write Bro/Zeek log file.
- Parameters
data (
IterableofModel) – Log records as anIterableofModelper line.filename (PathLike[str]) – Log file name.
format (
str) – Log format.*args – See
write_json()andwrite_ascii()for more information.args (Any) –
kwargs (Any) –
- Keyword Arguments
**kwargs – See
write_json()andwrite_ascii()for more information.- Raises
WriterFormatError – If
formatis not supported.- Return type
-
zlogging.dump(data, file, format, *args, **kwargs)[source] Write Bro/Zeek log file.
- Parameters
data (
IterableofModel) – Log records as anIterableofModelper line.format (
str) – Log format.file (TextFile) – Log file object opened in text mode.
*args – See
dump_json()anddump_ascii()for more information.args (Any) –
kwargs (Any) –
- Keyword Arguments
**kwargs – See
dump_json()anddump_ascii()for more information.- Raises
WriterFormatError – If
formatis not supported.- Return type
-
zlogging.dumps(data, format, *args, **kwargs)[source] Write Bro/Zeek log string.
- Parameters
data (
IterableofModel) – Log records as anIterableofModelper line.format (
str) – Log format.*args – See
dumps_json()anddumps_ascii()for more information.args (Any) –
kwargs (Any) –
- Keyword Arguments
**kwargs – See
dumps_json()anddumps_ascii()for more information.- Raises
WriterFormatError – If
formatis not supported.- Return type
-
zlogging.parse(filename, *args, **kwargs)[source] Parse Bro/Zeek log file.
- Parameters
filename (PathLike[str]) – Log file name.
*args – See
parse_json()andparse_ascii()for more information.**kwargs – See
parse_json()andparse_ascii()for more information.args (Any) –
kwargs (Any) –
- Return type
- Returns
The parsed JSON log data.
- Raises
ParserError – If the format of the log file is unknown.
-
zlogging.load(file, *args, **kwargs)[source] Parse Bro/Zeek log file.
- Parameters
file (BinaryFile) – Log file object opened in binary mode.
*args – See
load_json()andload_ascii()for more information.**kwargs – See
load_json()andload_ascii()for more information.args (Any) –
kwargs (Any) –
- Return type
- Returns
The parsed JSON log data.
- Raises
ParserError – If the format of the log file is unknown.
-
zlogging.loads(data, *args, **kwargs)[source] Parse Bro/Zeek log string.
- Parameters
data (AnyStr) – Log string as binary or encoded string.
*args – See
loads_json()andloads_ascii()for more information.**kwargs – See
loads_json()andloads_ascii()for more information.args (Any) –
kwargs (Any) –
- Return type
- Returns
The parsed JSON log data.
- Raises
ParserError – If the format of the log file is unknown.
-
class
zlogging.Model(*args, **kwargs)[source] Bases:
objectLog data model.
- Variables
__fields__ (
OrderedDictmappingstrandBaseType) – Fields of the data model.__record_fields__ (
OrderedDictmappingstrandRecordType) – Fields ofrecorddata type in the data model.__empty_field__ (bytes) – Placeholder for empty field.
__unset_field__ (bytes) – Placeholder for unset field.
__set_separator__ (bytes) – Separator for set/vector fields.
- Warns
BroDeprecationWarning – Use of
bro_*type annotations.- Raises
ModelValueError – In case of inconsistency between field data types, or values of
unset_field,empty_fieldandset_separator.ModelTypeError – Wrong parameters when initialisation.
Note
Customise the
Model.__post_init__method in your subclassed data model to implement your own ideas.Example
Define a custom log data model using the prefines Bro/Zeek data types, or subclasses of
BaseType:class MyLog(Model): field_one = StringType() field_two = SetType(element_type=PortType)
Or you may use type annotations as PEP 484 introduced when declaring data models. All available type hints can be found in
zlogging.typing:class MyLog(Model): field_one: zeek_string field_two: zeek_set[zeek_port]
However, when mixing annotations and direct assignments, annotations will take proceedings, i.e. the
Modelclass shall process first annotations then assignments. Should there be any conflicts,ModelErrorwill be raised.See also
See
expand_typing()for more information about processing the fields.-
property
fields fields of the data model
-
__call__(format)[source] Serialise data model with given format.
- Parameters
format (
str) – Serialisation format.- Return type
Any
- Returns
The serialised data.
- Raises
ModelFormatError – If
formatis not supproted, i.e.Mode.to{format}()does not exist.
-
tojson()[source] Serialise data model as JSON log format.
- Return type
OrderedDict[str, Any]
- Returns
An
OrderedDictmapping each field and serialised JSON serialisable data.
-
toascii()[source] Serialise data model as ASCII log format.
-
asdict(dict_factory=None)[source] Convert data model as a dictionary mapping field names to field values.
-
zlogging.new_model(name, **fields)[source] Create a data model dynamically with the appropriate fields.
- Parameters
name (
str) – data model name**fields – defined fields of the data model
fields (Any) –
- Returns
created data model
- Return type
Model
Examples
Typically, we define a data model by subclassing the
Modelclass, as following:class MyLog(Model): field_one = StringType() field_two = SetType(element_type=PortType)
when defining dynamically with
new_model(), the definition above can be rewrote to:MyLog = new_model('MyLog', field_one=StringType(), field_two=SetType(element_type=PortType))
-
class
zlogging.AddrType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
addrdata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, IPAddress]) – raw data
- Return type
Optional[IPAddress]
- Returns
The parsed IP address. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[IPAddress]) – raw data
- Returns
The JSON serialisable IP address string.
- Return type
-
class
zlogging.BoolType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
booldata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
Type[bool]
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“bool”]
-
parse(data)[source] Parse
datafrom string.
-
tojson(data)[source] Serialize
dataas JSON log format.
-
class
zlogging.CountType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
countdata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
Type[uint64]
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“count”]
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, uint64]) – raw data
- Return type
Optional[uint64]
- Returns
The parsed numeral data. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[uint64]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.DoubleType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
doubledata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
Type[Decimal]
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“double”]
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, Decimal]) – raw data
- Return type
Optional[Decimal]
- Returns
The parsed numeral data. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[Decimal]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.EnumType(empty_field=None, unset_field=None, set_separator=None, namespaces=None, bare=False, enum_hook=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
enumdata type.- Parameters
empty_field (
bytesorstr, optional) – Placeholder for empty field.unset_field (
bytesorstr, optional) – Placeholder for unset field.set_separator (
bytesorstr, optional) – Separator forset/vectorfields.namespaces (
List[str], optional) – Namespaces to be loaded.bare (
bool, optional) – IfTrue, do not loadzeeknamespace by default.enum_hook (
dictmapping ofstrandenum.Enum, optional) – Additional enum to be included in the namespace.*args – Variable length argument list.
**kwargs – Arbitrary keyword arguments.
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, Enum]) – raw data
- Return type
Optional[Enum]
- Returns
The parsed enum data. If
datais unset,Nonewill be returned.- Warns
ZeekValueWarning – If
dateis not defined in the enum namespace.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[Enum]) – raw data
- Returns
The JSON serialisable enum data.
- Return type
-
class
zlogging.IntervalType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
intervaldata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
Type[TimeDeltaType]
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“interval”]
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, TimeDeltaType]) – raw data
- Return type
Optional[TimeDeltaType]
- Returns
The parsed numeral data. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[TimeDeltaType]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.IntType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
intdata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
Type[int64]
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“int”]
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, int64]) – raw data
- Return type
Optional[int64]
- Returns
The parsed numeral data. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[int64]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.PortType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
portdata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
Type[uint16]
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“port”]
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, uint16]) – raw data
- Return type
Optional[uint16]
- Returns
The parsed port number. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[uint16]) – raw data
- Returns
The JSON serialisable port number string.
- Return type
-
class
zlogging.RecordType(empty_field=None, unset_field=None, set_separator=None, *args, **element_mapping)[source] Bases:
zlogging.types._VariadicTypeBro/Zeek
recorddata type.- Parameters
empty_field (
bytesorstr, optional) – Placeholder for empty field.unset_field (
bytesorstr, optional) – Placeholder for unset field.set_separator (
bytesorstr, optional) – Separator forset/vectorfields.*args – Variable length argument list.
**kwargs – element_mapping (
dictmappingstrandBaseTypeinstance): Data type of container’s elements.
- Variables
- Raises
ZeekTypeError – If
element_mappingis not supplied.ZeekValueError – If
element_mappingis not a valid Bro/Zeek data type; or in case of inconsistency fromempty_field,unset_fieldandset_separatorof each field.
Note
A valid
element_mappingshould be a simple or generic data type, i.e. a subclass of_SimpleTypeor_GenericType.See also
See
_aux_expand_typing()for more information about processing the fields.-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“record”]
-
element_mapping: OrderedDict[str, Union[_SimpleType, _GenericType]]
-
class
zlogging.SetType(empty_field=None, unset_field=None, set_separator=None, element_type=None, *args, **kwargs)[source] Bases:
zlogging.types._GenericType,Generic[zlogging.types._S]Bro/Zeek
setdata type.- Parameters
empty_field (
bytesorstr, optional) – Placeholder for empty field.unset_field (
bytesorstr, optional) – Placeholder for unset field.set_separator (
bytesorstr, optional) – Separator forset/vectorfields.element_type (
BaseTypeinstance) – Data type of container’s elements.*args – Variable length argument list.
**kwargs – Arbitrary keyword arguments.
- Variables
- Raises
ZeekTypeError – If
element_typeis not supplied.ZeekValueError – If
element_typeis not a valid Bro/Zeek data type.
Example
As a generic data type, the class supports the typing proxy as introduced PEP 484:
>>> SetType[StringType]
which is the same at runtime as following:
>>> SetType(element_type=StringType())
Note
A valid
element_typeshould be a simple data type, i.e. a subclass of_SimpleType.-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, Set[_S]]) – raw data
- Return type
Optional[Set[_S]]
- Returns
The parsed set data. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[Set[_S]]) – raw data
- Returns
The JSON serialisable set data.
- Return type
-
class
zlogging.StringType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
stringdata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“string”]
-
parse(data)[source] Parse
datafrom string.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[ByteString]) – raw data
- Returns
The JSON serialisable string data encoded in ASCII.
- Return type
-
class
zlogging.SubnetType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
subnetdata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, IPNetwork]) – raw data
- Return type
Optional[IPNetwork]
- Returns
The parsed IP network. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[IPNetwork]) – raw data
- Returns
The JSON serialisable IP network string.
- Return type
-
class
zlogging.TimeType(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleTypeBro/Zeek
timedata type.- Parameters
- Variables
-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
Type[DateTimeType]
-
property
zeek_type Corresponding Zeek type name.
- Type
- Return type
Literal[“time”]
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, DateTimeType]) – raw data
- Return type
Optional[DateTimeType]
- Returns
The parsed numeral data. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[DateTimeType]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.VectorType(empty_field=None, unset_field=None, set_separator=None, element_type=None, *args, **kwargs)[source] Bases:
zlogging.types._GenericType,Generic[zlogging.types._S]Bro/Zeek
vectordata type.- Parameters
empty_field (
bytesorstr, optional) – Placeholder for empty field.unset_field (
bytesorstr, optional) – Placeholder for unset field.set_separator (
bytesorstr, optional) – Separator forset/vectorfields.element_type (
BaseTypeinstance) – Data type of container’s elements.*args – Variable length argument list.
**kwargs – Arbitrary keyword arguments.
- Variables
- Raises
ZeekTypeError – If
element_typeis not supplied.ZeekValueError – If
element_typeis not a valid Bro/Zeek data type.
Example
As a generic data type, the class supports the typing proxy as introduced PEP 484:
>>> VectorType[StringType]
which is the same at runtime as following:
>>> VectorType(element_type=StringType())
Note
A valid
element_typeshould be a simple data type, i.e. a subclass of_SimpleType.-
property
python_type Corresponding Python type annotation.
- Type
Any
- Return type
-
parse(data)[source] Parse
datafrom string.- Parameters
data (Union[AnyStr, List[_S]]) – raw data
- Return type
Optional[List[_S]]
- Returns
The parsed list data. If
datais unset,Nonewill be returned.
-
tojson(data)[source] Serialize
dataas JSON log format.- Parameters
data (Optional[List[_S]]) – raw data
- Returns
The JSON serialisable list data.
- Return type