Bro/Zeek Logging Framework for Python¶
Table of Contents¶
- Dumpers
- Loaders
- Data Model
- Data Types
- Typing Annotations
- Data Classes
- Exceptions & Warnings
- Internal Auxiliary Functions
- Enum Namespace
- Module Contents
- Namespaces
Broker
NamespaceCluster
NamespaceDCE_RPC
NamespaceHTTP
NamespaceInput
NamespaceIntel
NamespaceJSON
NamespaceKnown
NamespaceLoadBalancing
NamespaceLog
NamespaceMOUNT3
NamespaceMQTT
NamespaceNFS3
NamespaceNetControl
NamespaceNotice
NamespaceOpenFlow
NamespaceProtocolDetector
NamespaceReporter
NamespaceSMB
NamespaceSOCKS
NamespaceSSL
NamespaceSignatures
NamespaceSoftware
NamespaceSumStats
NamespaceTunnel
NamespaceWeird
NamespaceZeekygenExample
Namespacezeek
Namespace
Module Contents¶
Bro/Zeek logging framework.
-
zlogging.
write
(data, filename, format, *args, **kwargs)[source] Write Bro/Zeek log file.
- Parameters
data (
Iterable
ofModel
) – Log records as anIterable
ofModel
per line.filename (PathLike[str]) – Log file name.
format (
str
) – Log format.*args – See
write_json()
andwrite_ascii()
for more information.args (Any) –
kwargs (Any) –
- Keyword Arguments
**kwargs – See
write_json()
andwrite_ascii()
for more information.- Raises
WriterFormatError – If
format
is not supported.- Return type
-
zlogging.
dump
(data, file, format, *args, **kwargs)[source] Write Bro/Zeek log file.
- Parameters
data (
Iterable
ofModel
) – Log records as anIterable
ofModel
per line.format (
str
) – Log format.file (TextFile) – Log file object opened in text mode.
*args – See
dump_json()
anddump_ascii()
for more information.args (Any) –
kwargs (Any) –
- Keyword Arguments
**kwargs – See
dump_json()
anddump_ascii()
for more information.- Raises
WriterFormatError – If
format
is not supported.- Return type
-
zlogging.
dumps
(data, format, *args, **kwargs)[source] Write Bro/Zeek log string.
- Parameters
data (
Iterable
ofModel
) – Log records as anIterable
ofModel
per line.format (
str
) – Log format.*args – See
dumps_json()
anddumps_ascii()
for more information.args (Any) –
kwargs (Any) –
- Keyword Arguments
**kwargs – See
dumps_json()
anddumps_ascii()
for more information.- Raises
WriterFormatError – If
format
is not supported.- Return type
-
zlogging.
parse
(filename, *args, **kwargs)[source] Parse Bro/Zeek log file.
- Parameters
filename (PathLike[str]) – Log file name.
*args – See
parse_json()
andparse_ascii()
for more information.**kwargs – See
parse_json()
andparse_ascii()
for more information.args (Any) –
kwargs (Any) –
- Return type
- Returns
The parsed JSON log data.
- Raises
ParserError – If the format of the log file is unknown.
-
zlogging.
load
(file, *args, **kwargs)[source] Parse Bro/Zeek log file.
- Parameters
file (BinaryFile) – Log file object opened in binary mode.
*args – See
load_json()
andload_ascii()
for more information.**kwargs – See
load_json()
andload_ascii()
for more information.args (Any) –
kwargs (Any) –
- Return type
- Returns
The parsed JSON log data.
- Raises
ParserError – If the format of the log file is unknown.
-
zlogging.
loads
(data, *args, **kwargs)[source] Parse Bro/Zeek log string.
- Parameters
data (AnyStr) – Log string as binary or encoded string.
*args – See
loads_json()
andloads_ascii()
for more information.**kwargs – See
loads_json()
andloads_ascii()
for more information.args (Any) –
kwargs (Any) –
- Return type
- Returns
The parsed JSON log data.
- Raises
ParserError – If the format of the log file is unknown.
-
class
zlogging.
Model
(*args, **kwargs)[source] Bases:
object
Log data model.
- Variables
__fields__ (
OrderedDict
mappingstr
andBaseType
) – Fields of the data model.__record_fields__ (
OrderedDict
mappingstr
andRecordType
) – Fields ofrecord
data type in the data model.__empty_field__ (bytes) – Placeholder for empty field.
__unset_field__ (bytes) – Placeholder for unset field.
__set_separator__ (bytes) – Separator for set/vector fields.
- Warns
BroDeprecationWarning – Use of
bro_*
type annotations.- Raises
ModelValueError – In case of inconsistency between field data types, or values of
unset_field
,empty_field
andset_separator
.ModelTypeError – Wrong parameters when initialisation.
Note
Customise the
Model.__post_init__
method in your subclassed data model to implement your own ideas.Example
Define a custom log data model using the prefines Bro/Zeek data types, or subclasses of
BaseType
:class MyLog(Model): field_one = StringType() field_two = SetType(element_type=PortType)
Or you may use type annotations as PEP 484 introduced when declaring data models. All available type hints can be found in
zlogging.typing
:class MyLog(Model): field_one: zeek_string field_two: zeek_set[zeek_port]
However, when mixing annotations and direct assignments, annotations will take proceedings, i.e. the
Model
class shall process first annotations then assignments. Should there be any conflicts,ModelError
will be raised.See also
See
expand_typing()
for more information about processing the fields.-
property
fields
fields of the data model
-
__call__
(format)[source] Serialise data model with given format.
- Parameters
format (
str
) – Serialisation format.- Return type
Any
- Returns
The serialised data.
- Raises
ModelFormatError – If
format
is not supproted, i.e.Mode.to{format}()
does not exist.
-
tojson
()[source] Serialise data model as JSON log format.
- Return type
OrderedDict[str, Any]
- Returns
An
OrderedDict
mapping each field and serialised JSON serialisable data.
-
toascii
()[source] Serialise data model as ASCII log format.
-
asdict
(dict_factory=None)[source] Convert data model as a dictionary mapping field names to field values.
-
zlogging.
new_model
(name, **fields)[source] Create a data model dynamically with the appropriate fields.
- Parameters
name (
str
) – data model name**fields – defined fields of the data model
fields (Any) –
- Returns
created data model
- Return type
Model
Examples
Typically, we define a data model by subclassing the
Model
class, as following:class MyLog(Model): field_one = StringType() field_two = SetType(element_type=PortType)
when defining dynamically with
new_model()
, the definition above can be rewrote to:MyLog = new_model('MyLog', field_one=StringType(), field_two=SetType(element_type=PortType))
-
class
zlogging.
AddrType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
addr
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, IPAddress]) – raw data
- Return type
Optional[IPAddress]
- Returns
The parsed IP address. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[IPAddress]) – raw data
- Returns
The JSON serialisable IP address string.
- Return type
-
class
zlogging.
BoolType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
bool
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
Type[bool]
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“bool”]
-
parse
(data)[source] Parse
data
from string.
-
tojson
(data)[source] Serialize
data
as JSON log format.
-
class
zlogging.
CountType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
count
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
Type[uint64]
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“count”]
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, uint64]) – raw data
- Return type
Optional[uint64]
- Returns
The parsed numeral data. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[uint64]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.
DoubleType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
double
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
Type[Decimal]
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“double”]
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, Decimal]) – raw data
- Return type
Optional[Decimal]
- Returns
The parsed numeral data. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[Decimal]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.
EnumType
(empty_field=None, unset_field=None, set_separator=None, namespaces=None, bare=False, enum_hook=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
enum
data type.- Parameters
empty_field (
bytes
orstr
, optional) – Placeholder for empty field.unset_field (
bytes
orstr
, optional) – Placeholder for unset field.set_separator (
bytes
orstr
, optional) – Separator forset
/vector
fields.namespaces (
List[str]
, optional) – Namespaces to be loaded.bare (
bool
, optional) – IfTrue
, do not loadzeek
namespace by default.enum_hook (
dict
mapping ofstr
andenum.Enum
, optional) – Additional enum to be included in the namespace.*args – Variable length argument list.
**kwargs – Arbitrary keyword arguments.
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, Enum]) – raw data
- Return type
Optional[Enum]
- Returns
The parsed enum data. If
data
is unset,None
will be returned.- Warns
ZeekValueWarning – If
date
is not defined in the enum namespace.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[Enum]) – raw data
- Returns
The JSON serialisable enum data.
- Return type
-
class
zlogging.
IntervalType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
interval
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
Type[TimeDeltaType]
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“interval”]
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, TimeDeltaType]) – raw data
- Return type
Optional[TimeDeltaType]
- Returns
The parsed numeral data. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[TimeDeltaType]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.
IntType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
int
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
Type[int64]
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“int”]
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, int64]) – raw data
- Return type
Optional[int64]
- Returns
The parsed numeral data. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[int64]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.
PortType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
port
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
Type[uint16]
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“port”]
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, uint16]) – raw data
- Return type
Optional[uint16]
- Returns
The parsed port number. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[uint16]) – raw data
- Returns
The JSON serialisable port number string.
- Return type
-
class
zlogging.
RecordType
(empty_field=None, unset_field=None, set_separator=None, *args, **element_mapping)[source] Bases:
zlogging.types._VariadicType
Bro/Zeek
record
data type.- Parameters
empty_field (
bytes
orstr
, optional) – Placeholder for empty field.unset_field (
bytes
orstr
, optional) – Placeholder for unset field.set_separator (
bytes
orstr
, optional) – Separator forset
/vector
fields.*args – Variable length argument list.
**kwargs – element_mapping (
dict
mappingstr
andBaseType
instance): Data type of container’s elements.
- Variables
- Raises
ZeekTypeError – If
element_mapping
is not supplied.ZeekValueError – If
element_mapping
is not a valid Bro/Zeek data type; or in case of inconsistency fromempty_field
,unset_field
andset_separator
of each field.
Note
A valid
element_mapping
should be a simple or generic data type, i.e. a subclass of_SimpleType
or_GenericType
.See also
See
_aux_expand_typing()
for more information about processing the fields.-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“record”]
-
element_mapping
: OrderedDict[str, Union[_SimpleType, _GenericType]]
-
class
zlogging.
SetType
(empty_field=None, unset_field=None, set_separator=None, element_type=None, *args, **kwargs)[source] Bases:
zlogging.types._GenericType
,Generic
[zlogging.types._S
]Bro/Zeek
set
data type.- Parameters
empty_field (
bytes
orstr
, optional) – Placeholder for empty field.unset_field (
bytes
orstr
, optional) – Placeholder for unset field.set_separator (
bytes
orstr
, optional) – Separator forset
/vector
fields.element_type (
BaseType
instance) – Data type of container’s elements.*args – Variable length argument list.
**kwargs – Arbitrary keyword arguments.
- Variables
- Raises
ZeekTypeError – If
element_type
is not supplied.ZeekValueError – If
element_type
is not a valid Bro/Zeek data type.
Example
As a generic data type, the class supports the typing proxy as introduced PEP 484:
>>> SetType[StringType]
which is the same at runtime as following:
>>> SetType(element_type=StringType())
Note
A valid
element_type
should be a simple data type, i.e. a subclass of_SimpleType
.-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, Set[_S]]) – raw data
- Return type
Optional[Set[_S]]
- Returns
The parsed set data. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[Set[_S]]) – raw data
- Returns
The JSON serialisable set data.
- Return type
-
class
zlogging.
StringType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
string
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“string”]
-
parse
(data)[source] Parse
data
from string.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[ByteString]) – raw data
- Returns
The JSON serialisable string data encoded in ASCII.
- Return type
-
class
zlogging.
SubnetType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
subnet
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, IPNetwork]) – raw data
- Return type
Optional[IPNetwork]
- Returns
The parsed IP network. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[IPNetwork]) – raw data
- Returns
The JSON serialisable IP network string.
- Return type
-
class
zlogging.
TimeType
(empty_field=None, unset_field=None, set_separator=None, *args, **kwargs)[source] Bases:
zlogging.types._SimpleType
Bro/Zeek
time
data type.- Parameters
- Variables
-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
Type[DateTimeType]
-
property
zeek_type
Corresponding Zeek type name.
- Type
- Return type
Literal[“time”]
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, DateTimeType]) – raw data
- Return type
Optional[DateTimeType]
- Returns
The parsed numeral data. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[DateTimeType]) – raw data
- Returns
The JSON serialisable numeral data.
- Return type
-
class
zlogging.
VectorType
(empty_field=None, unset_field=None, set_separator=None, element_type=None, *args, **kwargs)[source] Bases:
zlogging.types._GenericType
,Generic
[zlogging.types._S
]Bro/Zeek
vector
data type.- Parameters
empty_field (
bytes
orstr
, optional) – Placeholder for empty field.unset_field (
bytes
orstr
, optional) – Placeholder for unset field.set_separator (
bytes
orstr
, optional) – Separator forset
/vector
fields.element_type (
BaseType
instance) – Data type of container’s elements.*args – Variable length argument list.
**kwargs – Arbitrary keyword arguments.
- Variables
- Raises
ZeekTypeError – If
element_type
is not supplied.ZeekValueError – If
element_type
is not a valid Bro/Zeek data type.
Example
As a generic data type, the class supports the typing proxy as introduced PEP 484:
>>> VectorType[StringType]
which is the same at runtime as following:
>>> VectorType(element_type=StringType())
Note
A valid
element_type
should be a simple data type, i.e. a subclass of_SimpleType
.-
property
python_type
Corresponding Python type annotation.
- Type
Any
- Return type
-
parse
(data)[source] Parse
data
from string.- Parameters
data (Union[AnyStr, List[_S]]) – raw data
- Return type
Optional[List[_S]]
- Returns
The parsed list data. If
data
is unset,None
will be returned.
-
tojson
(data)[source] Serialize
data
as JSON log format.- Parameters
data (Optional[List[_S]]) – raw data
- Returns
The JSON serialisable list data.
- Return type