EvaluatorBuilder
A builder for constructing MetricEvaluator instances.
This builder allows for step-by-step configuration of the MetricEvaluator, including adding metrics with specific reducers and options. Or, alternatively, loading configuration from a dictionary.
Source code in metrics_toolbox/builder.py
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 | |
add_metric(metric, reducers=(ReducerEnum.LATEST,), **kwargs)
Add a Metric to the evaluator.
Metric can be specified by name or enum.
Details
By default, the metric is added with the LATEST reducer and no options. Reducers and options can be specified via kwargs. Reducers can be provided as strings or ReducerEnum values.
| Parameters: |
|
|---|
| Returns: |
|
|---|
Source code in metrics_toolbox/builder.py
build(class_to_instantiate=None)
Execute the builder and produce a MetricEvaluator.
| Parameters: |
|
|---|
| Returns: |
|
|---|
Source code in metrics_toolbox/builder.py
from_dict(cfg)
Configure the builder from a dictionary.
| Parameters: |
|
|---|
| Returns: |
|
|---|
Examples:
>>> cfg = {
... "<metric name>": {"reducers": ["mean", "min"], <metric specific kwargs>: ...},
... }
>>> builder = EvaluatorBuilder().from_dict(cfg)