Metric

The base class for all metrics.

Details

  • Each Metric subclass must implement the compute method.

  • Define the class private attributes to specify what kind of metric it is

  • Allow to pass possible options during initialization.

Attributes:
  • _name (MetricNameEnum) –

    The name of the metric.

  • _scope (MetricScopeEnum) –

    The scope of the metric (micro, macro, class).

  • _type (MetricTypeEnum) –

    The type of data the metric operates on (probabilities or labels).
Source code in metrics_toolbox/metrics/base_metric.py 
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
class Metric:
    """The base class for all metrics.

    Details
    -------

    - Each Metric subclass must implement the compute method.

    - Define the class private attributes to specify what kind of metric it is

    - Allow to pass possible options during initialization.

    Attributes
    ----------
    _name: MetricNameEnum
        The name of the metric.
    _scope: MetricScopeEnum
        The scope of the metric (micro, macro, class).
    _type: MetricTypeEnum
        The type of data the metric operates on (probabilities or labels).
    """

    _name: MetricNameEnum = None
    _scope: MetricScopeEnum = None
    _type: MetricTypeEnum = None

    def __init__(self, **kwargs):
        pass

    def __repr__(self) -> str:
        return f"Metric(name={self.name.value}, scope={self.scope.value}, type={self.type.value})"

    @property
    def name(self) -> MetricNameEnum:
        """The name of the metric.

        Returns
        -------
        MetricNameEnum
            The name of the metric.
        """
        return self._name

    @property
    def scope(self) -> MetricScopeEnum:
        """The scope of the metric.

        Returns
        -------
        MetricScopeEnum
            The scope of the metric.
        """
        return self._scope

    @property
    def type(self) -> MetricTypeEnum:
        """The type of data the metric operates on.

        Returns
        -------
        MetricTypeEnum
            The type of data the metric operates on.
        """
        return self._type

    @property
    def id(self) -> str:
        """Unique identifier for the metric instance.

        Child classes can override this to make more specific ids if needed.

        Returns
        -------
        str
            The unique identifier for the metric instance.
        """
        return f"{self.name.value}_{self.scope.value}"

    def compute(
        self, y_true: np.ndarray, y_pred: np.ndarray, column_names: list[str]
    ) -> MetricResult:
        """A method to compute the metric.

        Different metrics will have different implementations of this method,
        but they all share the same signature.

        y_true and y_pred are always 2D arrays with the same shape,
        where each column corresponds to a different prediction value,
        and column_names specify the names of these columns.

        It is possible to use Metrics directly, but the assumed way is to use them
        via a Evaluator class, and thus, all input validation is done there.

        Parameters
        ----------
        y_true : np.ndarray
            The ground truth target values.
        y_pred : np.ndarray
            The predicted target values.
        column_names : list[str]
            The names of the columns the input arrays correspond to.
        """
        raise NotImplementedError

id property

Unique identifier for the metric instance.

Child classes can override this to make more specific ids if needed.

Returns:
  • str –

    The unique identifier for the metric instance.

name property

The name of the metric.

Returns:

scope property

The scope of the metric.

Returns:

type property

The type of data the metric operates on.

Returns:

compute(y_true, y_pred, column_names)

A method to compute the metric.

Different metrics will have different implementations of this method, but they all share the same signature.

y_true and y_pred are always 2D arrays with the same shape, where each column corresponds to a different prediction value, and column_names specify the names of these columns.

It is possible to use Metrics directly, but the assumed way is to use them via a Evaluator class, and thus, all input validation is done there.

Parameters:
  • y_true (ndarray) –

    The ground truth target values.

  • y_pred (ndarray) –

    The predicted target values.

  • column_names (list[str]) –

    The names of the columns the input arrays correspond to.
Source code in metrics_toolbox/metrics/base_metric.py 
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
def compute(
    self, y_true: np.ndarray, y_pred: np.ndarray, column_names: list[str]
) -> MetricResult:
    """A method to compute the metric.

    Different metrics will have different implementations of this method,
    but they all share the same signature.

    y_true and y_pred are always 2D arrays with the same shape,
    where each column corresponds to a different prediction value,
    and column_names specify the names of these columns.

    It is possible to use Metrics directly, but the assumed way is to use them
    via a Evaluator class, and thus, all input validation is done there.

    Parameters
    ----------
    y_true : np.ndarray
        The ground truth target values.
    y_pred : np.ndarray
        The predicted target values.
    column_names : list[str]
        The names of the columns the input arrays correspond to.
    """
    raise NotImplementedError