rapidsai/cuml

[FEA] Model precision with precision_score

Open

#1,522 opened on Dec 30, 2019

View on GitHub
 (6 comments) (0 reactions) (1 assignee)Python (644 forks)auto 404
feature requestgood first issue

Repository metrics

Stars
 (5,223 stars)
PR merge metrics
 (PR metrics pending)

Description

I'd like to be able to use cuML to calculate the precision score of an estimator in Python. I didn't see a C++/CUDA implementation in the metrics/ header files, so this may require both C++/CUDA and Python/Cython bindings eventually. Didn't see an existing issue but will close if I find one.

Currently, this requires going to the CPU if we want to use an existing API (relying on NEP-18 for dispatch doesn't work here). With about three million rows, sklearn's precision_score can take about 1.5 seconds. A stopgap GPU implementation for binary precision using CuPy can be 500x+ faster.

def cupy_precision_score(y, y_pred):
    
    pos_pred_ix = cupy.where(y_pred == 1)
    tp_sum = (y_pred[pos_pred_ix] == y[pos_pred_ix]).sum()
    fp_sum = (y_pred[pos_pred_ix] != y[pos_pred_ix]).sum()
​
    return (tp_sum / (tp_sum + fp_sum)).item()
​
%time print(cupy_precision_score(y, yp.values))
%time print(precision_score(y_host, y_pred))
0.9998934583422118
CPU times: user 2.59 ms, sys: 0 ns, total: 2.59 ms
Wall time: 2.05 ms
0.9998934583422118
CPU times: user 1.47 s, sys: 59.5 ms, total: 1.53 s
Wall time: 1.5 s

Cross-listing to the tracking issue #608

Contributor guide