online-ml/river

Online GapEncoder

Open

#1,439 opened on Nov 3, 2023

View on GitHub
 (4 comments) (0 reactions) (0 assignees)Python (553 forks)batch import
Good first issueNew feature

Repository metrics

Stars
 (4,574 stars)
PR merge metrics
 (Avg merge 47d 6h) (46 merged PRs in 30d)

Description

skrub is a wonderful new project related to scikit-learn. You can see Gaël Varoquaux present it here. They have a transformer called GapEncoder: it's a way to embed fuzzy strings. This could be really powerful online, say for classifying Tweets or Twitch messages, where typos are aplenty.

We already have a way to do online TD-IDF/count vectorization. But we don't have Gamma-Poisson matrix factorization. It is doable online though. Once we have it, we could assemble the two into a nice GapEncoder class. See paper here.

This is related to #1412. Indeed, maybe this works well without Gamma-Poisson matrix factorization. For instance, we could use decomposition.LDA, which we already have.

Contributor guide

Research direction
Explore online Gamma Poisson matrix factorization or online LDA to implement an online GapEncoder.
Tech stack
python
Domain
machine learning
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Needs investigation
Prerequisites
PythonMachine learning basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
20