piskvorky/gensim

Improving computational time in LDASeqModel

Open

#1,545 opened on Aug 22, 2017

View on GitHub
 (9 comments) (1 reaction) (0 assignees)Python (4,349 forks)batch import
Hacktoberfestdifficulty mediumfeaturegood first issueperformance

Repository metrics

Stars
 (15,144 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Hi,

I was training a LDASeqModel on ~40k documents through 20 years (20 time slices in my case) and after 2 days of computation, still didn't finish.

I realized that only LDAModel is used in this class. Shouldn't be better to use LDAMulticore since it can also handle the case of single core ?

Thanks

Contributor guide

Research direction
Examine the LDASeqModel class in gensim/models/ldaseqmodel.py. Identify where LDAModel is instantiated and how it is used. Investigate the LDAMulticore class to understand its interface and whether it can replace LDAModel while maintaining the same functionality. Consider performance benchmarks and potential parameter differences. Propose a change to use LDAMulticore when available, falling back to LDAModel if needed.
Tech stack
python
Domain
machine learningai
Issue type
Performance
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75