FluxML/Flux.jl

'cell_clip' and 'proj_clip' for Flux: LSTM

Open

#653 opened on Mar 1, 2019

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Julia (619 forks)batch import
enhancementhelp wanted

Repository metrics

Stars
 (4,725 stars)
PR merge metrics
 (Avg merge 3d 6h) (6 merged PRs in 30d)

Description

Looking through the docs. , I found that this feature is missing for LSTM layers. It comes as an optional parameter in tensorflow LSTMCell This will be useful for porting stuff to flux while achieving the same results as from tensorflow implementations for some popular algorithms (like ELMo). Looking in the code base of tensorflow if says that it clip the value of cell states between [-cell_clip, cell_clip] in it's clip_by_value method.

Contributor guide

Research direction
Study the current LSTM implementation in Flux.jl and add support for cell clip and proj clip parameters following TensorFlow's implementation.
Tech stack
julia
Domain
machine learning
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Julia
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75