Terminal slows down after enabling this plugin

General infomation

Terminal: Konsole OS: Ubuntu 20.04 ZSH framework: oh-my-zsh ZSH version 5.8 ZVM version 0.8.3

Basic examination

• I have read through the README page
• I have the latest version of zsh-vi-mode
• I have tested with another terminal program

Problem description

Enabling this plugin makes prompt 2.5 times slow. Am I doing something wrong?

Without any plugins

********************************************************************
                      Prompt Benchmark Results
********************************************************************
Warmup duration      4s
Benchmark duration   2.011s
Benchmarked prompts  112
Time per prompt      17.95ms  <-- prompt latency (lower is better)
********************************************************************

Nearly 15 plugins

********************************************************************
                      Prompt Benchmark Results
********************************************************************
Warmup duration      4s
Benchmark duration   2.001s
Benchmarked prompts  101
Time per prompt      19.81ms  <-- prompt latency (lower is better)
********************************************************************

15 other plugins and zsh-vi-mode

********************************************************************
                      Prompt Benchmark Results
********************************************************************
Warmup duration      4s
Benchmark duration   2.044s
Benchmarked prompts  41
Time per prompt      49.84ms  <-- prompt latency (lower is better)
********************************************************************

Here is all my settings related with this plugin:

ZVM_LAZY_KEYBINDINGS=false
ZVM_VI_INSERT_ESCAPE_BINDKEY=kj
plugins=(... zsh-vi-mode)
source $ZSH/oh-my-zsh.sh

# https://github.com/jeffreytse/zsh-vi-mode#execute-extra-commands
function my_bindings() {
    # easy bindings instead of ctrl + arrow keys
    bindkey '^j' backward-word
    bindkey '^f' forward-word
    bindkey '^g' autosuggest-accept
    bindkey "\e\e" sudo-command-line
}


zvm_after_init_commands+=('[ -f ~/.fzf.zsh ] && source ~/.fzf.zsh')
zvm_after_init_commands+=(my_bindings)
# Always starting with insert mode for each command line
ZVM_LINE_INIT_MODE=$ZVM_MODE_INSERT
ZVM_INSERT_MODE_CURSOR=$ZVM_CURSOR_BLOCK

Reproduction steps

1. Install and enable zsh-prompt-bencmark
2. Benchmark with zsh-vi-mode enabled
3. Benchmark with zsh-vi-mode not enabled
4. Compare the results and see this plugin takes too much time

Expected behavior

I expect this plugin to be not take too much time like the other ones.