bytedeco/javacv

FFmpegFrameGrabber.grab() occasionally skipped when extract all frames from video with HW codec in android

Open

#1,895 opened on Sep 28, 2022

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Java (1,583 forks)batch import
bughelp wanted

Repository metrics

Stars
 (6,985 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

[Environment] JavaCV: 1.5.7 OS: Android 30 Device: LG V50(LM-V500N)

[What i'm trying to do]

  • Extract all frames from MP4 video
  • Run on background thread

[Description of the problem situation] My program works fine when I use software codecs. I got the same result N times(93 frames were extracted from the reference video).

When the program is executed after writing the code to use the hardware codec, the first execution is extracted with the same number of frames as when using the software codec.

But FFmpegFrameGrabber.grab() occasionally skipped(returned null) sometimes. As a result, the total number of frames is different each time. 1st: 93 frames 2nd: 91 frames 3rd: 88 frames ...

[Video extraction code]

FFmpegFrameGrabber.tryLoad()
val bitmapConverter = AndroidFrameConverter()
val inputStream = FileInputStream(path)
val grabber = FFmpegFrameGrabber(inputStream)

try {
    FFmpegLogCallback.set()
    grabber.pixelFormat = AV_PIX_FMT_RGBA
    grabber.videoCodecName = "h264_mediacodec"
    grabber.setOption("fflags", "genpts")
    grabber.setVideoOption("threads", "0")
    grabber.start()

    var totalFrameCount = 0
    for (currFrameIndex in 0 until grabber.lengthInVideoFrames) {
        val frame = grabber.grabImage()
        val src = bitmapConverter.convert(frame) ?: continue
        totalFrameCount++
     }
    logD("DEBUG", "total frame count : $totalFrameCount")
} catch (e: Exception) {
    throw e
} finally {
    inputStream.close()
    grabber.release()
    bitmapConverter.close()
}

[Solutions I tried but failed]

  1. Use startUnsafe() instead of start()
  2. Use grabAtFrameRate() instead of grab() (https://github.com/bytedeco/javacv/commit/b29601401964831630b64ad0fba504a4dc8977e3)

Is there any other way I can try?

Contributor guide