FFmpegFrameGrabber.grab() occasionally skipped when extract all frames from video with HW codec in android
#1,895 opened on Sep 28, 2022
Repository metrics
- Stars
- (6,985 stars)
- PR merge metrics
- (No merged PRs in 30d)
Description
[Environment] JavaCV: 1.5.7 OS: Android 30 Device: LG V50(LM-V500N)
[What i'm trying to do]
- Extract all frames from MP4 video
- Run on background thread
[Description of the problem situation] My program works fine when I use software codecs. I got the same result N times(93 frames were extracted from the reference video).
When the program is executed after writing the code to use the hardware codec, the first execution is extracted with the same number of frames as when using the software codec.
But FFmpegFrameGrabber.grab() occasionally skipped(returned null) sometimes. As a result, the total number of frames is different each time. 1st: 93 frames 2nd: 91 frames 3rd: 88 frames ...
[Video extraction code]
FFmpegFrameGrabber.tryLoad()
val bitmapConverter = AndroidFrameConverter()
val inputStream = FileInputStream(path)
val grabber = FFmpegFrameGrabber(inputStream)
try {
FFmpegLogCallback.set()
grabber.pixelFormat = AV_PIX_FMT_RGBA
grabber.videoCodecName = "h264_mediacodec"
grabber.setOption("fflags", "genpts")
grabber.setVideoOption("threads", "0")
grabber.start()
var totalFrameCount = 0
for (currFrameIndex in 0 until grabber.lengthInVideoFrames) {
val frame = grabber.grabImage()
val src = bitmapConverter.convert(frame) ?: continue
totalFrameCount++
}
logD("DEBUG", "total frame count : $totalFrameCount")
} catch (e: Exception) {
throw e
} finally {
inputStream.close()
grabber.release()
bitmapConverter.close()
}
[Solutions I tried but failed]
- Use startUnsafe() instead of start()
- Use grabAtFrameRate() instead of grab() (https://github.com/bytedeco/javacv/commit/b29601401964831630b64ad0fba504a4dc8977e3)
Is there any other way I can try?