bytedeco/javacv

Failed to mix 2 pcm mono audio files using FFmpegFrameFilter class and "amix" filter option

Open

#1,315 opened on Oct 14, 2019

View on GitHub
 (22 comments) (0 reactions) (0 assignees)Java (1,583 forks)batch import
enhancementhelp wantedquestion

Repository metrics

Stars
 (6,985 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

Hello, I am not able to mix two mono pcm ulaw files using FFmpegFrameFilter class with the "amix" option. The output file size is much smaller (78 bytes) as comapred to input (225 KB & 225 KB) and the does not play at all.

Intially I created a mix audio running ffmpeg commandline so that I can compare the results with the output from the java program

ffmpeg -i xmit.wav -i recv.wav -filter_complex [0:a][1:a]amix=inputs=2 ffmpeg-amix-audio.wav

Commad

ffmpeg -i ffmpeg-amix-audio.wav

Console output

Output #0, wav, to 'ffmpeg-amix-audio.wav':
  Metadata:
    ISFT            : Lavf58.12.100
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 8000 Hz, mono, s16, 128 kb/s (default)
    Metadata:
      encoder         : Lavc58.18.100 pcm_s16le
size=     451kB time=00:00:28.88 bitrate= 128.0kbits/s speed= 310x
video:0kB audio:451kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.016880%

My sample java program to mix two audio files using javacv-1.5.1.jar

public static void amixTest() throws Exception, org.bytedeco.javacv.FrameFilter.Exception, org.bytedeco.javacv.FrameRecorder.Exception {
		String xmit = "C:/media/audio/xmit.wav"; //mono, 8KHz, 64 kb/s PCM ulaw
		String recv = "C:/media/audio/recv.wav"; //mono, 8KHz, 64 kb/s PCM ulaw
		String output = "C:/media/audio/javacv-amix-audio.wav";
	
		// Grab input samples
		FFmpegFrameGrabber gbXmit = new FFmpegFrameGrabber(xmit);
        FFmpegFrameGrabber gbRecv = new FFmpegFrameGrabber(recv);
        gbXmit.setAudioChannels(1);
        gbXmit.start();
        gbRecv.setAudioChannels(1);
        gbRecv.start();
        
        // Start the recorder
        FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(output,1); // output is a mono audio channel
		recorder.setFormat("wav");
		recorder.start();
		
		// Filter the Input
		// Note: the filter string from ffmpeg official site did not work (https://ffmpeg.org/ffmpeg-filters.html#amix)
		// e.g. amix=inputs=2:duration=first
		// Refer to discussion https://github.com/bytedeco/javacv/issues/1082
		// New filter string "[0:a][1:a]amix=inputs=2[a]"
		FFmpegFrameFilter filter = new FFmpegFrameFilter("[0:a][1:a]amix=inputs=2[a]", 1);
		filter.setAudioInputs(2);
        filter.start();
        
        Frame outFrame = null; Frame xmitFrame = null; Frame recvFrame = null;
        long xmitFrCt = 0; long recvFrCt = 0; long recFrCt = 0;
        boolean readAllXmitFrames= false; boolean readAllRecvFrames = false;
        
        // process frames
        while(true) {
        	
        	xmitFrame = gbXmit.grabSamples();
        	recvFrame = gbRecv.grabSamples();
        	if(xmitFrame!=null) {
        		xmitFrCt++;
        		System.out.println("Xmit frame timestamp: " + xmitFrame.timestamp);
        		filter.push(0, xmitFrame);
        	} else {
        		readAllXmitFrames = true;
        	}
        	if(recvFrame!=null) {
        		recvFrCt++;
        		System.out.println("Recv frame timestamp: " + recvFrame.timestamp);
        		filter.push(1, recvFrame);
        	} else {
        		readAllRecvFrames = true;
        	}
        	
        	while ((outFrame = filter.pullSamples()) != null) {
        		recFrCt++;
        		System.out.println("Output frame timestamp: " + outFrame.timestamp);
        		recorder.record(outFrame);
            }
        	
        	if(readAllXmitFrames && readAllRecvFrames) {
        		System.out.println("Xmit Frame Count: " + xmitFrCt + ", Recv Frame Count: " + recvFrCt + ", Recording Frame count: " + recFrCt);
        		break;
        	}
        }

Eclipse console output does not give any error, however the program failed to read the last sample from each input and the output frame/sample timestamps are different than input. Also the output stream info does not match with ffmpeg output

Xmit frame timestamp: 0
Recv frame timestamp: 0
Output frame timestamp: 0
Xmit frame timestamp: 512000
Recv frame timestamp: 512000
…..
….
Xmit frame timestamp: 28160000
Recv frame timestamp: 28160000
Output frame timestamp: 5108390
Xmit frame timestamp: 28672000
Recv frame timestamp: 28672000
Xmit Frame Count: 57, Recv Frame Count: 57, Recording Frame count: 56
Input #0, wav, from 'C:/media/audio/xmit.wav':
  Duration: 00:00:28.88, bitrate: 64 kb/s
    Stream #0:0: Audio: pcm_mulaw ([7][0][0][0] / 0x0007), 8000 Hz, 1 channels, s16, 64 kb/s
Input #0, wav, from 'C:/media/audio/recv.wav':
  Duration: 00:00:28.68, bitrate: 64 kb/s
    Stream #0:0: Audio: pcm_mulaw ([7][0][0][0] / 0x0007), 8000 Hz, 1 channels, s16, 64 kb/s
Output #0, wav, to 'C:/media/audio/javacv-amix-audio.wav':
  Metadata:
    ISFT            : Lavf58.20.100
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, mono, s16, 705 kb/s

Here is the ffmpeg output from directly quering the javacv generated file Command ffmpeg -i javacv-amix-audio.wav Console Output

Input #0, wav, from 'javacv-amix-audio.wav':
  Metadata:
    encoder         : Lavf58.20.100
  Duration: 00:00:05.20, bitrate: 705 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, mono, s16, 705 kb/s

Please suggets how to fix the java program.

Contributor guide