Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FFmpegFrameGrabber FFmpegFrameRecorder may lead to off-heap memory leak #2256

Closed
ZhanPro opened this issue Jul 26, 2024 · 11 comments
Closed

Comments

@ZhanPro
Copy link

ZhanPro commented Jul 26, 2024

When I use JavaCV's FFmpegFrameGrabber and FFmpegFrameRecorder in Java to perform audio format conversion, an off-heap memory leak occurred.

H�ere is my code :
`@Slf4j
public class JavaCVDecoder implements IDecoder {

FFmpegFrameGrabber grabber;

FFmpegFrameRecorder recorder;

FFmpegFrameRecorder videoRecorder;

private int inputAudioChannels;

private AudioCodec inputAudioCodec;

private int inputSampleRate;

private long duration;

private int outputAudioChannels;

private AudioCodec outputAudioCodec;

private int outputSampleRate;

private String inputFormat;

private String outputFormat;

public int imageWidth;

public double frameRate;

public int imageHeight;

public int totalFrames;

public double totalSeconds;

public int bitrate;

@Override
public double getFrameRate() {
    return frameRate;
}

@Override
public int getImageWidth() {
    return imageWidth;
}

@Override
public int getImageHeight() {
    return imageHeight;
}


public JavaCVDecoder(InputStream inputStream, OutputStream outputStream, int audioChannels) {
    this.grabber = new FFmpegFrameGrabber(inputStream);
    this.recorder = new FFmpegFrameRecorder(outputStream, audioChannels);
}

public JavaCVDecoder(String inputFileName, String outputFileName, int audioChannels) {
    this.grabber = new FFmpegFrameGrabber(inputFileName);
    this.recorder = new FFmpegFrameRecorder(outputFileName, audioChannels);
}

public JavaCVDecoder(FFmpegFrameGrabber grabber, FFmpegFrameRecorder recorder) {
    this.grabber = grabber;
    this.recorder = recorder;
}

public JavaCVDecoder(ByteArrayInputStream inputStream, ByteArrayOutputStream audioOutput,
                     ByteArrayOutputStream videoOutput) {
    this.grabber = new FFmpegFrameGrabber(inputStream);
    this.recorder = new FFmpegFrameRecorder(audioOutput, 1);
    this.videoRecorder = new FFmpegFrameRecorder(videoOutput, this.grabber.getImageWidth(),
            this.grabber.getImageHeight());
}

@Override
public void start() {
    try {
        this.grabber.start();
        this.recorder.start();

        int inputCodecId = this.grabber.getAudioCodec();
        this.inputAudioCodec = this.getAudioCodecByCodecId(inputCodecId);

        this.inputAudioChannels = this.grabber.getAudioChannels();
        this.inputSampleRate = this.grabber.getSampleRate();
        this.duration = this.grabber.getLengthInTime() / Constant.MICROSECONDS_PER_MILLISECOND;
        this.inputFormat = this.grabber.getFormat();

        int outputCodecId = this.recorder.getAudioCodec();
        this.outputAudioCodec = this.getAudioCodecByCodecId(outputCodecId);
        this.outputAudioChannels = this.recorder.getAudioChannels();
        this.outputSampleRate = this.recorder.getSampleRate();
        this.outputFormat = this.recorder.getFormat();

        Frame frame;
        while ((frame = this.grabber.grabFrame()) != null) {
            if (frame.type == Frame.Type.AUDIO) {
                this.recorder.record(frame);
                frame.close();
            }
        }
    } catch (FFmpegFrameGrabber.Exception e) {
        log.error("ffmpeg frame grabber start exception:{}", e.getMessage(), e);
        throw new RuntimeException(e);
    } catch (FFmpegFrameRecorder.Exception e) {
        log.error("ffmpeg frame recorder start exception:{}", e.getMessage(), e);
        throw new RuntimeException(e);
    } catch (FrameGrabber.Exception e) {
        log.error("frame grabber exception:{}", e.getMessage(), e);
        throw new RuntimeException(e);
    }
}

`

@ZhanPro
Copy link
Author

ZhanPro commented Jul 26, 2024

The version is 1.5.8

org.bytedeco
javacv-platform
1.5.8

@saudet
Copy link
Member

saudet commented Jul 26, 2024

Please try again with the snapshots: http://bytedeco.org/builds/

@ZhanPro
Copy link
Author

ZhanPro commented Jul 26, 2024

Oh, thanks, I‘ll try it

@ZhanPro
Copy link
Author

ZhanPro commented Jul 26, 2024

Please try again with the snapshots: http://bytedeco.org/builds/
So it's a known issue in 1.5.8 ?

@saudet
Copy link
Member

saudet commented Jul 26, 2024

Duplicate of #2255

@saudet saudet marked this as a duplicate of #2255 Jul 26, 2024
@saudet saudet closed this as completed Jul 26, 2024
@ZhanPro
Copy link
Author

ZhanPro commented Jul 29, 2024

I have followed the instructions to upgrade the version of JavaCV to 1.5.11-SNAPSHOT. However, I encountered an issue: after the upgrade, my program runs fine on macOS when packaged into a JAR, but it fails to run on Linux, with the following error:

Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.bytedeco.ffmpeg.global.avutil
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.bytedeco.javacpp.Loader.load(Loader.java:1289)
at org.bytedeco.javacpp.Loader.load(Loader.java:1234)
at org.bytedeco.javacpp.Loader.load(Loader.java:1210)
at org.bytedeco.ffmpeg.avformat.Read_packet_Pointer_BytePointer_int.(Read_packet_Pointer_BytePointer_int.java:45)
at org.bytedeco.javacv.FFmpegFrameGrabber.(FFmpegFrameGrabber.java:372)
at com.sankuai.ai.speech.platform.codec.decoder.javacvdecoder.JavaCVDecoder.(JavaCVDecoder.java:84)
at com.sankuai.ai.speech.platform.codec.uitls.DecodeUtils.decode(DecodeUtils.java:98)
at com.sankuai.ai.speech.platform.api.gateway.component.AudioNormalizeComponent.process(AudioNormalizeComponent.java:79)

The dependencies I am using are:

`
org.bytedeco
javacv
1.5.11-SNAPSHOT

        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacpp</artifactId>
            <version>1.5.11-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>ffmpeg-platform</artifactId>
            <version>7.0-1.5.11-SNAPSHOT</version>
        </dependency>

`

What is the reason for this?

@ZhanPro
Copy link
Author

ZhanPro commented Jul 29, 2024

操作系统
CentOS 7.6.1810
内核版本
Linux 5.10.0-136.16.0.mt20230627.508.mt2203sp1.x86_64
CPU 型号
Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz

@ZhanPro
Copy link
Author

ZhanPro commented Jul 29, 2024

I sincerely look forward to your reply.

@saudet
Copy link
Member

saudet commented Jul 29, 2024 via email

@ZhanPro
Copy link
Author

ZhanPro commented Jul 31, 2024

Please try again with Ubuntu

On Mon, Jul 29, 2024, 17:15 ZhanPro @.> wrote: I sincerely look forward to your reply. — Reply to this email directly, view it on GitHub <#2256 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZMQFYWVWMI5ASYXS6FJB3ZOX22BAVCNFSM6AAAAABLPV2KGWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENJVGI4TEMBYGU . You are receiving this because you modified the open/close state.Message ID: @.>

However, all our online servers are running on CentOS.

@ZhanPro
Copy link
Author

ZhanPro commented Jul 31, 2024

so, is there any other way?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants