macSubtitleOCR

Overview

macSubtitleOCR converts bitmap subtitles into the SubRip subtitle format (SRT) using the macOS Vision framework to perform OCR. Currently PGS and VobSub subtitles are supported.

For more details on OCR accuracy, refer to the Accuracy section below.

Release and Nightly builds are provided for arm64. Unfortunately, GitHub Actions doesn't provide a runner I can build x86_64 binaries on, sorry. However, you should be able to compile for x86_64 on you own machine by following the directions in the Building the Project section below.

Features

• Export raw JSON output from Vision for further analysis
• Save .png images of subtitles for manual correction of OCR output
• Optional support for FFmpeg in case of any issues with internal decoder

Supported Formats

• PGS (.mkv, .sup)
• VobSub (.mkv, .sub, .idx)

Building the Project

Important

macSubtitleOCR requires Swift 6 support to compile

To make and install the project, follow the directions below:

Build Internal Decoder Only

To build macSubtitleOCR, follow these steps:

git clone https://github.com/ecdye/macSubtitleOCR
cd macSubtitleOCR
swift build --configuration release

Build With FFmpeg Decoder

To build with FFmpeg support, follow these steps:

brew install ffmpeg
git clone https://github.com/ecdye/macSubtitleOCR
cd macSubtitleOCR
USE_FFMPEG=1 swift build --configuration release -Xswiftc -DFFMPEG

Running Tests

The testing process compares OCR output against known correct results. Tests aim for at least 95% accuracy, as there are slight differences in Vision results between machines.

swift test

Accuracy

In general, Vision produces a highly accurate output for almost all subtitles. If you find an edge case with degraded performance, open an issue so it can be investigated.

In tests comparing Vision's output with Tesseract, Vision consistently gave better results, particularly with tricky cases like properly recognizing I.

While some tools, like SubtitleEdit, may use binary image compare for marginally better accuracy, Vision offers more flexibility with built-in language support.

Contribution and TODO

For information on how to contribute to the project, please refer to CONTRIBUTING.md.

If you're interested in working on specific features or improvements, check out issues tagged as enhancements.

References

• Presentation Graphic Stream (PGS) Files
• DVD Subtitle Stream (VobSub) Files
• Matroska Technical Specifications