macOS - Native Apple Silicon version without Rosetta?

[obitwrite]

Dear developers / @kaixxx – thank you for all the work on noScribe, it is so important and valuable to have a trustworthy local transcription app!

As macOS user, I'm wondering if a completely native Apple Silicon / aarch64-version is possible. I read somewhere in the issues that this is because of ffmpeg not providing an aarch64-version?

Although it isn't in the near future, but I read that Apple is going to phase out Rosetta support: https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment/

Looking forward to your ideas!

[gernophil]

Yes, you're right. Right now we still use the intel version of ffmpeg even in the arm64 version. There are some options like switching to pyav, which bundles its own ffmpeg, but we haven't looked deeper into this. This probably won't happen for version 0.7, but I think as soon as apple will drop Rosetta2 (which is a ghost spooking around since some time already) many arm64 ffmpeg versions will pop out. Thanksgiving the reminder anyway. We'll take up the thinking on this again :).

[obitwrite]

Thanks for your reply @gernophil ! I'm not a developer, but came across this page: https://ffmpeg.martin-riedl.de/. I don't know the developer, but perhaps this can help?

[gernophil]

Thanks @obitwrite, we are aware of Martin Riedel and osxexperts.net. We just haven't found the time to evaluate, if those binaries and their licenses suit our purpose :).

[obitwrite]

I see! Thanks again for your replies, and looking forward to future versions of NoScribe! 😄

[mutlusun]

There is already a PR moving noscribe to use pyav as ffmpeg backend (#236 ). Thus, this issue will be fixed by it but won't be merged before the new release.

[obitwrite]

Thanks for the update @mutlusun – that's good to hear!

[gernophil]

@mutlusun, do you know, if pyav also provides ffmpeg as isolated binary somewhere in it's package? Then we could maybe just replace the current one with this.

[mutlusun]

@mutlusun, do you know, if pyav also provides ffmpeg as isolated binary somewhere in it's package? Then we could maybe just replace the current one with this.

Not as far as I know. I think this is also not necessary. The PR moves the whole logic to pyav and separates it from the rest of the code.

[obitwrite]

Just to check @mutlusun – did I understand it correctly, that the intention is to implement pyav in version 0.70? And that the implementation of pyav will make NoScribe completely native for Apple Silicon (so no Rosetta needed)?

[gernophil]

I just learned that pyannote4 switched from torchaudio to torchcodec and torchcodec is "Relying on FFmpeg to do the decoding and encoding. TorchCodec uses the version of FFmpeg you already have installed.". Will it still be possible to use pyav or do we need a real ffmpeg for this?

[kaixxx]

I just learned that pyannote4 switched from torchaudio to torchcodec

I've managed to avoid that. I am loading the audio with torchaudio into the memory and send this waveform to pyannote instead of the filename:

noScribe/pyannote_mp_worker.py

Line 81 in e4e54ae

waveform, sample_rate = torchaudio.load(audio_file)

[risa2000]

More out of curiosity, just got to this project, I am confused about the statement:

Yes, you're right. Right now we still use the intel version of ffmpeg even in the arm64 version. There are some options like switching to pyav, which bundles its own ffmpeg, but we haven't looked deeper into this. This probably won't happen for version 0.7, but I think as soon as apple will drop Rosetta2 (which is a ghost spooking around since some time already) many arm64 ffmpeg versions will pop out. Thanksgiving the reminder anyway. We'll take up the thinking on this again :).

There is native arm64 version of ffmpeg available on homebrew for quite some time. I guess there are probably some other intricacies why you cannot use it, but from the discussion here it looks like it is because it is not available.

[gernophil]

The Homebrew version is a dynamic build, so we can't simply bundle it with a standalone app. We would need a static build. There are static builds for arm64 available from OSXExperts and Martin Riedel, but so far we didn't find the time to check their licenses, if we are allowed to use them and how trustworthy they are.

[risa2000]

@gernophil
I see what you mean. However, after venturing to the Intel binaries page and reading this:
https://evermeet.cx/ffmpeg/apple-silicon-arm
I guess, your best bet is to get out of this dependency ASAP.
I believe (correct me if I am wrong) that you do not need any video codecs as the only task for ffmpeg in noScribe context should be converting an (arbitrary) input audio stream to the one accepted by whisper, right? (Which, if I read the requirements correctly is basically PCM 16bit).
This could simplify the build substantially and you might be able to build the static build on your own.
Am I right?