audio stream

[chocolateonly]

1. There may be noise when entering the conversation. Is there a solution;
2. Turn on the camera and I feel like it's not shooting as clearly as a native camera;
3. Can the audio stream be exposed for translation and other processing purposes;
   We look forward to your reply.

[lambiengcode]

Hi @chocolateonly , thank you for your feedback and questions!

1. There may be noise when entering the conversation. Is there a solution?

Could you please let us know what platform or browser you're using (e.g., Chrome on Windows, Safari on macOS, etc.) and how to reproduce this issue? This will help us investigate further. Also, is the noise from the microphone (audio input) or something else?

2. Turn on the camera and I feel like it's not shooting as clearly as a native camera

The video you see in a WebRTC-based app (local track) is not the raw image directly from the camera. Instead, it's a video track that has been created from the camera source with a new configuration — typically using a resolution like 1280x720. This resolution is lower than what many modern mobile devices support natively, but it's a good balance for video calls, helping reduce bandwidth usage and latency. Also, the local video you see has a small delay due to WebRTC processing. This delay can slightly increase if you apply virtual backgrounds or beauty filters.

3. Can the audio stream be exposed for translation and other processing purposes

Yes! For this, we recommend looking into waterbus-rs, our backend implementation in Rust. It allows access to RTP audio packets, which you can use for speech-to-text (STT), translation, or other audio processing. After processing, you can emit the translated or transcribed results to users via a signaling channel. Contributions on this are very welcome!

Let us know if you need help getting started with waterbus-rs or implementing audio processing. Thanks again for your interest!

[chocolateonly]

Thank you for your reply!

1. I used two apps running on my phone, HUWEIP40 PRO Harmony OS 4.2.0 and vivo X100 Pro Android. 11. When entering the meeting, I did not turn on the microphone, but there was noise on both phones. It seems that although the microphone is not turned on when entering the conference, it is in the on state and can hear the other person's voice. I tried to switch the microphone button and found that it could be turned off and on normally. Did the microphone not turn off correctly during initialization.

Future<void> _displayDialogJoinMeeting(Meeting meeting) async {
await _waterbusSdk.prepareMedia();
bool isDismissWithoutJoin = true;
showDialogWaterbus(
alignment: Alignment.bottomCenter,
paddingBottom: 56.sp,
onlyShowAsDialog: true,
child: DialogPrepareMeeting(
meeting: meeting,
handleJoinMeeting: () {
isDismissWithoutJoin = false;
AppNavigator.popUntil(Routes.homeRoute);
add(MeetingJoinedEvent(meeting: meeting));
},
),
).then((value) {
if (isDismissWithoutJoin) {
add(MeetingDisposed());
}
});
}

2. On HUWEIP40 PRO, the graining is quite obvious, and before entering the conference, the pixels shown in the picture are quite grainy. It will be better after entering the meeting. Other phones are functioning normally. The code is in the preview_camera_card file:

participant.cameraSource! .mediaView(
objectFit:
RTCVideoViewObjectFit.RTCVideoViewObjectFitCover,
mirror: true,
)

3. Plan to implement it using node instead of rust.
4. The operation of modifying and saving settings did not take effect, and the console did not report any errors. For example, should the microphone be turned off during silent startup? There is currently no response triggered:

Future<void> applyCallSettings(CallSetting setting) async {
if (_callSetting.videoQuality == setting.videoQuality) {
if (_callSetting.e2eeEnabled != setting.e2eeEnabled) {
await _enableEncryption(setting.e2eeEnabled);
}
_callSetting = setting;
return;
}
_callSetting = setting;
if (_localStream == null || _mParticipant == null) return;
final MediaStream?  newStream = await _getUserMedia(onlyStream: true);
await _replaceMediaStream(newStream);
if (!(_mParticipant?.isVideoEnabled ?? true)) {
await toggleVideo(forceValue: _mParticipant?.isVideoEnabled);
}
if (!(_mParticipant?.isAudioEnabled ?? true)) {
await toggleAudio(forceValue: _mParticipant?.isAudioEnabled);
}
}

D/skia    ( 7458): SkJpegCodec::onGetPixels -
I/org.webrtc.Logging( 7458): CameraStatistics: Camera fps: 7.
I/GED     ( 7458): ged_boost_gpu_freq, level 100, eOrigin 2, final_idx 24, oppidx_max 24, oppidx_min 0
D/skia    ( 7458): onFlyCompress
D/skia    ( 7458): SkJpegCodec::onGetPixels +
D/skia    ( 7458): SkJpegCodec::onGetPixels -
I/com.bc.message( 7458): Background concurrent copying GC freed 3200(290KB) AllocSpace objects, 34(21MB) LOS objects, 49% free, 13MB/27MB, paused 93us total 142.169ms
I/GED     ( 7458): ged_boost_gpu_freq, level 100, eOrigin 2, final_idx 24, oppidx_max 24, oppidx_min 0
D/skia    ( 7458): onFlyCompress
D/skia    ( 7458): SkJpegCodec::onGetPixels +

5. I tried to modify the properties of the virtual backgrounds or beauty filters you mentioned, but it did not take effect. The error is as follows.

Build fingerprint: 'wowi/full_l39_n_cm_l669_bcall/i14Pro:11/RP1A.200720.011/mp1V83031:user/test-keys'
Revision: '0'
ABI: 'arm64'
Timestamp: 2025-07-18 15:43:17+0800
pid: 7458, tid: 17044, name: Thread-152  >>> com.bc.messager <<<
uid: 10227
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
Abort message: 'F0000 00:00:1752824596.366941   17044 image_frame.cc:291] Format UNKNOWN = 0;
 is not valid in this situation.
'
    x0  0000000000000000  x1  0000000000004294  x2  0000000000000006  x3  00000079c9f418e0
    x4  0000007f99f9d000  x5  0000007f99f9d000  x6  0000007f99f9d000  x7  000000000fa8577a
    x8  00000000000000f0  x9  0000007f938f3800  x10 ffffff80fffffbdf  x11 0000000000000001
    x12 0000000000000000  x13 000000000000000e  x14 00000079c9f40750  x15 000057e1e589ef1d
    x16 0000007f9398cc80  x17 0000007f9396e3a0  x18 0000007997982000  x19 00000000000000ac
    x20 0000000000001d22  x21 00000000000000b2  x22 0000000000004294  x23 00000000ffffffff
    x24 0000007cf45c90a0  x25 00000079c9f43fc0  x26 000000000000000b  x27 0000000000000002
    x28 00000079c9f41ad0  x29 00000079c9f41960
    lr  0000007f93921544  sp  00000079c9f418c0  pc  0000007f93921574  pst 0000000000001000
backtrace:
      #00 pc 000000000004e574  /apex/com.android.runtime/lib64/bionic/libc.so (abort+180) (BuildId: 3dd99fe7a181e7428f58a10f32f9fd0e)
      #01 pc 0000000000612f20  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #02 pc 00000000006132a0  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #03 pc 0000000000613040  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #04 pc 0000000000612ed8  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #05 pc 00000000006132fc  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #06 pc 0000000000525900  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #07 pc 00000000004d801c  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000) (Java_com_google_mediapipe_framework_PacketGetter_nativeGetImageWidthFromImageList+64)
      #08 pc 000000000013ced4  /apex/com.android.art/lib64/libart.so (art_quick_generic_jni_trampoline+148) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #09 pc 00000000001337e8  /apex/com.android.art/lib64/libart.so (art_quick_invoke_static_stub+568) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #10 pc 00000000001a9804  /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+228) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #11 pc 000000000031c050  /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*)+376) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #12 pc 0000000000312238  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+912) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #13 pc 0000000000689a6c  /apex/com.android.art/lib64/libart.so (MterpInvokeStatic+548) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #14 pc 000000000012d994  /apex/com.android.art/lib64/libart.so (mterp_op_invoke_static+20) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #15 pc 0000000000341b64  [anon:dalvik-classes27.dex extracted in memory from /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk!classes27.dex] (com.google.mediapipe.framework.PacketGetter.getImageWidthFromImageList+8)
      #16 pc 00000000003094e0  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.llvm.8567976207078191854)+264) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #17 pc 0000000000311850  /apex/com.android.art/lib64/libart.so (art::interpreter::ArtInterpreterToInterpreterBridge(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*, art::JValue*)+200) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #18 pc 000000000031221c  /apex/com.android.art/lib64/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+884) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #19 pc 0000000000689a6c  /apex/com.android.art/lib64/libart.so (MterpInvokeStatic+548) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #20 pc 000000000012d994  /apex/com.android.art/lib64/libart.so (mterp_op_invoke_static+20) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #21 pc 00000000003a8086  [anon:dalvik-classes27.dex extracted in memory from /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk!classes27.dex] (com.google.mediapipe.tasks.vision.imagesegmenter.ImageSegmenter$1.convertToTaskResult+154)
      #22 pc 0000000000686db0  /apex/com.android.art/lib64/libart.so (MterpInvokeVirtual+1520) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #23 pc 000000000012d814  /apex/com.android.art/lib64/libart.so (mterp_op_invoke_virtual+20) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #24 pc 00000000003a7fd0  [anon:dalvik-classes27.dex extracted in memory from /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk!classes27.dex] (com.google.mediapipe.tasks.vision.imagesegmenter.ImageSegmenter$1.convertToTaskResult)
      #25 pc 00000000006887f4  /apex/com.android.art/lib64/libart.so (MterpInvokeInterface+1812) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #26 pc 000000000012da14  /apex/com.android.art/lib64/libart.so (mterp_op_invoke_interface+20) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #27 pc 000000000036f4ba  [anon:dalvik-classes27.dex extracted in memory from /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk!classes27.dex] (com.google.mediapipe.tasks.core.OutputHandler.run+6)
      #28 pc 0000000000686db0  /apex/com.android.art/lib64/libart.so (MterpInvokeVirtual+1520) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #29 pc 000000000012d814  /apex/com.android.art/lib64/libart.so (mterp_op_invoke_virtual+20) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #30 pc 0000000000370494  [anon:dalvik-classes27.dex extracted in memory from /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk!classes27.dex] (com.google.mediapipe.tasks.core.TaskRunner.lambda$create$0)
      #31 pc 0000000000689d10  /apex/com.android.art/lib64/libart.so (MterpInvokeStatic+1224) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #32 pc 000000000012d994  /apex/com.android.art/lib64/libart.so (mterp_op_invoke_static+20) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #33 pc 000000000036ffe8  [anon:dalvik-classes27.dex extracted in memory from /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk!classes27.dex] (com.google.mediapipe.tasks.core.TaskRunner$$ExternalSyntheticLambda0.process+8)
      #34 pc 00000000003094e0  /apex/com.android.art/lib64/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.llvm.8567976207078191854)+264) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #35 pc 0000000000675510  /apex/com.android.art/lib64/libart.so (artQuickToInterpreterBridge+776) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #36 pc 000000000013cff8  /apex/com.android.art/lib64/libart.so (art_quick_to_interpreter_bridge+88) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #37 pc 0000000000133564  /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+548) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #38 pc 00000000001a97e8  /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+200) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #39 pc 000000000055db44  /apex/com.android.art/lib64/libart.so (art::JValue art::InvokeVirtualOrInterfaceWithVarArgs<art::ArtMethod*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, art::ArtMethod*, std::__va_list)+468) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #40 pc 000000000055dce4  /apex/com.android.art/lib64/libart.so (art::JValue art::InvokeVirtualOrInterfaceWithVarArgs<_jmethodID*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, std::__va_list)+92) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #41 pc 000000000041a1f8  /apex/com.android.art/lib64/libart.so (art::JNI<true>::CallVoidMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list)+656) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #42 pc 000000000037debc  /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CallMethodV(char const*, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, std::__va_list, art::Primitive::Type, art::InvokeType)+2532) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #43 pc 000000000036bc60  /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CallVoidMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list)+72) (BuildId: 1ad871ae595961595bfa64be35e125bc)
      #44 pc 00000000004cff00  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #45 pc 00000000004d010c  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #46 pc 0000000000504474  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #47 pc 0000000000530f54  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #48 pc 000000000051b25c  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #49 pc 000000000051af28  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #50 pc 000000000052d2c8  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #51 pc 000000000052cee0  /data/app/~~C32BNdHM082fSjf6kYUVXA==/com.bc.messager-vnWTzdC1D3qtfHiNssInXA==/base.apk (offset 0xb0e4000)
      #52 pc 00000000000b0bd8  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+64) (BuildId: 3dd99fe7a181e7428f58a10f32f9fd0e)
      #53 pc 00000000000505d0  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: 3dd99fe7a181e7428f58a10f32f9fd0e)

6.Is there a way to close the logs? Unable to print logs, keep outputting.
7.Also, the official demo is not running locally, is the service no longer running?

static const String baseUrl = ' https://service-v2.waterbus.tech/busapi/v1/ ';
static const String wsUrl = ' https://sfu-v2.waterbus.tech ';

Looking forward to your reply, thank you！

[lambiengcode]

@chocolateonly Sorry for the late response. You can handle the audio track and process the translation in track.ts. btw, service-v2 is deprecated and I'm currently migrating from NestJS to Rust.