android(call): Increment 2a — native LiveKit call audio, background service, speaker/BT routing

Replaces WebView call media on Android with a native LiveKit connection (iOS
parity), so call audio survives backgrounding and the output can be switched
to Speaker/Bluetooth/Earpiece — the #3 (missing speaker control) and #4
(background audio) items.

Gated behind a new, default-off server flag ANDROID_NATIVE_MEDIA (independent
of ANDROID_CALLS, which only governs the native ring). Flag off = today's
WebView media, video included; flag on = native audio (video is Increment 2b,
so enable it for VOICE testing first). No behavior change until flipped.

Native (mobile/plugins/native-call/android):
- build.gradle: add kotlin-android + io.livekit:livekit-android 2.29.2 (+ twilio
  audioswitch, coroutines) + JitPack repo; minSdk 24 (LiveKit floor).
- CallMedia.kt: LiveKit Room wrapper — connect/mic/disconnect + AudioSwitchHandler
  routing (auto + selectDevice) and route-change events.
- CallForegroundService.java: microphone-type foreground service for background
  audio (no-ops without RECORD_AUDIO so it never crashes the call).
- NativeCallPlugin.java: real bodies for reportOutgoingCall/reconnectRoom/setMuted
  /endCall + new setAudioRoute/getAudioRoute; fires callConnected/callError/audioRoute.
- AndroidManifest: FG service + FOREGROUND_SERVICE(_MICROPHONE)/RECORD_AUDIO/
  MODIFY_AUDIO_SETTINGS/BLUETOOTH_CONNECT.
- android-patch.js: bump the generated app minSdk to 24.

Server:
- config.js/routes.js: ANDROID_NATIVE_MEDIA → /api/meetings/config.androidNativeMedia.

Web (home.html):
- Android flips to the native-media path when androidNativeMedia is set; the
  existing outgoing/answer/mic/end code then drives the plugin unchanged.
- Speaker button wired for Android native: cycles speaker/earpiece/bluetooth via
  NC.setAudioRoute and reflects the real output from the audioRoute event.

Video stays no-op on the native path (Increment 2b).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
SravanandClaude Opus 4.8 committed 2026-10-30 17:25:18 +05:30
1 parent 08eff043ec
commit aa4aeeaba1
9 files changed
+444 -24

No files matched your search

@@ -1,32 +1,48 @@
// Android build for the native-call plugin. Mirrors the official @capacitor/* plugin module layout (see
// node_modules/@capacitor/status-bar/android/build.gradle) so it drops into the CI-generated app exactly like
// the plugins that already build. Java (not Kotlin) to match the proven official-plugin structure — no extra
// kotlin-gradle-plugin wiring in the library module, one less thing to fail when authored blind.
// node_modules/@capacitor/status-bar/android/build.gradle) so it drops into the CI-generated app like the
// plugins that already build.
//
// INCREMENT 2 adds native LiveKit call media (background audio + Speaker/Bluetooth routing), which is a Kotlin +
// coroutines SDK — so this module now also applies kotlin-android (the ring/plugin glue stays Java). Capacitor
// pulls the Kotlin stdlib via a BOM but does NOT apply the Kotlin Gradle plugin at the root, so we bring it here
// (the standard community-plugin pattern); the classpath is additive to the generated root's AGP.
ext {
junitVersion = project.hasProperty('junitVersion') ? rootProject.ext.junitVersion : '4.13.2'
androidxCoreVersion = project.hasProperty('androidxCoreVersion') ? rootProject.ext.androidxCoreVersion : '1.12.0'
androidxAppCompatVersion = project.hasProperty('androidxAppCompatVersion') ? rootProject.ext.androidxAppCompatVersion : '1.6.1'
// Same FCM version the @capacitor/push-notifications plugin pulls in, so we don't fork the Firebase version.
firebaseMessagingVersion = project.hasProperty('firebaseMessagingVersion') ? rootProject.ext.firebaseMessagingVersion : '23.3.1'
// Follow the project's Kotlin (Capacitor pins kotlin_version at the root); fall back to a recent stable.
bzcKotlinVersion = rootProject.hasProperty('kotlin_version') ? rootProject.ext.kotlin_version : '1.9.24'
// LiveKit Android SDK — native background call media (Increment 2). Pinned; bump deliberately.
livekitVersion = '2.29.2'
audioswitchVersion = '1.2.0'
coroutinesVersion = '1.8.1'
}
buildscript {
ext.bzc_kotlin = rootProject.hasProperty('kotlin_version') ? rootProject.ext.kotlin_version : '1.9.24'
repositories {
google()
mavenCentral()
}
dependencies {
classpath 'com.android.tools.build:gradle:8.2.1'
// Kotlin Gradle plugin so this module can compile the .kt LiveKit wrapper. Additive to the root's AGP.
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$bzc_kotlin"
}
}
apply plugin: 'com.android.library'
apply plugin: 'kotlin-android'
android {
namespace "com.bizgaze.connect.nativecall"
compileSdk project.hasProperty('compileSdkVersion') ? rootProject.ext.compileSdkVersion : 34
// LiveKit 2.29 compiles against a recent platform; honor the CI root (Capacitor 8 = 35) and floor at 35.
compileSdk project.hasProperty('compileSdkVersion') ? rootProject.ext.compileSdkVersion : 35
defaultConfig {
minSdkVersion project.hasProperty('minSdkVersion') ? rootProject.ext.minSdkVersion : 23
// LiveKit Android's floor. The app min is bumped to 24 in CI (android-patch.js) so the merge holds.
minSdkVersion 24
targetSdkVersion project.hasProperty('targetSdkVersion') ? rootProject.ext.targetSdkVersion : 34
versionCode 1
versionName "1.0"
@@ -44,11 +60,16 @@ android {
sourceCompatibility JavaVersion.VERSION_17
targetCompatibility JavaVersion.VERSION_17
}
kotlinOptions {
jvmTarget = '17'
}
}
repositories {
google()
mavenCentral()
// LiveKit pulls transitive artifacts (its WebRTC fork) that live on JitPack.
maven { url 'https://jitpack.io' }
}
dependencies {
@@ -62,4 +83,11 @@ dependencies {
// RemoteMessage / FirebaseMessaging (transitive from the push plugin, but declared so this module compiles
// against it directly; same version → no resolution conflict).
implementation "com.google.firebase:firebase-messaging:$firebaseMessagingVersion"
// Kotlin + coroutines for the native LiveKit media wrapper (CallMedia.kt).
implementation "org.jetbrains.kotlin:kotlin-stdlib:$bzcKotlinVersion"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-android:$coroutinesVersion"
// Native call media (background audio) + audio-device routing (speaker/bluetooth/earpiece).
implementation "io.livekit:livekit-android:$livekitVersion"
// AudioDevice types used by LiveKit's AudioSwitchHandler; declared so they're on our compile classpath.
implementation "com.twilio:audioswitch:$audioswitchVersion"
}
@@ -7,6 +7,16 @@
<uses-permission android:name="android.permission.VIBRATE" />
<uses-permission android:name="android.permission.WAKE_LOCK" />
<!-- Increment 2 (native LiveKit call media): a foreground service keeps call audio alive in the background;
the microphone type lets the call keep publishing the mic. RECORD_AUDIO/MODIFY_AUDIO_SETTINGS for capture
+ routing (also added app-side by android-patch.js — merge dedups); BLUETOOTH_CONNECT (API 31+) so the
audio-device switch can route to a Bluetooth headset. -->
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<application>
<!-- Our FirebaseMessagingService REPLACES the @capacitor/push-notifications one: it handles call-wake
DATA messages natively (to ring when the app is closed) and forwards everything else to Capacitor's
@@ -32,5 +42,12 @@
android:showWhenLocked="true"
android:turnScreenOn="true"
android:theme="@android:style/Theme.DeviceDefault.NoActionBar" />
<!-- Keeps the call process alive for background audio (Increment 2). The microphone type is required on
Android 14+ for a service that uses the mic. -->
<service
android:name="com.bizgaze.connect.nativecall.CallForegroundService"
android:exported="false"
android:foregroundServiceType="microphone" />
</application>
</manifest>
@@ -0,0 +1,97 @@
package com.bizgaze.connect.nativecall;
import android.Manifest;
import android.app.Notification;
import android.app.NotificationChannel;
import android.app.NotificationManager;
import android.app.PendingIntent;
import android.app.Service;
import android.content.Context;
import android.content.Intent;
import android.content.pm.PackageManager;
import android.content.pm.ServiceInfo;
import android.os.Build;
import android.os.IBinder;
import android.util.Log;
import androidx.core.app.NotificationCompat;
import androidx.core.content.ContextCompat;
// Keeps the call's process alive while the app is backgrounded so native LiveKit audio keeps flowing (Increment 2).
// Android 14+ requires a TYPED foreground service; we use the "microphone" type (a call publishes the mic). That
// type needs RECORD_AUDIO at startForeground time, so start() no-ops without it (the call still works in the
// foreground; the plugin requests the grant so the next call gets background audio). Stopped on call end.
public class CallForegroundService extends Service {
private static final String CHANNEL_ID = "bzc_ongoing_call";
private static final int NOTIF_ID = 2749;
// Start the ongoing-call foreground service — but only if RECORD_AUDIO is granted (the microphone FG type
// requires it; starting without it throws on Android 14+). No-op otherwise so we never crash the call.
static void start(Context ctx) {
Context app = ctx.getApplicationContext();
if (ContextCompat.checkSelfPermission(app, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
Log.i("bzcall", "FG service skipped — RECORD_AUDIO not granted yet (foreground audio only)");
return;
}
try {
ContextCompat.startForegroundService(app, new Intent(app, CallForegroundService.class));
} catch (Exception e) { Log.w("bzcall", "FG service start failed: " + e.getMessage()); }
}
static void stop(Context ctx) {
try { ctx.getApplicationContext().stopService(new Intent(ctx.getApplicationContext(), CallForegroundService.class)); }
catch (Exception ignored) {}
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
ensureChannel();
Notification n = buildNotification();
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
startForeground(NOTIF_ID, n, ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE);
} else {
startForeground(NOTIF_ID, n);
}
Log.i("bzcall", "ongoing-call foreground service started");
} catch (Exception e) {
// Must still call startForeground or the OS kills us with an ANR; fall back to the untyped form.
Log.w("bzcall", "startForeground(microphone) failed: " + e.getMessage());
try { startForeground(NOTIF_ID, n); } catch (Exception ignored) {}
}
return START_NOT_STICKY; // don't resurrect after the call/process ends
}
private void ensureChannel() {
if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) return;
NotificationManager nm = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
if (nm == null || nm.getNotificationChannel(CHANNEL_ID) != null) return;
NotificationChannel ch = new NotificationChannel(CHANNEL_ID, "Ongoing call", NotificationManager.IMPORTANCE_LOW);
ch.setDescription("Shown while a Biz Connect call is in progress");
ch.setShowBadge(false);
ch.setSound(null, null);
nm.createNotificationChannel(ch);
}
private Notification buildNotification() {
NotificationCompat.Builder b = new NotificationCompat.Builder(this, CHANNEL_ID)
.setSmallIcon(android.R.drawable.sym_action_call)
.setContentTitle("Ongoing call")
.setContentText("Tap to return to the call")
.setOngoing(true)
.setCategory(NotificationCompat.CATEGORY_CALL)
.setPriority(NotificationCompat.PRIORITY_LOW);
try {
Intent launch = getPackageManager().getLaunchIntentForPackage(getPackageName());
if (launch != null) {
launch.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_SINGLE_TOP);
int flags = PendingIntent.FLAG_UPDATE_CURRENT;
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) flags |= PendingIntent.FLAG_IMMUTABLE;
b.setContentIntent(PendingIntent.getActivity(this, 0, launch, flags));
}
} catch (Exception ignored) {}
return b.build();
}
@Override public IBinder onBind(Intent intent) { return null; }
}
@@ -0,0 +1,117 @@
package com.bizgaze.connect.nativecall
import android.content.Context
import android.util.Log
import com.twilio.audioswitch.AudioDevice
import io.livekit.android.LiveKit
import io.livekit.android.LiveKitOverrides
import io.livekit.android.audio.AudioOptions
import io.livekit.android.audio.AudioSwitchHandler
import io.livekit.android.room.Room
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.launch
// Native LiveKit call media for Android (Increment 2) — mirrors the iOS plugin's single-Room handling. A
// process-level singleton so the connection survives a WebView reload (e.g. answering from a killed app); the
// foreground service (CallForegroundService) keeps the process alive so audio continues when backgrounded.
//
// Audio routing (speaker / bluetooth / earpiece / wired) is handled by LiveKit's AudioSwitchHandler: it
// auto-selects by priority (bluetooth > wired > speaker > earpiece) and we expose selectDevice() for the manual
// Speaker/Bluetooth control. Playback (hearing the other side) needs no permission; publishing the mic needs
// RECORD_AUDIO — but we answer MUTED (the web re-applies the real mic state on callConnected), so a user with no
// mic grant still hears audio and only needs the grant to unmute.
object CallMedia {
private const val TAG = "bzcall"
private val scope = CoroutineScope(Dispatchers.Main + SupervisorJob())
private var room: Room? = null
private var audio: AudioSwitchHandler? = null
// Java-friendly callback the plugin implements to forward to notifyListeners(...). All invoked on main.
interface Events {
fun onConnected()
fun onError(message: String)
fun onRoute(selected: String, available: List<String>)
}
@JvmStatic var events: Events? = null
@JvmStatic val isConnected: Boolean get() = room != null
// Connect (or re-home) the call's LiveKit room. startMuted mirrors the iOS "answer muted" house rule.
@JvmStatic
fun connect(context: Context, url: String, token: String, startMuted: Boolean) {
val app = context.applicationContext
scope.launch {
try {
room?.let { try { it.disconnect() } catch (_: Throwable) {} }
val handler = AudioSwitchHandler(app)
audio = handler
val r = LiveKit.create(
app,
overrides = LiveKitOverrides(audioOptions = AudioOptions(audioHandler = handler))
)
room = r
handler.registerAudioDeviceChangeListener { _, _ -> emitRoute() }
r.connect(url, token)
r.localParticipant.setMicrophoneEnabled(!startMuted)
Log.i(TAG, "native media connected (startMuted=$startMuted)")
events?.onConnected()
emitRoute()
} catch (e: Throwable) {
Log.w(TAG, "native media connect failed: ${e.message}")
events?.onError(e.message ?: "connect failed")
}
}
}
@JvmStatic
fun setMicEnabled(enabled: Boolean) {
val r = room ?: return
scope.launch {
try { r.localParticipant.setMicrophoneEnabled(enabled) }
catch (e: Throwable) { Log.w(TAG, "setMic failed: ${e.message}") }
}
}
// Manual output selection for the Speaker/Bluetooth/Earpiece control. If the requested device isn't present,
// the current (auto) selection is left alone.
@JvmStatic
fun setRoute(route: String) {
val h = audio ?: return
val target: AudioDevice? = when (route) {
"speaker" -> h.availableAudioDevices.firstOrNull { it is AudioDevice.Speakerphone }
"earpiece" -> h.availableAudioDevices.firstOrNull { it is AudioDevice.Earpiece }
"bluetooth" -> h.availableAudioDevices.firstOrNull { it is AudioDevice.BluetoothHeadset }
"wired" -> h.availableAudioDevices.firstOrNull { it is AudioDevice.WiredHeadset }
else -> null
}
if (target != null) h.selectDevice(target)
emitRoute()
}
@JvmStatic fun currentRoute(): String = routeName(audio?.selectedAudioDevice)
@JvmStatic fun availableRoutes(): List<String> = audio?.availableAudioDevices?.map { routeName(it) } ?: emptyList()
private fun emitRoute() {
val h = audio ?: return
events?.onRoute(routeName(h.selectedAudioDevice), h.availableAudioDevices.map { routeName(it) })
}
private fun routeName(d: AudioDevice?): String = when (d) {
is AudioDevice.Speakerphone -> "speaker"
is AudioDevice.Earpiece -> "earpiece"
is AudioDevice.BluetoothHeadset -> "bluetooth"
is AudioDevice.WiredHeadset -> "wired"
else -> "unknown"
}
@JvmStatic
fun disconnect() {
val r = room
room = null
audio = null
if (r != null) scope.launch { try { r.disconnect() } catch (_: Throwable) {} }
}
}
@@ -1,5 +1,12 @@
package com.bizgaze.connect.nativecall;
import android.Manifest;
import android.content.pm.PackageManager;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;
import com.getcapacitor.JSArray;
import com.getcapacitor.JSObject;
import com.getcapacitor.Plugin;
import com.getcapacitor.PluginCall;
@@ -32,6 +39,7 @@ public class NativeCallPlugin extends Plugin {
@Override
public void load() {
instance = this;
wireMedia();
if (pendingAnswer != null) {
JSObject a = pendingAnswer;
pendingAnswer = null;
@@ -39,6 +47,45 @@ public class NativeCallPlugin extends Plugin {
}
}
// Bridge CallMedia's (Kotlin, LiveKit) events to the shared web listeners. Routed through the live plugin
// instance so a WebView reload doesn't fire into a dead bridge. The event names/payloads mirror the iOS plugin
// (callConnected / callError), plus audioRoute for the Android Speaker/Bluetooth control.
private void wireMedia() {
CallMedia.setEvents(new CallMedia.Events() {
@Override public void onConnected() {
NativeCallPlugin p = instance;
if (p != null) p.notifyListeners("callConnected", new JSObject().put("ok", true));
}
@Override public void onError(String message) {
NativeCallPlugin p = instance;
if (p != null) p.notifyListeners("callError", new JSObject().put("error", message == null ? "" : message));
}
@Override public void onRoute(String selected, java.util.List<String> available) {
NativeCallPlugin p = instance;
if (p == null) return;
JSObject o = new JSObject();
o.put("output", selected == null ? "unknown" : selected);
JSArray arr = new JSArray();
if (available != null) for (String s : available) arr.put(s);
o.put("available", arr);
p.notifyListeners("audioRoute", o);
}
});
}
// LiveKit needs RECORD_AUDIO to publish the mic, and the microphone foreground service needs it to start.
// We answer MUTED (playback needs no permission), so this is best-effort: it prompts without blocking, and the
// grant persists so unmute + background audio work from this call on (or the next).
private void ensureMicPermission() {
try {
if (getContext() != null
&& ContextCompat.checkSelfPermission(getContext(), Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED
&& getActivity() != null) {
ActivityCompat.requestPermissions(getActivity(), new String[]{ Manifest.permission.RECORD_AUDIO }, 7731);
}
} catch (Exception ignored) {}
}
@Override
protected void handleOnDestroy() {
super.handleOnDestroy();
@@ -90,32 +137,79 @@ public class NativeCallPlugin extends Plugin {
call.resolve();
}
// Outgoing call from the web app. Increment 1: media stays in the WebView, so there is nothing to ring or
// connect natively here — the callee is rung by the server's FCM call-wake. Just record it so a later
// endCall/cancel can match. (Increment 2 will start a foreground service + native LiveKit here.)
// Outgoing call from the web app. Increment 2 (native media): start the ongoing-call foreground service and
// connect the LiveKit room with the url/token the web minted (/api/meetings/token). Connect MUTED — the web
// re-applies the real mic state on the callConnected event. The callee is rung by the server's FCM call-wake.
@PluginMethod
public void reportOutgoingCall(PluginCall call) {
String uuid = call.getString("callUUID", "");
if (!uuid.isEmpty()) CallStore.markActive(uuid);
startNativeMedia(call.getString("url", ""), call.getString("token", ""));
call.resolve();
}
// Stop ringing / clear the incoming UI for a call (remote hung up, we ended, or another device answered).
// Stop ringing / clear the incoming UI AND tear down native media (remote hung up, we ended, or another device
// answered). Idempotent: disconnect()/stop() are safe no-ops when nothing is connected/running.
@PluginMethod
public void endCall(PluginCall call) {
String uuid = call.getString("callUUID", "");
if (!uuid.isEmpty()) CallStore.markEnded(uuid);
CallStore.cancel(getContext());
IncomingCallActivity.dismissIfShowing();
CallMedia.disconnect();
CallForegroundService.stop(getContext());
call.resolve();
}
// Increment 1: mute is handled by the WebView (it owns the media). No-op so the shared bridge doesn't reject.
@PluginMethod public void setMuted(PluginCall call) { call.resolve(); }
// Mute/unmute the native mic (Increment 2 — the plugin owns the media now).
@PluginMethod
public void setMuted(PluginCall call) {
boolean muted = Boolean.TRUE.equals(call.getBoolean("muted", false));
CallMedia.setMicEnabled(!muted);
call.resolve();
}
// --- Increment 2+ methods (video / screen-share / hole-punch / #12 reconnect / #5 transcript). Safe no-ops
// for now so the shared web bridge never rejects on Android; real bodies land with the native-media build. ---
@PluginMethod public void reconnectRoom(PluginCall call) { call.resolve(); }
// Connect (answer path) or re-home (#12 multi-device) the native LiveKit room with web-minted creds. On
// Android this is the primary connect on answer: the FCM call-wake carries no livekit creds, so the web calls
// this (bzNativeReconnectMedia) with a peerId token once the meeting WebSocket has joined.
@PluginMethod
public void reconnectRoom(PluginCall call) {
startNativeMedia(call.getString("url", ""), call.getString("token", ""));
call.resolve();
}
// Select the call audio output (Speaker / Bluetooth / Earpiece / Wired) for the in-call control. Resolves the
// route actually in effect so the web button can reflect it.
@PluginMethod
public void setAudioRoute(PluginCall call) {
String route = call.getString("route", "");
if (!route.isEmpty()) CallMedia.setRoute(route);
JSObject ret = new JSObject();
ret.put("output", CallMedia.currentRoute());
call.resolve(ret);
}
// Report the current audio output + the devices available to switch to (used to seed/refresh the control).
@PluginMethod
public void getAudioRoute(PluginCall call) {
JSObject ret = new JSObject();
ret.put("output", CallMedia.currentRoute());
JSArray arr = new JSArray();
for (String s : CallMedia.availableRoutes()) arr.put(s);
ret.put("available", arr);
call.resolve(ret);
}
// Shared connect helper for the outgoing + answer paths.
private void startNativeMedia(String url, String token) {
if (url == null || url.isEmpty() || token == null || token.isEmpty()) return;
ensureMicPermission();
CallForegroundService.start(getContext());
CallMedia.connect(getContext(), url, token, true); // start muted; web re-applies on callConnected
}
// --- Increment 2b methods (video tiles / screen-share / hole-punch / #5 transcript). Safe no-ops for now so
// the shared web bridge never rejects on Android; real bodies land with the native-video build. ---
@PluginMethod public void setCamera(PluginCall call) { call.resolve(); }
@PluginMethod public void switchCamera(PluginCall call) { call.resolve(); }
@PluginMethod public void startScreenShare(PluginCall call) { call.resolve(); }