feat: Manual Focus point

[CmdrDats]

Plugin(s)

• Barcode Scanning
• Face Detection
• Face Mesh Detection
• Selfie Segmentation
• Subject Segmentation
• Translation

Current problem

When we're using the barcode scanner, our camera view needs to be in the top 30% of the screen, leaving the rest of the screen for other UI elements. The focal point of the camera remains in the center of the screen under the UI which makes focussing barcodes on smaller products difficult.

Preferred solution

Add a setFocusPoint method that accepts normalized coordinates (0.0-1.0) and maps them to native camera focus points on both Android and iOS.

Alternative options

If we could control the underlying camera viewport position and size, that might also work but seems a lot more fiddly.

Additional context

Cursor provides this as a possible implementation plan - I am happy to implement, test and PR this after feedback and if approved.

API Design

TypeScript Interface:

interface BarcodeScannerPlugin {
  /**
   * Set the camera's focus point using normalized coordinates.
   * @param options { x: number, y: number } - Screen coordinates (0.0-1.0)
   */
  setFocusPoint(options: SetFocusPointOptions): Promise<void>;
}

interface SetFocusPointOptions {
  /** X coordinate (0.0 = left, 1.0 = right) */
  x: number;
  /** Y coordinate (0.0 = top, 1.0 = bottom) */  
  y: number;
}

Implementation Plan

1. Android (CameraX)

File: BarcodeScanner.java

• Add method to convert normalized coordinates to MeteringPoint
• Use existing getCameraControl().startFocusAndMetering() API
• Implementation:

public void setFocusPoint(float x, float y) {
    if (camera == null || previewView == null) return;
    
    MeteringPointFactory factory = previewView.getMeteringPointFactory();
    MeteringPoint point = factory.createPoint(x * previewView.getWidth(), 
                                            y * previewView.getHeight());
    FocusMeteringAction action = new FocusMeteringAction.Builder(point).build();
    camera.getCameraControl().startFocusAndMetering(action);
}

File: BarcodeScannerPlugin.java

• Add @PluginMethod for setFocusPoint
• Parse coordinates and validate range [0.0-1.0]

2. iOS (AVFoundation)

File: BarcodeScanner.swift

• Add method using existing getCaptureDevice() pattern
• Convert normalized coordinates to camera coordinate system
• Implementation:

@objc public func setFocusPoint(_ options: SetFocusPointOptions) throws {
    guard let device = cameraView?.getCaptureDevice(),
          let previewLayer = cameraView?.getPreviewLayer() else { return }
    
    guard device.isFocusPointOfInterestSupported else {
        throw RuntimeError("Focus point not supported")
    }
    
    let point = CGPoint(x: options.getX(), y: options.getY())
    let devicePoint = previewLayer.captureDevicePointConverted(fromLayerPoint: point)
    
    try device.lockForConfiguration()
    device.focusPointOfInterest = devicePoint
    device.focusMode = .autoFocus
    device.unlockForConfiguration()
}

3. TypeScript Updates

• Add interfaces to definitions.ts
• Export method in index.ts
• Add comprehensive JSDoc documentation

4. Coordinate System

• Input: Normalized coordinates (0.0-1.0)
• Mapping:
  • Android: Multiply by preview dimensions for MeteringPoint
  • iOS: Use captureDevicePointConverted(fromLayerPoint:) for proper transformation
• Benefits: Resolution-independent, orientation-agnostic, cross-platform consistent

Before submitting

• I have read and followed the feature request guidelines.
• I have attached links to possibly related issues and discussions.

[robingenz]

Feel free to create a PR. I will then take a look at it. However, I cannot promise that we will merge it.

[CmdrDats]

cool - happy to do that - wanted to sense check that I'm not being silly and missing something.

[CmdrDats]

I have tested this on iOS (on a physical device) and on an android simulator (will get access to an android device for testing on friday)

Patch I used to test on the capacitor-mlkit-demo:

diff --git a/src/app/modules/barcode-scanning/barcode-scanning-modal.component.ts b/src/app/modules/barcode-scanning/barcode-scanning-modal.component.ts
index ab6b18a..3a9fbec 100644
--- a/src/app/modules/barcode-scanning/barcode-scanning-modal.component.ts
+++ b/src/app/modules/barcode-scanning/barcode-scanning-modal.component.ts
@@ -32,7 +32,7 @@ import { InputCustomEvent } from '@ionic/angular';
       </ion-toolbar>
     </ion-header>
 
-    <ion-content>
+    <ion-content (click)="onTapToFocus($event)">
       @if (isWeb) {
         <video #video autoplay class="video"></video>
       }
@@ -52,6 +52,13 @@ import { InputCustomEvent } from '@ionic/angular';
           </ion-fab-button>
         </ion-fab>
       }
+      @if (focusIndicator) {
+        <div 
+          class="focus-indicator"
+          [style.left.px]="focusIndicator.x"
+          [style.top.px]="focusIndicator.y"
+        ></div>
+      }
     </ion-content>
   `,
   styles: [
@@ -88,6 +95,33 @@ import { InputCustomEvent } from '@ionic/angular';
         transform: translateX(-50%);
         width: 50%;
       }
+
+      .focus-indicator {
+        position: absolute;
+        width: 50px;
+        height: 50px;
+        margin-left: -25px;
+        margin-top: -25px;
+        border: 2px solid #fff;
+        border-radius: 50%;
+        pointer-events: none;
+        animation: focusIndicatorAnimation 0.6s ease-out;
+      }
+
+      @keyframes focusIndicatorAnimation {
+        0% {
+          transform: scale(1.5);
+          opacity: 0.8;
+        }
+        50% {
+          transform: scale(1.2);
+          opacity: 1;
+        }
+        100% {
+          transform: scale(1);
+          opacity: 0;
+        }
+      }
     `,
   ],
 })
@@ -107,6 +141,7 @@ export class BarcodeScanningModalComponent implements AfterViewInit, OnDestroy {
   public isWeb = Capacitor.getPlatform() === 'web';
   public minZoomRatio: number | undefined;
   public maxZoomRatio: number | undefined;
+  public focusIndicator: { x: number; y: number } | null = null;
 
   constructor(
     private readonly dialogService: DialogService,
@@ -146,6 +181,49 @@ export class BarcodeScanningModalComponent implements AfterViewInit, OnDestroy {
     await BarcodeScanner.toggleTorch();
   }
 
+  public async onTapToFocus(event: MouseEvent): Promise<void> {
+    // Skip if this is a web platform as focus is handled differently
+    if (this.isWeb) {
+      return;
+    }
+
+    // Get the tap coordinates relative to the screen
+    const rect = (event.currentTarget as HTMLElement).getBoundingClientRect();
+    const x = event.clientX - rect.left;
+    const y = event.clientY - rect.top;
+
+    // Convert to normalized coordinates (0.0 - 1.0)
+    const normalizedX = x / rect.width;
+    const normalizedY = y / rect.height;
+
+    // Clamp to valid range
+    const clampedX = Math.max(0, Math.min(1, normalizedX));
+    const clampedY = Math.max(0, Math.min(1, normalizedY));
+
+    try {
+      // Call the focus point API
+      await BarcodeScanner.setFocusPoint({
+        x: clampedX,
+        y: clampedY,
+      });
+
+      // Show focus indicator
+      this.showFocusIndicator(x, y);
+    } catch (error) {
+      console.error(error);
+      console.log('Focus not supported or failed:', error);
+    }
+  }
+
+  private showFocusIndicator(x: number, y: number): void {
+    this.focusIndicator = { x, y };
+    
+    // Hide the indicator after animation completes
+    setTimeout(() => {
+      this.focusIndicator = null;
+    }, 600);
+  }
+
   private async startScan(): Promise<void> {
     // Hide everything behind the modal (see `src/theme/variables.scss`)
     document.querySelector('body')?.classList.add('barcode-scanning-active');

[CmdrDats]

Confirmed working properly on an Android device (Xiaomi Redmi A2)