GeoCtrl (2026)

Abstract

We propose a geometry-based virtual object rotation technique that leverages triangular configuration of fingertips and a speed-responsive Control-Display (CD) gain. Rather than replicating real-world physics which often lead to unnatural mid-air behavior in the absence of physical feedback, we leverage finger-level geometry to enable low-effort object rotation. We map the orientation and morphology of a triangle formed around fingertips directly to the object’s rotation, assuming that users tend to trace the object’s intended motion when physical constraints are missing. To enhance the proposed interaction and accommodate natural motor limits, we implement an adaptive CD gain that scales geometric input according to the fingertip movement speed. In a user evaluation, our method yielded shorter task completion time than physics-based rotation, though longer than a commodity hand-interaction technique, while significantly reducing physical demand relative to both reference conditions without increasing overall workload. We demonstrate that geometry-driven mapping and dynamic gain adaptation enable low-effort, finger-level rotational control of virtual objects in immersive environments.

System Overview

GeoCtrl is a bare-hand in-hand rotation technique built on Unity and Meta Quest 3. Instead of simulating physics, it reads the geometry of three fingertips and maps it to the object, so users can rotate virtual objects with coarse gestural approximations rather than the exact motor precision a physics engine demands.

1. Geometry-based Rotation

The system forms an imaginary reference triangle from the thumb, index, and middle fingertips, the minimal configuration affording stable grasp control. The triangle’s orientation Q_orientation is built from a forward vector d_index and the normal d_index × d_middle, anchored at the thumb. Because this alone misses in-plane motion of the middle finger, an offset term Q_offset is derived from the angle θ between d_index and d_middle, capturing morphological change of the triangle.

The frame-to-frame change of the combined quaternion Q_curr = Q_orientation ∗ Q_offset is applied directly to the object. All computation runs in the local wrist coordinate frame, decoupling finger manipulation from wrist rotation, and a 1-Euro filter (f_c = 3.0, β = 0.66) on each fingertip suppresses tracking jitter.

2. Speed-Adaptive Control-Display Gain

The geometric mapping does not require a 1:1 ratio, so a speed-responsive CD gain is layered on top to improve efficiency and controllability. Following the two-phase model of motor control, fast fingertip motion (ballistic) is amplified and slow motion (corrective) is attenuated via a sigmoid of the maximum fingertip speed v:

The rotational increment is converted to angle-axis form and its angle scaled by g(v).

3. Grab, Clutching, and Stability Handling

Supporting mechanisms keep the interaction stable without any physics-based contact handling:

• Grab and release. A grab triggers when all three fingertips contact the object, establishing the initial reference triangle; release occurs when contact is lost for two or more fingertips.
• Grip-anchored positioning. The object snaps to the angle-weighted centroid of the triangle. Because the thumb vertex subtends the sharpest interior angle in a precision grip, inverse-angle weighting gives it the greatest influence, keeping the anchor at the center of the grip rather than drifting toward the two close fingers.
• Dwell-based clutching. Reaching a kinematic limit naturally induces a pause, so a fingertip pause over 0.15 s decouples input from output, letting users reset their hand pose without moving the object. It exits automatically on release.
• Triangle-area thresholding. Rotation is deactivated when the triangle area falls below an empirically set threshold (5th percentile of observed areas), suppressing the unwanted rotation that occurs when fingertips are too close together.

4. Evaluation Testbed

A docking task testbed was built to evaluate the technique: two 4 cm semi-transparent dice, threshold-based dock-and-hold completion (2 cm / 10°, held 1 s), with grab, clutching, and docked states surfaced through outline feedback. Two reference conditions were deployed in the same environment for comparison — HPTK+, a state-of-the-art physics-based hand simulation, and HandLevel, the Touch Hand Grab feature of the Meta XR Interaction SDK.

Key Results

We ran a user study to evaluate the system (n=24) comparing GeoCtrl against the physics-based and commodity references.

• Faster than physics simulation: GeoCtrl significantly reduced task completion time and time-on-target against the physics-based reference, confirming more effective rotational control without haptic feedback.
• Lowest physical demand: Rated significantly below both references on NASA-TLX physical demand, with no difference from the commodity technique on any other subscale — the added finger-level DOF cost no extra workload.
• Finger-level, not arm-level: GeoCtrl produced the greatest object rotation with far less wrist movement, confining manipulation to a significantly smaller workspace than both references.
• User preference: Preferred by 67% of participants, and rated more realistic and more physically comfortable than the physics-based reference, at the cost of a longer completion time than the commodity technique.

Citation

@inproceedings{oh2026geoctrl,
  title={GeoCtrl: Finger-based Geometric Mapping and Speed-Adaptive Gain for In-Hand Object Rotation in AR/VR},
  author={Oh, Seo Young and Son, Taejun and Lee, Juyoung and Yoon, Sang Ho and Woo, Woontack},
  booktitle={2026 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
  year={2026},
  publisher={IEEE}
}