3D ToF Camera senSPure™ C11U Adopted for Interactive Gaming in Indonesia

Executive Summary

TOPPAN’s 3D ToF camera, senSPure™ C11U (C11U), was selected by Toei Techno International Inc. for its high robustness in interactive commercial content. The C11U maintains stable performance regardless of lighting conditions, visitor clothing, or the number of people. With a ceiling-mounted setup, the camera provides stable 3D sensing over a wide area. This allows for long-term, stable operation with minimal on-site adjustments, even for multiplayer game experiences.

Background

Context and Motivation

In recent years, commercial facilities and event venues require more than just basic human presence detection. There is a high demand for interactive content where visuals and effects change in response to visitor movements. Since the pandemic, the need for “touchless” experiences based on body position and motion has grown. These spaces often face drastic changes in lighting. Visitors also vary in height and wear diverse clothing. These factors make stable human detection difficult for conventional methods. A sensing technology that can detect and track multiple people simultaneously is essential. Toei Techno International Inc. previously worked with various 2D LiDARs and 3D cameras. They recognized these market trends and began development of solutions to create memorable experiences through intuitive participation.

Technical Callenges

Interactive content for commercial spaces requires high stability. However, conventional sensing methods faced several technical hurdles in real-world environments.

Challenge 1: Data Loss from Ambient and Sunlight

Intense stage lights or sunlight from windows constantly change the lighting conditions. RGB and stereo cameras suffer from depth data loss or noise due to brightness fluctuations and shadows. Standard ToF sensors also struggle with infrared noise from sunlight, which makes measurement unstable near windows.

Challenge 2: Data Loss for Low-Reflectivity Objects

In commercial spaces, visitor clothing and belongings vary in material and color. With RGB or stereo cameras, black or dark objects lack contrast against the background. This often results in a failure to recognize the person.

Challenge 3: Real-time Limits of AI and Skeletal Estimation

AI-based skeletal estimation and machine learning are common, but they require high computational power. This causes processing delays in crowded environments. Interactive content requires zero lag between movement and visual effects. Real-time performance was a major challenge.

Challenge 4: Constraints on Placement and Range

2D LiDARs are often restricted to floor-level installation. This creates blind spots due to permanent objects or overlapping people. Other 3D cameras have field-of-view limits. To cover a wide area, they require multiple sensors, which increases cost and installation complexity.

Benefits of C11U Integration

Benefit 1: Stable 3D Data even with Sunlight

The C11U uses TOPPAN’s unique hybrid ToF^TM sensor. This method removes non-signal infrared components as noise. It obtains 3D data without loss, even in environments with strong sunlight from windows. As a result, the system achieves stable human detection at all times, even without total control over lighting.

See more

Benefit 2: Reliable Data Regardless of Clothing or Attributes

Dark or black objects are typically difficult for ToF sensors because of low reflectivity. However, the senSPure™ SDK allows users to adjust parameters such as LightTimes and IR Thresh. By fine-tuning these parameters for the local environment, Toei Techno minimized on-site adjustments. The flexibility of the SDK ensures stable detection, even for visitors in black clothing.

Benefit 3: Real-time Performance via Lightweight Processing

Distance images and point cloud data from the C11U have a simpler structure than RGB images used for AI. This reduces the processing load on the application. This project utilizes an application for human detection and tracking based on C11U 3D data. It links movement and effects in real-time, even with many people present. This setup runs without high-end GPUs and ensures a stable system for permanent installations.

Benefit 4: Wide Detection Range and Flexible Placement

The C11U offers a range of 0.5m to 7.0m and a wide field of view of 90°(H) x 67°(V). This allows for flexible adjustment of installation height and position. One single camera covers a large area. This achieves stable human tracking in crowded situations without the need for multiple sensors on floors or walls. It is an ideal configuration for commercial maintenance and cost-efficiency.

System Overview

This system uses a custom C++ application to fetch depth and point cloud data from the C11U. It converts and writes this data to textures in real-time. This texture is shared with TouchDesigner via high-speed memory share, where Blob Tracking occurs. The final coordinates go to Unity to drive the content. By distributive processing across specialized tools and memory sharing, the system eliminates bottlenecks. It achieves ultra-low latency without dependence on RGB analysis or skeletal estimation.

The content is an interactive quiz game. Visitors move to specific zones on the floor to answer questions projected on the wall. When a visitor enters a zone, the C11U captures the 3D position, and the application judges the answer. Particle effects follow the visitor’s tracks instantly. This zero-lag response provides an intuitive and pleasant user experience.

Future Outlook

From Interactive Media to Industrial Automation The technical value proven in this interactive project extends beyond entertainment. TOPPAN’s hybrid ToF^TM technology is highly compatible with Industrial Automation, specifically for AMR (Autonomous Mobile Robots) and AGV (Automated Guided Vehicles).

Potential applications include:

— Simultaneous detection of people and goods in warehouses or factories.
— Obstacle recognition in environments with inconsistent lighting.
— Spatial recognition of dark parts or low-reflectivity objects.

TOPPAN will continue to expand the SDK and hardware options for industrial use. This project is the first step to prove the C11U as a foundation for high-precision machine automation.

Customer Profile

Company	Toei Techno International Inc.
Business	LED Lighting, Digital Signage, Spatial Design, Digital Solution Development.
Website	https://www.toeitechno.com

Buy Now: 3D ToF Camera C11U [↗]