Real-time Video Focus Control System Based on Eye Squinting Degree

Introduction

In the previous development of intelligent AI glasses for drone control, I explored the idea: whether operators’ subtle facial expressions can be used to assist in adjusting visual focus or zooming the field of view, so as to improve the intuitiveness and convenience of manipulation.

As a myopic person, the common behavior of squinting to obtain clearer vision provides intuitive inspiration for this concept. Squinting is essentially a natural visual adjustment. If it can be transformed into a contactless interactive command, the naturalness of human-computer interaction will be greatly improved.

In recent years, with the rapid development of computer vision, real-time face detection and eye keypoint localization have become mature and stable, laying a reliable technical foundation for this idea.

This project designs and implements a real-time video focus control system based on eye opening and closing degree. The system captures frames through a camera, calculates the Eye Aspect Ratio (EAR) to quantify the squinting level, and dynamically adjusts the video clarity accordingly: the image remains blurred when eyes are open (simulating myopia), and becomes sharp when squinting (simulating manual focusing). The system runs entirely on local computing with real-time response, no network required, featuring low latency and strong privacy protection.

System Features

1. Real-time Response: The system can detect eye status in real time and adjust video clarity immediately.
2. Physiological Principle: Based on the actual squinting movement of the eyes, which conforms to natural human behavior.
3. Fully Offline: All calculations are completed locally without the need for network connection.
4. Customizable Threshold: The response threshold can be adjusted according to the eye characteristics of different users.
5. Lightweight Implementation: Based on Python and MediaPipe, with concise and efficient code.
6. Open-Source Code: The complete code is open-source, facilitating learning and improvement.

Hardware Environment

1. Development Board: Raspberry Pi 5 (4GB RAM)
2. Camera: Ordinary USB camera (supports 640x480 resolution)
3. Display: Any display for showing the processed video
4. Power Supply: Standard USB power supply
   

   

   The hardware connection diagram is drawn using the Digi-Key Scheme-it online design tool, ensuring clear and standardized connection logic, which can be directly used for project deployment and debugging reference.

   Software Environment

   1. Operating System: Raspberry Pi OS
   2. Python Version: 3.9
   3. Main Dependent Libraries and Versions:

                     OpenCV: 4.12.0.88

   4. MediaPipe: 0.10.14
   5. NumPy: 2.0.2

Inference Framework: MediaPipe Face Mesh (a lightweight facial key point detection framework), which is used to capture eye status in real time, quantify the degree of eye opening, and provide technical support for real-time focus adjustment of the system.

System Workflow：

1. The camera captures real-time video frames.
2. MediaPipe Face Mesh detects the face and extracts eye key points.
3. Calculate the Eye Aspect Ratio (EAR) to reflect the degree of eye opening.
4. Calculate the focus level (ranging from 0.0 to 1.0) based on the EAR value.
5. Apply the corresponding degree of blur effect according to the focus level.
6. Display the processed video frames and provide real-time data visualization.

Key Technologies

Eye Aspect Ratio (EAR) Calculation

EAR is a key indicator for measuring the degree of eye opening, and its calculation formula is as follows:

Among them, p1-p6 are the positions of 6 key points around the eye:

- p1, p4: Left and right corners of the eye

- p2, p6: Central points of the upper eyelid

- p3, p5: Central points of the lower eyelid

def calculate_eye_aspect_ratio(self, eye_landmarks):
    if len(eye_landmarks) < 6:
        return 0.0
    
    p1 = eye_landmarks[0]
    p2 = eye_landmarks[1]
    p3 = eye_landmarks[2]
    p4 = eye_landmarks[3]
    p5 = eye_landmarks[4]
    p6 = eye_landmarks[5]
    
    A = np.linalg.norm(p2 - p6)
    B = np.linalg.norm(p3 - p5)
    C = np.linalg.norm(p1 - p4)
    
    if C == 0:
        return 0.0
    
    ear = (A + B) / (2.0 * C)
    return ear

def calculate_focus(self, ear_value):
    # 将EAR值限制在预定义的范围内
    clamped_ear = np.clip(ear_value, self.min_ear, self.max_ear)
    ear_range = self.max_ear - self.min_ear
    
    if ear_range <= 0:
        ear_range = 0.15
    
    # 焦点水平：0.0（完全模糊）到1.0（完全清晰）
    focus_level = (self.max_ear - clamped_ear) / ear_range
    focus_level = np.clip(focus_level, 0, 1)
    
    # 计算模糊半径
    blur_range = self.max_blur_radius - self.min_blur_radius
    target_blur_radius = self.max_blur_radius - (focus_level * blur_range)
    
    return focus_level, int(target_blur_radius)

Real-time Blur Processing

According to the calculated blur radius, apply Gaussian blur to the video frame:

def apply_blur(self, frame, blur_radius):
    if blur_radius <= 1:
        return frame.copy()
    
    # 确保内核大小为奇数
    kernel_size = max(1, int(blur_radius * 2 + 1))
    if kernel_size % 2 == 0:
        kernel_size += 1
    
    # 应用高斯模糊
    return cv2.GaussianBlur(frame, (kernel_size, kernel_size), blur_radius)

def main():
    # 初始化摄像头
    camera_index = 0
    cap = cv2.VideoCapture(camera_index)
    
    # 设置摄像头参数
    cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
    cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
    
    # 创建焦点控制器
    controller = SimpleEyeSquintController()
    
    print("系统启动：睁眼=模糊，眯眼=清晰")
    
    while True:
        # 读取帧
        ret, frame = cap.read()
        if not ret:
            break
        
        # 镜像翻转
        frame = cv2.flip(frame, 1)
        
        # 处理帧
        processed_frame, focus_info = controller.process_frame(frame)
        
        # 显示处理后的帧
        cv2.imshow('Eye Squint Focus Control', processed_frame)
        
        # 处理按键
        key = cv2.waitKey(1) & 0xFF
        if key == ord('q'):
            break
        elif key == ord('+'):
            controller.adjust_max_blur(controller.max_blur_radius + 3)
        elif key == ord('-'):
            controller.adjust_max_blur(controller.max_blur_radius - 3)
    
    # 释放资源
    cap.release()
    controller.release()
    cv2.destroyAllWindows()

def __init__(self):
    # EAR值平滑
    self.EAR_SMOOTHING = 5
    self.left_ear_history = deque(maxlen=self.EAR_SMOOTHING)
    self.right_ear_history = deque(maxlen=self.EAR_SMOOTHING)
    
    # 模糊半径平滑
    self.focus_smoothing = 0.3
    self.blur_history = deque(maxlen=3)

Notes

• Parameter Tuning
• EAR Threshold Setting
• Through experiments, the appropriate EAR threshold range for most people has been identified:
  • Eye-open State (Blurred): EAR ≈ 0.38
  • Squinting State (Clear): EAR ≈ 0.23
   

Response Sensitivity Adjustment

• EAR_SMOOTHING: Increasing it makes the response smoother; decreasing it makes the response more sensitive.
• focus_smoothing: Controls the smoothness of blur transition.
• max_blur_radius: The maximum blur level, simulating different "degrees of myopia".

Effect Demonstration

Operation Effect

• Real-time EAR value (eye opening degree)
• Current focus level (0%-100%)
• Current blur radius
• System status (BLURRY or CLEAR)
• Focus level progress bar

Interaction Experience

• Normal eye opening: The video is presented in a blurred state to simulate myopia.
• Gradually squinting: As the degree of squinting deepens, the video gradually becomes clearer.
• Full squinting: The video reaches the clearest state.
• Opening eyes again: The video gradually returns to a blurred state.

Control Functions

• Press the "q" key: Exit the program.
• Press the "+" key: Increase the maximum blur level (simulate more severe myopia).
• Press the "-" key: Decrease the maximum blur level (simulate milder myopia).

Performance Optimization

1. Computational Efficiency

• MediaPipe Face Mesh runs efficiently on the CPU.
• Gaussian blur is implemented with OpenCV optimization.
• Only necessary eye key points are calculated to reduce computational load.

2. Memory Usage

• Use deque to implement a sliding window, ensuring constant memory usage.
• Process in real time without storing historical frames.
• Lightweight UI overlay, without significant additional overhead.

3. Real-Time Performance

• Achieves 15-20 FPS on Raspberry Pi 5.
• Response latency < 100ms.

Application Scenarios

1. Vision Training

    

   Through the feedback loop of squinting for clarity, help users understand their ability to control eye muscles.

    
2. Interactive Demonstration

    

   Vividly show how myopic patients can improve vision by squinting, which is used for popular science education in ophthalmology.

    
3. Game Control

    

   Use squinting as a novel game control method to enhance the sense of immersion in the game.

    
4. Assistive Technology

    

   Provide a contactless interaction method for people with special needs.

Improvement Directions

1. Personalized Calibration

    

   Add a calibration function to automatically adapt to individual differences of users.

    
2. Multi-Level Myopia Simulation

    

   Provide simulated myopia of different degrees, from mild to severe.

    
3. Astigmatism Simulation

    

   Add directional blur to more realistically simulate astigmatism.

    
4. Eye Fatigue Detection

    

   Remind users of prolonged squinting to prevent eye fatigue.

    
5. Cross-Platform Support

    

   Port to mobile devices and embedded platforms.

Summary

• Technological Innovation: Convert physiological movements into interactive commands.
• Educational Significance: Intuitively show the impact of myopia on vision.
• Practical Value: Provide new ideas for contactless interaction.
• Open-Source Sharing: The complete code is open-source to promote technical exchange.