Presentation
Vision–Language Models for Ergonomic Assessment of Manual Lifting Tasks: Estimating Horizontal and Vertical Hand Distances from RGB Video
SessionWednesday Poster Session
DescriptionManual lifting tasks are a major contributor to work-related musculoskeletal disorders, and effective ergonomic risk assessment is essential for quantifying physical exposure and informing ergonomic interventions. The Revised NIOSH Lifting Equation (RNLE) is a widely used ergonomic risk assessment tool for lifting tasks that relies on six task variables, including horizontal (H) and vertical (V) hand distances; such distances are typically obtained through manual measurement or specialized sensing systems and are difficult to use in real-world environments. We evaluated the feasibility of using vision-language models (VLMs) to non-invasively estimate H and V from RGB video streams. Two multi-stage VLM-based pipelines were developed: a text-guided detection-only pipeline and a detection-plus-segmentation pipeline. Both pipelines used text-guided localization of task-relevant regions of interest, visual feature extraction, and transformer-based temporal regression to estimate H and V at the start and end of a lift. Estimation performance was evaluated using leave-one-subject-out validation across the two pipelines and seven camera view conditions. Results varied across pipelines and camera view conditions, with the segmentation-based, multi-view pipeline yielding the smallest errors. Pixel-level segmentation reduced estimation error relative to the detection-only pipeline. These findings support the feasibility of VLM-based pipelines for video-based estimation of RNLE distance parameters.
Event Type
Poster
TimeWednesday, October 21st5:30pm - 6:30pm PDT
LocationSummit Pavilion