[1]LI Guopeng,WANG Lianqing,HAN Kun,et al.Action segmentation based on multigraph fusion constraint semi-nonnegative matrix factorization[J].CAAI Transactions on Intelligent Systems,2023,18(6):1223-1232.[doi:10.11992/tis.202303049]
Copy

Action segmentation based on multigraph fusion constraint semi-nonnegative matrix factorization

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 6 Page number: 1223-1232 Column: 学术论文—机器感知与模式识别 Public date: 2023-11-05
Title:
Action segmentation based on multigraph fusion constraint semi-nonnegative matrix factorization
Author(s):
LI Guopeng1;  WANG Lianqing2; 3;  HAN Kun1;  WANG Yuhong4;  SONG Dan1;  YU Li1; 2
1. Test Center, National University of Defense Technology, Xi’an 710106, China;
2. College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China;
3. College of Information and Communication, National University of Defense Technology, Wuhan 430035, China;
4. China Industrial Control Systems Cyber Emergency Response Team, Beijing 100040, China
Keywords:
action segmentation; clustering; semi-NMF; multigraph fusion constraint; structural similarity; measurement similarity; low-dimensional representation; k-nearest neighbor graph
CLC:
TP391
DOI:
10.11992/tis.202303049
Abstract:
Most clustering-based action segmentation methods mainly exploit the structure similarity information between adjacent frames (points) in the sequence to improve the accuracy of action segmentation. These methods improve the consistency of segmentation inside each action but introduce potential issues for accurately segmenting action boundaries. Hence, this paper presents a novel action segmentation method based on multigraph fusion constraint semi-nonnegative matrix factorization (MGSeNMF). In this method, the structural and measurement similarity information is fused to build a multigraph fusion constraint term, which is fused to semi-NMF to obtain a low-dimensional representation. A k-nearest neighbor graph is also generated for the action sequences, realizing accurate segmentation using the graph cut method. Experimental results on two kinds of real-action datasets show that MGSeNMF can accurately divide the boundary of actions while maintaining consistent segmentation inside each action. Thus, the proposed method improves the accuracy of segmentation and efficiency of running time significantly.
References:
[1] DU Zexing, WANG Xue, ZHOU Guoqing, et al. Fast and unsupervised action boundary detection for action segmentation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 3313-3322.
[2] CHAKRABORTY B K, SARMA D, BHUYAN M K, et al. Review of constraints on vision-based gesture recognition for human-computer interaction[J]. IET computer vision, 2018, 12(1): 3–15.
[3] 邵振洲, 赵红发, 渠瀛, 等. 一种面向多模态手术轨迹的快速无监督分割方法[J]. 小型微型计算机系统, 2018, 39(10): 2296–2302
SHAO Zhenzhou, ZHAO Hongfa, QU Ying, et al. Fast unsupervised approach for multi-modality surgical trajectory segmentation[J]. Journal of Chinese computer systems, 2018, 39(10): 2296–2302
[4] CHEN Wanjun, ZHANG Erhu, ZHANG Zhuomin. A Laplacian structured representation model in subspace clustering for enhanced motion segmentation[J]. Neurocomputing, 2016, 208: 174–182.
[5] LI Sheng, LI Kang, FU Yun. Temporal subspace clustering for human motion segmentation[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2016: 4453-4461.
[6] GAO Shanghua, HAN Qi, LI Zhongyu, et al. Global2Local: efficient structure search for video action segmentation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 16800-16809.
[7] ZHOU Tao, FU Huazhu, GONG Chen, et al. Consistency and diversity induced human motion segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(1): 197–210.
[8] LEI Peng, TODOROVIC S. Temporal deformable residual networks for action segmentation in videos[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6742-6751.
[9] ELHAMIFAR E, VIDAL R. Sparse subspace clustering: algorithm, theory, and applications[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 35(11): 2765–2781.
[10] ZHENG Jianwei, YANG Ping, SHEN Guojiang, et al. Enhanced low-rank constraint for temporal subspace clustering and its acceleration scheme[J]. Pattern recognition, 2021, 111: 107678.
[11] DING C H Q, LI Tao, JORDAN M I. Convex and semi-nonnegative matrix factorizations[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 32(1): 45–55.
[12] GAO Hongbo, LYU Chen, ZHANG Tong, et al. A structure constraint matrix factorization framework for human behavior segmentation[J]. IEEE transactions on cybernetics, 2022, 52(12): 12978–12988.
[13] SARFRAZ M S, MURRAY N, SHARMA V, et al. Temporally-weighted hierarchical clustering for unsupervised action segmentation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 11220-11229.
[14] WANG Xiumei, GUO Dingning, CHENG Peitao. Support structure representation learning for sequential data clustering[J]. Pattern recognition, 2022, 122: 108326.
[15] CAI Deng, HE Xiaofei, HAN Jiawei, et al. Graph regularized nonnegative matrix factorization for data representation[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 33(8): 1548–1560.
[16] CAI Yongda, HUANG J Z, YIN Jianfei. A new method to build the adaptive k-nearest neighbors similarity graph matrix for spectral clustering[J]. Neurocomputing, 2022, 493: 191–203.
[17] SHI Jianbo, MALIK J. Normalized cuts and image segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2000, 22(8): 888–905.
[18] HE Yangcheng, LU Hongtao, HUANG Lei, et al. Pairwise constrained concept factorization for data representation[J]. Neural networks, 2014, 52: 1–17.
[19] XIA Guiyu, CHEN Beijia, SUN Huaijiang, et al. Nonconvex low-rank kernel sparse subspace learning for keyframe extraction and motion segmentation[J]. IEEE transactions on neural networks and learning systems, 2021, 32(4): 1612–1626.
[20] LIU Haijun, CHENG Jian, WANG Feng. Sequential subspace clustering via temporal smoothness for sequential data segmentation[J]. IEEE transactions on image processing, 2018, 27(2): 866–878.
[21] ZHANG Kexin, ZHAO Xuezhuan, PENG Siyuan. Multiple graph regularized semi-supervised nonnegative matrix factorization with adaptive weights for clustering[J]. Engineering applications of artificial intelligence, 2021, 106: 104499.
[22] WANG Lichen, DING Zhengming, FU Yun. Low-rank transfer human motion segmentation[J]. IEEE transactions on image processing, 2019, 28(2): 1023–1034.
[23] JIANG Zhuolin, LIN Zhe, DAVIS L. Recognizing human actions by learning and matching shape-motion prototype trees[J]. IEEE transactions on pattern analysis and machine intelligence, 2012, 34(3): 533–547.
[24] ZHU Xiaofeng, ZHANG Shichao, HE Wei, et al. One-step multi-view spectral clustering[J]. IEEE transactions on knowledge and data engineering, 2019, 31(10): 2022–2034.
[25] BAI Yue, WANG Lichen, LIU Yunyu, et al. Human motion segmentation via velocity-sensitive dual-side auto-encoder[J]. IEEE transactions on image processing, 2023, 32: 524–536.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems