No Access

An Efficient Method for Video Object Detection in Automatic Scoring of Physical Experimental Operations

Wenbin Zeng

School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

E-mail Address: wbzeng@tju.edu.cn

Search for more papers by this author

Jichang Guo

https://orcid.org/0000-0003-3130-1685

School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

E-mail Address: jcguo@tju.edu.cn

Corresponding author

Search for more papers by this author

Luguo Hao

School of Information Engineering, Guangdong University of Technology, Guangzhou, Guangdong 510006, China

E-mail Address: haolg@gdut.edu.cn

Search for more papers by this author

, and

Jianfei Liu

School of Electronic and Information Engineering, Hebei University of Technology, Tianjin 300401, China

E-mail Address: jfliu@hebut.edu.cn

Search for more papers by this author

https://doi.org/10.1142/S0218213023500409Cited by:0 (Source: Crossref)

Abstract

Automatic scoring of students’ physical experimental operations is a very practical application which has not been researched deeply. The common method for automatic scoring of students’ experimental operations is to infer the behavior of experimental operations through the state of experimental instruments. Video object detection is the basic task of detecting the state of experimental instruments, and the problem of missed detection or false detection in video multi-object detection is one of the main reasons leading to the error of automatic scoring results. However, existing methods of video object detection mainly improve the accuracy of the model in public datasets, which has the disadvantage of not correcting false detection while improving accuracy. Therefore, an efficient video object detection method composed of YOLOv5 and a logical reasoning post-processing method was proposed to fill this gap. We compared our method with other state-of-the-art methods on three independent datasets of physical experimental instruments. We established a pipeline for automatic scoring of students’ experimental operations, designed flow charts and state score tables of three physics experiments, and compared the automatic scoring results with the average scores of six experimental teachers. The results show that our method is more robust and efficient in this application scenario. We hope this report can promote the application of logical reasoning methods in video object detection.

Keywords:

Remember to check out the Most Cited Articles!
Check out Notable Titles in Artificial Intelligence.

References

1. W. Zeng et al., Evaluation of physical electrical experiment operation process based on YOLOv5 and ResNeXt cascade networks, J. Neural Process Letters (2022), https://doi.org/10.1007/s11063-022-10952-0. Google Scholar
2. J. Redmon and A. Farhadi, YOLOv3: An incremental improvement, J. IEEE Trans. Pattern Anal. (2018) 15: 1125–1131. Google Scholar
3. A. Bochkovskiy, C.-Y. Wang and H.-Y. M. Liao, YOLOv4: Optimal speed and accuracy of object detection, arXiv preprint (2020), https://doi.org/10.48550/arXiv.2004.10934. Google Scholar
4. C. Y. Wang, A. Bochkovskiy and H. Y. M. Liao, Scaled-YOLOv4: Scaling cross stage partial network, in Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2021), pp. 13029–13038. Crossref, Google Scholar
5. G. Jocher, K. Nishimura, T. Mineeva and R. Vilariño, YOLOv5 release v6.1, latest version available at https://github.com/ultralytics/yolov5 (2022). Google Scholar
6. W. Liu et al., SSD: Single Shot MultiBox Detector, in Computer Vision — ECCV 2016, Lecture Notes in Computer Science, Vol. 9905 (2016). Google Scholar
7. S. Ren, K. He, R. Girshick and J. Sun, Faster R-CNN: Towards realtime object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell. 39 (6) (2017) 1137–1149. Crossref, Web of Science, Google Scholar
8. R. Girshick, Fast R-CNN, in IEEE Int. Conf. on Computer Vision (ICCV) (2015). Crossref, Google Scholar
9. Y. Chen et al., Memory enhanced global-local aggregation for video object detection, in CVPR (2020). Google Scholar
10. M. Han, Y. Wang, X. Chang and Y. Qiao, Mining inter-video proposal relations for video object detection, in ECCV (2020). Google Scholar
11. M. Shvets, W. Liu and A. C. Berg, Leveraging long-range temporal relationships between proposals for video object detection, in ICCV (2019). Google Scholar
12. J. Deng, Y. Pan, T. Yao, W. Zhou, H. Li and T. Mei, Relation distillation networks for video object detection, in ICCV (2019). Google Scholar
13. H. Wu, Y. Chen, N. Wang and Z. Zhang, Sequence level semantics aggregation for video object detection, in ICCV (2019). Google Scholar
14. S. Wang, Y. Zhou, J. Yan and Z. Deng, Fully motion-aware network for video object detection, in ECCV (2018). Google Scholar
15. G. Bertasius, L. Torresani and J. Shi, Object detection in video with spatiotemporal sampling networks, in ECCV (2018). Google Scholar
16. F. Xiao and Y. J. Lee, Video object detection with an aligned spatial-temporal memory, in ECCV (2018). Google Scholar
17. O. Russakovsky et al., ImageNet large scale visual recognition challenge, IJCV 115 (3) (2015) 211–252. Crossref, Web of Science, Google Scholar
18. X. Zhu et al., Deep feature flow for video recognition, in CVPR (2017). Google Scholar
19. X. Zhu et al., Flow-guided feature aggregation for video object detection, in ICCV (2017), pp. 408–417. Google Scholar
20. X. Zhu et al., Towards high performance video object detection, in CVPR (2018), pp. 7210–7218. Google Scholar
21. F. Xiao and Y. J. Lee, Video object detection with an aligned spatial-temporal memory, in ECCV (2018), pp. 494–510. Google Scholar
22. S. Wang et al., Fully motion-aware network for video object detection, in ECCV (2018). Google Scholar
23. J. Deng et al., Relation distillation networks for video object detection, in ICCV (2019). Google Scholar
24. H. Deng et al., Object guided external memory network for video object detection, in ICCV (2019). Google Scholar
25. M. Shvets et al., Leveraging long-range temporal relationships between proposals for video object detection, in ICCV (2019), pp. 9755–9763. Google Scholar
26. H. Wu et al., Sequence level semantics aggregation for video object detection, in ICCV (2019). Google Scholar
27. W. Han et al., Seq-nms for video object detection, CoRR (2016), abs/1602.08465. Google Scholar
28. L. Ye et al., Video object detection with a convolutional regression tracker, ISPRS Journal of Photogrammetry and Remote Sensing 176 (3) (2021) 139–150. Google Scholar
29. C. Feichtenhofer et al., Detect to track and track to detect, in ICCV (2017). Google Scholar
30. T. Y. Lin et al., Feature pyramid networks for object detection, in CVPR (2017). Google Scholar
31. K. Wang et al., PANet: Few-shot image semantic segmentation with prototype alignment, in ICCV (2019), pp. 9197–9206. Google Scholar
32. S. Nowozin, Optimal decisions from probabilistic models: The intersection-overunion case, in CVPR (2014). Google Scholar
33. K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, in CVPR (2016), pp. 770–778. Google Scholar
34. G. Huang et al., Densely connected convolutional networks, in CVPR (2017), pp. 4700–4708. Google Scholar
35. S. Xie et al., Aggregated residual transformations for deep neural networks, in CVPR (2017), pp. 1492–1500. Google Scholar