An Enhanced Visual Object Tracking Approach based on Combined Features of Neural Networks, Wavelet Transforms, and Histogram of Oriented Gradients
Received: 28 April 2022 | Revised: 7 May 2022 | Accepted: 12 May 2022 | Online: 14 May 2022
Corresponding author: M. Bourennane
Abstract
In this paper, a new Visual Object Tracking (VOT) approach is proposed to overcome the main problem the existing approaches encounter, i.e. the significant appearance changes which are mainly caused by heavy occlusion and illumination variation. The proposed approach is based on a combination of Deep Convolutional Neural Networks (DCNNs), Histogram of Oriented Gradient (HOG) features, and discrete wavelet packet transforms. The problem of illumination variation is solved by incorporating the coefficients of the image discrete wavelet packet transform instead of the image template to handle the case of images with high saturation in the input of the used CNN, whereas the inverse discrete wavelet packet transforms are used at the output for extracting the CNN features. By combining four learned correlation filters with the convolutional features, the target location is deduced using multichannel correlation maps at the CNN output. On the other side, the maximum value of the resulting maps from the correlation filters with convolutional features produced by the previously obtained HOG feature of the image template are calculated and are used as an updating parameter of the correlation filters extracted from CNN and from HOG. The major aim is to ensure long-term memory of the target appearance so that the target item may be recovered if tracking fails. In order to increase the performance of HOG, the coefficients of the discrete packet wavelet transform are employed instead of the image template. The obtained results demonstrate the superiority of the proposed approach.
Keywords:
Visual tracking, deep convolution neural networks, wavelet transform, HOG featuresDownloads
References
F. A. Dharejo et al., "A deep hybrid neural network for single image dehazing via wavelet transform," Optik, vol. 231, Apr. 2021, Art. no. 166462.
M. Y. Abbass, K.-C. Kwon, N. Kim, S. A. Abdelwahab, F. E. A. El-Samie, and A. A. M. Khalaf, "Efficient object tracking using hierarchical convolutional features model and correlation filters," The Visual Computer, vol. 37, no. 4, pp. 831–842, Apr. 2021. DOI: https://doi.org/10.1007/s00371-020-01833-5
C. Ma, J.-B. Huang, X. Yang, and M.-H. Yang, "Hierarchical Convolutional Features for Visual Tracking," in IEEE International Conference on Computer Vision, Santiago, Chile, Dec. 2015, pp. 3074–3082. DOI: https://doi.org/10.1109/ICCV.2015.352
C. Ma, J.-B. Huang, X. Yang, and M.-H. Yang, "Robust Visual Tracking via Hierarchical Convolutional Features," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 11, pp. 2709–2723, Aug. 2019. DOI: https://doi.org/10.1109/TPAMI.2018.2865311
A. Zgaren, W. Bouachir, and R. Ksantini, "Coarse-to-Fine Object Tracking Using Deep Features and Correlation Filters," in 15th International Symposium on Visual Computing, San Diego, CA, USA, Nov. 2020, pp. 517–529. DOI: https://doi.org/10.1007/978-3-030-64556-4_40
Y. Said, M. Barr, and H. E. Ahmed, "Design of a Face Recognition System based on Convolutional Neural Network (CNN)," Engineering, Technology & Applied Science Research, vol. 10, no. 3, pp. 5608–5612, Jun. 2020. DOI: https://doi.org/10.48084/etasr.3490
P. Chakraborty and C. Tharini, "Pneumonia and Eye Disease Detection using Convolutional Neural Networks," Engineering, Technology & Applied Science Research, vol. 10, no. 3, pp. 5769–5774, Jun. 2020. DOI: https://doi.org/10.48084/etasr.3503
S. Alqethami, B. Almtanni, W. Alzhrani, and M. Alghamdi, "Disease Detection in Apple Leaves Using Image Processing Techniques," Engineering, Technology & Applied Science Research, vol. 12, no. 2, pp. 8335–8341, Apr. 2022. DOI: https://doi.org/10.48084/etasr.4721
J. Zhang, J. Sun, J. Wang, and X.-G. Yue, "Visual object tracking based on residual network and cascaded correlation filters," Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 8, pp. 8427–8440, Aug. 2021. DOI: https://doi.org/10.1007/s12652-020-02572-0
Y. Bai, T. Xu, B. Huang, and R. Yang, "Deep Deblurring Correlation Filter for Object Tracking," IEEE Access, vol. 8, pp. 68623–68637, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.2986311
Y. Qi et al., "Hedged Deep Tracking," in IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, Jun. 2016, pp. 4303–4311. DOI: https://doi.org/10.1109/CVPR.2016.466
C. Ma, Y. Xu, B. Ni, and X. Yang, "When Correlation Filters Meet Convolutional Neural Networks for Visual Tracking," IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1454–1458, Jul. 2016. DOI: https://doi.org/10.1109/LSP.2016.2601691
D. E. Touil, N. Terki, and S. Medouakh, "Hierarchical convolutional features for visual tracking via two combined color spaces with SVM classifier," Signal, Image and Video Processing, vol. 13, no. 2, pp. 359–368, Mar. 2019. DOI: https://doi.org/10.1007/s11760-018-1364-z
B. Latreche, S. Saadi, M. Kious, and A. Benziane, "A novel hybrid image fusion method based on integer lifting wavelet and discrete cosine transformer for visual sensor networks," Multimedia Tools and Applications, vol. 78, no. 8, pp. 10865–10887, Apr. 2019. DOI: https://doi.org/10.1007/s11042-018-6676-z
M. X. Bastidas Rodriguez et al., "Deep Adaptive Wavelet Network," in IEEE Winter Conference on Applications of Computer Vision, Snowmass, CO, USA, Mar. 2020, pp. 3100–3108. DOI: https://doi.org/10.1109/WACV45572.2020.9093580
S. Fujieda, K. Takayama, and T. Hachisuka, "Wavelet Convolutional Neural Networks," arXiv, arXiv:1805.08620, May 2018.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely Connected Convolutional Networks," in IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, Jul. 2017, pp. 2261–2269. DOI: https://doi.org/10.1109/CVPR.2017.243
H. Lu, H. Wang, Q. Zhang, D. Won, and S. W. Yoon, "A Dual-Tree Complex Wavelet Transform Based Convolutional Neural Network for Human Thyroid Medical Image Segmentation," in IEEE International Conference on Healthcare Informatics, New York, NY, USA, Jun. 2018, pp. 191–198. DOI: https://doi.org/10.1109/ICHI.2018.00029
F. Cotter and N. Kingsbury, "Deep Learning in the Wavelet Domain," arXiv, arXiv:1811.06115, Nov. 2018.
W. Yun, D. Kim, B. Song, and H. Yoon, "Block comparison based face identification using HOG feature," in 18th IEEE International Symposium on Robot and Human Interactive Communication, Toyama, Japan, Oct. 2009, pp. 484–487. DOI: https://doi.org/10.1109/ROMAN.2009.5326203
W. Zhang, G. Zelinsky, and D. Samaras, "Real-time Accurate Object Detection using Multiple Resolutions," in IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil, Oct. 2007, pp. 1–8. DOI: https://doi.org/10.1109/ICCV.2007.4409057
M. Villamizar, F. Moreno-Noguer, J. Andrade-Cetto, and A. Sanfeliu, "Efficient rotation invariant object detection using boosted Random Ferns," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, Jun. 2010, pp. 1038–1045. DOI: https://doi.org/10.1109/CVPR.2010.5540104
Y. Wei, Q. Tian, and T. Guo, "An Improved Pedestrian Detection Algorithm Integrating Haar-Like Features and HOG Descriptors," Advances in Mechanical Engineering, vol. 5, Jan. 2013, Art. no. 546206. DOI: https://doi.org/10.1155/2013/546206
D. E. Touil, N. Terki, and S. Medouakh, "Learning spatially correlation filters based on convolutional features via PSO algorithm and two combined color spaces for visual tracking," Applied Intelligence, vol. 48, no. 9, pp. 2837–2846, Sep. 2018. DOI: https://doi.org/10.1007/s10489-017-1120-z
K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," arXiv, arXiv:1409.1556, Apr. 2015.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database," in IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, Jun. 2009, pp. 248–255. DOI: https://doi.org/10.1109/CVPR.2009.5206848
M. Danelljan, G. Häger, F. S. Khan, and M. Felsberg, "Learning Spatially Regularized Correlation Filters for Visual Tracking," in IEEE International Conference on Computer Vision, Santiago, Chile, Dec. 2015, pp. 4310–4318. DOI: https://doi.org/10.1109/ICCV.2015.490
D. S. Bolme, J. R. Beveridge, B. A. Draper, and Y. M. Lui, "Visual object tracking using adaptive correlation filters," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, Jun. 2010, pp. 2544–2550. DOI: https://doi.org/10.1109/CVPR.2010.5539960
K. Zhang, L. Zhang, and M.-H. Yang, "Real-Time Compressive Tracking," in 12th European Conference on Computer Vision, Florence, Italy, Oct. 2012, pp. 864–877. DOI: https://doi.org/10.1007/978-3-642-33712-3_62
M. Danelljan, G. Hager, F. Khan, and M. Felsberg, "Accurate Scale Estimation for Robust Visual Tracking," in British Machine Vision Conference, Nottingham, UK, Sep. 2014. DOI: https://doi.org/10.5244/C.28.65
H. K. Galoogahi, T. Sim, and S. Lucey, "Multi-channel Correlation Filters," in IEEE International Conference on Computer Vision, Sydney, NSW, Australia, Dec. 2013, pp. 3072–3079. DOI: https://doi.org/10.1109/ICCV.2013.381
J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, "High-Speed Tracking with Kernelized Correlation Filters," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp. 583–596, Mar. 2015. DOI: https://doi.org/10.1109/TPAMI.2014.2345390
V. N. Boddeti, T. Kanade, and B. V. K. V. Kumar, "Correlation Filters for Object Alignment," in IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, Jun. 2013, pp. 2291–2298. DOI: https://doi.org/10.1109/CVPR.2013.297
F. A. Dharejo et al., "A deep hybrid neural network for single image dehazing via wavelet transform," Optik, vol. 231, Apr. 2021, Art. no. 166462. DOI: https://doi.org/10.1016/j.ijleo.2021.166462
Y. Wu, J. Lim, and M.-H. Yang, "Object Tracking Benchmark," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1834–1848, Sep. 2015. DOI: https://doi.org/10.1109/TPAMI.2014.2388226
A. Vedaldi and K. Lenc, "MatConvNet: Convolutional Neural Networks for MATLAB," in 23rd ACM international conference on Multimedia, Brisbane, Australia, Oct. 2015, pp. 689–692. DOI: https://doi.org/10.1145/2733373.2807412
X. Jia, H. Lu, and M.-H. Yang, "Visual tracking via adaptive structural local sparse appearance model," in IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, Jun. 2012, pp. 1822–1829. DOI: https://doi.org/10.1109/CVPR.2012.6247880
J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, "Exploiting the Circulant Structure of Tracking-by-Detection with Kernels," in 12th European Conference on Computer Vision, Florence, Italy, Oct. 2012, pp. 702–715. DOI: https://doi.org/10.1007/978-3-642-33765-9_50
J. Zhang, S. Ma, and S. Sclaroff, "MEEM: Robust Tracking via Multiple Experts Using Entropy Minimization," in 13th European Conference, Zurich, Switzerland, Sep. 2014, pp. 188–203. DOI: https://doi.org/10.1007/978-3-319-10599-4_13
Z. Hong, Z. Chen, C. Wang, X. Mei, D. Prokhorov, and D. Tao, "MUlti-Store Tracker (MUSTer): A cognitive psychology inspired approach to object tracking," in IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, Jun. 2015, pp. 749–758. DOI: https://doi.org/10.1109/CVPR.2015.7298675
Y. Li and J. Zhu, "A Scale Adaptive Kernel Correlation Filter Tracker with Feature Integration," in Computer Vision - ECCV 2014 Workshops, Zurich, Switzerland, Sep. 2014, pp. 254–265. DOI: https://doi.org/10.1007/978-3-319-16181-5_18
S. Hare et al., "Struck: Structured Output Tracking with Kernels," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 10, pp. 2096–2109, Jul. 2016. DOI: https://doi.org/10.1109/TPAMI.2015.2509974
L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. S. Torr, "Fully-Convolutional Siamese Networks for Object Tracking," in Computer Vision – ECCV 2016 Workshops, Amsterdam, Netherlands, Oct. 2016, pp. 850–865. DOI: https://doi.org/10.1007/978-3-319-48881-3_56
L. Bertinetto, J. Valmadre, S. Golodetz, O. Miksik, and P. H. S. Torr, "Staple: Complementary Learners for Real-Time Tracking," in IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, Jun. 2016, pp. 1401–1409. DOI: https://doi.org/10.1109/CVPR.2016.156
S. Hong, T. You, S. Kwak, and B. Han, "Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network," in 32nd International Conference on Machine Learning, Lille, France, Jul. 2015, pp. 597–606.
C. Ma, X. Yang, C. Zhang, and M.-H. Yang, "Long-term correlation tracking," in IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, Jun. 2015, pp. 5388–5396. DOI: https://doi.org/10.1109/CVPR.2015.7299177
Z. Kalal, K. Mikolajczyk, and J. Matas, "Tracking-Learning-Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 7, pp. 1409–1422, Jul. 2012. DOI: https://doi.org/10.1109/TPAMI.2011.239
H. K. Galoogahi, A. Fagg, and S. Lucey, "Learning Background-Aware Correlation Filters for Visual Tracking," in IEEE International Conference on Computer Vision, Venice, Italy, Oct. 2017, pp. 1144–1152. DOI: https://doi.org/10.1109/ICCV.2017.129
M. Danelljan, G. Hager, F. S. Khan, and M. Felsberg, "Convolutional Features for Correlation Filter Based Visual Tracking," in IEEE International Conference on Computer Vision Workshop, Santiago, Chile, Dec. 2015, pp. 621–629. DOI: https://doi.org/10.1109/ICCVW.2015.84
X. Li, Q. Liu, N. Fan, Z. Zhou, Z. He, and X. Jing, "Dual-regression model for visual tracking," Neural Networks, vol. 132, pp. 364–374, Dec. 2020. DOI: https://doi.org/10.1016/j.neunet.2020.09.011
T. Yang and A. B. Chan, "Visual Tracking via Dynamic Memory Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, pp. 360–374, Jan. 2021.
M. Danelljan, G. Hager, F. S. Khan, and M. Felsberg, "Adaptive Decontamination of the Training Set: A Unified Formulation for Discriminative Visual Tracking," in IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, Jun. 2016, pp. 1430–1438. DOI: https://doi.org/10.1109/CVPR.2016.159
Downloads
How to Cite
License
Copyright (c) 2022 M. Bourennane, N. Terki, M. Hamiane, A. Kouzou
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
