Hybrid Spatial-Temporal Deep Learning Architectures for FMCW Radar-Based Human Activity Recognition

Daffa Ahmadhan  Khusumah; Fiky Yosef Suratman; Hesty Susanti

doi:10.55681/armada.v4i7.3001

Authors

Daffa Ahmadhan Khusumah S2 Teknik Elektro, Fakultas Teknik Elektro, Telkom University, Indonesia
Fiky Yosef Suratman S2 Teknik Elektro, Fakultas Teknik Elektro, Telkom University, Indonesia
Hesty Susanti S2 Teknik Elektro, Fakultas Teknik Elektro, Telkom University, Indonesia

DOI:

https://doi.org/10.55681/armada.v4i7.3001

Keywords:

FMCW Radar, Human Activity Recognition, CNN, Bi-LSTM, Dilated Convolution, Spatial–Temporal Learn-ing

Abstract

Human Activity Recognition (HAR) supports intelligent healthcare, surveillance, assisted living, and human–machine interaction. Vision-based methods are often limited by privacy concerns, illumination changes, and occlusion. This study proposes a hybrid spatial–temporal deep learning framework for FMCW radar-based HAR using micro-Doppler spectrograms. Four architectures are compared: 3D CNN–LSTM, 3D Bi-LSTM–CNN, CNN–Dilated Convolution–LSTM, and a Hybrid Ensemble CNN-LSTM with a Decision Tree classifier. Radar processing includes beat-frequency extraction, Range FFT, Doppler FFT, clutter suppression, and spectrogram generation. Convolutional layers extract spatial features, while LSTM and Bi-LSTM networks model temporal dependencies; dilated convolution expands the receptive field efficiently. Experimental results show that the hybrid models outperform conventional CNN and standalone LSTM approaches in accuracy, robustness, and generalisation. The hybrid ensemble achieves the best performance by combining spatial–temporal learning with ensemble optimisation while remaining effective in noisy environments and preserving user privacy.

Downloads

Download data is not yet available.

References

Chen, V. C. (2019). The micro-Doppler effect in radar (2nd ed.). Artech House.

Chen, Z., Sun, Y., & Qu, L. (2025). Research on cross-scene human activity recognition based on radar and Wi-Fi multimodal fusion. Electronics, 14(8), 1518. https://doi.org/10.3390/electronics14081518

Cohen, L. (1995). Time-frequency analysis. Prentice Hall.

Diraco, G., Rescio, G., & Leone, A. (2025). Radar-based activity recognition in strictly privacy-sensitive settings through deep feature learning. Biomimetics, 10(4), 243. https://doi.org/10.3390/biomimetics10040243

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16 × 16 words: Transformers for image recognition at scale. In International Conference on Learning Representations. https://openreview.net/forum?id=YicbFdNTTy

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 770–778). IEEE. https://doi.org/10.1109/CVPR.2016.90

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

Kakuba, S., Colaco, S. J., Kim, J. H., Lee, D.-G., Yoon, Y., & Han, D. S. (2024). Dilated causal convolution based human activity recognition using voxelized point cloud radar data. In 2024 International Conference on Artificial Intelligence in Information and Communication (ICAIIC) (pp. 812–815). IEEE. https://doi.org/10.1109/ICAIIC60209.2024.10463502

Kim, Y., & Ling, H. (2009). Human activity classification based on micro-Doppler signatures using a support vector machine. IEEE Transactions on Geoscience and Remote Sensing, 47(5), 1328–1337. https://doi.org/10.1109/TGRS.2009.2012849

Lara, O. D., & Labrador, M. A. (2013). A survey on human activity recognition using wearable sensors. IEEE Communications Surveys & Tutorials, 15(3), 1192–1209. https://doi.org/10.1109/SURV.2012.110112.00192

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539

Lim, S., Park, C., Lee, S., & Jung, Y. (2024). Human activity recognition based on point clouds from millimeter-wave radar. Applied Sciences, 14(22), 10764. https://doi.org/10.3390/app142210764

Lin, Y., Li, H., & Faccio, D. (2024). Human multi-activities classification using mmWave radar: Feature fusion in time-domain and PCANet. Sensors, 24(16), 5450. https://doi.org/10.3390/s24165450

Qian, Y., Chen, C., Tang, L., Jia, Y., & Cui, G. (2023). Parallel LSTM-CNN network with radar multispectrogram for human activity recognition. IEEE Sensors Journal, 23(2), 1308–1317. https://doi.org/10.1109/JSEN.2022.3224083

Rokach, L. (2010). Ensemble-based classifiers. Artificial Intelligence Review, 33(1–2), 1–39. https://doi.org/10.1007/s10462-009-9124-7

Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-C. (2015). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, & R. Garnett (Eds.), Advances in neural information processing systems (Vol. 28, pp. 802–810). Curran Associates.

Shrestha, A., Li, H., Le Kernec, J., & Fioranelli, F. (2020). Continuous human activity classification from FMCW radar with Bi-LSTM networks. IEEE Sensors Journal, 20(22), 13607–13619. https://doi.org/10.1109/JSEN.2020.3006386

Skolnik, M. I. (2001). Introduction to radar systems (3rd ed.). McGraw-Hill.

Tan, T.-H., Tian, J.-H., Sharma, A. K., Liu, S.-H., & Huang, Y.-F. (2024). Human activity recognition based on deep learning and micro-Doppler radar data. Sensors, 24(8), 2530. https://doi.org/10.3390/s24082530

Trinh, T. H., Dai, A. M., Luong, M.-T., & Le, Q. V. (2018). Learning longer-term dependencies in RNNs with auxiliary losses. In J. Dy & A. Krause (Eds.), Proceedings of the 35th International Conference on Machine Learning (Vol. 80, pp. 4965–4974). Proceedings of Machine Learning Research. https://proceedings.mlr.press/v80/trinh18a.html

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems (Vol. 30, pp. 5998–6008). Curran Associates.

Wu, X., Ling, Z., Zhang, X., Ma, Z., & Deng, W. (2025). Human similar activity recognition using millimeter-wave radar based on CNN-BiLSTM and class activation mapping. Eng, 6(3), 44. https://doi.org/10.3390/eng6030044

Yu, F., & Koltun, V. (2016). Multi-scale context aggregation by dilated convolutions. In International Conference on Learning Representations. https://arxiv.org/abs/1511.07122

Zhang, Y., Tang, H., Wu, Y., Wang, B., & Yang, D. (2024). FMCW radar human action recognition based on asymmetric convolutional residual blocks. Sensors, 24(14), 4570. https://doi.org/10.3390/s24144570

Zhao, M., Adib, F., & Katabi, D. (2016). Emotion recognition using wireless signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking (pp. 95–108). Association for Computing Machinery. https://doi.org/10.1145/2973750.2973762