EVALUATION OF VISION TRANSFORMER ON WEATHER IMAGE RECOGNITION
Keywords:
image, weather, convolutional neutral network (CNN), Vision Transformer (ViT)
Abstract
This study implements Vision Transformer 16x16 Words model for weather images classification. Its performance is compared with other traditional convolutional neural network (CNN) architectures, namely EfficientNetB2, DenseNet201, EfficientNetB7 and MobileNetV2. These models are implemented by transfer learning techniques for classification of images. In order to ensure the comparative performance, the same hyper-parameters of their models, such as dropout rate, optimizer and learning rate are employed identically. Furthermore, the same dataset on weather image phenomena applied on all those models with the same training, validation and testing dataset of weather images classification. The dataset of 11 different image classes that are collected from different resources of weather images with various kinds of weather phenomena are employed. The test results of performance show that the Vision Transformer gives the best results at 86.20%, which is suitable for application in evaluating weather images classification problem.
Downloads
Download data is not yet available.
References
[1] National Weather Service (NWS).
Advances in radars and satellites.
https://www.weather.gov/mqt/fitz_remote [Accessed
12th April 2023].
[2] Jaseena KU, Kovoor BC. Deterministic weather forecasting models based on intelligent predictors: A
survey. Journal of King Saud University-Computer
and Information Sciences. 2022;34(6): 3393–3412.
https://doi.org/10.1016/j.jksuci.2020.09.009.
[3] Xiao H, Zhang F, Shen Z, Wu K, Zhang J.
Classification of weather phenomenon from images by using deep convolutional neural network. Earth and Space Science. 2021;8(5): 1–9.
https://doi.org/10.1029/2020EA001604.
[4] Naufal MF, Kusuma SF. Weather image classification using convolutional neural network with transfer
learning. In: Parung J, Pah ND, Sutrisna PD, Suciadi
MFS. (eds.) International conference on informatics,
technology, and engineering 2021 (InCITE 2021):
Leveraging smart engineering, 25-26 August 2021,
Surabaya, Indonesia. New York, United States: AIP
Publishing; 2022. https://doi.org/10.1063/5.0080195.
[5] Ibrahim MR, Haworth J, Cheng T. WeatherNet:
Recognising weather and visual conditions from
street-level images using deep residual learning.
ISPRS International Journal of Geo-Information.
2019;8(12): 549. https://doi.org/10.3390/ijgi8120549.
[6] Khan MN, Ahmed MM. Weather and surface condition detection based on road-side webcams: Application of pre-trained convolutional neural network.
International Journal of Transportation Science and
Technology. 2022;11(3): 468–483.
[7] Minhas S, Khanam Z, Ehsan S, McDonald-Maier
K, Hernández-Sabaté A. Weather classification by
utilizing synthetic data. Sensors. 2022;22(9): 3193.
https://doi.org/10.3390/s22093193.
[8] Kang LW, Feng TZ, Fu RH. Inception networkbased weather image classification with pre-filtering
process. In: 23rd International Computer Symposium.
Singapore: Springer Singapore; 2018. p.368–375.
https://doi.org/10.1007/978-981-13-9190-3_38
(2019).
[9] Kiet Tran-Trung, Ha Duong Thi Hong , Vinh Truong
Hoang. Weather forecast based on color cloud image recognition under the combination of local image descriptor and histogram selection. Electronics.
2022;11(21): 3460. https://doi.org/10.3390 /electronics11213460.
[10] Horváth J, Baireddy S, Hao H, Montserrat DM,
Delp EJ. Manipulation detection in satellite images
using vision transformer. In: 2021 IEEE/CVF
conference on computer vision and pattern
recognition workshops (CVPRW), Nashville,
TN, USA. IEEE Xplore; 2021. p.1032-1041.
https://doi.org/10.1109/CVPRW53098.2021.00114.
[11] Li J, Luo X. A study of weather-image classification combining vit and a dual enhancedattention module. Electronics. 2023;12(5): 1213.
https://doi.org/10.3390/electronics12051213.
[12] Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16x16 words: Transformers for image recognition at scale. To be published in
ICLR 2021. arXiv. [Preprint] 2021. Version 2.
https://doi.org/10.48550/arXiv.2010.11929.
[13] Huang G, Liu Z, Van DML, Weinberger KQ. Densely
Connected Convolutional Networks. In: 2017 IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. IEEE Xplore; 2017.
p.2261-2269. doi: 10.1109/CVPR.2017.243.
[14] Tan M, Le Q. Efficientnet: Rethinking model scaling
for convolutional neural networks. To be published
in PMLR 97. arXiv. [Preprint] 2020. Version 5.
https://doi.org/10.48550/arXiv.1905.11946.
[15] Howard AG, Zhu M, Chen B, Kalenichenko D, Wang
W, Weyand T, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications.
Computer Vision and Pattern Recognition. arXiv;
2017. https://doi.org/10.48550 /arXiv.1704.04861.
[16] Xiao H. Weather phenomenon database
(WEAPD). V1. Harvard Dataverse; 2021.
https://doi.org/10.7910/DVN/M8JQCR.
Advances in radars and satellites.
https://www.weather.gov/mqt/fitz_remote [Accessed
12th April 2023].
[2] Jaseena KU, Kovoor BC. Deterministic weather forecasting models based on intelligent predictors: A
survey. Journal of King Saud University-Computer
and Information Sciences. 2022;34(6): 3393–3412.
https://doi.org/10.1016/j.jksuci.2020.09.009.
[3] Xiao H, Zhang F, Shen Z, Wu K, Zhang J.
Classification of weather phenomenon from images by using deep convolutional neural network. Earth and Space Science. 2021;8(5): 1–9.
https://doi.org/10.1029/2020EA001604.
[4] Naufal MF, Kusuma SF. Weather image classification using convolutional neural network with transfer
learning. In: Parung J, Pah ND, Sutrisna PD, Suciadi
MFS. (eds.) International conference on informatics,
technology, and engineering 2021 (InCITE 2021):
Leveraging smart engineering, 25-26 August 2021,
Surabaya, Indonesia. New York, United States: AIP
Publishing; 2022. https://doi.org/10.1063/5.0080195.
[5] Ibrahim MR, Haworth J, Cheng T. WeatherNet:
Recognising weather and visual conditions from
street-level images using deep residual learning.
ISPRS International Journal of Geo-Information.
2019;8(12): 549. https://doi.org/10.3390/ijgi8120549.
[6] Khan MN, Ahmed MM. Weather and surface condition detection based on road-side webcams: Application of pre-trained convolutional neural network.
International Journal of Transportation Science and
Technology. 2022;11(3): 468–483.
[7] Minhas S, Khanam Z, Ehsan S, McDonald-Maier
K, Hernández-Sabaté A. Weather classification by
utilizing synthetic data. Sensors. 2022;22(9): 3193.
https://doi.org/10.3390/s22093193.
[8] Kang LW, Feng TZ, Fu RH. Inception networkbased weather image classification with pre-filtering
process. In: 23rd International Computer Symposium.
Singapore: Springer Singapore; 2018. p.368–375.
https://doi.org/10.1007/978-981-13-9190-3_38
(2019).
[9] Kiet Tran-Trung, Ha Duong Thi Hong , Vinh Truong
Hoang. Weather forecast based on color cloud image recognition under the combination of local image descriptor and histogram selection. Electronics.
2022;11(21): 3460. https://doi.org/10.3390 /electronics11213460.
[10] Horváth J, Baireddy S, Hao H, Montserrat DM,
Delp EJ. Manipulation detection in satellite images
using vision transformer. In: 2021 IEEE/CVF
conference on computer vision and pattern
recognition workshops (CVPRW), Nashville,
TN, USA. IEEE Xplore; 2021. p.1032-1041.
https://doi.org/10.1109/CVPRW53098.2021.00114.
[11] Li J, Luo X. A study of weather-image classification combining vit and a dual enhancedattention module. Electronics. 2023;12(5): 1213.
https://doi.org/10.3390/electronics12051213.
[12] Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16x16 words: Transformers for image recognition at scale. To be published in
ICLR 2021. arXiv. [Preprint] 2021. Version 2.
https://doi.org/10.48550/arXiv.2010.11929.
[13] Huang G, Liu Z, Van DML, Weinberger KQ. Densely
Connected Convolutional Networks. In: 2017 IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. IEEE Xplore; 2017.
p.2261-2269. doi: 10.1109/CVPR.2017.243.
[14] Tan M, Le Q. Efficientnet: Rethinking model scaling
for convolutional neural networks. To be published
in PMLR 97. arXiv. [Preprint] 2020. Version 5.
https://doi.org/10.48550/arXiv.1905.11946.
[15] Howard AG, Zhu M, Chen B, Kalenichenko D, Wang
W, Weyand T, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications.
Computer Vision and Pattern Recognition. arXiv;
2017. https://doi.org/10.48550 /arXiv.1704.04861.
[16] Xiao H. Weather phenomenon database
(WEAPD). V1. Harvard Dataverse; 2021.
https://doi.org/10.7910/DVN/M8JQCR.
Published
29-September-2023
How to Cite
1.
Phi H, Tran N. EVALUATION OF VISION TRANSFORMER ON WEATHER IMAGE RECOGNITION. journal [Internet]. 29Sep.2023 [cited 22Dec.2024];13(3). Available from: https://journal.tvu.edu.vn/tvujs_old/index.php/journal/article/view/2431
Section
Articles
