Information and communication technologies have been widely used to achieve the objective of smart city development. A smart air quality sensing and forecasting system is an important part of a smart city. One of the major challenges in designing such a forecast system is ensuring high accuracy and an acceptable computation time. In this paper, we show that it is possible to accurately forecast fine particulate matter (PM2.5) concentrations with low computation time by using different clustering techniques. An Internet of Things framework comprising of Airbox devices for PM2.5 monitoring has been used to acquire the data. Our main focus is to achieve high forecasting accuracy with reduced computation time. We use a hybrid model to do the forecast and a grid based system to cluster the monitoring stations based on the geographical distance. The experiments and evaluation is done using Airbox devices data from 557 stations deployed all over Taiwan. We are able to demonstrate that a proper clustering based on geographical distance can reduce the forecasting error rate and also the computation time. Also, in order to further evaluate our system, we have applied wavelet-based clustering to group the monitoring stations. A final comparative analysis is done for different clustering schemes with respect to accuracy and computational time.