English  |  正體中文  |  简体中文  |  Post-Print筆數 : 11 |  Items with full text/Total items : 88657/118248 (75%)
Visitors : 23504068      Online Users : 236
RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.
Scope Tips:
  • please add "double quotation mark" for query phrases to get precise results
  • please goto advance search for comprehansive author search
  • Adv. Search
    HomeLoginUploadHelpAboutAdminister Goto mobile version
    政大機構典藏 > 商學院 > 統計學系 > 學位論文 >  Item 140.119/125517
    Please use this identifier to cite or link to this item: http://nccur.lib.nccu.edu.tw/handle/140.119/125517

    Title: 基於支持向量機計算的相互熵之特徵選取
    A Feature Selection Study based on SVM and Mutual Entropy
    Authors: 游上葦
    Yu, Shang-Wei
    Contributors: 周珮婷
    Yu, Shang-Wei
    Keywords: 機器學習

    Machine learning
    Feature selection
    Dimension reduction
    Support Vector Machine
    Shannon Entropy
    Mutual Entropy
    Date: 2019
    Issue Date: 2019-09-05 15:42:06 (UTC+8)
    Abstract: 特徵選取為機器學習領域中一重要部分,適當的選取特徵(變數),除了減少機器運算時間、人力與金錢外,也可以避免模型過度配適或是欠擬和的情況發生。雖然經過多年發展已有很多特徵選取的方法,但同一種模型,不一定適用所有資料情況,因此提出新方法希望在特徵選取上會有更多選擇。
    Feature selection technique plays a significant role in machine learning. Selecting features (variables) adequately can not only reduce the expenditure, operating time in machine and the cost of labor but also prevent under fitting or overfitting. Although lots of feature selection methods have been developed for decades, it is impossible to apply a unique method to all types of data sets. In this study, we propose a new method to calculate the correlation between variables based on the Shannon entropy from information theory and SVM classifier. Variables are grouped into several clusters and selected by the new correlation measurement. Besides, we define the importance of variable by the test statistic of KS test using Gaussian mixed model and E-M algorithm for the propose of result assessment. The performance of proposed method on two simulated data and five real data are demonstrated and compared with other feature selection methods. The predicted results are stable through the proposed method with a reduced dataset.
    Reference: Akay, M.F., (2009). Support vector machines combined with feature selection for breast cancer diagnosis. Expert Systems with Applications, 36 (2), 3240–3247.
    Arunasakthi, K., KamatchiPriya, & L., Askerunisa, A., (2014). Fisher Score Dimensionality Reduction for SVM Classification. 2014 International Conference on Innovations in Engineering and Technology, 3(3), 1900-1904.
    Blum, A., Langley, P., Selection of relevant features and examples in machine learning. Artificial Intelligence, 97(1-2), 245-271.
    Bennasar, M., Hicks, Y., & Setchi, R., (2015). Feature selection using Joint Mutual Information Maximisation. Expert Systems with Applications, 42, 8520-8532.
    Boser, B.E., Guyon, I.M., & Vapnil, V.N., (1992). A Training Algorithm for Optimal Margin Classiers. Proceedings of the fifth annual workshop on Computational learning theory, 144-152.
    Breiman, L., (1996). Bagging Predictors. Machine Learning, 24, 123–140.
    Breiman, L., (2001). Random Forests. Machine Learning, 45(1), 5-32. doi:10.1023/a:1010933404324
    Chen, Y.W., & Lin, C.J., (2005). Combining SVMs with Various Feature Selection Strategies. Department of Computer Science, National Taiwan University.
    Cortes, C., & Vapnik, V., (1995). Support-vector networks. Machine Learning, 20(3), 273-297. doi:10.1007/bf00994018
    Dash, M., & Liu, H., (1997). Feature selection for classification. Intelligent Data Analysis, 1(1-4), 131-156.
    Díaz-Uriarte, R., & Alvarez de Andrés, S., (2006). Gene selection and classification of microarray data using random forest. BMC Bioinformatics, 7(1), 3.
    Guyon, I., & Elisseeff, A., (2003). An introduction to variable and feature selection. Journal of machine learning research, 3(March), 1157-1182.
    Haldurai, H., Madhubala, T., & Rajalakshmi, R., (2016). A Study on Genetic Algorithm and its Applications. International Journal of Computer Sciences and Engineering, 4(10), 139-143.
    Ho, T.K., (1995). Random decision forests. Proceedings of the Third International Conference on Document Analysis and Recognition, 1, 278.
    Ho, T.K., (1998). The Random Subspace Method for Constructing Decision Forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8), 832-844.
    Holland, J.H., (1992). Genetic Algorithms. Scientific American, 267(1), 66-73.
    Hsieh, F., Liu, S.Y., Hsieh, Y.C., McCowan, B., (2018). From patterned response
      dependency to structured covariate dependency: Entropy based categorical-
      pattern-matching. PLoS ONE, 13(6).
    Hsu, C.W., Chang, C.C., & Lin, C.J., (2003). Machine Learning with Applications in Breast Cancer Diagnosis and Prognosis. Department of Computer Science, National Taiwan University. https://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf
    Huang, C.L., & Wang, C.J., (2006). A GA-based feature selection and parameters optimization for support vector machines, Expert Systems with Applications, 31, 231-240.
    Jolliffe, I.T., & Cadima, J., (2016). Principal component analysis: A review and recent developments. Philosophical Transactions of The Royal Society A Mathematical Physical and Engineering Sciences, 374(2065), 20150202.
    Kumbhar, P., & Mali, M., (2016). A Survey on Feature Selection Techniques and Classification Algorithms for Efficient Text Classification. International Journal of Science and Research, 5(5), 1267-1275.
    Popovic, M., (2018). Researchers in an Entropy Wonderland: A Review of the Entropy Concept. Thermal Science, 22(2), 1163-1178.
    Raymer, M.L., Punch, W.F., Goodman, E.D., Kuhn, L.A., & Jain, A.K., (2000). Dimensionality reduction using genetic algorithms. IEEE Transactions on Evolutionary Computation, 4(2), 164-171.
    Kohavi, R., John, G., (1997). Wrappers for feature selection. Artificial Intelligence, 97(1-2), 273-324.
    Genuer, R., Poggi, J. M., & Tuleau-Malot, C., (2010). Variable selection using Random Forests. Pattern Recognition Letters, 31(14), 2225-2236.
    Saeys, Y., Inza, I., & Larrañaga, P., (2007). A review of feature selection techniques in bioinformatics. Bioinformatics, 23(19), 2507-2517.
    Scho ̈lkopf, B., Sung, K.K., Burges, Chris J.C., Girosi, F., Niyogi, P., Poggio, T., & Vapnik, V., (1997). Comparing support vector machines with Gaussian kernels to radial basis function classifiers. IEEE Transactions on Signal Processing, 45(11), 2758-2765.
    Shannon, C. E., (1948). A Mathematical Theory of Communication. The Bell System Technical Journal. 27, 379-432, 623-656.
    Srivastava, S., Joshi, N., & Gaur, M., (2013). A Review Paper on Feature Selection Methodologies and Their Applications. International Journal of Engineering Research and Development, 7(6), 57-61.
    Tibshirani, R., (1996). Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1), 267-288.
    Xu, X., (2006). Adaptive Intrusion Detection Based on Machine Learning: Feature Extraction, Classifier Construction and Sequential Pattern Prediction. International Journal of Web Services Practices, 2(1-2), 49-58.
    Yu, L., & Liu, H., (2003). Feature selection for high-dimensional data: A fast correlation based filter solution. Department of Computer Science & Engineering, Arizona State University.
    Description: 碩士
    Source URI: http://thesis.lib.nccu.edu.tw/record/#G0106354024
    Data Type: thesis
    DOI: 10.6814/NCCU201900996
    Appears in Collections:[統計學系] 學位論文

    Files in This Item:

    File SizeFormat
    402401.pdf1767KbAdobe PDF0View/Open

    All items in 政大典藏 are protected by copyright, with all rights reserved.

    社群 sharing

    DSpace Software Copyright © 2002-2004  MIT &  Hewlett-Packard  /   Enhanced by   NTU Library IR team Copyright ©   - Feedback