Vol 8, No 4 (2017) > Electrical, Electronics and Computer Engineering >

Sin Activation Structural Tolerance of Online Sequential Circular Extreme Learning Machine

Sarutte Atsawaraungsuk, Tatpong Katanyukul


Abstract: This article discusses the development of the online sequential circular extreme learning machine (OS-CELM) and structural tolerance OS-CELM (STOS-CELM). OS-CELM is developed based on the circular extreme learning machine (CELM) to enable sequential learning. It can update a new chunk of data by spending less training time to update the chunk than the batch CELM. STOS-CELM is developed based on an idea similar to that of OS-CELM, but with a Householder block exact inverse QR decomposition (QRD) recursive least squares (QRD-RLS) algorithm to allow sequential learning and mitigate the criticality of deciding the number of hidden nodes. In addition, our experiments have shown that given the same hidden node setting, STOS-CELM can deliver accuracy comparable to a batch CELM approach and also has higher accuracy than the original online sequential extreme learning machine (OS-ELM) and structural tolerance OS-ELM (STOS-ELM) in classification problems, especially those involving high dimension datasets.
Keywords: Circular extreme learning machine; Extreme learning machine; Householder block exact QRD recursive least squares algorithm; Online sequential extreme learning machine

Full PDF Download


Atsawaraungsuk S., Horata, P., 2015. Evolutionary Circular-ELM for the Reduced-reference Assessment of Perceived Image Quality. Lecture Notes in Electrical Engineering, Volume 339, pp. 657–664

Decherchi, S., Gastaldo, P., Zunino, R., Cambria, E., Redi, J., 2013. Circular-ELM for the Reduced-reference Assessment of Perceived Image Quality. Neurocomputing, Volume 102, pp. 78–89

Gastaldo, P., Zounino, R., Heynderickx I., Vicario, E., 2002. Circular Back-propagation Networks for Measuring Displayed Image Quality. Lecture Notes in Computer Science, Volume 2415, pp. 1219–1224

Golub, G.H., Loan, C.F.V., 1996. Matrix Computations (3rd ed). Johns Hopkins University, Maryland

Horata, P., Chiewchanwattana, S., Sunat, K., 2013. Robust Extreme Learning Machine. Neurocomputing, Volume 102, pp. 31–44

Horata, P., Chiewchanwattana, S., Sunat, K., 2015. Enhancement of Online Sequential Extreme Learning Machine based on the Householder Block Exact Inverse QRD Recursive Least Squares. Neurocomputing, Volume 149, pp. 239–252

Huang, G.-B., Wang, D.H., Lan, Y., 2011. Extreme Learning Machines: A Survey. International Journal of Machine Learning and Cybernetics, Volume 2(2), pp. 107–122

Huang, G.-B., Zhu, Q.-Y., Siew, C.-K., 2004. Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks. In: Proceedings of the IEEE International Joint Conference on Neural Networks 2004, Budapest, 25 July, Hungary, pp. 985–990

Huang, G.-B., Zhu, Q.-Y., Siew, C.-K., 2006. Extreme Learning Machine: Theory and Applications. Neurocomputing, Volume 70(1-3), pp. 489–501

Liang, N.-Y., Huang, G.-B., Saratchandran, P., Sundararajan, N., 2006. A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks. IEEE Transactions on Neural Networks, Volume 17(6), pp. 1411–1423

Lim, J., 2013. Partitioned Online Sequential Extreme Learning Machine for Large Ordered System Modeling. Neurocomputing, Volume 102, pp. 59–64

Lu, J., Huang, J., Lu, F., 2017. Time Series Prediction based on Adaptive Weight Online Sequential Extreme Learning Machine. Applied Sciences, Volume 7(3), pp. 217–231

Moonen, M., Vandewalle, J., 1991. A Square Root Covariance Algorithm for Constrained Recursive Least Squares Estimation. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, Volume 3(3), pp 163–172

Pan, C.-T., Plemmons, R., 1989. Least Squares Modifications with Inverse Factorizations: Parallel Implications. Journal of Computational and Applied Mathematics, Volume 27(1–2), pp. 109–127

Ridella, S., Rovetta S., Zunino, R., 1997. Circular Backpropagation Networks for Classification. IEEE Transactions on Neural Networks, Volume 8, pp. 81–97

Rontogiannis, A.A., Theodoridis, S., 2009. QRD-RLS Adaptive Filtering. In: J.A. Apolinário Jr. (ed.). Householder-based RLS Algorithms, Springer, pp. 9–64

Stefani, A., Xenos, M., 2009. Meta-metric Evaluation of E-Commerce-related Metrics. Electronic Notes in Theoretical Computer Science, Volume 233, pp. 59–72

Trefethen, L.N., Bau, D., 1997. Numerical Linear Algebra. Philadelphia, Penn.: SIAM (Society for Industrial and Applied Mathematics)

Yang, Z., Chen, L., Zhang, P., 2015. OS-ELM Based Real-time RFID Indoor Positioning System for Shop-floor Management. In: Proceedings of ELM-2014, Volume 2, pp. 233–241