https://oldena.lpnu.ua/handle/ntb/52483| Title: | Unsupervised Real-Time Stream-Based Novelty Detection Technique |
| Authors: | Vergeles, Anna Prokopenko, Dmytro Khaya, Alexander Manakova, Nataliia |
| Affiliation: | Cloud Operations Oracle |
| Bibliographic description (Ukraine): | Unsupervised Real-Time Stream-Based Novelty Detection Technique / Anna Vergeles, Dmytro Prokopenko, Alexander Khaya, Nataliia Manakova // Data stream mining and processing : proceedings of the IEEE second international conference, 21-25 August 2018, Lviv. — Львів : Lviv Politechnic Publishing House, 2018. — P. 166–170. — (Dynamic Data Mining & Data Stream Mining). |
| Bibliographic description (International): | Unsupervised Real-Time Stream-Based Novelty Detection Technique / Anna Vergeles, Dmytro Prokopenko, Alexander Khaya, Nataliia Manakova // Data stream mining and processing : proceedings of the IEEE second international conference, 21-25 August 2018, Lviv. — Lviv Politechnic Publishing House, 2018. — P. 166–170. — (Dynamic Data Mining & Data Stream Mining). |
| Is part of: | Data stream mining and processing : proceedings of the IEEE second international conference, 2018 |
| Conference/Event: | IEEE second international conference "Data stream mining and processing" |
| Issue Date: | 28-Feb-2018 |
| Publisher: | Lviv Politechnic Publishing House |
| Place of the edition/event: | Львів |
| Temporal Coverage: | 21-25 August 2018, Lviv |
| Keywords: | high-load cloud SaaS telemetry sensors logs real-time monitoring streaming data changepoint novelty anomaly detection unsupervised |
| Number of pages: | 5 |
| Page range: | 166-170 |
| Start page: | 166 |
| End page: | 170 |
| Abstract: | A highly loaded cloud application environment requires the highest stability and operability, generates large telemetry data streams. These are obvious and actual prerequisites to develop a workload shift detector for the failures prevention aim. Having studied the previous works, the authors developed an approach to the detection of changepoints based on the specific conditions of the streaming telemetry data. The simulation of data center workload has allowed us to generate telemetry data under specific workload, thus we can evaluate the performance of the detector under various conditions. The conducted experiment has shown the viability of the proposed approach as well as directions for further study and improvement. |
| URI: | https://ena.lpnu.ua/handle/ntb/52483 |
| ISBN: | © Національний університет „Львівська політехніка“, 2018 © Національний університет „Львівська політехніка“, 2018 |
| Copyright owner: | © Національний університет “Львівська політехніка”, 2018 |
| URL for reference material: | http://arxiv.org/pdf/1610.07677v1 |
| References (Ukraine): | [1] M. Dias de Assunção, A. da Silva Veith, and R. Buyya, “Distributed data stream processing and edge computing: A survey on resource elasticity and future directions,” Journal of Network and Computer Applications, vol. 103, pp. 1–17, 2018. [2] M. Harvan, T. Locher, and A. C. Sima, “Cyclone: Unified Stream and Batch Processing,” in 2016 45th International Conference on Parallel Processing Workshops (ICPPW), Philadelphia, PA, USA, pp. 220–229, 2016. [3] W. Li, D. Niu, Y. Liu, S. Liu, and B. Li, “Wide-Area Spark Streaming: Automated Routing and Batch Sizing,” IEEE International Conference on Autonomic Computing (ICAC), Columbus, OH, USA, pp. 33–38, 2017. [4] K. Vidyasankar, “On Atomic Batch Executions in Stream Processing,” Procedia Computer Science, vol. 98, pp. 72–79, 2016. [5] C. Klein, B. Donnellan, and M. Helfert, Eds., Correlation-ModelBased Reduction of Monitoring Data in Data Centers, Setúbal: SCITEPRESS - Science and Technology Publications Lda, 2016. [6] P.-Y. Chen, S. Yang, and J. A. McCann, “Distributed Real-Time Anomaly Detection in Networked Industrial Sensing Systems,” IEEE Trans. Ind. Electron., vol. 62, no. 6, pp. 3832–3842, 2015. [7] S. Y. Shin and J. C. Maldonado, Novelty detection algorithm for data streams multi-class problems. Coimbra, Portugal, ACM, March 18-22, 2013. [8] D. Hong, D. Zhao, and Y. Zhang, “The Entropy and PCA Based Anomaly Prediction in Data Streams,” Procedia Computer Science, vol. 96, pp. 139–146, 2016. [9] C. C. Olson, K. P. Judd, and J. M. Nichols, “Manifold learning techniques for unsupervised anomaly detection,” Expert Systems with Applications, vol. 91, pp. 374–385, 2018. [10] Sajjad Kamali Siahroudi, Poorya Zare Moodi, and Hamid Beigy, “Detection of evolving concepts in non-stationary data streams: A multiple kernel learning approach,” Expert Systems With Applications, pp. 187-197, 2018. [11] S. Ahmad, A. Lavin, S. Purdy, and Z. Agha, “Unsupervised real-time anomaly detection for streaming data,” Neurocomputing, vol. 262, pp. 134–147, 2017. [12] M. Tennant, F. Stahl, O. Rana, and J. B. Gomes, “Scalable real-time classification of data streams with concept drift,” Future Generation Computer Systems, vol. 75, pp. 187–199, 2017. [13] E. Yu and P. Parekh, “A Bayesian Ensemble for Unsupervised Anomaly Detection,” [Online] Available: http://arxiv.org/pdf/1610.07677v1. [14] B. Krawczyk, L. L. Minku, J. Gama, J. Stefanowski, and M. Woźniak, “Ensemble learning for data stream analysis: A survey,” Information Fusion, vol. 37, pp. 132–156, 2017. [15] Z. Ding and M. Fei, “An Anomaly Detection Approach Based on Isolation Forest Algorithm for Streaming Data using Sliding Window,” IFAC Proceedings Volumes, vol. 46, no. 20, pp. 12–17, 2013. [16] A. Bifet, G. Holmes, B. Pfahringer, R. Kirkby, and R. Gavaldà, “New ensemble methods for evolving data streams,” 15th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '09, Paris, France, p. 139, 2009. [17] K. Noto, C. Brodley, and D. Slonim, “Anomaly Detection Using an Ensemble of Feature Models,” (eng), IEEE International Conference on Data Mining, pp. 953–958, 2010. [18] N. Golyandina and A. Korobeynikov, “Basic Singular Spectrum Analysis and forecasting with R,” Computational Statistics & Data Analysis, vol. 71, pp. 934–954, 2014. |
| References (International): | [1] M. Dias de Assunção, A. da Silva Veith, and R. Buyya, "Distributed data stream processing and edge computing: A survey on resource elasticity and future directions," Journal of Network and Computer Applications, vol. 103, pp. 1–17, 2018. [2] M. Harvan, T. Locher, and A. C. Sima, "Cyclone: Unified Stream and Batch Processing," in 2016 45th International Conference on Parallel Processing Workshops (ICPPW), Philadelphia, PA, USA, pp. 220–229, 2016. [3] W. Li, D. Niu, Y. Liu, S. Liu, and B. Li, "Wide-Area Spark Streaming: Automated Routing and Batch Sizing," IEEE International Conference on Autonomic Computing (ICAC), Columbus, OH, USA, pp. 33–38, 2017. [4] K. Vidyasankar, "On Atomic Batch Executions in Stream Processing," Procedia Computer Science, vol. 98, pp. 72–79, 2016. [5] C. Klein, B. Donnellan, and M. Helfert, Eds., Correlation-ModelBased Reduction of Monitoring Data in Data Centers, Setúbal: SCITEPRESS - Science and Technology Publications Lda, 2016. [6] P.-Y. Chen, S. Yang, and J. A. McCann, "Distributed Real-Time Anomaly Detection in Networked Industrial Sensing Systems," IEEE Trans. Ind. Electron., vol. 62, no. 6, pp. 3832–3842, 2015. [7] S. Y. Shin and J. C. Maldonado, Novelty detection algorithm for data streams multi-class problems. Coimbra, Portugal, ACM, March 18-22, 2013. [8] D. Hong, D. Zhao, and Y. Zhang, "The Entropy and PCA Based Anomaly Prediction in Data Streams," Procedia Computer Science, vol. 96, pp. 139–146, 2016. [9] C. C. Olson, K. P. Judd, and J. M. Nichols, "Manifold learning techniques for unsupervised anomaly detection," Expert Systems with Applications, vol. 91, pp. 374–385, 2018. [10] Sajjad Kamali Siahroudi, Poorya Zare Moodi, and Hamid Beigy, "Detection of evolving concepts in non-stationary data streams: A multiple kernel learning approach," Expert Systems With Applications, pp. 187-197, 2018. [11] S. Ahmad, A. Lavin, S. Purdy, and Z. Agha, "Unsupervised real-time anomaly detection for streaming data," Neurocomputing, vol. 262, pp. 134–147, 2017. [12] M. Tennant, F. Stahl, O. Rana, and J. B. Gomes, "Scalable real-time classification of data streams with concept drift," Future Generation Computer Systems, vol. 75, pp. 187–199, 2017. [13] E. Yu and P. Parekh, "A Bayesian Ensemble for Unsupervised Anomaly Detection," [Online] Available: http://arxiv.org/pdf/1610.07677v1. [14] B. Krawczyk, L. L. Minku, J. Gama, J. Stefanowski, and M. Woźniak, "Ensemble learning for data stream analysis: A survey," Information Fusion, vol. 37, pp. 132–156, 2017. [15] Z. Ding and M. Fei, "An Anomaly Detection Approach Based on Isolation Forest Algorithm for Streaming Data using Sliding Window," IFAC Proceedings Volumes, vol. 46, no. 20, pp. 12–17, 2013. [16] A. Bifet, G. Holmes, B. Pfahringer, R. Kirkby, and R. Gavaldà, "New ensemble methods for evolving data streams," 15th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '09, Paris, France, p. 139, 2009. [17] K. Noto, C. Brodley, and D. Slonim, "Anomaly Detection Using an Ensemble of Feature Models," (eng), IEEE International Conference on Data Mining, pp. 953–958, 2010. [18] N. Golyandina and A. Korobeynikov, "Basic Singular Spectrum Analysis and forecasting with R," Computational Statistics & Data Analysis, vol. 71, pp. 934–954, 2014. |
| Content type: | Conference Abstract |
| Appears in Collections: | Data stream mining and processing : proceedings of the IEEE second international conference |
| File | Description | Size | Format | |
|---|---|---|---|---|
| 2018_Vergeles_A-Unsupervised_Real_Time_166-170.pdf | 292.31 kB | Adobe PDF | View/Open | |
| 2018_Vergeles_A-Unsupervised_Real_Time_166-170__COVER.png | 500.14 kB | image/png | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.