A Framework for Detecting and Cleaning the Errors in Big Sensor Data

Abstract

Big sensor data is popular in both industry and scientific research applications. where the data is generated with large areas and high speed it is very difficult to process using database management tools or traditional data processing applications. Cloud computing provides a very good platform to support the addressing of this challenge as it provides a extensible stack of large computing services, massive storage , and software services in a scalable manner at very less cost. Some techniques have been developed in recent years for comprising the sensor data on cloud, such as sensor-cloud. However, these techniques do not show efficient effect on fast detection and locating of errors in big sensor data sets. For fast data error detection in big sensor datasets , in this paper, we develop a novel data error detection in large node datasets , in this paper , we develop a novel data error detection approach which exploits the full estimation potential of cloud and the network feature of Wireless sensor network.  Primarily, a group of node data error types are classified and defined. Based on that classification, the network component of a clustered wireless sensor is introduced and analyzed to support fast error detection and location. Specifically, in our proposed process, the error detection is depends on the scale- free topology and most of detection operations can be conducted in limited blocks rather than a whole large dataset. Hence the detection and location process can be dramatically increased. Moreover, the detection and location challenges can be distributed to cloud platform to fully exploit the processing power and huge storage. Experimental results shows that our introduced system shows that it reduces the time for error detection and location in big sensor data sets generated by large scale node network systems with acceptable error detecting accuracy.