Data systems are computerized systems that keep information about students, educators and schools and allow users to access the data and manage it, as well as analyze it. They go by many names including learning management system, student information system (SIS), decision support system data warehouse, and many more.
Data system design is designed to improve the way data is collected, stored and returned to an organisation. It involves determining which methods for retrieval and storage are the most efficient, creating data models and schemas and creating secure security. Data system design is about determining the tools and technologies that are most effective for storing, delivering and processing data.
Big sensor data systems rely on a set of different data sources, sourced from a range of sensors that are physical and not, such as wireless and mobile devices and wearables, telecommunication networks, and public databases. Each of these sources produces an array of sensor readings, each with their own metric values. The main challenge is to find a time resolution that is suitable for the data, and also an aggregate method that lets the sensor data to be portrayed in a single format using the same metric.
In order to ensure the accuracy of data analysis, it is essential to ensure that the data can be properly understood. This requires preprocessing the process of preparing all the activities that prepare data for subsequent analysis and transformations. This includes formatting, combing and replication. Preprocessing can be either stream or batch based.
