Subject description
Data collection: smart phones, sensors and internet-connected devices, web, cleaning and preparation of data, data anonymization and de-identification.
Data retention; scalable relational databases, NoSQL databases, understanding the compromise between the consistency of data, performance and availability.
Data processing: event-oriented processing, processing parallelization (map-reduce), extraction of structured data from unstructured.
Analyses: efficient algorithms for processing and analysis of data, machine learning
Visualization, procedures and challenges of visualizing large amounts of data, other modalities of presentation of data (soundification, etc.).
Applications of the presented techniques: systems for context detection, smart systems (applications of smart cities, smart transport, etc.), medical applications, social networks, financial systems
The subject is taught in programs
Objectives and competences
Is familiar with the concept of "big data". Able to evaluate the amount of data, the rate of events, their diversity, and the key challenges associated with large amounts of data.
Knows the difference and can choose among relational or NoSQL database, and evaluate the appropriateness of use.
Knows the strengths and weaknesses of map-reduce model and evaluates it in comparison with relational databases.
Can apply basic analytical and visualization techniques for working with large amounts of data in a use-case.
Teaching and learning methods
Lectures or mentoring
Seminar
Expected study results
Understanding the concept of "big data": data volume, events and their diversity, and key challenges associated with large amounts of data.
Understanding of relational databases, their capabilities and limitations.
Understanding the capabilities, strengths and weaknesses of NoSQL databases.
Understanding of map-reducer model, its strengths and weaknesses, as well as a comparison with relational databases.
Understanding of basic analytical and visualization techniques for working with large amounts of data.
Basic sources and literature
- European Commission: http://www.internet-of-things-research.eu/pdf/Converging_Technologies_for_Smart_Environments_and_Integrated_Ecosystems_IERC_Book_Open_Access_2013.pdf
- Tom White: Hadoop: The Definitive Guide, 3rd Edition; Storage and Analysis at Internet Scale; O'Reilly Media
- Jure Leskovec, Anand Rajaraman, Jeffrey D. Ullman: Mining of Massive Datasets, http://i.stanford.edu/~ullman/mmds/book.pdf
- Jimmy Lin, Chris Dyer: Data-Intensive Text Processing with MapReduce, http://lintool.github.io/MapReduceAlgorithms/MapReduce-book-final.pdf
- Tamara Munzner: Visualization Analysis and Design (2014 Draft) http://www.cs.ubc.ca/~tmm/courses/533/book/vispmp-draft.pdf
- Scott Murray: Interactive Data Visualization for the Web: An Introduction to Designing with D3, O'Reilly Media
Stay up to date
University of Ljubljana, Faculty of Electrical Engineering Tržaška cesta 25, 1000 Ljubljana