IMR: MM-1C: Enabling Continual Passive Estimation of Performance of Internet Transfers: Online Measurement and Classification Methods

This project focuses on developing techniques that can monitor network traffic and reliably estimate the performance experienced by Internet users, while only looking at headers of network packets and not looking at any privacy-violating information. They main novelty is in also inferring what type of network connection, what type of application, and what type of device the user is using for accessing the Internet, and in being able to do all of the analysis fast enough to keep up with high speed with which traffic is carried by the Internet. The innovations proposed in this project include: (i) The design of deep learning frameworks for classifying end-user segments, based on access networks, client platforms, and application usage, which will enhance understanding of Internet performance. (ii) The design of online sampling, hashing, and sketching techniques, which will enable other types of passive analysis to also be conducted in an online and light-weight manner. Currently, most passive analysis studies are relegated to storing and working with large sets of traces, which limits the their deployment. (iii) Passive estimation of whether network bandwidth constrains an Internet transfer, which has never been attempted before in a setting as diverse and evolving as the Internet.This collaborative project brings together investigators from the fields of Computer Science and Statistics, and is expected to transform several domains: (i) The proposed efforts in user segmentation will also aid in understanding how other properties of the Internet (not just performance) differ across different user segments. This will aid in ensuring equitable and ubiquitous Internet access for all citizens. (ii) The proposed monitoring techniques will be an important source of insights on network systems management that can help alleviate system bottlenecks and improve performance. (iii) Experience in experimentation, measurements, and scientific analysis of big data is invaluable to federal, commercial, and academic institutions that are involved in mining for information in large data-sets. The proposed efforts in emulation and analysis of massive amounts of traffic data will be an excellent source of undergraduate and graduate students trained in these aspects. (iv) The proposed efforts in involving minorities and undergraduates in research will help broaden the diversity and capabilities of the Data Science (Computer Science/Statistics) work force. (v) The proposed outreach efforts (to middle- and high-schoolers) will help increase community engagement with science and technology.The website for this project can be found at: https://sites.google.com/cs.unc.edu/real-time-passive-traffic-anal/home. This website will be updated with new developments and resources, until the end of the project activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.