Dagstuhl-Seminar 13472
Global Measurement Framework
( 17. Nov – 20. Nov, 2013 )
Permalink
Organisatoren
- Philip Eardley (British Telecom R&D - Ipswich, GB)
- Marco Mellia (Polytechnic University of Torino, IT)
- Jörg Ott (Aalto University, FI)
- Jürgen Schönwälder (Jacobs Universität - Bremen, DE)
- Henning Schulzrinne (Columbia University - New York, US)
Kontakt
- Susanne Bach-Bernhard (für administrative Fragen)
The Internet has a history of unexpected and often unpredictable behaviors due to manifold interactions of thousands of networks, and billions of components and devices and users. The resulting complexity requires measurements to understand how the network is performing, to observe how it is evolving, and to determine where failures or degradations occur. Especially with constantly evolving applications and their interaction paradigms, new phenomena occur and need to be factored into operations and management: one example is the substantial effort going into defining interfaces to assist peer-to-peer applications so that the amount of cross-ISP traffic is reduced. Measurements thus form an integral part of network operator tool sets to keep the net up and running. This Dagstuhl seminar seeks contributing to improving Internet measurements in several respects:
- improving the expressiveness of measurement metrics (and develop appropriate new ones) beyond throughput, loss rate, and RTT so that the actual application-specific user quality of experience can be assessed;
- expanding the reach, scale, and diversity of measurements and the corresponding data analysis to obtain a more comprehensive view on the performance of networks and applications;
- structuring the otherwise mostly disconnected measurement activities to allow interfacing between them and/or providing defined access methods to them, for both carrying out measurements and accessing measurement results (offline and in real-time);
- providing ways to better instrument and more broadly utilize measurement infrastructure, inside operators, for end users, and at third parties.
We will bring together researchers from industry, academia, and regulators across continents and across different backgrounds to discuss the state of the art in measurements and their exploitation, measurement and analysis techniques, privacy and anonymization. We aim at identifying and initiating next steps towards a coherent global measurement framework for the Internet.
The topics to be discussed include:
- Measurement infrastructures and metrics
We will assess the (open) measurement infrastructures in operation today as well as those being set up and their measurement architectures, methods, and tools to understand how far those complement or overlap with each other and share experience about measurement results. We will discuss which metrics should be used to characterize network performance at which layers and understand how the information gathered from different infrastructures could be combined to offer richer insights. - Interfaces to measurement infrastructures
We will further discuss interfaces to diverse measurement infrastructures that would allow interaction between infrastructures as well as internal and external access to measurement data (including possible policy constraints). The seminar will cover at least interfaces for data mining and planning (non-real-time), for applications for real-time adaptation (real-time), for network operators for real-time troubleshooting & network management (real-time), for interacting with probes to carry out measurements (real-time) and control interfaces so that an operator can define what measurements are carried out (perhaps in near real-time). These interfaces might be global, so allowing operators or regulators to control measurement probes in other networks. This ties in with recent activity in the Internet Engineering Task Force and the Broadband Forum who both explore potential standardization paths for Internet measurements. - Using measurement results
Building upon item 2), which addresses access to the data, we will discuss what use cases benefit from global measurements and what their requirements on data collection and access (e.g., which timescales) would be. Besides network and data center management, information for regulators and end user applications, also the measurement infrastructures themselves could benefit, e.g., in terms of calibrating and validating their operation. We will discuss the implications of a ubiquitous, global measurement platform, where for example every broadband hub, WiFi base station and 4G mobile has an embedded measurement capability. - Data protection, anonymization, and privacy preservation
Maintaining the privacy and security of user data is undoubtedly important. Data collection and data analysis require carefully chosen anonymization methods to prevent retroactive identification of users and their behavior. This becomes even more important as measurement results might be shared across operators, because broader aggregation with other data becomes possible, and the data now leaves one security domain. But also aggregate information about operator performance requires a careful definition of sharing policies so that operators can strike a reasonable balance between being as open as possible (and thus most helpful for Internet-scale optimizations) on the one hand and not revealing too much information to their competitors or the public on the other.
The seminar will run for three days with the rough structure outlined below. Some invited talks are meant to set the scene for the group work and stir the discussion. The seminar will be structured into two theme days, the first one about creating a global measurement platform, the second on using a global measurement platform. The third day will be dedicated to a recap and a call for actions. Participants may also present ad-hoc ideas for discussion.
The organizers of the Dagstuhl seminar will produce a summary report to be published, e.g., in the IEEE Communications Magazine or in the editorial zone of ACM Computer Communications Review.
Sunday
- Evening
- Informal opening and discussion
Monday: Creating the global measure-ment platform
- Morning
- Welcome and organization Introduction of participants (1-2min each, no slides) Invited talks on today’s measurement infrastructures and their capabilities (30 mins each), specifically covering aspects relevant for creating a global measurement framework.
- Afternoon
-
Group discussion I: Creating the global measurement platform
- metrics and measurements
- infrastructure and interfaces
- Evening
- Informal continuation and group summary preparation
Tuesday: Using the platform
- Morning
- Presentation and discussion of group findings Invited talks on the regulator, operator, and user views and on applications using measurement data (20 min each)
- Afternoon
-
Group discussion II: Using the global measurement platform
- operator use cases
- regulator use cases
- end user use cases
- Evening
- Social
Wednesday
- Morning
- Presentation and discussion of group results II Ad-hoc talks and discussion Call for action: Recommendations and proposed next steps for: Operators, Regulators, Standards bodies, Researchers…
- Afternoon
- Conclusion and seminar observations
The Internet has a history of unexpected and often unpredictable behaviors due to manifold interactions of thousands of networks, and billions of components and devices and users. The resulting complexity requires measurements to understand how the network is performing, to observe how it is evolving, and to determine where failures or degradations occur. Especially with constantly evolving applications and their interaction paradigms, new phenomena occur and need to be factored into operations and management: one example is the substantial effort going into defining interfaces to assist peer-to-peer applications so that the amount of cross-ISP traffic is reduced. Measurements thus form an integral part of network operator tool sets to keep the net up and running.
But measurements are equally important for the research community to understand network traffic as well as protocol and application dynamics and their evolution. And they assist in quantifying application and (access) network performance and thus provide a tool for end users and regulators to monitor operators and their service level agreements. Tools such as speedtest.net have become widely used for individual measurements and basic ISP rating. Measurement service providers such as SamKnows or RIPE offer networks of probes, i.e., separate devices or embedded software on access routers, for continuous background measurements at the end users. These help ISPs and regulators in their work. Standards bodies such as the IETF and the Broadband Forum have established working groups to define a global measurement architecture and common interfaces and to extend the set of metrics describing communication properties.
This Dagstuhl seminar brought together researchers from industry, academia, and regulators to discuss the state of the art in measurements and their exploitation, measurement and analysis techniques, privacy and anonymization, and to contribute to a common understanding in a number of areas, including:
- improving the expressiveness of measurement metrics (and develop appropriate new ones) beyond throughput, loss rate, and RTT so that the actual application-specific user emph{quality of experience} can be assessed;
- expanding the reach, scale, and diversity of measurements and the corresponding data analysis to obtain a more comprehensive view on the performance of networks and applications;
- structuring the otherwise mostly disconnected measurement activities to allow interfacing between them and/or providing defined access methods to them, for both carrying out measurements and accessing measurement results (offline and in real-time);
- providing ways to better instrument and more broadly utilize measurement infrastructure, inside operators, for end users, and at third parties.
A side effect of this organization is that there were virtually no individual talks and hence no talk abstracts were collected.
We focused on two complementary aspects of a global measurement framework: 1) creating a global measurement framework and 2) using such a framework. Both were introduced by panels, with a lot of discussion contributing to these overviews, as described in the respective introduction to the following two sections.
- Saba Ahsan (Aalto University, FI) [dblp]
- Vaibhav Bajpai (Jacobs Universität - Bremen, DE) [dblp]
- Arthur W. Berger (Akamai Technologies - Cambridge, US) [dblp]
- Ernst Biersack (EURECOM - Biot, FR) [dblp]
- Trevor Burbridge (British Telecom R&D - Ipswich, GB) [dblp]
- Fabián E. Bustamante (Northwestern University - Evanston, US) [dblp]
- Pedro Casas (FTW Forschungszentrum Telekommunikation Wien GmbH, AT) [dblp]
- Benoit Claise (CISCO Systems Belgium, BE) [dblp]
- Sam Crawford (SamKnows Ltd. - London, GB) [dblp]
- Philip Eardley (British Telecom R&D - Ipswich, GB) [dblp]
- Daniel Karrenberg (RIPE NCC - Amsterdam, NL) [dblp]
- Mirja Kühlewind (Universität Stuttgart, DE) [dblp]
- Abdelkader Lahmadi (INRIA Lorraine - Nancy, FR) [dblp]
- Jukka Manner (Aalto University, FI) [dblp]
- Marco Mellia (Polytechnic University of Torino, IT) [dblp]
- Al Morton (AT&T - Middletown, US) [dblp]
- Jörg Ott (Aalto University, FI) [dblp]
- Fabio Ricciato (AIT Austrian Institute of Technology - Wien, AT) [dblp]
- Dario Rossi (Télécom Paris Tech, FR) [dblp]
- Ramin Sadre (Aalborg University, DK) [dblp]
- Jürgen Schönwälder (Jacobs Universität - Bremen, DE) [dblp]
- Henning Schulzrinne (Columbia University - New York, US) [dblp]
- Andrea Soppera (British Telecom R&D - Ipswich, GB) [dblp]
- Anna Sperotto (University of Twente, NL) [dblp]
- Burkhard Stiller (Universität Zürich, CH) [dblp]
- Tivadar Szemethy (Netvisor - Budapest, HU) [dblp]
- Brian Trammell (ETH Zürich, CH) [dblp]
Verwandte Seminare
- Dagstuhl-Seminar 16012: Global Measurements: Practice and Experience (2016-01-04 - 2016-01-07) (Details)
Klassifikation
- modelling / simulation
- networks
- world wide web / internet
Schlagworte
- Internet measurements
- quality of experience
- network management
- traffic engineering