Revealing the Evolution of a Cloud Provider Through its Network Weather Map

Piraux, Maxime; Navarre, Louis; Rybowski, Nicolas; Bonaventure, Olivier; Donnet, Benoît

doi:10.1145/3517745.3561462

Download

Paper published in a book (Scientific congresses and symposiums)

Revealing the Evolution of a Cloud Provider Through its Network Weather Map

Piraux, Maxime; Navarre, Louis; Rybowski, Nicolas et al.

2022 • In Proc. ACM Internet Measurement Conference

Peer reviewed

Permalink
https://hdl.handle.net/2268/295386

DOI
10.1145/3517745.3561462

Files (1)Send to Details Statistics Bibliography Similar publications

Files

Full Text

paper.pdf

Author preprint (1.04 MB)

Download

All documents in ORBi are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Cloud infrastructure; network topology; dataset

Abstract :

[en] Researchers often face the lack of data on large operational networks to understand how they are used, how they behave, and sometimes how they fail. This data is crucial to drive the evolution of Internet protocols and develop techniques such as traffic engineering, DDoS detection and mitigation. Companies that have access to measurements from operational networks and services leverage this data to improve the availability, speed, and resilience of their Internet services. Unfortunately, the availability of large datasets, especially collected regularly over a long period of time, is a daunting task that remains scarce in the literature. We tackle this problem by releasing a dataset collected over roughly two years of observations of a major cloud company (OVH). Our dataset, called OVH Weather dataset, represents the evolution of more than 180 routers, 1,100 internal links, 500 external links, and their load percentages in the backbone network over time. Our dataset has a high density with snapshots taken every five minutes, totaling more than 500,000 files. In this paper, we also illustrate how our dataset could be used to study the backbone networks evolution. Finally, our dataset opens several exciting research questions that we make available to the research community.

Disciplines :

Computer science

Author, co-author :

Piraux, Maxime

Navarre, Louis

Rybowski, Nicolas

Bonaventure, Olivier

Donnet, Benoît ; Université de Liège - ULiège > Département d'électricité, électronique et informatique (Institut Montefiore) > Algorithmique des grands systèmes

Language :

English

Title :

Revealing the Evolution of a Cloud Provider Through its Network Weather Map

Publication date :

October 2022

Event name :

ACM Internet Measurement Conference

Event date :

Du 25 au 27 octobre

Audience :

International

Main work title :

Proc. ACM Internet Measurement Conference

Publisher :

ACM

Peer reviewed :

Peer reviewed

Additional URL :

https://weathermapdataset.info.ucl.ac.be

Data Set :

https://weathermapdataset.info.ucl.ac.be

Available on ORBi :

since 03 October 2022

Statistics

Number of views

44 (4 by ULiège)

Number of downloads

366 (2 by ULiège)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

Bibliography

2022. Network Weathermap: Open Source Network Visualisation. (May 2022). Retrieved May 18, 2022 from https://www.network-weathermap.com
2022. OVHcloud: Network & Infrastructure Status. (May 2022). Retrieved May 18, 2022 from https://network.status-ovhcloud.com
2022. OVHcloud Network Weathermap. (May 2022). Retrieved May 18, 2022 from http://weathermap.ovh.net
2022. Scaleway Netmap. (Sep 2022). Retrieved Sep 12, 2022 from https://netmap.scaleway.com
P. Ambati, D. Irwin, and P. Shenoy. 2020. No Reservations: A First Look at Amazon’s Reserved Instance Marketplace. In Proc. 12th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud). USENIX.
T. Benson, A. Akella, and D. A. Maltz. 2010. Network Traffic Characteristics of Data Centers in the Wild. In Proc. ACM Internet Measurement Conference (IMC). https://doi.org/10.1145/1879141.1879175
RIPE Network Coordination Center. 2010. Atlas. (2010). See https://atlas.ripe.net.
Y. Cheng, Z. Chai, and A. Anwar. 2018. Characterizing Co-located Datacenter Workloads: An Alibaba Case Study. In Proc. 9th Asia-Pacific Workshop on Systems (APSys). https://doi.org/10.1145/3265723.3265742
K. Cho, K. Mitsuya, and A. Kato. 2000. Traffic Data Repository at the WIDE Project. In Proc. USENIX Annual Technical Conference. USENIX. https://doi.org/10.5555/1267724.1267775
kc claffy, Y. Hyun, K. Keys, M. Fomenkov, and D. Krioukov. 2009. Internet Mapping: from Art to Science. In Proc. IEEE Cybersecurity Application and Technologies Conference for Homeland Security (CATCH). https://doi.org/10.1109/CATCH.2009. 38
Amogh Dhamdhere, David D Clark, Alexander Gamero-Garrido, Matthew Luckie, Ricky KP Mok, Gautam Akiwate, Kabir Gogia, Vaibhav Bajpai, Alex C Snoeren, and Kc Claffy. 2018. Inferring persistent interdomain congestion. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication. 1–15.
B. Donnet and T. Friedman. 2007. Internet Topology Discovery: a Survey. IEEE Communications Surveys & Tutorials 9, 4 (December 2007), 2–15. https://doi.org/10.1109/COMST.2007.4444750
Benoit Donnet, Matthew Luckie, Pascal Mérindol, and Jean-Jacques Pansiot. 2012. Revealing MPLS tunnels obscured from traceroute. ACM SIGCOMM Computer Communication Review 42, 2 (2012), 87–93.
C. Dovrolis, K. Gummadi, A. Kuzmanovic, and S. D. Meinrath. 2010. Measurement lab: Overview and an invitation to the research community. ACM SIGCOMM Computer Communication Review 40, 3 (2010), 53–56. https://doi.org/10.1145/1823844.1823853
Andrew D Ferguson, Jordan Place, and Rodrigo Fonseca. 2013. Growth analysis of a large isp. In Proceedings of the 2013 conference on Internet measurement conference. 347–352.
S. Gay, P. Schaus, and S. Vissichio. 2017. REPETITA: Repeatable Experiments for Performance Evaluation of Traffic-Engineering Algorithms. cs.NI 1710.08665. arXiv.
P. Gill, C. Diot, L. Y. Ohlsen, M. Mathis, and S. Soltesz. 2022. M-Lab: user initiated Internet data for the research community. (2022), 34–37 pages. https://doi.org/10.1145/1823844.1823853
K. Keys. 2010. Internet-Scale IP Alias Resolution Techniques. ACM SIGCOMM Computer Communication Review 40, 1 (January 2010), 50–55. https://doi.org/10.1145/1672308.1672318
K. Keys, Y. Hyun, M. Luckie, and kc claffy. 2013. Internet-Scale IPv4 Alias Resolution with MIDAR. IEEE/ACM Transactions on Networking 21, 2 (April 2013), 383–399. https://doi.org/10.1109/TNET.2012.2198887
S. Knight, X. N. Hung, N. Falkner, R. Bowden, and M. Roughan. 2011. The Internet Topology Zoo. IEEE Journal on Selected Areas in Communications 29, 9 (October 2011), 1765–1775. https://doi.org/10.1109/JSAC.2011.111002
W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson. 1994. On the self-similar nature of Ethernet traffic (extended version). IEEE/ACM Transactions on networking 2, 1 (1994), 1–15. https://doi.org/10.1109/90.282603
A. Lodhi, N. Larson, A. Dhamdhere, C. Dovrolis, and kc Claffy. 2014. Using peeringDB to understand the peering ecosystem. ACM (SIGCOMM) Computer Communication Review 44, 2 (2014), 20–27. https://doi.org/10.1145/2602204.2602208
R. Mahajan, N. Spring, D. Wetherall, and T. Anderson. 2022. Inferring Link Weights Using End-to-End Measurements. In Proc. ACM SIGCOMM Workshop on Internet measurment (IMW). ACM. https://doi.org/10.1145/637201.637237
A. Marder. 2020. APPLE: Alias Pruning by Path Length Estimation. In Proc. Passive and Active Measurement Conference (PAM). https://doi.org/10.1007/978-3-030- 44081- 7_15
E. Marechal, P. Mérindol, and B. Donnet. 2022. ISP Probing Reduction with Anaximander. In Proc. Passive and Active Measurement Conference (PAM). https://doi.org/10.1007/978-3-030-98785-5_20
P. Mérindol, V. Van den Schriek, B. Donnet, O. Bonaventure, and J.-J. Pansiot. 2009. Quantifying ASes Multiconnectivity Using Multicast Information. In Proc. ACM Internet Measurement Conference (IMC). https://doi.org/10.1145/1644893.1644937
Giovane CM Moura, Ricardo de O Schmidt, John Heidemann, Wouter B de Vries, Moritz Muller, Lan Wei, and Cristian Hesselman. 2016. Anycast vs. DDoS: Evaluating the November 2015 root DNS event. In Proceedings of the 2016 Internet Measurement Conference. 255–270.
S. Orlowski, R. Wessaly, M. Pióro, and A. Tomaszewski. 2010. SDNlib 1.0-Survivable Network Design Library. Networks: an International Journal 55, 3 (May 2010), 276–286. https://doi.org/10.1002/net.20371
Victor-Alexandru Pădurean, Oliver Gasser, Randy Bush, and Anja Feldmann. 2022. SRv6: Is There Anybody Out There? arXiv preprint arXiv:2205.04193 (2022).
R. Pastor-Satorras and A. Vespignani. 2004. Evolution and Structure of the Internet: A Statistical Physics Approach. Cambridge University Press.
V. Paxson and S. Floyd. 1995. Wide area traffic: the failure of Poisson modeling. IEEE/ACM Transactions on networking 3, 3 (1995), 226–244. https://doi.org/10.1109/90.392383
PeeringDB. 2004–2022. The Interconnection Database. (2004–2022). see https://www.peeringdb.com.
M. Peuster, S. Schneider, and H. Karl. 2019. The Softwarised Network Data Zoo. In Proc. IEEE/IFIP 15th International Conference on Network and Service Management (CNSM). IFIP. https://doi.org/10.23919/CNSM46954.2019.9012740
A. Roy, H. Zeng, J. Bagga, G. Porter, and A. C. Snoeren. 2015. Inside the Social Network’s (Datacenter) Network. In Proc. ACM SIGCOMM. ACM. https://doi.org/10.1145/2785956.2787472
Y. Shavitt and E. Shir. 2005. DIMES: Let the Internet Measure Itself. ACM SIGCOMM Computer Communication Review 35, 5 (2005). https://doi.org/10.1145/1096536.1096546
J. Sherry, E. Katz-Bassett, M. Pimenova, H. V. Madhyastha, T. Anderson, and A. Krishnamurthy. 2010. Resolving IP Aliases with Prespecified Timestamps. In Proc. ACM Internet Measurement Conference (IMC). https://doi.org/10.1145/1879141. 1879163
H. Soroush, K. Sung, E. Learned-Miller, B. N. Levine, and M. Liberatore. 2013. Disabling GPS is Not Enough: Cellular location leaks over the Internet. In Proc. Privacy Enhancing Technologies Symposium (PETS). https://doi.org/10.1007/978-3-642-39077-7_6
N. Spring, R. Mahajan, and D. Wetherall. 2002. Measuring ISP Topologies with Rocketfuel. In Proc. ACM SIGCOMM. https://doi.org/10.1145/633025.633039
N. Spring, L. Peterson, A. Bavier, and V. Pai. 2006. Using PlanetLab for network research: myths, realities, and best practices. ACM SIGOPS Operating Systems Review 40, 1 (2006), 17–24. https://doi.org/10.1145/1113361.1113368
Staff, RIPE NCC. 2015. RIPE Atlas: A global Internet measurement network. Internet Protocol Journal 18, 3 (2015).
R. Teixeira, K. Marzullo, S. Savage, and G. Voelker. 2003. In Search of Path Diversity in ISP Networks. In Proc. ACM Internet Measurement Conference (IMC). ACM. https://doi.org/10.1145/948205.948247
Caida. 2002–2022. The UCSD Network Telescope. (2002–2022). https://www.caida.org/projects/network_telescope/.
Caida. 2010–2022. Macroscopic Internet Topology Data Kit (ITDK). (2010–2022). https://www.caida.org/catalog/datasets/internet-topology-data-kit/.
RIPE. [n. d.]. RIPE RIS, Routing Information Service. ([n. d.]). See https://www.ripe.net/analyse/internet-measurements/routing-information-service-ris.
P. Tune and M. Roughan. 2017. Controlled synthesis of traffic matrices. IEEE/ACM Transactions on Networking 25, 3 (2017), 1582–1592. https://doi.org/10.1109/TNET.2016.2639066
Steve Uhlig, Bruno Quoitin, Jean Lepropre, and Simon Balon. 2006. Providing public intradomain traffic matrices to the research community. ACM SIGCOMM Computer Communication Review 36, 1 (2006), 83–86. https://doi.org/10.1145/1111322.1111341
University of Oregon. [n. d.]. Route Views, University of Oregon Route Views Project. ([n. d.]). See http://www.routeviews.org/routeviews/.
Yves Vanaubel, Pascal Mérindol, Jean-Jacques Pansiot, and Benoit Donnet. 2017. Through the wormhole: Tracking invisible MPLS tunnels. In Proceedings of the 2017 Internet Measurement Conference. 29–42.
Y. Zhang, Z. Ge, A. Greenberg, and M. Roughan. 2005. Network Anomography. In Proc. ACM Internet Measurement Conference (IMC). ACM. https://doi.org/10.5555/1251086.1251116