Efficient and Time Scale-Invariant Detection of Correlated Activity in Communication Networks

Brian Thompson, James Abello

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In many real-world networks, interactions between entities are observed at specific moments in continuous time, such as email, SMS messaging, and IP traffic. The majority of methods for analyzing such data first aggregate communication over designated time blocks, resulting in one or more discrete time series, to which existing tools can be applied. However, regardless of how the block lengths are chosen, discretizing time inherently introduces information loss and biases analysis towards patterns occurring at the designated time scale, effects which can be especially pronounced in networks with a high degree of temporal variability. Due to this, there has been increasing interest in using stochastic point processes to model network activity. We present a novel approach based on such models to detect times and sets of entities with temporally correlated recent activity. We develop efficient algorithms and compare our approach to existing and baseline methods through experiments on synthetic and real-world data.

Original languageEnglish (US)
Title of host publicationProceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015
EditorsXindong Wu, Alexander Tuzhilin, Hui Xiong, Jennifer G. Dy, Charu Aggarwal, Zhi-Hua Zhou, Peng Cui
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages532-539
Number of pages8
ISBN (Electronic)9781467384926
DOIs
StatePublished - Jan 29 2016
Externally publishedYes
Event15th IEEE International Conference on Data Mining Workshop, ICDMW 2015 - Atlantic City, United States
Duration: Nov 14 2015Nov 17 2015

Publication series

NameProceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015

Other

Other15th IEEE International Conference on Data Mining Workshop, ICDMW 2015
CountryUnited States
CityAtlantic City
Period11/14/1511/17/15

Fingerprint

Telecommunication networks
Electronic mail
Telecommunication traffic
Time series
Communication
Experiments

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering
  • Computer Science Applications

Keywords

  • anomaly detection
  • dynamic networks
  • graph algorithms
  • stochastic models
  • streaming
  • temporal correlation

Cite this

Thompson, B., & Abello, J. (2016). Efficient and Time Scale-Invariant Detection of Correlated Activity in Communication Networks. In X. Wu, A. Tuzhilin, H. Xiong, J. G. Dy, C. Aggarwal, Z-H. Zhou, & P. Cui (Eds.), Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015 (pp. 532-539). [7395714] (Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDMW.2015.24
Thompson, Brian ; Abello, James. / Efficient and Time Scale-Invariant Detection of Correlated Activity in Communication Networks. Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015. editor / Xindong Wu ; Alexander Tuzhilin ; Hui Xiong ; Jennifer G. Dy ; Charu Aggarwal ; Zhi-Hua Zhou ; Peng Cui. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 532-539 (Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015).
@inproceedings{143a93e837274dbca8a4714097dfb504,
title = "Efficient and Time Scale-Invariant Detection of Correlated Activity in Communication Networks",
abstract = "In many real-world networks, interactions between entities are observed at specific moments in continuous time, such as email, SMS messaging, and IP traffic. The majority of methods for analyzing such data first aggregate communication over designated time blocks, resulting in one or more discrete time series, to which existing tools can be applied. However, regardless of how the block lengths are chosen, discretizing time inherently introduces information loss and biases analysis towards patterns occurring at the designated time scale, effects which can be especially pronounced in networks with a high degree of temporal variability. Due to this, there has been increasing interest in using stochastic point processes to model network activity. We present a novel approach based on such models to detect times and sets of entities with temporally correlated recent activity. We develop efficient algorithms and compare our approach to existing and baseline methods through experiments on synthetic and real-world data.",
keywords = "anomaly detection, dynamic networks, graph algorithms, stochastic models, streaming, temporal correlation",
author = "Brian Thompson and James Abello",
year = "2016",
month = "1",
day = "29",
doi = "https://doi.org/10.1109/ICDMW.2015.24",
language = "English (US)",
series = "Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "532--539",
editor = "Xindong Wu and Alexander Tuzhilin and Hui Xiong and Dy, {Jennifer G.} and Charu Aggarwal and Zhi-Hua Zhou and Peng Cui",
booktitle = "Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015",
address = "United States",

}

Thompson, B & Abello, J 2016, Efficient and Time Scale-Invariant Detection of Correlated Activity in Communication Networks. in X Wu, A Tuzhilin, H Xiong, JG Dy, C Aggarwal, Z-H Zhou & P Cui (eds), Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015., 7395714, Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015, Institute of Electrical and Electronics Engineers Inc., pp. 532-539, 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015, Atlantic City, United States, 11/14/15. https://doi.org/10.1109/ICDMW.2015.24

Efficient and Time Scale-Invariant Detection of Correlated Activity in Communication Networks. / Thompson, Brian; Abello, James.

Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015. ed. / Xindong Wu; Alexander Tuzhilin; Hui Xiong; Jennifer G. Dy; Charu Aggarwal; Zhi-Hua Zhou; Peng Cui. Institute of Electrical and Electronics Engineers Inc., 2016. p. 532-539 7395714 (Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Efficient and Time Scale-Invariant Detection of Correlated Activity in Communication Networks

AU - Thompson, Brian

AU - Abello, James

PY - 2016/1/29

Y1 - 2016/1/29

N2 - In many real-world networks, interactions between entities are observed at specific moments in continuous time, such as email, SMS messaging, and IP traffic. The majority of methods for analyzing such data first aggregate communication over designated time blocks, resulting in one or more discrete time series, to which existing tools can be applied. However, regardless of how the block lengths are chosen, discretizing time inherently introduces information loss and biases analysis towards patterns occurring at the designated time scale, effects which can be especially pronounced in networks with a high degree of temporal variability. Due to this, there has been increasing interest in using stochastic point processes to model network activity. We present a novel approach based on such models to detect times and sets of entities with temporally correlated recent activity. We develop efficient algorithms and compare our approach to existing and baseline methods through experiments on synthetic and real-world data.

AB - In many real-world networks, interactions between entities are observed at specific moments in continuous time, such as email, SMS messaging, and IP traffic. The majority of methods for analyzing such data first aggregate communication over designated time blocks, resulting in one or more discrete time series, to which existing tools can be applied. However, regardless of how the block lengths are chosen, discretizing time inherently introduces information loss and biases analysis towards patterns occurring at the designated time scale, effects which can be especially pronounced in networks with a high degree of temporal variability. Due to this, there has been increasing interest in using stochastic point processes to model network activity. We present a novel approach based on such models to detect times and sets of entities with temporally correlated recent activity. We develop efficient algorithms and compare our approach to existing and baseline methods through experiments on synthetic and real-world data.

KW - anomaly detection

KW - dynamic networks

KW - graph algorithms

KW - stochastic models

KW - streaming

KW - temporal correlation

UR - http://www.scopus.com/inward/record.url?scp=84964793141&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964793141&partnerID=8YFLogxK

U2 - https://doi.org/10.1109/ICDMW.2015.24

DO - https://doi.org/10.1109/ICDMW.2015.24

M3 - Conference contribution

T3 - Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015

SP - 532

EP - 539

BT - Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015

A2 - Wu, Xindong

A2 - Tuzhilin, Alexander

A2 - Xiong, Hui

A2 - Dy, Jennifer G.

A2 - Aggarwal, Charu

A2 - Zhou, Zhi-Hua

A2 - Cui, Peng

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Thompson B, Abello J. Efficient and Time Scale-Invariant Detection of Correlated Activity in Communication Networks. In Wu X, Tuzhilin A, Xiong H, Dy JG, Aggarwal C, Zhou Z-H, Cui P, editors, Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015. Institute of Electrical and Electronics Engineers Inc. 2016. p. 532-539. 7395714. (Proceedings - 15th IEEE International Conference on Data Mining Workshop, ICDMW 2015). https://doi.org/10.1109/ICDMW.2015.24