|
|
| Line 1: |
Line 1: |
| − | ''Zhou
| + | This a [[Category::Paper]] discussed in Social Media Analysis 10-802 in Spring 2010. |
| − | http://link.aps.org/doi/10.1103/PhysRevE.67.041908
| + | |
| | == Citation == | | == Citation == |
| | | | |
| − | PhysRevE.67.041908,
| + | Neighborhood Formation and Anomaly Detection in Bipartite Graphs, |
| − | title = {Network landscape from a Brownian particle's perspective},
| + | Jimeng Sun, Huiming Qu, Deepayan Chakrabarti, Christos Faloutsos, ICDM 2005 |
| − | author = {Zhou, Haijun },
| |
| − | journal = {Phys. Rev. E},
| |
| − | volume = {67},
| |
| − | number = {4},
| |
| | | | |
| − | == Abstract from the paper ==
| |
| − |
| |
| − | Given a complex biological or social network, how many clusters should it be decomposed into? We define
| |
| − | the distance di, j from node i to node j as the average number of steps a Brownian particle takes to reach j from
| |
| − | i. Node j is a global attractor of i if di, j<di,k for any k of the graph; it is a local attractor of i if j\inEi (the set
| |
| − | of nearest neighbors of i) and di, j<di,l for any l\in Ei . Based on the intuition that each node should have a high
| |
| − | probability to be in the same community as its global (local) attractor on the global ~local! scale, we present a
| |
| − | simple method to uncover a network’s community structure. This method is applied to several real networks
| |
| − | and some discussion on its possible extensions is made.
| |
| | == Online version == | | == Online version == |
| | | | |
| − | [http://www.mpikg-golm.mpg.de/theory/people/zhou/works/PREv67p041908.pdf pdf link to the paper] | + | [http://www.cs.cmu.edu/~deepay/mywww/papers/icdm05.pdf Neighborhood Formation and Anomaly Detection in Bipartite Graphs] |
| − | | |
| − | === Summary ===
| |
| − | "All things are made of atoms — little particles that move around in perpetual motion, attracting each other when they are a little distance apart, but repelling upon being squeezed into one another." - Feynman
| |
| − | | |
| − | | |
| − | | |
| − | In this [[Category::Paper|article]], Zhou proposes Brownian perspective of [[AddressesProblem::Community Detection|community formation in a network]].
| |
| − | | |
| − | * The main idea of this paper is to show how communities are formed due to diffusion like phenomenon in the network.
| |
| − | | |
| − | * The purpose of brownian perspective is to establish the notion of local attractors and global attractor in a network.
| |
| − | | |
| − | * A node that is closely associated with a local attractor would contribute to the stability of the community which in turn determines the tendency of a local attractor to be a global attractor. [[UsesMethod::Netwalk|Custom method]]
| |
| − | | |
| − | * There is a strong resemblance between the structure of global community and local community, whenever the size of network is huge.
| |
| − | | |
| − | | |
| − | The main results given are:
| |
| − | * [[UsesDataset::Football_networks|football fan networks]] 115 nodes and 613 unweighted edges, based on the connection pattern the method divides the network into 15 L communities.
| |
| − | * [[UsesDataset::Karate network|Zachary's karate network]] 34 nodes and 77 weighted edges and it was
| |
| − | * [[UsesDataset::Santa Fe Institute network|scientific collaboration network]] 118 nodes and 200 weighted edges. Divides this network into six communities. L_{3} has stronger direct interaction with community L_{6}.
| |
| − | * [[UsesDataset::Protein interaction network|yeast core of baker's yeast]], the largest of the datasets used in the experiments in this paper. It has been reported that there are 1471 proteins and 2770 unweighted edges. Divides this giant component into 14 G communities and 69 L communities. G-attractor has the major attention in the network, which makes the network vulnerable once that particular protein/attractor is removed, thus leaving the system perturbed.
| |
| − | | |
| − | == Background ==
| |
| − | This paper gives a nice impression of how well a physical process can be reformulated to fit a social science problem. To begin with the brownian perspective, we shall look into the brownian motion to understand the motivation behind this interesting formulation. The brownian motion, has been observed by Brown a botanist for the first time, while he was studying paramecium in the water. He noticed that some of the suspended particles in the solvent moves in a jiggly fashion. Hence, the observation of the motion is called Brownian motion. Later, in 1905 Albert Einstein at ETHZ, studied and explained the mechanics behind this jiggly motion of suspended particles in a fluid. The core findings mentioned in his paper are as following:
| |
| − | | |
| − | * When a solute is dissolved in the solvent that is contained in a semi-permeable (permeable to the solvent only) container then the pressure exerted on the walls of the container is due to solute molecules which is known as osmotic pressure. Earlier this was not the idea according to the concept of "free energy".
| |
| − | | |
| − | * He went on to explain the cause of this osmotic pressure with the help of molecular kinetic theory of heat.
| |
| − | # Seperation between particles is large
| |
| − | # They have random velocities
| |
| − | # Unless they collide they are no attraction forces between them and their position at time is almost mutually independent. Yes, their collisions are elastic.
| |
| − | | |
| − | * He also explained the correlation between diffusion and the irregular movement of these particles. He found that the distribution of displacements after time t is interesting same as that of coincidental or fortuitous error. But, the coefficients in the exponential term are related to diffusion coefficient in his findings.
| |
| − | | |
| − | * Based on his findings, he derived the formula to determine the mean displacement of an atom and insight towards a way to calculate the size of an atom.
| |
| − | | |
| − | == What's the brownian perspective in this context? ==
| |
| − | As we have understood from Einstein's paper that a suspended particle over a time displaces itself going through elastic collisions and by occasionally hitting the walls. Although, it is irregular it has been observed that it obeys the homogeneous liquid environment. In that context, if an intelligent suspended particle lives in a network and does brownian motion for a long time and measured positions after displacement is considered to be nodes of the network.
| |
| − | | |
| − | So, every edge weight is determined by the displacement of the suspended particle. This idea, is nothing but the well known random walk.
| |
| − | | |
| − | Thereby, the community structure is actually determined by the next node visited by the particle. Although it will be interesting to see if all suspended particles would think that this is the community because their motion is mutually independent over a period of time.
| |
| − | | |
| − | For the ease of quantification and lack of knowledge about the network, the displacement or jumping probability of the particle is assumed to be Pij = Aij / Sum-over-all-nearest neighbours Ail. Authors describe it as transfer matrix.
| |
| − | | |
| − | == Observations ==
| |
| − | * On artificial networks – ensemble, competitive results vs Girvan and Newman Proc.
| |
| − | * But, not effective on hierarchical communities O(N^3) for N < =10^3
| |
| − | * Scale free property in real world - Decompose large networks into sub-networks
| |
| − | * Say, Include nearest and next nearest neighbours in a subnet.
| |
| − | * On Sparse Networks - Computationally better when N < 10^3
| |
| − | * Linearly biased brownian motion is considerably superior to those of unbiased and quadratic-biased. (refer to Gitterman E.62 2000)
| |
| | | | |
| | + | == Summary == |
| | + | Physiological studies have suggested that participants in conversation accommodate in dimensions such as style, utterance length, gesture, speaking rate etc. In this paper authors investigate accommodation in twitter. They propose a novel probabilistic framework to compute measures such as stylistic cohesion,stylistic accommodation and stylistic influence and symmetry. |
| | | | |
| | + | == Evaluation == |
| | | | |
| − | == Related Papers ==
| + | They evaluate their methods by asking following 4 questions : |
| − | * Zhou, H., Lipowsky, R.: Network Brownian Motion: A New Method to Measure Vertex-Vertex Proximity and toIdentify Communities and Subcommunities Phys. Rev. E 67 (2003) 041908
| + | - Does NF find out meaningful neighborhoods? |
| − | * Zhou, H.: Distance, dissimilarity index, and network community structure. Phys. Rev. E 67 (2003) 061901
| + | - How close ispproximate NF to exact NF? |
| | + | - Can AD detect injected anomalies? |
| | + | - How much time these methods take to run on graphs of varying sizes? |
| | | | |
| | + | == Discussion == |
| | + | This paper poses two important social problems related to bipartite social graphs and explained how those problems can be solved efficiently using random walks. |
| | | | |
| − | == Study Plan ==
| + | They also claim that the neighborhoods over nodes can represent personalized clusters depending on different perspectives. |
| − | Papers you may want to read to understand this paper.
| |
| | | | |
| − | * Girvan, M., Newman, M. E. J.: Community structure in social and biological networks. Proc. Natl. Acad. Sci. USA. 99 (2002), 7821-7826 [http://pds12.egloos.com/pds/200901/09/95/7821.full.pdf pdf]
| + | During presentation one of the audiences raised question about is anomaly detection in this paper similar to betweenness of edges defined in Kleinber's text as discussed in [[Class Meeting for 10-802 01/26/2010]]. I think they are similar. In the texbook they propose, detecting edges with high betweenness and using them to partition the graph. In this paper they first try to create neighbourhood partitions based on random walk prbabilities and which as a by product gives us nodes and edges with high betweenness value. |
| − | ** [http://en.wikipedia.org/wiki/Community_structure Community Structure]
| |
| − | ** [http://en.wikipedia.org/wiki/Biological_network Biological Network]
| |
| | | | |
| − | * Newman, M.E.J., Girvan, M.: Finding and evaluating community structure in networks. e-print: cond-mat/0308217 [http://arxiv.org/pdf/cond-mat/0308217v1.pdf pdf] | + | == Related papers == |
| | + | There has been a lot of work on anomaly detection in graphs. |
| | + | * The paper by [[RelatedPaper::Moonesinghe and Tan ICTAI06]] finds the clusters of outlier objects by doing random walk on the weighted graph. |
| | + | * The paper by [[RelatedPaper::Aggarwal SIGMOD 2001]] proposes techniques for projecting high dimensional data on lower dimensions to detect outliers. |
| | | | |
| − | * [http://en.wikipedia.org/wiki/Brownian_motion Brownian Motion] | + | == Study plan == |
| | + | * Article:Bipartite graph:[http://en.wikipedia.org/wiki/Bipartite_graph] |
| | + | * Article:Anomaly detection:[http://en.wikipedia.org/wiki/Anomaly_detection] |
| | + | * Paper:Topic sensitive pagerank:[http://dl.acm.org/citation.cfm?id=511513] |
| | + | **Paper:The PageRank Citation Ranking: Bringing Order to the Web:[http://ilpubs.stanford.edu:8090/422/] |
| | + | * Paper:Multilevel k-way Partitioning Scheme for Irregular Graphs:[http://glaros.dtc.umn.edu/gkhome/node/81] |
This a Paper discussed in Social Media Analysis 10-802 in Spring 2010.
Citation
Neighborhood Formation and Anomaly Detection in Bipartite Graphs,
Jimeng Sun, Huiming Qu, Deepayan Chakrabarti, Christos Faloutsos, ICDM 2005
Online version
Neighborhood Formation and Anomaly Detection in Bipartite Graphs
Summary
Physiological studies have suggested that participants in conversation accommodate in dimensions such as style, utterance length, gesture, speaking rate etc. In this paper authors investigate accommodation in twitter. They propose a novel probabilistic framework to compute measures such as stylistic cohesion,stylistic accommodation and stylistic influence and symmetry.
Evaluation
They evaluate their methods by asking following 4 questions :
- Does NF find out meaningful neighborhoods?
- How close ispproximate NF to exact NF?
- Can AD detect injected anomalies?
- How much time these methods take to run on graphs of varying sizes?
Discussion
This paper poses two important social problems related to bipartite social graphs and explained how those problems can be solved efficiently using random walks.
They also claim that the neighborhoods over nodes can represent personalized clusters depending on different perspectives.
During presentation one of the audiences raised question about is anomaly detection in this paper similar to betweenness of edges defined in Kleinber's text as discussed in Class Meeting for 10-802 01/26/2010. I think they are similar. In the texbook they propose, detecting edges with high betweenness and using them to partition the graph. In this paper they first try to create neighbourhood partitions based on random walk prbabilities and which as a by product gives us nodes and edges with high betweenness value.
Related papers
There has been a lot of work on anomaly detection in graphs.
- The paper by Moonesinghe and Tan ICTAI06 finds the clusters of outlier objects by doing random walk on the weighted graph.
- The paper by Aggarwal SIGMOD 2001 proposes techniques for projecting high dimensional data on lower dimensions to detect outliers.
Study plan
- Article:Bipartite graph:[1]
- Article:Anomaly detection:[2]
- Paper:Topic sensitive pagerank:[3]
- Paper:The PageRank Citation Ranking: Bringing Order to the Web:[4]
- Paper:Multilevel k-way Partitioning Scheme for Irregular Graphs:[5]
