AI News, Graph-based machine learning: Part 2
- On Tuesday, June 5, 2018
- By Read More
Graph-based machine learning: Part 2
In my previous post, we discussed the foundation of community detection using modularity optimization.
Two great properties emerge from this approach These are important points to highlight as they make distributed computation a prime candidate for this memory problem.
The best way to do this at scale (when you don’t know where the information ultimately is on disk) is by using distributed transactions (aka passing messages).
In the context of community detection, each node sends a message to its neighbors with content along the lines of: “Hey I’m your friendly neighbor Node 3 from Community 12” If you’ve ever worked with graphs you’re likely to be very familiar with the concepts of vertices and edges.
Independently, and in parallel, each node waits for its turn to reduce all its messages into a coherent local sum of weighted edges, and make a decision based on the local modularity deltaQ of each neighboring community.
This is done by creating a new graph with a new set of nodes (corresponding to each community) and edges being inferred from the edges during the previous computation (e.g.
Note that additional information, say the internal coherence within a community, can be propagated in a similar fashion to the condensed node and provide valuable information.
Finally, here we have a fully functional procedure to perform modularity optimization on graphs of ridiculously large size, assuming we have enough computers to store all the information on disk.
The intermediary step should therefore be saved for further investigation as they likely yield valuable information on the structural complexity of the data.
This change of scales, previously limited by RAM, opens exciting perspectives as the self modular structure of complex systems have been shown to hold crucial information to understanding their nature.
Being an unsupervised learning technique and an initial starting point for a lot of analysis, the low barriers of entry make this approach applicable to a wide range of datasets.
- On Thursday, March 21, 2019
24. Community Detection in Graphs - Motivation | Stanford University
Copyright Disclaimer Under Section 107 of the Copyright Act 1976, allowance is made for "FAIR USE" for purposes such as criticism, comment, news reporting, ...
Example to illustrate the calculation of Edge Betweenness using BFS
Introduction to SNA. Lecture 5. Network communities.
Cohesive subgroups. Graph cliques. Network communities. Graph partitioning. Modularity. Edge Betweenness. Spectral partitioning. Modularity maximization.
Gephi Modularity Tutorial
A quick tutorial on how to use gephi's modularity feature to detect communities and color code them in graphs.
Using Gephi to visualise and understand communities
Quick video to show how you can use Gephi (a free graph visualisation tool) to quickly visualise communities in a network and use some of the features in Gephi ...
My PhD thesis in ICTEAM: Detecting Communities
In this short video, researcher Arnaud Browet gives an overview of his PhD thesis «Algorithms for community and role detection in networks», which was ...
Network Clustering & Connectedness
Follow along with the course eBook: See the full course: The way in which a network is connected plays a .
Detecting Communities in a Network Using Girvan Newman Algorithm (Python Implementation)
Implementation using Networkx package of Python.
Network Analysis. Lecture 8. Network communitites
Cohesive subgroups. Graph cliques, k-plexes, k-cores. Network communities. Vertex similarity matrix. Similarity based clustering. Agglomerative clustering.
GTAC 2014: Impact of Community Structure on SAT Solver Performance
Zack Newsham (University of Waterloo) Modern CDCL SAT solvers routinely solve very large in- dustrial SAT instances in relatively short periods of time.