Survey · 2016 IEEE International Conference on Big Data (Big Data)
A Survey on Semantic Web and Big Data Technologies for Social Network Analysis
Summary
At a Glance
Social network analysis (SNA) needs two things at once: integrating heterogeneous social data into one consistent graph, and processing that graph at scale. The survey maps the technologies for both.
RDF, RDFS, OWL and SPARQL give social data a shared, machine-readable graph model.
Parallel frameworks (Hadoop, Spark, Pregel-style systems) and NoSQL stores make large graphs tractable.
Distributed triplestores that run SPARQL as parallel jobs.
I · Introduction
Why Social Network Analysis Is Hard
- A social network is a graph: nodes are real-world entities, edges their relations. SNA finds hidden connections and characterizes the structure, mostly with graph algorithms.
- Typical questions: strategic positions, communities, influence, homophily and transitivity, longest distances.
- Many graph mining problems, e.g. partitioning a graph into subgraphs, are NP-hard, so systems use heuristics and keep data distributed.
- Social data grows fast in size, variety and complexity, and is mostly unstructured.
Process very large datasets in reasonable time → parallel frameworks and graph databases.
Merge distinct datasets into a larger, semantically consistent one → RDF and ontologies.
II · Semantic Web Technologies
The Semantic Web Toolkit
Graph data model. A statement is a triple <subject, predicate, object>: a labelled, directed edge.
Vocabulary to describe classes and properties, i.e. relationships between classes.
Richer primitives than RDFS for machines to reason over classes and properties.
Pattern-matching query language for RDF graphs; all SNA algorithms can be written in it (Lee et al.).
Newer SPARQL features matter for SNA: property paths for path queries (Kostylev et al.) and a proposed recursion operator (Reutter et al.).
II · Fig. 1 · Interactive
RDF Lab: Query the Graph of Fig. 1
The seven triples of the paper's example graph, matched in your browser. Highlighted: the edges and nodes used by at least one solution.
II.A · TripleStores
Triplestores: Databases for RDF
RDF is a graph data model, SPARQL a graph query language, and a triplestore a graph database that exposes a SPARQL endpoint. Performance matters for real-time use; FEASIBLE (Saleem et al.) benchmarks them.
Jena TDB, RDF4J (formerly Sesame), RDFSuite, AllegroGraph, RDF-3X, Hexastore, RDFox
Virtuoso, EAGRE, TriAD, 4Store, TripleRush
II.B–C · RDF Vocabularies, Prototypes and Frameworks
Vocabularies and Prototype Systems
- FOAF: user profiles and friendships; merges personal data from several sources.
- SKOS: sharing and linking concepts and their relations.
- SIOC: activities of online communities (blogs, forums, wikis); extensions SIOCT and ActOnto.
- MOAT: the meaning of a tag, defined collaboratively.
- SemSNI: visits and private messages between members.
- Also SCOT, RELATIONSHIP, MUTO, AclOnto (access control), InterestOnto (tagging), OntoSNA.
- Flink (Mika): extracts, aggregates and visualizes social networks with Semantic Web reasoning.
- LinkProbe: predicts links between members.
- RDF graph SNA (Opuszko and Ruhland): better link prediction with a semantic similarity measure.
- SemTagP: detects communities in RDF graphs.
- OBDA (Calvanese et al.): answers SPARQL over RDF graphs via ontologies.
- Similar Twitter accounts with Virtuoso and FOAF (Razis et al.).
III · Big Data Architectures
Big Data: Five Vs
For graph processing: MapReduce, Hadoop, Spark, Flink, Pregel-style systems.
To store and access (RDF) data: tabular stores and graph databases. Schema-free, no joins, scale horizontally.
III · Table I · Interactive
What Do Social Platforms Run On?
Click a technology to see which services use it, or a service to see its stack.
MongoDB and Neo4j appear most often (5 of 12 services each), followed by Cassandra and HBase (4 each).
Bars: number of the 12 services in Table I that list the technology. Store types added for orientation.
III.A · Parallel Processing Frameworks
From MapReduce to Spark
Google's data-parallel model; Hadoop is the open-source implementation with HDFS, replicated blocks and fault tolerance. Good for batch, weak for interactive or real-time queries. Twitter's "Who to Follow" runs on Hadoop and Pig.
Resilient Distributed Datasets keep data in memory and minimize disk access; iterative computations become easy. Spark processes distributed RDF queries more efficiently than Hadoop (HAQWA).
Pig Latin (Yahoo), HiveQL (Facebook, Netflix) and LINQ for Dryad. SPARQL can be translated to MapReduce jobs; Presto scales better than Hive for RDF.
III.A · Interactive
Why Iterative Graph Jobs Hurt on Hadoop
A toy model with made-up unit costs, not a measurement. It follows the survey's argument: Hadoop reads from and writes to HDFS in every iteration, while Spark keeps intermediate results in memory.
III.A · Graph Processing Systems · Interactive
Graph Processing Systems
Pegasus turns graph mining into MapReduce jobs; Pregel is vertex-centric and in-memory for iterative graph work, with Giraph as its open-source Hadoop-based version; GraphX extends Spark RDDs; GraphLab uses shared memory. Hover a bar.
- GraphX is best for iterative operations such as connected components; Urika wins non-iterative ones such as degree distribution (Lim et al.).
- Hadoop is the worst platform for multi-iteration algorithms (Guo et al.), who also warn that dataset, algorithm and platform diversity make such comparisons hard.
III.B · NoSQL Database Related Technologies
Tabular Stores and Graph Databases
HBase (BigTable-like, real-time random access; used by eBay, Yahoo, Facebook), Cassandra (from Facebook; CumulusRDF builds an RDF store on it), Accumulo, MongoDB. For Giraph, HDFS and Hive storage beat HBase (Barnawi et al.).
Best for complex, densely connected data: nodes link directly, query time does not depend on graph size. RDF triplestores, Neo4j, DEX, InfiniteGraph, HyperGraphDB, FlockDB, OrientDB, Titan.
Of Neo4j, FlockDB, OrientDB, InfoGrid and AllegroGraph on SNA workloads, Neo4j performs best (Mathew and Kumar).
Cypher, Gremlin, GraphQL, GraphLog, QGraph and SNQL: performance on SNA depends heavily on them.
IV · Future Research Directions · Interactive
Open Problems
- Privacy and trust: social data is personal; preserving privacy is still open.
- Partial graphs: services do not hand out the whole graph, so algorithms must work on limited data.
- Profile matching of one user across several networks.
- Scalable RDF processing: single machines are not enough, and Hadoop was not designed for RDF or iterative graph work, so systems are moving to Spark (Facebook ported from Hive to Spark).
Distributed RDF systems named in Section IV, by platform. Click a bar.
V · Conclusion
Conclusion
- Computing measures such as degree centrality or clustering coefficient on today's networks requires big data tools.
- Big data is not only size but diversity: heterogeneous formats and sources, decentralized control.
- The right platform depends on the task: Hadoop for batch jobs on very large data, Spark for interactive jobs and real-time queries.
- Ontologies integrate schema-less, heterogeneous data; SPARQL queries must become parallel jobs, and decentralized triplestores are the key to large-scale RDF.
V · Conclusion
Check Questions
- What are the two big challenges of SNA named by the survey, and which technology family addresses each?
- Write the triple for "sercan knows omer" and explain its parts.
- Why are iterative graph algorithms slow on Hadoop MapReduce?
- When would you choose a graph database over a tabular NoSQL store?
- Name two open research problems in SNA from Section IV.
Show answers
- Processing very large data in reasonable time (Big Data) and integrating distinct datasets consistently (Semantic Web).
- <:sercan, :knows, :omer>: subject, predicate (the labelled edge) and object.
- Each iteration reads its input from HDFS and writes its result back; many map and reduce jobs are needed.
- For complex, densely connected data with path or neighbourhood queries, since nodes link directly and query time does not grow with graph size.
- Any two of: privacy and trust, analysis on partial graphs, matching profiles across networks, scalable RDF processing.
References
Selected References
- Lee, S., Sukumar, S. R., Lim, S.-H. (2015). Graph mining meets the semantic web. ICDE Workshops, 53–58.
- Mika, P. (2005). Flink: Semantic web technology for the extraction and analysis of social networks. J. Web Semantics, 3(2), 211–223.
- Bello-Orgaz, G., Jung, J. J., Camacho, D. (2016). Social big data: Recent achievements and new challenges. Information Fusion, 28, 45–59.
- Xin, R. S. et al. (2013). GraphX: A resilient distributed graph system on Spark. GRADES Workshop.
- Batarfi, O. et al. (2015). Large scale graph processing systems: Survey and an experimental evaluation. Cluster Computing, 18(3), 1189–1213.
- Bizer, C., Schultz, A. (2009). The Berlin SPARQL benchmark. Int. J. Semantic Web Inf. Syst., 5(2), 1–24.
See the full paper for all 104 references. Charts on these slides are redrawn from the paper's Fig. 1, Table I and text.