Wide-column, graph, and specialised stores
Wide-column stores handle enormous write volumes, graph databases excel at relationships, and specialised engines target time series, search, and vectors.
Wide-column stores
Wide-column stores, such as Apache Cassandra, HBase, and Google Bigtable, organise data into tables of rows identified by a key, where each row can have many, sparse columns. Cassandra rows are grouped by a partition key and sorted within the partition by clustering columns. Queries must usually specify the partition key, so tables are designed around queries.
Log-structured storage
Many write-heavy stores use log-structured merge trees (LSM trees): writes go to an in-memory table and a log, then are flushed to immutable sorted files on disk, which are merged in the background (compaction). Writes are very fast and sequential; reads may check several files, helped by Bloom filters. Cassandra, RocksDB, LevelDB, and HBase use LSM-based storage.
Tunable consistency
Cassandra lets each query choose a consistency level, such as ONE, QUORUM, or ALL replicas, trading latency and availability against consistency. Using QUORUM for both writes and reads with a replication factor of 3 gives strongly consistent reads in normal operation. This flexibility suits workloads like messaging, sensor data, and activity logs at very large scale.
Graph databases
Graph databases store nodes, relationships, and properties on both. Neo4j's property graph model and its query language Cypher express patterns directly: MATCH (a:Person)-[:FRIEND]->(b)-[:FRIEND]->(c) RETURN c finds friends of friends. Traversing relationships is fast because each node points to its neighbours, whereas the same query in SQL needs repeated self-joins.
Graph use cases
Graphs suit highly connected data: social networks, recommendation engines (people who bought this also bought), fraud detection (rings of accounts sharing devices or addresses), network and IT infrastructure maps, knowledge graphs, and access-control hierarchies. RDF triple stores, queried with SPARQL, represent knowledge as subject-predicate-object statements for semantic web and linked-data applications.
Time-series and search engines
Time-series databases, such as InfluxDB, TimescaleDB, and Prometheus, optimise for append-heavy timestamped data like metrics and sensor readings, with compression, retention policies, and time-based aggregation. Search engines, such as Elasticsearch and OpenSearch, built on Apache Lucene's inverted indexes, provide full-text search with relevance ranking, fuzzy matching, and fast filtering over logs and documents.
Vector databases
Machine-learning models turn text, images, and audio into embeddings: lists of hundreds or thousands of numbers where similar items are close together. Vector databases and extensions such as pgvector store embeddings and answer nearest-neighbour queries using approximate indexes like HNSW graphs. They power semantic search and retrieval-augmented generation for AI applications.
Worked example
Fraud detection: accounts, devices, phone numbers, and addresses are nodes; 'used' and 'registered with' are relationships. A query finds accounts connected within three hops to a known fraudulent account through shared devices or addresses. In a graph database this is a short pattern match touching only nearby nodes; in SQL it requires several self-joins over large tables whose cost grows quickly with each hop.
Key takeaways
Wide-column stores such as Cassandra, HBase, and Bigtable use partition keys and clustering columns, designed around queries. LSM trees make writes fast and sequential, with compaction and Bloom filters aiding reads. Cassandra's tunable consistency chooses ONE, QUORUM, or ALL per query. Graph databases such as Neo4j store nodes and relationships, with Cypher patterns making multi-hop traversals fast; RDF stores use SPARQL. Graphs suit social, recommendation, fraud, and knowledge applications. Time-series databases handle metrics; Lucene-based search engines handle full text; vector databases answer nearest-neighbour queries over embeddings for AI.
Flashcard
Rows with many sparse columns
Check your understanding
Why are friends-of-friends-of-friends queries typically faster in a graph database than in a relational one?