One file. Relationship traversal, vector similarity, BM25 full-text search, durable streams, and ACID transactions in one local engine and one query layer.
Think SQLite, but for connected data you want to query by relationship, semantics, and text.
curl -fsSL https://raw.githubusercontent.com/jeffhajewski/latticedb/main/dist/install.sh | bash
pip install latticedb
npm install @hajewski/latticedb
Vector similarity, full-text search, and graph traversal in a single Cypher query.
-- Find chunks similar to a query, traverse to their document, then to the author MATCH (chunk:Chunk)-[:PART_OF]->(doc:Document)-[:AUTHORED_BY]->(author:Person) WHERE chunk.embedding <=> $query_vector < 0.3 AND doc.content @@ "neural networks" RETURN doc.title, chunk.text, author.name ORDER BY chunk.embedding <=> $query_vector LIMIT 10
Medians for 0.17.0, measured through the public C API with the default settings on an Apple M5. Method, spreads and full tables.
| Operation | LatticeDB | SQLite |
|---|---|---|
| One-row durable commit | 44 µs | 42 µs |
| Inserts, 1,000 per transaction | 332K nodes/s | 1.84M rows/s |
| Lookup by an indexed property, 100K rows | 2.3 µs | 3.0 µs |
| 2-hop count, 100K nodes | 8.8 µs | 8.9 µs |
| 3-hop count, 100K nodes | 21 µs | 29 µs |
| Paths of 1 to 5 hops, 100K nodes | 600 µs | 1,361 µs |
Medians, each database through its own C API: LatticeDB 0.17.0 with its defaults,
SQLite 3.51.0 with WAL and synchronous=FULL, joins on indexed columns
and a recursive CTE for the last row. SQLite is level on short traversals and
several times faster at bulk inserts; LatticeDB is about twice as fast on long
paths. How it was measured.
Other systems' published figures are cited in the docs, apart from these
measurements, with an honest account of what each alternative does better:
:memory: and never touch the diskRead, replace and remove a node's vector, from every binding and from Cypher.
get_vectorremove_vector takes it out of search and keeps the nodeSET n.embedding = $vector stores the node's vectorembedding key in CREATE and MERGEREMOVE n.embedding and SET n.embedding = null remove itNode.get_vector and Node.set_vector are deprecatedThis graph is rendered from the exact creation/query pattern shown below.
These statements build the graph shown below.
CREATE (alice:Person {name: "Alice"}) CREATE (doc:Document {title: "Attention Is All You Need"}) CREATE (c1:Chunk {text: "Self-attention..."}) CREATE (c2:Chunk {text: "Transformer blocks..."}) CREATE (c1)-[:PART_OF]->(doc) CREATE (c2)-[:PART_OF]->(doc) CREATE (doc)-[:AUTHORED_BY]->(alice) MATCH (chunk:Chunk)-[:PART_OF]->(doc:Document)-[:AUTHORED_BY]->(author:Person) RETURN doc.title, chunk.text, author.name;
pip install latticedbfrom latticedb import Database from latticedb.embedding import hash_embed with Database("knowledge.db", create=True, enable_vectors=True, vector_dimensions=128) as db: with db.write() as txn: alice = txn.create_node(labels=["Person"], properties={"name": "Alice"}) doc = txn.create_node(labels=["Document"], properties={"title": "Attention Is All You Need"}) chunk = txn.create_node(labels=["Chunk"], properties={"text": "Self-attention..."}) txn.set_vector(chunk.id, "embedding", hash_embed("transformer", dimensions=128)) txn.create_edge(chunk.id, doc.id, "PART_OF") txn.create_edge(doc.id, alice.id, "AUTHORED_BY") txn.commit() results = db.query(""" MATCH (chunk:Chunk)-[:PART_OF]->(doc:Document)-[:AUTHORED_BY]->(author:Person) WHERE chunk.embedding <=> $query < 0.5 RETURN doc.title, chunk.text, author.name ORDER BY chunk.embedding <=> $query LIMIT 5 """, parameters={"query": hash_embed("attention mechanism", dimensions=128)}) for row in results: print(f"{row['doc.title']} by {row['author.name']}")
npm install @hajewski/latticedbimport { Database } from "@hajewski/latticedb"; import { hashEmbed } from "@hajewski/latticedb/embedding"; const db = new Database("knowledge.db", { create: true, enableVectors: true, vectorDimensions: 128, }); await db.open(); await db.write(async (txn) => { const alice = await txn.createNode({ labels: ["Person"], properties: { name: "Alice" } }); const doc = await txn.createNode({ labels: ["Document"], properties: { title: "Attention Is All You Need" } }); const chunk = await txn.createNode({ labels: ["Chunk"], properties: { text: "Self-attention..." } }); await txn.setVector(chunk.id, "embedding", hashEmbed("transformer", 128)); await txn.createEdge(chunk.id, doc.id, "PART_OF"); await txn.createEdge(doc.id, alice.id, "AUTHORED_BY"); }); const results = await db.query( `MATCH (chunk:Chunk)-[:PART_OF]->(doc:Document)-[:AUTHORED_BY]->(author:Person) WHERE chunk.embedding <=> $query < 0.5 RETURN doc.title, chunk.text, author.name ORDER BY chunk.embedding <=> $query LIMIT 5`, { query: hashEmbed("attention mechanism", 128) } ); for (const row of results.rows) { console.log(`${row["doc.title"]} by ${row["author.name"]}`); } await db.close();