Skip to content

Commit 68b85c3

Browse files
Jonathan D.A. Jewellclaude
Jonathan D.A. Jewell
and
claude
committed
feat: integrate Lithoglyph core-zig with Zig FFI
Replaces placeholder implementations with real Lithoglyph database calls. **Changes:** 1. **build.zig**: Added Lithoglyph core-zig as module dependency - Reads LITHOGLYPH_PATH env var (defaults to relative path) - Imports bridge.zig from Lithoglyph core 2. **src/main.zig**: Integrated real Lithoglyph API - Import lithoglyph module (FdbDb, FdbTxn, types) - Removed placeholder Database/Transaction structs - Updated all 9 NIF functions to call Lithoglyph: * formdb_nif_db_open → lithoglyph.fdb_db_open * formdb_nif_db_close → lithoglyph.fdb_db_close * formdb_nif_txn_begin → lithoglyph.fdb_txn_begin * formdb_nif_txn_commit → lithoglyph.fdb_txn_commit * formdb_nif_txn_abort → lithoglyph.fdb_txn_abort * formdb_nif_apply → lithoglyph.fdb_apply - Added proper error handling with FdbBlob error messages - Extract provenance data from FdbResult 3. **README.md**: Complete build and integration documentation - Prerequisites (Zig, Lithoglyph, Erlang) - Build commands and environment variables - API reference for all 9 functions - Lithoglyph integration examples - CBOR encoding documentation **Status:** - ✅ Idris2 ABI with formal proofs - ✅ Zig FFI structure - ✅ Lithoglyph core-zig integration - ⏳ Schema/journal retrieval (placeholder - TODO) - ⏳ CBOR result parsing (basic - TODO) **Next Steps:** - Build and test NIF library - Wire up Gleam client - Implement schema/journal retrieval - Add CBOR result parsing Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1 parent 8b1e231 commit 68b85c3

File tree

3 files changed

+258
-106
lines changed

3 files changed

+258
-106
lines changed

‎server/ffi/zig/README.md‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
# Glyphbase Zig FFI - Lithoglyph Integration
2+
3+
This directory contains the Zig FFI implementation that integrates Glyphbase with the Lithoglyph database.
4+
5+
## Architecture
6+
7+
Following the **hyperpolymath ABI/FFI Universal Standard**:
8+
9+
- **ABI Layer**: `../../../src/abi/*.idr` (Idris2 with formal proofs)
10+
- **FFI Layer**: This directory (Zig with C-compatible exports)
11+
- **Database Engine**: Lithoglyph core-zig (imported as dependency)
12+
13+
## Building
14+
15+
### Prerequisites
16+
17+
1. **Zig** (0.13.0 or later)
18+
2. **Lithoglyph repository** cloned at `~/Documents/hyperpolymath-repos/lithoglyph`
19+
3. **Erlang/OTP** (for NIF headers)
20+
21+
### Environment Variables
22+
23+
```bash
24+
# Optional: Override Lithoglyph path
25+
export LITHOGLYPH_PATH=~/Documents/hyperpolymath-repos/lithoglyph/formdb/database/core-zig
26+
27+
# Optional: Override ERTS include directory
28+
export ERTS_INCLUDE_DIR=/usr/lib/erlang/usr/include
29+
```
30+
31+
### Build Commands
32+
33+
```bash
34+
# Build the NIF shared library
35+
zig build
36+
37+
# Output: ../../priv/libformdb_nif.so (Linux)
38+
# ../../priv/libformdb_nif.dylib (macOS)
39+
# ../../priv/formdb_nif.dll (Windows)
40+
41+
# Run unit tests
42+
zig build test
43+
44+
# Run integration tests
45+
zig build test-integration
46+
```
47+
48+
## Integration with Gleam
49+
50+
The compiled NIF library is loaded by the Gleam server via `src/formdb/nif_ffi.gleam`:
51+
52+
```gleam
53+
@external(erlang, "formdb_nif", "db_open")
54+
pub fn nif_db_open(path: BitArray) -> DbHandle
55+
```
56+
57+
## API Functions
58+
59+
All functions are exported with C calling convention:
60+
61+
| Function | Description |
62+
|----------|-------------|
63+
| `formdb_nif_version` | Get NIF version (0.1.0) |
64+
| `formdb_nif_db_open` | Open database connection |
65+
| `formdb_nif_db_close` | Close database connection |
66+
| `formdb_nif_txn_begin` | Begin transaction (read-only or read-write) |
67+
| `formdb_nif_txn_commit` | Commit transaction |
68+
| `formdb_nif_txn_abort` | Abort transaction |
69+
| `formdb_nif_apply` | Apply CBOR-encoded operation |
70+
| `formdb_nif_schema` | Get database schema (CBOR-encoded) |
71+
| `formdb_nif_journal` | Get journal entries since timestamp |
72+
73+
## Lithoglyph Core Integration
74+
75+
This FFI wraps the Lithoglyph core-zig library (`lithoglyph/formdb/database/core-zig`):
76+
77+
```zig
78+
const lithoglyph = @import("lithoglyph");
79+
80+
// Use Lithoglyph types
81+
const FdbDb = lithoglyph.FdbDb;
82+
const FdbTxn = lithoglyph.FdbTxn;
83+
const FdbStatus = lithoglyph.types.FdbStatus;
84+
85+
// Call Lithoglyph functions
86+
const status = lithoglyph.fdb_db_open(
87+
path.ptr,
88+
path.len,
89+
null, // options
90+
0, // options_len
91+
&out_db,
92+
&out_err,
93+
);
94+
```
95+
96+
## Memory Management
97+
98+
- **Allocator**: Uses `std.heap.GeneralPurposeAllocator` for all allocations
99+
- **Handle Management**: Lithoglyph core-zig maintains handle registries
100+
- **Error Blobs**: CBOR-encoded error messages allocated by Lithoglyph
101+
- **Provenance Data**: Returned as CBOR blobs from operations
102+
103+
## Error Handling
104+
105+
All functions return status codes or NULL pointers on error:
106+
107+
```zig
108+
// Success: returns non-null pointer
109+
const db = formdb_nif_db_open("/path/to/database.ldb");
110+
111+
// Error: returns null
112+
if (db == null) {
113+
// Error logged via std.log.err
114+
}
115+
```
116+
117+
## CBOR Encoding
118+
119+
Operations, results, and provenance use CBOR encoding:
120+
121+
- **Operations**: `{"op": "insert", "collection": "docs", "data": {...}}`
122+
- **Results**: `{"status": "ok", "doc_id": 123}`
123+
- **Provenance**: `{"actor": "user", "timestamp": "...", "rationale": "..."}`
124+
125+
## TODO
126+
127+
- [ ] Implement schema retrieval (currently returns empty map)
128+
- [ ] Implement journal retrieval (currently returns empty array)
129+
- [ ] Add CBOR parsing for extracting block IDs from results
130+
- [ ] Add comprehensive error handling with detailed error messages
131+
- [ ] Add performance benchmarks
132+
- [ ] Add fuzzing tests
133+
134+
## See Also
135+
136+
- [ABI Documentation](../../../src/abi/README.md) - Idris2 ABI with proofs
137+
- [Lithoglyph Core](~/Documents/hyperpolymath-repos/lithoglyph/formdb/database/core-zig) - Database engine
138+
- [Server Integration](../../README.md) - Gleam server using this NIF

‎server/ffi/zig/build.zig‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,19 @@ pub fn build(b: *std.Build) void {
2222
) catch "/usr/lib/erlang/usr/include";
2323
lib.addIncludePath(.{ .cwd_relative = erts_include });
2424

25+
// Add Lithoglyph core-zig as a module dependency
26+
const lithoglyph_path = std.process.getEnvVarOwned(
27+
b.allocator,
28+
"LITHOGLYPH_PATH"
29+
) catch "../../../../lithoglyph/formdb/database/core-zig";
30+
31+
const lithoglyph_core = b.addModule("lithoglyph", .{
32+
.root_source_file = .{ .cwd_relative =
33+
b.pathJoin(&.{lithoglyph_path, "src/bridge.zig"})
34+
},
35+
});
36+
lib.root_module.addImport("lithoglyph", lithoglyph_core);
37+
2538
// Install to priv directory for Erlang to find
2639
const install_artifact = b.addInstallArtifact(lib, .{
2740
.dest_dir = .{

0 commit comments

Comments
 (0)