Skip to content

Add LazyFs for lazy in-memory files#67

Open
defrex wants to merge 4 commits intovercel-labs:mainfrom
defrex:main
Open

Add LazyFs for lazy in-memory files#67
defrex wants to merge 4 commits intovercel-labs:mainfrom
defrex:main

Conversation

@defrex
Copy link

@defrex defrex commented Jan 29, 2026

I've ben loving just-bash! It's solved a ton of problems for me by enabling filesystem agents without a full sandbox.

However, I'm loading so many files in there that initializing the tool has gotten quite slow, and most of the time the agent doesn't even need the files.

The new LazyFs class has allowed me to quickly initialize the virtual filesystem and only load files that directories on an as-needed bases, which has greatly improved performance.

In addition to the unit tests in the PR, I've integrated this into my agents and run through my evals, etc confirming that this is working great for me.

README Snippet

LazyFs - Lazy-loading filesystem that loads content on-demand via callbacks. Ideal for remote content, API-backed storage, or large datasets where you don't want to load everything upfront:

import { Bash, LazyFs, MountableFs, InMemoryFs } from "just-bash";

const lazyFs = new LazyFs({
  // Called when reading a directory - return entries or null
  listDir: async (dirPath) => {
    // Replace with your API call, e.g.: await fetchDirectoryFromAPI(dirPath)
    return [
      { name: "file.txt", type: "file" as const },
      { name: "subdir", type: "directory" as const },
    ];
  },
  // Called when reading a file - return content or null
  loadFile: async (filePath) => {
    // Replace with your API call, e.g.: await fetchFileFromAPI(filePath)
    return { content: "file content here" };
  },
  allowWrites: true, // default, writes go to in-memory cache
});

// Mount it at a path
const fs = new MountableFs({ base: new InMemoryFs() });
fs.mount("/mnt/remote", lazyFs);

const bash = new Bash({ fs });
await bash.exec("ls /mnt/remote"); // triggers listDir("/")
await bash.exec("cat /mnt/remote/data.txt"); // triggers loadFile("/data.txt")

Content is cached after first access. Writes and deletes stay in memory and shadow the lazy-loaded content.

defrex and others added 3 commits January 28, 2026 17:02
Implements a new filesystem that loads content on-demand via user-provided
loader functions. Content is cached in memory using InMemoryFs as the backing
store. Designed for AI agents needing lazy access to remote or virtual content.

Features:
- Lazy loading of files and directories via listDir/loadFile callbacks
- Caching with separate negative cache for files vs directories
- Write/delete operations shadow lazy content
- Full symlink support with proper resolution
- Implements complete IFileSystem interface

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Document the new lazy-loading filesystem implementation with usage
examples showing listDir/loadFile callbacks and caching behavior.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix TypeScript compilation error in LazyFs documentation example
  by using inline stub values instead of undefined placeholder functions
- Use correct type "directory" instead of "dir" to match LazyDirEntry
- Add .claude/ to gitignore (local Claude Code settings)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vercel
Copy link

vercel bot commented Jan 29, 2026

@defrex is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

When writing files to non-existent paths, parent directories created by
InMemoryFs were invisible to LazyFs operations. Added markParentsModified()
helper to track implicitly created parent directories, and fixed readdir
to handle locally-created directories that don't exist in the lazy source.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@cramforce
Copy link
Contributor

Should OverlayFS just be lazy?

@defrex
Copy link
Author

defrex commented Jan 30, 2026

Should OverlayFS just be lazy?

That seems like a good default for OverlayFS, since it could work the same while skipping unnecessary IO.

Though to clarify, that wouldn't solve my use-case since I specifically need to load data from a database, not the filesystem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants