Skip to content

Start documentation section for Butterfly #460

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
SoryRawyer wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Start documentation section for Butterfly #460

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4b152ce
start documentation section for Butterfly
SoryRawyer Jun 2, 2025
ff2fd38
Apply suggestions from code review
SoryRawyer Jan 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions docs/technical-reference/architecture-before-4.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ This architecture provides a good separation of concerns (data vs. UI); allows t

The server-side (back-end) part of OpenRefine is implemented in Java as one single servlet which is executed by the [Jetty](http://jetty.codehaus.org/jetty/) web server and servlet container. The use of Java strikes a balance between performance and portability across operating systems (there is very little OS-specific code and has mostly to do with starting the application).

The functional extensibility of OpenRefine is provided by a fork of the [SIMILE Butterfly](https://github.com/OpenRefine/simile-butterfly) modular web application framework. With this framework, extensions are able to provide new functionality both in the
server- and client-side. A [list of known extensions](https://openrefine.org/extensions) is maintained on our website and we have [specific documentation for extension developers](technical-reference/writing-extensions.md).

The client-side part of OpenRefine is implemented in HTML, CSS and plain Javascript. It primariy uses the following libraries:
* [jQuery](http://jquery.com/)
* [Wikimedia's jQuery.i18n](https://github.com/wikimedia/jquery.i18n)
Expand All @@ -26,6 +23,16 @@ Those are fetched at build time via [Apache Maven](https://maven.apache.org/).
The data storage and processing architecture is being transformed. Up to version 3.x, OpenRefine uses an in-memory storage, where the entire project grid is loaded in the Java heap, with operations mutating that state. From 4.x on, OpenRefine uses a
different architecture, where data is stored on disk by default and cached in memory if the project is small enough.

### Butterfly {#butterfly}
The functional extensibility of OpenRefine is provided by a fork of the [MIT SIMILE Butterfly](https://github.com/OpenRefine/simile-butterfly) modular web application framework. With this framework, extensions are able to provide new functionality both in the
server- and client-side. A [list of known extensions](https://openrefine.org/extensions) is maintained on our website and we have [specific documentation for extension developers](technical-reference/writing-extensions.md).

Butterfly organizes back-end functionality using a collection of modules. The only module exposed by core OpenRefine (not including the bundled extensions) is the `core` module, which provides extension points for other modules to hook into. Butterfly modules can declare their dependencies using the `requires=` property in the `module/MOD-INFO/module.properties file.

Each module uses a JavaScript file named `controller.js` to connect different parts of the backend. Butterfly uses a JavaScript interpreter to run this and other provided JavaScript files to configure the back-end. Extensions rely on `controller.js`, and specifically an `init` method within that file, to leverage the aforementioned extension points.
Copy link
Member

@tfmorris tfmorris Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth explaining how the Javascript/Java "glue" works here (or at least point to the extension docs).

More generally some of the non-obvious conceptual stuff should receive a significant portion of the attention.


While Butterfly does allow for modules to manage their own API endpoints, extension modules typically register their endpoints through the form of commands. HTTP requests for those commands are then dispatched by the `core` module.
Copy link
Member

@tfmorris tfmorris Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True for HTTP commands, but there are many other extension registries for importers, exporters, GREL functions, etc.


## Server-side architecture {#server-side-architecture}

OpenRefine's server-side is written entirely in Java (`main/src/`) and its entry point is the Java servlet `com.google.refine.RefineServlet`. By default, the servlet is hosted in the lightweight Jetty web server instantiated by `server/src/com.google.refine.Refine`. Note that the server class itself is under `server/src/`, not `main/src/`; this separation leaves the possibility of hosting `RefineServlet` in a different servlet container.
Expand Down