The Atom Syndication Database (Harmonizing bookmarks structure)

[sjehuda]

Good day, everyone!

I would want to have a discussion of standardizing bookmark databases/structures, including of extending them.

Projects with a common purpose

• buku
• omnom https://github.com/asciimoo/omnom @asciimoo
• Blasta https://git.xmpp-it.net/sch/Blasta
• linkding https://github.com/sissbruecker/linkding @sissbruecker
• Mynij https://nlnet.nl/project/Mynij/
• Postmill https://postmill.xyz/
• Shaarli https://github.com/shaarli/Shaarli @shaarli
• RBMenu https://github.com/DevHyperCoder/rbmenu
• Postmarks https://github.com/ckolderup/postmarks @ckolderup
• and many others...

---

What are the most essential elements that bookmark databases should include?

[asciimoo]

Interesting idea. Could you explain why would it be beneficial to the projects/users?
The only thing that comes to my mind is easier migration between services, but it can be quickly solved by simple import/export command line tools and/or APIs.

[sjehuda]

Could you explain why would it be beneficial to the projects/users?

Of course.

Yes, that would be beneficial for migrating between software and services.

Easier sharing of data from any mean and software, regardles to the brand.

Issue

This is an issue that has happened some years ago: Some browser vendor has changed its database structure by removing the description field, and consequently all descriptions of databases were removed. Some people who did not expect that change and has lost their changes, were foisted to use that browser for months until they had found a viable alternative system.

Solution

It would also allow and easy integration, synchronization etc. with new services, without compromizing the integrity of bookmark databases.

XMPP

Considering XMPP as a synchronization platform, due to XEP-0060: Publish-Subscribe which allows to store data on XMPP account.

As I intend to work on a buku plugin for the browser Falkon, I further intend to add bookmarks, history and session synchronization for Falkon, and other browsers (e.g. Dillo, Ladybord, Otter Browser etc.), which would be based on a system which is similar to Blasta.

That is to say, to utilize XMPP accounts to synchronize browser session. No need for a new and special API. Everything is already taken care of by XMPP, including privacy.

---

P.S. I did not even standartize Blasta, and since Omnom appears to provide more features than Blasta (e.g. snapshots), then I sense that starting to communicate about it as sooner be better than later.

[sjehuda]

I have added a couple of projects:

• linkding, an annotation system
• Postmil, a publishing platform which is similar to Lemmy.

I have notified project linkding

• Standartize structure of bookmarks sissbruecker/linkding#923

I have announced of my intention to be compatible with XMPP publishing systems such as Libervia and Movim

• https://mail.jabber.org/hyperkitty/list/standards@xmpp.org/thread/QPHHLQKQJBHOE4ZW5IQFVLNXXABH26DW/

I have wrote a post about it at my journal

• https://portal.mozz.us/gemini/woodpeckersnest.space/~schapps/journal/2025-01-06-blasta-libervia-and-movim.gmi
• gemini://woodpeckersnest.space/~schapps/journal/2025-01-06-blasta-libervia-and-movim.gmi

---

Even though I was aware that HTML based annotation system is in fact a publishing system, I did not intend for Blasta to be absolutely compatible with publishing systems, yet the structure, of posts of Libervia and Movim which are based upon Atom Syndication Format, is identical by almost 100% if not by a 100%.

@asciimoo Because your system is compatible with ActivityPub, how close is it to be an optional publishing system?

[asciimoo]

@sjehuda I don't know what is the definition of a publishing system, but omnom is a functional ActivityPub actor. If you follow an Omnom user via ActivityPub, you get the new bookmarks.

[sjehuda]

I define bookmarks management systems as publishing systems. I do intend to add an ActivityPub intreface to Blasta, in addition to its XMPP interface, so that it interoperates with Omnom and Postmarks.

[sjehuda]

I am still thinking of it, and I am thinking, for sometime already, to harmonize the bookmarks database of Blasta (and other projects such as Slixfeed) with the structure of The Atom Syndication Format (RFC 4287).

I think that a database which would conform with The Atom Syndication Format is hthe best that we can have, because The Atom Syndication Format has all the required elements for posts, including "related" links, "enclosures" links.

So, just as with project "linkding/" we can bookmark URI https://omnom.zone/ with a description (i.e. atom:summary), and even further notes of our own (i.e. atom:content), and add URI https://nlnet.nl/project/Omnom/ and URI https://*******.com/asciimoo/omnom as related links; and this is standardized by conforming with The Atom Syndication Format, which is the great advantage.

I intend to do so; and I hope that the developers here would be fond of it.

Internal references.

#484 (comment)

#845

[FrederickGeek8]

Is part of the goal with this idea, @sjehuda, for interoperability between services in terms of "following" users? As someone who has been thinking about these technologies recently, and has been playing about with linkhut, something that been a bit sad is the lack interoperability in terms of following & aggregation.

To address part of @asciimoo's comments, ActivityPub (to my ignorant knowledge) is a different beast in so much as it requires dynamic actions on the part of the server that are not possible with statically generated sites or simple clients. If we are able to standardize on a (auxiliary or otherwise) Atom feed format, then it opens us up to passive listeners or statically generated/transient systems should be able to consume and aggregate data more akin to RSS readers.

Although this is an technical oversimplification and misrepresentation, ActivityPub feels like a Atom/RSS-superset that allows for user interaction that, while it is more powerful, is technically more complex & demanding. However, I think in a bookmark sharing system this doesn't necessarily need to be the case. For example, in linkhut, I can just copy a user's posted link into my personal link feed. All that feature requires is a RSS/Atom feed but without a standardized interface across similar platforms it could be difficult to have bridges across these systems.

In terms of a project like buku, Atom could easily be an export backend. If I export the Atom xml to a path on my webserver (or a static-site host), then I can serve my bookmarks to consumers without needing a full ActivityPub instance. From there, users on other services (or in their RSS/Atom readers) could aggregate what I'm publishing and copy links as they please.

Did I mischaracterize the spirit of this idea @sjehuda ?

[asciimoo]

I've never said that ActivityPub is superior or preferable over RSS/Atom nor vice versa. Both have their advantages and weaknesses, that's why Omnom supports both (and standard JSON REST API as well). Either users who are part of the Fediverse or who prefer RSS readers or just quickly want to grab the results with a curl + jq oneliner combo, can enjoy the output this way.

This is an issue that has happened some years ago: Some browser vendor has changed its database structure by removing the description field, and consequently all descriptions of databases were removed.

No standardization can solve the issue of a data provider deciding to filter/limit the provided data.

Instead of talking about standardization, every system should provide as much data/format as possible and then users can filter it how they want - in my opinion.

[sjehuda]

On Tue, 04 Nov 2025 07:45:43 -0800 Frederick Morlock ***@***.***> wrote: Is part of the goal with this idea, @sjehuda, for interoperability between services in terms of "following" users? As someone who has been thinking about these technologies recently, and has been playing about with [linkhut](https://linkhut.org/), something that been a bit sad is the lack interoperability in terms of following & aggregation.

I intend to explore linkhut.

To address part of @asciimoo's comments, ActivityPub (to my ignorant knowledge) is a different beast in so much as it requires dynamic actions on the part of the server that are not possible with statically generated sites or simple clients. If we are able to standardize on a (auxiliary or otherwise) Atom feed format, then it opens us up to passive listeners or statically generated/transient systems should be able to consume and aggregate data more akin to RSS readers. Although this is an technical oversimplification and misrepresentation, ActivityPub feels like a Atom/RSS-superset that allows for user interaction that, while it is more powerful, is technically more complex & demanding. However, I think in a bookmark sharing system this doesn't _necessarily_ need to be the case. For example, in linkhut, I can just copy a user's posted link into my personal link feed. All that feature requires is a RSS/Atom feed but without a standardized interface across similar platforms it could be difficult to have bridges across these systems.

The Atom Syndication Format has everythig which is required for both publishing and telecommunication. I am currently working on a publishing system which is entirely made of Atom documents. https://journal.woodpeckersnest.eu/ https://git.xmpp-it.net/sch/Rivista And there is also a potential for telecommunication from XMPP (Atom Over XMPP), FTP, HTTP et cetera, due to the fine structure of Atom documents.

**In terms of a project like buku,** Atom could easily be an export backend. If I export the Atom xml to a path on my webserver (or a static-site host), then I can serve my bookmarks to consumers without needing a full ActivityPub instance. From there, users on other services (or in their RSS/Atom readers) could aggregate what I'm publishing and copy links as they please. Did I mischaracterize the spirit of this idea @sjehuda ?

No. Not at al. I think that your descripion fits with my intentions. However, I deem that The Atom Syndication Format (RFC 4278) and also Feed Paging and Archiving (RFC 5005) have the means for best management of bookmarks, bibliographies and other references, and for sharing of bookmarks (e.g. YaCy, Searx, Mwmbl, et cetera), which I intend to implement with "Blasta" (a similar system to linkhut). Therefore, I think that designing the databse inline with Atom documents, as specified in RFC 4287, be best to do.

[sjehuda]

On Tue, 04 Nov 2025 08:24:09 -0800 Adam Tauber ***@***.***> wrote: No standardization can solve the issue of a data provider deciding to filter/limit the provided data. Instead of talking about standardization, every system should provide as much data/format as possible and users can filter it how they want - in my opinion.

I agree. This is more of a recommendation for bookmarks databases. Database design require thinking, so designing a database which would mostly fit to most, should be a good objective, I suppose. It would also be easier to import and export, would it not?

[sjehuda]

Friends.

I have realized a structure of a database with 46 tables which resembles RFC 4287: The Atom Syndication Format.

Please advise.

• Table "ele_entry" and "ele_feed" are representations of entries and feeds;

• Every property which may have multiple instances has a table of its own;

• Tables "aff_*" are to coordinate data of tables of properties which may have multiple instances; and

• Tables "txt_*" are primitive or pointers of primitives or association of primitives (txt_lang is an association of txt_code and txt_locale).

• "aff" means "affiliation" to associate datas of tables; "ele" means "element" of XML elements; and "txt" means "text" which is of XML attributes and primitves.

• There is a peculiar table "txt_lang" which is comprised of "txt_code" and "txt_locale", as an attempt to save storage. It might be preferable to manage all locales inside table "txt_lang" rather than dividing it into two tables.

I made the directives of the database to which I call subscriber.sql

https://github.com/user-attachments/files/24052057/subscriber.sql

[sjehuda]

Yes, that table requires some amount of time to realize, yet that is only done once; because, the table is meant to comply with RFC 4287, with additional tables of each project (e.g. buku), and RFC 4287 provide anything that every bookmarking and publishing systems are expected to have.

The longer the "good" times last, the greater the correction will be.

-- Ron Paul - End The Fed (Chapter 13: The Economic Case).

We do this task once, and we do this task for the longer term; and, I advise to do so sooner than later.

Yes. Indeed. It requires time, and I am working on it for three days, already.

I think, that the element "atom:link, along with its attributes "hreflang" (localization), "type" (MIME-Type), and "rel" (Relation), is significantly advantageous to a great extent; and, I think, that it ought be considered by every bookmarking system software.

The reason for the length of the table is to be able to comply with RFC 4287 while utilizing instances once, and referring to them by column "id", respectively.

Our cause is to eliminate the need tof contemporary spyware which is called "internet browser" (i.e. sipder "web" browser) which entraps people instead of empower people, and I think that the system Omnom is currently the closest to that important cause.

[FrederickGeek8]

I will look more at your proposal later, sjehuda, but I agree with LeXofLeviafan's thoughts on the size of the database. I could probably critique the table design (once I read closer into the "spec") but also, to LeXofLeviafan's point...

…Why are you trying to convert an XML schema into an SQL database anyway? It's meant for network APIs, not for relational DBs 😅

You have proposed a database schema, which seems to be a little counter to what I read your original goals to be (and what I've been pondering). Should we not focus on the external or abstract interfaces to the underlying bookmark data? E.g., an Atom feed for bookmark data must contain this minimal set of attributes attached to some set of Atom tags?

I don't think that database schema -- or whether there actually exists a database -- matters. It probably makes sense to focus more on what data is accessible through a Atom feed, Netscape bookmark export, etc., rather than be prescriptive of how the data is stored on the application side. For example, you proposed 46 disconnected database tables instead of a simple "list of all the attributes that should be attached to an bookmark entry". Attaching an entry to a common consumption format, like Atom, makes it easy consume and migrate (especially over a network).

To me, specifying an ideal "minimum spec" of data attached to a bookmark entry, and perhaps a recommended container format like an Atom-entry, is more in-line with your original goals (and at least what I envisioned). Do you agree? I also could have misunderstood the purpose of proposing a database schema rather than a more abstract or entry-grounded format (like Atom).

Regardless, thank you for putting this together. I will take a closer look later.

[sjehuda]

First of all, I thank you for your encouraging and critique comments;

Second of all, I am a criminal lawyer, not a database specialist; and

Third of all, I am experimenting, and it was only a couple of years ago which I designed an SQLite table, in contrasiction to hacking SQLite tables.

To me, specifying an ideal "minimum spec" of data attached to a bookmark entry, and perhaps a recommended container format like an Atom-entry, is more in-line with your original goals (and at least what I envisioned). Do you agree?

Yes. I agree.

I do tend to assume of my approach to be radical, and it accounts to both RSS databases and activities of multiple people (e.g. both tables "ele_author" and "ele_contributor" are based on the three premitive tables "txt_email", "txt_name", and "txt_uri"; and, tables "ele_published" and "ele_updated" are based on the the same premitive table "txt_date"), so that storage be utilized best with INTEGER (i.e. references), instead of having identical values of type "TEXT".

It is, of course, possible to reduce the amount of tables, by setting columns of elements that appear once in tables "ele_entry" (relevant to buku) and "ele_feed" (relevant to RSS readers).

I will ignore table "ele_feed" and its elements, as buku is not an RSS database, yet.

Reducing these tables.

aff_entry_feed, aff_feed_author, aff_feed_category, aff_feed_contributor, aff_feed_link, ele_feed, ele_icon, ele_generator, ele_logo, ele_subtitle, txt_version

This means, that the amount is 35 (46 - 11).

Intsead of this atom:element table.

id, ele_id_id, ele_title_id, ele_content_id, ele_summary_id, ele_rights_id, ele_published_id, ele_updated_id

And reomving elements atom:author, atom:contributor, atom:id, atom:rights; and attributes "type", "xml:base", and "xm:lang".

Reducing these tables.

aff_entry_author, aff_entry_contributor, aff_entry_source, ele_author, ele_content, ele_contributor, ele_id, ele_published, ele_rights, ele_source, ele_summary, ele_title, ele_updated, txt_base, txt_code, txt_date, txt_email, txt_lang, txt_length, txt_locale, txt_name, txt_src, txt_text, txt_type

This means, that the amount be 16 (40 - 24).

We can have this atom:element table.

ele_entry
---------
id, title, content, summary, published, updated

And further centralizing the table of element atom:link.

Intsead of this atom:link table.

id, txt_uri_id, txt_text_id, txt_length_id, txt_rel_id, txt_mimetype_id, txt_lang_id

Eventually, the amount of tables be concluded into 11.

We can have this atom:link table

ele_link
--------
id, txt_uri_id, title, txt_rel_id, txt_mimetype_id

And these associated tables.

aff_entry_category
------------------
id, ele_entry_id, ele_category_id

aff_entry_link
--------------
id, ele_entry_id, ele_link_id

ele_category
------------
id, txt_label_id, txt_scheme_id, txt_term_id

txt_label
---------
id, label

txt_mimetype
------------
id, mimetype

txt_scheme
----------
id, scheme

txt_term
--------
id, term

txt_rel
-------
id, relation

txt_uri
-------
id, uri

We can further reduce it to 10 by merging table txt_uri into table ele_link.

[LeXofLeviafan]

I don't think that database schema -- or whether there actually exists a database -- matters. It probably makes sense to focus more on what data is accessible through a Atom feed, Netscape bookmark export, etc., rather than be prescriptive of how the data is stored on the application side.

This. Implementation matters not; what matters is the API. What needs to be specified is how the data is provided externally, not in what manner it is used/stored internally.

If only there was a standard for how the data should be exchanged… Oh wait – there is! It's called "RFC 4287: The Atom Syndication Format" 😅

(There's even more than one such standard, in fact; though that's beside the point.)

that table requires some amount of time to realize, yet that is only done once

…That's exactly how programming doesn't work 🙃

Once you have something in your project, it's your responsibility to maintain for as long as it still exists there.

I am […] not a database specialist

…I'm not surprised at all, somehow. 😅

Again, you're going about this wrong. Even if you were to come up with a DB design that was sensible to implement, who's actually going to do it? Different kinds of projects will care about different kinds of data – the rest is simply redundant. And either way, if such a project needs a DB, chances are it already has one – designed by its author and very much unlikely to be even remotely similar to your design (at the very least because different people would have different approaches to DB design; as well as having different needs for those DBs). Why would they want to change that design on your say-so? In the end, it will only add them more work to do, without achieving any benefit by itself – since actually making use of such data would need to be implemented separately, one way or another.

…In fact, I can only think of one reason why anyone would want to have identical DB designs between various unrelated projects; i.e. if someone (you?) wanted to access those DBs externally, with a software of their own making (and to access it directly, instead of using an API/export functionality of the project in question) – thus, having identical DBs would make it much easier to accomplish. Problem is, if that's indeed the case, then the burden of supporting all those DBs should fall to you; as no one else would actually benefit from changing their DB designs to fit yours (even if it was a reasonable design in the first place).

[sjehuda]

I am mostly impressed by the element "atom:link, because it can specify types of links (related, alternate, photo, video, avatar et cetera.); so, I deem it helpful with many software.

For instance, setting attribute rel="iframe" and rel="video" is helpful (in a reStructuredText and Atom publication platform which I am currently working on), as it allows to set software (XSLT stylesheet) to realize those atom:link elements as xhtml:iframe or xhtml:video, instead of manually write HTML code; and

Attribute rel="feed" directes the server to add resulted atom:entry elements of the given URI as "related articles" similarly to "Openring".

Hence, the practice of atom:link has a significant importance to any software of RSS and bookmarks.

So, I am mostly focused at atom:link more of the entire RFC 4287 specification.

I do think that it would be useful to have a table which resembles elements of RFC4287, as it would encourage other people to explore this useful specification.