Idea: what about proto de/ser?

[Solverj]

Disclaimer: Sorry if this is not how you want ideas/challenges to be communicated, if this is wrong then how would I propose an idea? I couldn't find anything about this in your contributing guidelines.

Hey, I love your contributions to the open-source world. We're adopting wasmcloud in our organization but we have a problem.

We have to model for both external users and internal users, i.e., internal = WIT, external = proto. And modelling two places doesn't scale well, version-wise.

My wish would be:

1. Model in proto / wit
2. -> wit + proto
3. git push to "A"
4. world.wit (uses A.wit record through deps.toml)
5. further define wit with dependency to the generated records from 2 and 3
6. De/ser thought "A". proto but able to somehow convert to Wit records.
7. Push to some stream as proto

Do you guys see any solutions you know, or is this thought of and avoided? If so, why?

[ydnar]

Hi there! This is an interesting use case.

I could see a WIT → Protobuf generator that helps ease this process. The underlying structures in target language would be different.

That said, serializing WIT records into a confirming protobuf schema seems quite reasonable.

How would this look, ideally, to you?

[Solverj]

I'd like it to be a tool, where you'd write something like:

wit-bindgen-proto --out-dir 'some-folder' --package 'some-package' --wit(wit-recursive, wit-file) 'wit-folder' || 'wit-file' --records-only

The reasoning for it, is because then I could pipe it into buf and generate go|rust|python|whatever files from the proto definitions.

And the reason why I'd like wit-bindgen-proto not to also generate go files, is because I don't want to put everything in one folder, but have the choice to what I want to do with it further, e.g., centralized proto|wit definitions for a whole team. Specifically refered to by repositories / projects.

This would also make it such that you guys wouldn't have to do a lot of work to support this, and the guys at f.ex buf.build could take it further to support wit->proto or proto->wit.

[ydnar]

How do you imagine the generated Go types from WIT would interact with the generated proto types?

[Solverj]

I dont see them interacting. Just that theyre versioned as datamodels from same definition.

Proto for wireformat and contracts between apps external. Wit for internal communication.

[ydnar]

Do you plan to manually copy the data from the WIT types to the Proto types?

To be honest, if that's your plan, I'd recommend just writing a protoc plugin that emits WIT for a given proto file.

Protos have more information than WIT, e.g. field numbers, extensions, etc.