Refactor to support releases and versioning

[jakebeal]

The interfaces and capabilities of ETL tools are expected to be changing over the life of this project.

Thus, for reproducible execution of ETL processing toolchains, it will be important to be able to identify which version of a piece of software has been executed and which versions of other pieces of software it is intended to be executed against.

One way this might be accomplished is by the standard git / GitHub release tracking system, which gives nice persistent URIs, and which can be mapped onto well-defined versioning approaches such as SemVer. To do this effectively, however, we need each ETL reactor to live in its own repository, so that each one can be separately versioned in a way that appropriately tracks its interface.

[jakebeal]

Copying over discussion from Slack:

Mohammed Eslamid: im okay with this, but from my understanding - you can do that with one repo as well, it’s just a bit more involved.

Mark Weston
@jakebeal the reactors are already versioned, although I can see your argument for repositories if you want true git releases. I have a call now, but will follow up shortly

Jake Beal
In particular, I am wanting to have persistent URIs for releases.
and that will be much easier to do cleanly if we split the reactors
Also, t way one won't have to pull the whole half-gig repository for every instance (edited)

Mark Weston
An example of semantic versioning for an app is here: https://github.com/SD2E/reactors-etl/blob/master/reactors/lcms/lcms-0.1.0/VERSION which flows through to the docker image that gets built and turned into an app on TACC

But, tagging a release and having that flow through to the docker image produced and having that flow through to the app would obviously be very desirable

Want to move this discussion to the git issue? I don’t have much more to say - I support this.