Restore FHIR validation server

[costateixeira]

the FHIR validator server was removed - not sure why, if it was working for the tests.
Perhaps related to the json schema (which is not sufficient for FHIR validation)?

The validator should be restored

[bjungs]

The local FHIR server is still included in the base compose, for testing/dev purposes. However, the version for distribution should be configured to point to a "target" FHIR server to be used as SUT. Since the SUT will perform validations anyway, there is no point in maintaining a separate FHIR server that may or may not be compatible with the target.

[costateixeira]

This doesn't seem to make sense. The testing architecture we have in place includes the validation server. How else would we do validation? Was this discussed?

…

On Wed, Jan 15, 2025 at 9:39 AM Bernardo Nunes ***@***.***> wrote: The local FHIR server is still included in the base compose <https://github.com/imec-int/fhir-itb/blob/main/docker-compose.yml#L92>, for testing/dev purposes. However, the version for distribution <https://github.com/imec-int/fhir-itb/blob/main/dist/docker-compose.yml> should be configured to point to a "target" FHIR server to be used as SUT. Since the SUT will perform validations anyway, there is no point in maintaining a separate FHIR server that may or may not be compatible with the target. — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD3HUUHHN6PVWA7WYDL3GZL2KYNF5AVCNFSM6AAAAABU7G6V7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJRHE2DONZQG4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bjungs]

Validation can be done via the test cases. However, if I remember well, you said that validation was not necessary since the FHIR server itself would perform the validation. And you were right. There is no point of extra validation if the target FHIR server validates the request already. So instead of validating against a harcoded version of a specific implementation of the FHIR spec, we let the SUT itself handle the request and validate the response. This not only results in a more accurate interoperability testing (since we can validate the response directly from the SUT, and not some test-dummy), but also better maintainability on our side as it makes our code completely de-coupled from the SUT implementation.

Nevertheless, the compose files are just that. Docker Compose files.
For our current use-case we did not need the FHIR server so we removed it from the distribution file. However, anyone should feel free to create their own setups with the images we build in this repo. The distribution compose is just what we decided to use for the fhir-thon.

[costateixeira]

Did anyone decide that the FHIR server was not needed? This is not how the system was designed or how the interoperability testing would be done. Did you use or update the architecture for this? Anyway, our setup was the "master" and was broken for a demo, and I only raised the issue because I realized that this was due to these changes.

…

On Wed, Jan 15, 2025 at 11:44 AM Bernardo Nunes ***@***.***> wrote: Validation can be done via the test cases. However, if I remember well, you said that validation was not necessary since the FHIR server itself would perform the validation. And you were right. There is no point of extra validation if the target FHIR server validates the request already. So instead of validating against a harcoded version of a specific implementation of the FHIR spec, we let the SUT itself handle the request and validate the response. This not only results in a more accurate interoperability testing (since we can validate the response directly from the SUT, and not some test-dummy), but also better maintainability on our side as it makes our code completely de-coupled from the SUT implementation. Nevertheless, the compose files are just that. Docker Compose files. For our current use-case we did not need the FHIR server so we removed it from the distribution file. However, anyone should feel free to create their own setups with the images we build in this repo. The distribution compose is just what we decided to use for the fhir-thon. — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD3HUUCDXMTHUHALZEN3MVL2KY32PAVCNFSM6AAAAABU7G6V7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJSGI4TAOJQGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bjungs]

Since the introduction of the proxy the architecture has changed, yes.
The goal is to de-couple the testing system (ITB) from the SUTs (FHIR Server).
The user configures the SUT and connects their application (FHIR Client) to the proxy.

If you want to test against a local FHIR Server, you can add the local FHIR server to the setup and just configure that as the target.

In the near future we'll want to be able to support multiple users at the same time from the cloud. With the old architecture that was simply not possible, since we would need a new FHIR server per user.

Did anyone decide that the FHIR server was not needed?

I decided it was not needed anymore given the direction the project was going (proxy + multi-user cloud support) and the feedback given by you some time ago regarding the way a FHIR Server automatically validates requests. Which makes total sense.
Now, why would be want to run the same request in a local server if we can just check the response from the target server? The only reason it was useful was to use it as a test-dummy for the SUT, and it can still be used like that, if configured as such.

[costateixeira]

I presume you saw the architecture using 2 servers? From a testing perspective, I wouldn't expect the SUT to be validating its own content. My expectation is still that the FHIR validator should be in the testing platform. Anyway, we had a demo setup and I just realized why that was not working as I expected. I was assuming that I would still be expected to report issues, but I am not pushing for changes if someone else is making the decisions. I can only advise from a FHIR and interoperability perspective. For myself I just restored the working FHIR validating setup in a different repo, so I am not blocked.

…

On Wed, 15 Jan 2025, 12:39 Bernardo Nunes, ***@***.***> wrote: Since the introduction of the proxy the architecture has changed, yes. The goal is to de-couple the testing system (ITB) from the SUTs (FHIR Server). The user configures the SUT and connects their application (FHIR Client) to the proxy. If you want to test against a local FHIR Server, you can add the local FHIR server to the setup and just configure that as the target. In the near future we'll want to be able to support multiple users at the same time from the cloud. With the old architecture that was simply not possible, since we would need a new FHIR server per user. Did anyone decide that the FHIR server was not needed? I decided it was not needed anymore given the direction the project was going (proxy + multi-user cloud support) and the feedback given by you some time ago regarding the way a FHIR Server automatically validates requests. Which makes total sense. Why would be want to run the same request in a local server if we can just check the response from the target server? The only reason it was useful was to use it as a test-dummy for the SUT. — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD3HUUAIS6M5A4ZCHLZJVTL2KZCGDAVCNFSM6AAAAABU7G6V7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJSGQ3TENZYHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bjungs]

My expectation is still that the FHIR validator should be in the
testing platform.

The FHIR validator can easily be included but right now the test cases don't make use of it.
My point is just that it should not be part of the architecture itself but as an optional add-on, just like you can run any other service you like and be able to call them as services from within the test cases through their REST APIs.

[costateixeira]

The SUT should never be the validation server, and I presume doing FHIR testing should always include a FHIR validator, as most test cases would use it. So I am really sure I am missing something. I am sure to be missing something. But I just wanted to record that I was wondering why the initial demo was no longer working.

…

On Wed, Jan 15, 2025 at 3:36 PM Bernardo Nunes ***@***.***> wrote: My expectation is still that the FHIR validator should be in the testing platform. The FHIR validator can easily be included but right now the test cases don't make use of it. My point is just that it should not be part of the architecture itself but as an optional add-on, just like you can run any other service you like and be able to call them as services from within the test cases through their REST APIs. — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD3HUUEVE74M7IKJ6Q6KMTT2KZW7JAVCNFSM6AAAAABU7G6V7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOJTGAZTAMRVHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>