Improvement of vocabulary-tests needed?

The tests for vocabularies provided here:

iridium/tests/test_inveniordm_api.py

Lines 23 to 50 in 4d357dc

	def test_vocabularies(rdm):
	"""Test queries to vocabularies."""
	with pytest.raises(httpx.HTTPStatusError):
	rdm.query.vocabulary("invalid_voc")
	res = rdm.query.vocabulary(VocType.resource_types)
	assert res.hits.total == 33
	lcs = rdm.query.vocabulary(VocType.licenses)
	assert lcs.hits.total == 418
	sjs = rdm.query.vocabulary(VocType.subjects)
	assert sjs.hits.total == 48
	lgs = rdm.query.vocabulary(VocType.languages)
	assert lgs.hits.total == 7847
	assert rdm.query.vocabulary(VocType.languages, tags="extinct").hits.total == 619
	lg = rdm.query.vocabulary(VocType.licenses, suggest="gli")
	assert lg.hits.total == 1
	assert lg.hits.hits[0].id == "glide"
	with pytest.raises(httpx.HTTPStatusError):
	rdm.query.term("license", "invalid_license")
	gl = rdm.query.term(VocType.licenses, "glide")
	assert gl.id == "glide"

are only working on instances where the vocabularies aren't adjusted. For example, a freshly installed instance.

It is expected and wanted behaviour that vocabularies are adapted and changed: https://inveniordm.docs.cern.ch/customize/vocabularies/
Our instance, for example, has already some other subjects and we are working on different record-types.

I wonder whether the current tests need more improvement or whether we want to limit these tests down to a 'untouched' vocabularies folder and we expect the user to disable or adjust these tests for their own instances.

[apirogov]

Good question. I wrote tests against "vanilla", because Iridium currently has no awareness at all about customizations.

Do you have an idea how the test should work for customized vocabularies? I would only have the idea that you have to make your customization YAMLs available to the tests and use them as a basis.

But the next question is - is there an added value for testing Iridium by testing against custom vocabularies (except being able to run against your own instance)? All it would do is make sure that you correctly wired up your vocabularies in your instance, but the Iridium API does not care about what terms are in there.