dependencySet does not include transitive dependencies

[adamretter]

When building a complex Java extension which we wish to use EXPath XAR packaging for, the Java application will have dependencies other than its own class files which need to be included in the EXPath package.

These dependencies are correctly expressed in the pom.xml and we can for example use the Maven Shade (or even the Maven Assembly) plugin to assemble an uber/fat jar which contains all of the class files that we would need if we were distributing as a Java artifact.

However, when using the XAR plugin we need to also include all of the transitive dependencies of the project manually as it fails to detect these correctly whereas the Shade or Assembly plugins do. Including all these manually is frustrating as we have to repeat all the dependencies (which are effectively runtime scope) from the pom.xml into the xar-assembly.xml.

For examples see:

1. https://github.com/eXist-db/expath-file-module/blob/master/xar-assembly.xml#L26
2. https://github.com/adamretter/exist-sparql/blob/fix-build/xar-assembly.xml#L25

I can see two possible solutions:

1. Fix the XAR plugin so that it correctly also pulls in the transitive dependencies which are already defined in the pom.xml.
2. Add a classifier to the dependencySet in the xar-assembly.xml which allows us to specify the shaded artifact for our project (which will already include all transitive dependencies).

Ideally I think we should implement both facilities. I also think there is room for cleaning up the xar-assembly.xml and its dependencySet and fileSet features, to make them behave more on convention and to use more of the information that is already defined in the pom.xml without having to repeat ourselves.

I would be happy to undertake some work to do an overhaul of the plugin and improve it to be more Maven like ;-) Thoughts?

[claudius108]

I think that, for proposition 1, a parameter like includeTransitiveDependencies would work. I will try to add this in a few days.
Proposition 2 would also be nice. sure.

Thanks,
Claudius

[adamretter]

@claudius108 I think the default should be to include transitive dependencies. I don't see any reason for not having them, as the POM really sets up the deps. If you need to exclude some (I can't imagine why?) then you could add an excludes section like the POM, but er... just use the POM ;-)

[dizzzz]

@claudius108 @adamretter any updates/progress on this subject?

[adamretter]

@dizzzz Not that I know of. The manual copy/paste workaround will help you immediately.

[claudius108]

I am contemplating it.

[dizzzz]

One related aspect is.... it would be good to exclude specific transient dependencies as well... in pom.xml:

<dependency>
    <groupId>${project.groupId}</groupId>
    <artifactId>app</artifactId>
    <version>${project.version}</version>
    <exclusions>
        <exclusion>
            <groupId>*</groupId>
            <artifactId>*</artifactId>
        </exclusion>
    </exclusions>
</dependency>

[duncdrum]

i think our monex problems might be related see eXist-db/monex#92
@claudius108 how would you feel about adding an integration test, to see if dependency resolution actually works as expected from inside a running exist-db (base-x, …) instance?

[claudius108]

Hi,

I think that the plugin behaves correctly as to the EXPath Packaging System specification, namely one has to define every component to be included.

I also asked once @fgeorges what happens when they are tens of dependent files, and your issue with transitive dependencies is of the same kind.

One solution for this issue with transitive dependencies is to have a fat jar including everything needed. This is the solution. The Apache Maven Shade Plugin already creates a a distributable jar with all the dependencies, so I think it is better to not duplicate this functionality within this plugin, and keep it simple instead.

I think that any other solution would not be in the spirit of the above-mentioned specification. Maybe @fgeorges can share us this thoughts about the issue you are facing...

I think that Packaging specification could use the knowledge associated with the maven storage system, by using URNs for naming the components, as it is possible for this plugin, see an example below:

	<components>
		<resource>
			<public-uri>http://exist-db.org/xquery/jms</public-uri>
			<file>urn:java:class:org.exist.jms.xquery.JmsModule</file>
		</resource>
	</components>

I also suggested once this kind of storage for eXist to @wolfgangmm, so to avoid packing, for instance, commons-io-2.6.jar in my library when one can find it in $EXIST_DB/lib/core/.