Can't use Scala collections methods in code generation

[DavidGregory084]

For some reason as of Scala 2.13 it doesn't seem to be possible to use the Scala library when running javac as an annotation processor through Zinc. It simply results in NoSuchMethodErrors for every Scala collection method.

I have had to use -Yno-imports and Java 8 Streams in the codegen module, which has resulted in some terrible code. Who knew last was so difficult to write in Java 8.

[costa100]

Do you have more details about this, i.e. what is exactly the command that fails?

[DavidGregory084]

The CodeGenProcessor is used as an annotation processor in this build task, which generates the generated/ Scala code in each module based upon the Vert.x Java code for the corresponding module.

What I have observed is that if you use any Scala collections methods in that code (or really anything else from the Scala collections library), it will not be on the classpath when the annotation processor runs, regardless of what you do with the classpath for that task in build.sc.

It could be that the library ends up on the classpath twice, which can cause problems if there are two different versions, but I couldn't figure it out so I just used Java streams.

[costa100]

I am curious about this - just another quick question, what IDE did you use? I use IntelliJ but it shows me a lot of errors when I open the mill build script - I don't now it supports it. Just curious, how come did you use mill instead the ubiquitous sbt? Do you find it easier?

[costa100]

Never mind - I went to the mill page and they have instructions on how to get it working with IntelliJ.

[DavidGregory084]

Just curious, how come did you use mill instead the ubiquitous sbt? Do you find it easier?

I find sbt to be slow to run and difficult to configure outside of the most basic use cases, and it is not out of lack of experience with sbt, as I usually end up being the "build guy" on my team. For example, here I have configured mill to compile a different programming language. I don't know where I would even start to do that with sbt.

[costa100]

Thanks.

So, I created a separate scala (2.13.4) project using IJ 2020.3, I created an annotations processor in scala that calls scala Set[String] methods, I called that on a java file and it works. Do you want me to create a github repo for it, or I can place the whole thing in a zip and put it on dropbox? I packaged the whole thing i.e. scala libs in a single jar file along with TestAnnProcessor using the IJ artifact feature. Here is the scala code:

import java.util
import javax.annotation.processing.{AbstractProcessor, RoundEnvironment}
import javax.lang.model.SourceVersion
import javax.lang.model.element.TypeElement
import scala.jdk.CollectionConverters._


class TestAnnProcessor extends AbstractProcessor{
  override def process(annotations: util.Set[_ <: TypeElement], roundEnv: RoundEnvironment): Boolean = {
    val scColl = annotations.asScala.map(te => te.toString)
    println(scColl.mkString("\n"))
    println(scColl.size)
    println(scColl.headOption)
    true
  }

  override def getSupportedSourceVersion: SourceVersion = SourceVersion.latest()

  override def getSupportedAnnotationTypes: util.Set[String] = Set("interface", "Anno").asJava
}

I created this file, TestFile.java:

@interface Anno { }

@Anno
class AnnosWithoutProcessors { }

I ran this: javac -cp ./out/artifacts/TestAnnotationProcessor_jar/TestAnnotationProcessor.jar -Xlint:processing -processor TestAnnProcessor -proc:only TestFile.java and it produced this output:

Anno
1
Some(Anno)

0
None

Let me know if you have any suggestions about code in the annotations processor, i.e. should I try other collections apis? Also, any other combinations to try? I was thinking of packaging the class by itself and add the scala library to the cp option of javac and try it that way - it would closer to how it's run in your project.

Cheers

[costa100]

I was able to reproduce the error - it's quite a rabbit hole. The weird thing is that when I run javac by itself, and fed it the same class path & source path as the zincWorker.worker().compileJava command, it does not fail with that error. I have to investigate more...

[costa100]

I think I know what caused the exception, java.lang.NoSuchMethodError - I reproduced it isolated from mill. Mill uses scala 2.12. When you run javac to process the vertx annotations from the vertx code, the javac compiler runs in the context of scala 2.12 (it invokes the CodegenProcessor class dynamically), however, codegen was compiled with scala sdk 2.13.1. If you try to call scala 2.13 collection objects methods (the collections have been revamped in 2.13) you can get an error such as this one because these methods don't exist in scala 2.12.

java.lang.NoSuchMethodError: scala.collection.convert.AsScalaConverters.asScala$(Lscala/collection/convert/AsScalaConverters;Ljava/util/Iterator;)Lscala/collection/Iterator;

The AsScalaConverters class is in both sdks, but obviously the code inside changed.

What is the solution then? Imo, you have to compile the codegen module with scala 2.12 instead of 2.13, if you want to use the scala collections. However, I just tried that and the compilation fails. So now we have another problem. You can go back and fix the compilation errors, but at this point, I kind of doubt that you want to re-write your code.

Do you have any plans to support vertx 4? Do you have any incentives to keep this project going? If not, it would probably be best to put a note and archive it.

[DavidGregory084]

Thanks for looking at this @costa100 - I think in that case it might be as simple as porting the project to the latest version of mill (which is compiled against Scala 2.13)

I don't mind upgrading the codegen code to Scala 2.13, there isn't a lot of it and IME the Scala 2.12 -> 2.13 upgrade is pretty painless.

I will probably support vert.x 4 yes, unless the codegen mechanism has changed completely or something like that

[costa100]

I think in that case it might be as simple as porting the project to the latest version of mill (which is compiled against Scala 2.13)

That's a very good idea. Mill 0.6.1 reached EOL.

I will probably support vert.x 4 yes, unless the codegen mechanism has changed completely or something like that

This is great. Thanks!