Help with dotty compiler and classloading at runtime

[rvacaru]

Hi Dotty people! 👋
I'm Razvan a GSoC contributor at scalafix (https://summerofcode.withgoogle.com/myprojects/details/gQ08FcXb).
I implemented a compiler for custom rules by reading examples or tests in github where dotty classes . This is used by scalafix to compile external rules at runtime. I then manually invoke the RuleCompiler from console and it seems to be working, both when there are no compilation errors and when there are:

scala> import java.nio.file.Files
import dotty.tools.io.File
val r = new scalafix.internal.reflect.RuleCompiler(scalafix.internal.reflect.RuleCompilerClasspath.defaultClasspath)
                                                                                                                                                                                        
val mainText = """
      @main def hello: Unit = 
        println("Hello scala 3 world!")
        println(msg)
        println(2)
        println(power(2d, 0))
        println(power(2d, 1))
        println(power(2d, 2))
        println(power(2d, 3))
        println(power(2d, 35))

      
      def msg = "I was compiled by Scala 3 " + "..."
      inline def power(x: Double, inline n: Int): Double =
        inline if n == 0 then 1.0
        else inline if n % 2 == 1 then x * power(x, n - 1)
        else power(x * x, n / 2)
      """.stripMargin

val tmpFile = Files.createTempFile("Main-", ".scala")
val input = metaconfig.Input.VirtualFile(tmpFile.toString, mainText)
val cl = r.compile(input)
cl.isOk

// output below:

val r: scalafix.internal.reflect.RuleCompiler = scalafix.internal.reflect.RuleCompiler@33ed571e
val mainText: String = "
      @main def hello: Unit = 
        println("Hello scala 3 world!")
        println(msg)
        println(2)
        println(power(2d, 0))
        println(power(2d, 1))
        println(power(2d, 2))
        println(power(2d, 3))
        println(power(2d, 35))

      
      def msg = "I was compiled by Scala 3 " + "..."
      inline def power(x: Double, inline n: Int): Double =
        inline if n == 0 then 1.0
        else inline if n % 2 == 1 then x * power(x, n - 1)
        else power(x * x, n / 2)
      "
val tmpFile: java.nio.file.Path = /var/folders/8t/s8cs6f9s3p50pdf_1fy382kc0000gp/T/Main-18147230000312495277.scala
val input: metaconfig.Input.VirtualFile = Input.VirtualFile("/var/folders/8t/s8cs6f9s3p50pdf_1fy382kc0000gp/T/Main-18147230000312495277.scala", "...")
val cl: metaconfig.Configured[ClassLoader] = Ok(scala.reflect.internal.util.AbstractFileClassLoader@5bb799aa)
val res0: Boolean = true

But I'm now testing this and after compilation the test tries to load the class from the classpath, but the class isn't there. The same unit test works in scala2 with the scala2 compiler. I've debugged the code and found out where the test fails in scala3:

  def tryClassload(classloader: ClassLoader, fqn: String): Option[v1.Rule] = {
    try {
      Some(toRule(classloader.loadClass(fqn)))
    } catch {
      case _: ClassNotFoundException | _: NoSuchMethodException =>
        println("catch 1")
        try {
          val clz = classloader.loadClass(fqn + "$") // here is where it fails with class not found
          Some(toRule(clz))
        } catch {
          case _: ClassNotFoundException =>
            println("catch 2")
            None
        }
    }
  }

In the scala3 version catch 2 is printed out, while in scala2 only catch 1 is being printed and then class loading succeeds.

Would you know what is wrong or what to investigate further? Is there any documentation I should read on this?
What I'm also trying to do is look at the classes available in the classloader, tho I couldn't find out how to do it, would anyone know?

[rvacaru]

How to reproduce the issue

• Clone and check out scalafix reflect3 branch on my fork: https://github.com/scalacenter/scalafix/pull/1650/files or https://github.com/rvacaru/scalafix/tree/reflect3
• compile the unit test module for scala3:
  • unit3Target3/compile
  • unit3Target3/Test/compile
• run the broken unit test unit3Target3/testOnly scalafix.tests.reflect.ToolClasspathSuite
• the name of the broken test is --tool-classpath is respected when compiling from source

For the scala2 version of the test/code simply replace unit3Target3 with unit2_13Target2_13