Support Scala 3 #479
Comments
|
I created https://github.com/FasterXML/jackson-module-scala/tree/dotty but there are some remaining compile problems to solve |
|
I'll have a look at that branch. For now the above error is thrown, I think, because of the way Jackson determines if a class is a Scala class (https://github.com/FasterXML/jackson-module-scala/blob/master/src/main/scala/com/fasterxml/jackson/module/scala/util/Classes.scala#L10-L11) and this check always evaluates to false, since scala-reflect is not available when compiling with Scala 3. |
|
@DieBauer if you could find an alternative way to differentiate a Scala class from a plain Java class, we can support that too. We do need to avoid having jackson-module-scala try to work with non-Scala classes because it leads to unexpected behaviours when users have Scala and plain Java classes in their projects. |
|
@DieBauer which version if jackson-module-scala did you test with? v2.12.0 increases the reliance on the check for the ScalaSignature annotations that you highlighted. |
|
@pjfanning I checked that with 2.11.3, but same issue occurs on 2.12.0-rc2. https://scastie.scala-lang.org/UOWvdJ7xQkS64MV94WuJEQ Relying on the What kind of unexpected behaviours with non-scala classes are you talking about, if it turns out the runtime check cannot be done on scala 3? |
|
I may not have enough context here, but it might be easy enough to make "Plain Scala Object or not" detection pluggable, provide dynamically loaded default but allow override (or more specifically I suppose it should be provider for that detector). The reason there needs to be detection is that logic for figuring out construction of Plain Old Scala Objects is bit more advanced than that for POJOs -- more ways to construct things in Java, can not rely on certain things that Scala (and some other JVM langs like Kotlin) guarantee. |
|
I started a topic at https://users.scala-lang.org/t/recognising-scala-3-classes/6922 @DieBauer would you have any pointers to specific comments of the Dotty gitter channel relating to this? Otherwise, I might being the topic above to the gitter channel to see if I can see if the Dotty team would be amenable to adding a new annotation or a marker trait/interface. |
|
This is the question I asked https://gitter.im/lampepfl/dotty?at=5fbf67a566dcfa77e2a3cb08, only 1 person responded though. Indeed as mentioned in the topic, the classfile contains scala references. But that doesn't help you at runtime :) |
|
The dotty branch has a number of compile issues associated with higher-kinds - affects 3 classes, in the main. I might have time over the Christmas period to look at trying to support dotty compiled classes in the 2.13 build. I think this is a more useful starting point than supporting a scala 3 build. |
|
@DieBauer I created a test that shows that jackson-module-scala can handle a scala 3.0.0-M1 case class. See https://github.com/FasterXML/jackson-module-scala/tree/dotty-test -- the equivalent case object test would not compile though -- seems like the TastyReader doesn't support case objects in the way I expected. |
summon[@bishabosha] |
It seems the problem here is that I get the same error if I paste the following in scala 2 repl: scala> :paste
// Entering paste mode (ctrl-D to finish)
package com.github.pjfanning.scala3
class Scala3CaseObject {
val field1: String = "test"
val field2: Int = 42
}
// Exiting paste mode, now interpreting.
scala> val x = com.github.pjfanning.scala3.Scala3CaseObject
^
error: object Scala3CaseObject is not a member of package com.github.pjfanning.scala3
note: class Scala3CaseObject exists, but it has no companion object.
|
|
@bishabosha thanks for checking that - I have now fixed the Scala3CaseObject |
|
The 'dotty' branch now compiles (with a few test cases removed - ones related to ScalaObjectMapper which will not work in Scala3 - due to Manifest support being removed along with other scala-reflect features). Many tests fail due to scala/scala3#6349 |
|
@pjfanning scala/scala3#6349 is fixed by scala/scala3#10936. Maybe it could be unblocked. |
|
Are these changes totally dropping Scala 2 support? Poking around at the work being done here strongly suggests that this will ruin the experience for [some] scala 2 users. If the main branch is now going to be scala 3 only - then the changes make more sense to me, but you may want to drop all of the awkwardness around trying to support 2.11 / 2.12 / 2.13 and update the readme to be a bit more clear on your current vision for the project. |
|
@nbauernfeind no plan to drop scala 2 support - the dotty branch is purely for experimentation with scala 3 If we decide to try to support Scala 2 and 3 in same jackson-module-scala
|
|
I don't know too much about Scala 2/3, but one thing that may be useful is this: strong +1 for deprecating sub-classing of So that makes sense, regardless of other reasons. |
|
Thanks for the update. I am mostly just curious. (Also sad to see some of my favorite contributions disappear 😿.) The ScalaObjectMapper is/was a mixin that can be "attached" to any subclass of ObjectMapper as long as it is mixed in at the mapper's construction time. I'm not familiar with the changes in Scala3 (my focus has been elsewhere for far too long), but I am surprised to read that there is no TypeTag equivalent! The Manifest/TypeTag concepts have absolutely been one of my favorite features of Scala -- and I'm sad to see it disappear! It was extremely powerful for improving readability of boiler-plate-like code. </reminiscing> |
|
Ok, in that case change to sub-classing should be irrelevant. |
|
Upgrading dotty branch to scala 3.0.0-RC1 fixed some tests - still over 100 broken - scala/scala3#11486 is the cause of some of these test failures |
|
going back to using paranamer when scala3 is in use helps me work around scala/scala3#11486 so that leaves a little over 30 broken tests |
|
@pjfanning Interesting -- can Paranamer do something more than what JDK provides via bytecode? (Jackson-databind 3.0 has embedded code from |
|
@cowtowncoder scala compiler generates java classes and with scala 2.12 and 2.13, there is enough detail in the generated classes to avoid paranamer. Scala 2.11 and before did not have enough detail and require paranamer. Looks like Scala 3.0.0-RC1 needs paranamer again. Jackson-Module-Scala has explicit dependencies on paranamer (if the scala version needs it). |
|
@pjfanning ok, thanks. |
Seems like a regression... is there a ticket? |
|
@SethTisue I raised scala/scala3#11486 |
|
Latest situation is that most features of jackson-module-scala work with Scala 3.0.0-RC1. There is a snapshot release available for jackson-module-scala v2.13.0-SNAPSHOT (the code used to release this is in the dotty branch). |
|
The issues with the Scala 3 support
https://travis-ci.com/github/FasterXML/jackson-module-scala/builds/221433492 will give you an idea of the tests that are still failing |
|
jackson-module-scala 2.13.0-SNAPSHOT is now available for Scala 3.0.0-RC2 |
|
Any chance to get this released for Scala 3.0.0? I'm currently trying to port twitter/util to Scala 3 and it's used there. 🙂 edit: It's not super urgent, so please don't feel pressured or anything. There are lots of other blockers as well 😅 |
|
v2.13.0-SNAPSHOT is available for scala 3.0.0 |
Thank you and all the other contributors for your hard work on this fantastic library!! It is much, much appreciated. @pjfanning One question I have -- what's the functional difference between the 3.0 lib and the current one? <!-- JSON Serialization Dependencies -->
<dependency>
<groupId>com.github.pjfanning</groupId>
<artifactId>jackson-module-scala3-enum_3</artifactId>
<version>2.12.3</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.13</artifactId>
<version>2.12.3</version>
</dependency>And with stuff like this, mind-blowingly it "just works". I noticed some notes about how the 3.0 version has some problems/shortcomings -- would it be better just to continue using the 2.0 with Scala 3? import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.github.pjfanning.`enum`.EnumModule
import io.quarkus.jackson.ObjectMapperCustomizer
import javax.inject.Singleton
// https://quarkus.io/guides/rest-json#jackson
@Singleton
class Scala3ObjectMapperCustomizer extends ObjectMapperCustomizer:
def customize(mapper: ObjectMapper): Unit =
// General Scala support
// https://github.com/FasterXML/jackson-module-scala
mapper.registerModule(DefaultScalaModule)
// Suport for Scala 3 Enums
// https://github.com/pjfanning/jackson-module-scala3-enum
mapper.registerModule(EnumModule)import com.fasterxml.jackson.databind.ObjectMapper
import io.quarkus.test.junit.QuarkusTest
import org.junit.jupiter.api.Assertions.*
import org.junit.jupiter.api.{DisplayName, Test}
import javax.inject.Inject
import scala.collection.JavaConverters.*
enum AnEnum:
case A extends AnEnum
case B extends AnEnum
case class Other(foo: String)
case class Something(name: String, someEnum: AnEnum, other: Other)
@QuarkusTest
class Scala3ObjectMapperCustomizerTest:
@Inject
var objectMapper: ObjectMapper = null
@Test
def `Quarkus CDI Jackson ObjectMapper has Scala 3 modules registered`: Unit =
val moduleIds = objectMapper.getRegisteredModuleIds.asScala
val expectedModules =
List("com.fasterxml.jackson.module.scala.DefaultScalaModule", "com.github.pjfanning.enum.EnumModule$1")
expectedModules.foreach(it => {
assertTrue(moduleIds.contains(it), f"Expected ObjectMapper registered Module Ids to contain '$it'")
})
@Test
def `Jackson ObjectMapper can parse Scala 3 members`:
val sampleSomethingJSON: String = """
{
"name": "My Something",
"someEnum": "A",
"other": {
"foo": "bar"
}
}
"""
val parsed = objectMapper.readValue[Something](sampleSomethingJSON, classOf[Something])
assertEquals(parsed.name, "My Something")
assertEquals(parsed.someEnum, AnEnum.A)
assertEquals(parsed.other.foo, "bar") |
|
@GavinRay97 thanks for interest in this library. https://github.com/FasterXML/jackson-module-scala#scala-3 is the current status of Scala3 support. So far, the support is experimental and it is only by doing lots of testing in various scenarios that we will become more confident about whether jackson-module-scala works well with Scala3. If people run into issues and report them, we can look at seeing if they can be fixed. jackson-module-scala uses Java reflection to try to work out how to work out the structure of Scala types. Other Scala libs for JSON support use alternative approaches that probably mean you get better results when you have just to worry about Scala classes. jackson-module-scala is aimed more at use cases where you also have to worry about Java classes. |
|
Well for whatever it's worth, With this setup, I don't have to define manual decoders/encoder 👏 |
|
jackson-module-scala 2.13.0-rc1 is released |
Now that Scala 3 is in Milestone releases (https://dotty.epfl.ch/blog/2020/11/09/scala3-m1.html) it would be great if we can support Scala 3.
I tried to make it work with the
dottyCompatmode in thesbt-dottyplugin (https://scalacenter.github.io/scala-3-migration-guide/docs/tutorials/sbt-migration.html) but then things start to break :)The following snippet works on Scala 2.x, but fails with Scala 3.0.0-M1.
fails with:
This is the scastie to play with: https://scastie.scala-lang.org/2dYNr8u9TciEOU9rOCQJPQ
The text was updated successfully, but these errors were encountered: