Skip to content

Benchmarking Kotlin JSON Parsers: Jackson-Kotlin and Kotlinx Serialization

It’s once again… BENCHMARKING TIME

We’ll take different JSON Parsers, run them through tests, run them once, run them ten times, run them a thousand times, look at their speeds, and infer the reasons behind why one might be faster in certain situations.

Last time, we put Gson, Jackson, and Moshi to the test. I recommend giving that a read too.

But this time, due to popular demand on Reddit, it’s all about the Kotlin modules. We’re introducing Jackson again but this time, with its Kotlin module. Putting that against Kotlinx Serialization, a parser that boasts superpowers over the other parsers because it was build with and for Kotlin.

Once again, here’s how it goes.

Each test is going to be done within the context of Retrofit because that’s how most apps make network calls nowadays. We’ll be running the tests on the JVM and using MockWebServer to return responses of different lengths and we’ll have them returned as RxJava Singles.

I’ll be running each of these tests at least twice to make sure there aren’t any flukes.

Do note that this benchmarking test will only cover deserialisation JSON responses into Java (or well actually, Kotlin) objects and not the other way around.

Rules of the Test

I have 3 Retrofit interfaces set up, one for each parser. I also have 3 different JSON responses saved, all lists of the JsonPlaceholder Posts of varying lengths. The short one has 14 lines of JSON code, the medium one has 602, and the long one has 10106.

This class will of course be slightly varied between their Jackson and Kotlinx counterparts, but they all achieve the bare syntax needed to parse a post from the above API.

To avoid having the JVM startup time be a factor, I have one dummy test that runs before any of the actual tests. I also run a dummy test for each of the parsers to minimise the factor of any first-use-startup time affecting our tests.

I’ll be following the same tests as I did with the previous 3 parsers so we can compare these Kotlin-based parsers with them. If you want to see the comparisons, I recommend you check out that post too.

Considerations

I want to stress that in my gradle implementation for adding Kotlinx Serialization, I’m only using this:

implementation "com.jakewharton.retrofit:retrofit2-kotlinx-serialization-converter:0.5.0"

This is because I couldn’t otherwise find a way to use the standard library with Retrofit without my compiler giving me an error for duplicated modules. It’s possible that the standalone Kotlinx Serialization library or even a later version of Jake Wharton’s bundled one could perform better, but for now, because it’s the only one I could get working, this is what I settled with.

1 Iteration

Short

Medium

Long

JacksonK

10

16

40

Kotlinx

6

8

22

Parsing time in milliseconds

Kotlinx is here already boasting the fastest time not only between itself and Jackson Kotlin but also against the big 3 Java-based modules (which I’ll be calling the big 3 from now on). Its performance is ever so slightly than that of Moshi which performed best on the previous 1 iteration test.

Jackson Kotlin on the other hand is displaying the slowest time between these and the big 3, consistently performing worse than Gson on all json sizes. One run of this test even showed me a 61ms long parsing time which is pretty bad, however that’s more of an outlier.

This is however only a single iteration test and doesn’t tell us too much about their real performance.

1000 Iterations

Short

Medium

Long

JacksonK

0.41

0.62

4.16

Kotlinx

0.34

0.52

4.44

Parsing time in seconds

Again, Kotlinx is displaying the best performance on the short and medium tests, but falls in performance on the long test. Between itself and Jackson Kotlin on the long test, neither is ahead of the other as multiple runs of the test can show that Jackson Kotlin performs better than Kotlinx.

The noteworthy thing however is their performance against the big 3 which is across the board, worse, than the big 3. Moshi scored 3.57s, and Jackson scored 3.02 on this test.

Perhaps the claim of higher performance from being Kotlin-based modules may be invalid… but we’ll continue testing that.

5000 Iterations

Short

Medium

Long

JacksonK

1.82

2.70

20.92

Kotlinx

1.63

2.63

20.81

Parsing time in seconds

Not looking good here for the Kotlin modules. Short tests, better than Gson, but roughly equal with Jackson and Moshi. Medium tests perform similarly on all parsers (although multiple runs show the Kotlin parsers can perform worse).

Long tests however, not even Gson on the previous benchmarking hit 20 seconds. Both of the Kotlin modules have exceeded that, and they do so quite consistently across mulitple runs.

I can’t say things look good for Kotlinx at all, but Jackson Kotlin still has hope. On the previous benchmarking test, Jackson performed significantly better by dumping a bunch of random annotations on its entity. Let’s try doing the same with Jackson Kotlin.

5000 Iterations with JacksonK Annotations

Short

Medium

Long

JacksonK without Annotations

1.82

2.70

20.92

JacksonK with Annotations

1.90

3.31

29.94

Parsing time in seconds

I used the exact same annotations as I did with the previous benchmarking test on Jackson with annotations.

@JsonIgnoreProperties(value = ["someid"])
@JsonAutoDetect(fieldVisibility = JsonAutoDetect.Visibility.ANY)
data class PostJacksonKotlin(
    @JsonProperty("userId")
    val userId: Int,
    @JsonProperty("id") val id: Int,
    @JsonProperty("title") val title: String,
    @JsonProperty("body") val body: String,
    @JsonIgnore val ignoredField: String = ""
)

This time however, Jackson Kotlin’s performance is significantly worse than without the annotations. For the base Jackson module, I would guess the boost in performance happens because without including these annotations, the code on the compiled entity class would have to be identified and generated. Adding these annotations simply tells the compiler to generate them straight away.

But with the Kotlin module, there is probably more happening behind the scenes with Kotlin conversions, reflection, idk but what I can guess is the addition of the annotations is more of a hindrance or an extra step with the Kotlin module, unlike with the base module where it removes a step.

Do bear in mind that these annotations are complete oonga-boonga on my part. I didn’t add each of these annotations with delicate thought and increase in performance in mind, however these are the exact same annotations that increased the performance of base Jackson so take what you will from it.

Parsing Larger Objects

Up till now, we’ve been parsing posts which only have 4 fields. This time, let’s try parsing Users from the JsonPlaceholder API. This API has a more complex structure than the simple Post so let’s see if that makes a difference in our results.

This is a test of 5000 iterations but with new data classes for parsing a User object. Although you’d think that this would make the parsing time longer across the board, it was the complete opposite in the previous benchmarking test. Let’s see if the same holds true for the Kotlin parsers.

Short

Medium

Long

JacksonK

2.10

2.18

4.82

JacksonK (Annotations)

2.86

2.53

5.53

Kotlinx

1.79

2.51

5.46

Parsing time in seconds

Like the previous benchmarking, for some reason, a larger, more complex object was quicker to parse than the simple Post. But between how the parsers perform against each other, the Kotlin parsers didn’t actually do too bad. All of the parsers perform rather similarly (although Gson still seems to be the slowest).

Jackson Kotlin does slightly better than Kotlinx in this case, but still not as good as base Jackson with annotations. Multiple runs of this test can prove that Jackson Kotlin semi-consisteny reaches parsing times over 5 seconds on this test, while base Jackson with annotations is more consistent in its <4.5s parsing time.

Conclusion

There are a few key takeaways from here:

  • These Kotlin-based parsers aren’t necessarily faster than their Java-based counterparts
  • Jackson Kotlin doesn’t perform as well as base Jackson (which is still the best in performance)
  • Annotations don’t make Jackson Kotlin faster

I don’t want to say that them being Kotlin-based parsers is what’s making them slower than the others. That’s one hypothesis, but there’s still not enough evidence to make that conclusion.

Jackson-Kotlin is a module built on top of Jackson, and Kotlinx Serialization is still very new and most probably not as refined as the other parsers which have been around for years (even Moshi as it’s been rebuilt from Gson).

Not to mention, Kotlinx Serialization wasn’t built with Android or Retrofit in mind. It’s better used in more pure Kotlin-based frameworks like Kotlin Multiplatform or Kotlin Native.

I think we’ll have to wait a few more years at least for the Kotlin Parsers to mature, maybe see a new Jackson or Moshi completely refactored or rebuilt into Kotlin or a new Kotlin-based parser (with Android in mind) to surface before we can get stronger evidence as to whether or not Kotlin can make a json parser perform better.

For now however, stick with the big 3. ༼ つ ◕_◕ ༽つ

Tags: