Quarkus and GraalVM

#Quarkus#GraalVM#Java#C##ASP.NET Core

Because Quarkus and GraalVM got lots of attention in tech blogs in the last few months, I decided to test this combination by myself.

Quarkus claims to be A Kubernetes Native Java stack tailored for OpenJDK HotSpot and GraalVM, crafted from the best of breed Java libraries and standards and GraalVM advertises itself with Run Programs Faster Anywhere. This is exactly the combination that I need for one of my private applications. The focus of this article is not on Quarkus as a framework, but on the performance of the resulting native image. I implemented a small application in C# and ASP.NET Core 2.2 last year. I didn't choose C# because I am a fan of Microsoft, but because small prototypes in Java and Python used too much RAM on my small private server. As I am used to all the comfort offered by different Java frameworks, I thought that this application would be ideal for testing Quarkus. The code base is very small and the application uses only a few and small 3rd party frameworks.

The Application

The application is a REST service, that basically uses a Boyer-Moore algorithm in order to search for a string in a text file, which is 330 MB large. The file is being read in chunks of 5 MB into a byte array and then analyzed for the string. If the string is found, the next approximately 5500 lines are parsed as CSV data, in order to fill a small object structure. As reference: This object structure serialized as a JSON string is only 716 bytes long. There is no database involved, just file I/O and CPU.

The Test Setup

In the test scenario, the search string is at 91% of the file — rather at end. The test is always executed on the same Docker host in a Linux container. In exactly this setup, grep --max-count=1 takes 350 ms to find the string (of course without further parsing of CSV data). The measured time in the table below only includes searching and parsing, JSON serialization and sending of the data to the client is not measured. The duration was measured as mean value for 10 invocations, the RAM consumption was in all scenarios always constant after the very first invocation up to the 10th invocation.

The Result

Technology	RAM usage (“cold”)	RAM (after 1 invocation)	Duration
C# with ASP.NET Core 2.2	37 MB	61 MB	620 ms
Quarkus + OpenJDK 11	130 MB	179 MB	859 ms
Quarkus + OpenJDK 11 - OpenJ9	40 MB	80 MB	700 ms
Quarkus + native (GraalVM 19.3.1)	3 MB	113 MB	792 ms

The startup time of the native image is incredible fast, but for my scenario useless, because my use case is not application restarts. Surprisingly, the native image is not fast at all during execution time. I thought that I could reach something close to the grep time plus 50 ms at worst for the CSV parsing. More about this performance “problem” in the section “Conclusion”.

Also interesting is the fact, that the C# image releases lots of memory back to the OS, if the application is not used. After some time, the RAM usage goes down to 11 MB. This was not the case for any of the Java images. There are some JVM parameters to tell the GC to release memory more aggressively, but I did not fiddle around with them in this test.

After having collected all this numbers, I am very surprised by the performance of the C# image. I always thought that the JVM with HotSpot and its JIT play in the top league. But apparently Microsoft is not sleeping.

Update Just while writing this article, I also ported my application from ASP.Net Core 2.2 to 3.1, because .NET Core 3 should offer another performance boost. The results are impressive:

Technology	RAM usage (“cold”)	RAM (after 1 invocation)	Duration
C# with ASP.NET Core 3.1	36 MB	57 MB	501 ms

Problems

It was quite cumbersome to get GraalVM for Java 11 with Quarkus to work:

• Only Quarkus 1.2.0.Final (from 23.01.2020) supports GraalVM 19.3.1 and Java 11. Previous versions only supported GraalVM with Java 8, which is not really an option end 2019.
• Even with 1.2.0.Final, I was not able to build a native application under Windows, because Quarkus calls a GraalVM tool with incorrectly quoted parameters under Windows. This bug already existed in Quarkus 1.1.1.Final with Java 8 and GraalVM 19.2.1.
• Only with Windows + WSL (and thus GraalVM for Linux) I was able to build a native image under Windows.
• GraalVM used up to 9.9 GB RAM while building the application. This is also the reason why I could not build on a Linux host, because non of the available hosts had this large amount of free memory.

Conclusion

I was really surprised by the results:

1. I thought that my application with the native image would be close to grep, which was not at all the case.
2. I did not expect C# to be faster than the JVM.

Concerning the first item, I found a very interesting article, that should be titled wrong expectations. In short: A native image is not necessarily faster than an application executed in the JVM with JIT.