The product indexes chemistry. Reactions, substances, and the articles and patents they appear in, which means the interesting part is not the database — it is the annotator: the thing that reads a document, finds the places where a substance or a reaction is named, and writes down what it found and at which character offset.

That annotator is C#. The search tier next to it is Java: Lucene, with Spring in front of it. Two runtimes in one product is a real cost, somebody says so at least once a year, and in January somebody said it with a budget attached. Move the annotator onto the JVM, retire the .NET deployment, and the platform team maintains one set of build agents, one profiler, one set of container base images.

I had no argument against any of that. I had one question, which is how long the nightly pass would take afterwards.

It took two hours and sixteen minutes instead of forty, and that is why there is no Java annotator. What follows is the more useful half of the story: why the C# version was fast, and why almost none of the reasons had a cheap translation.

What the pipeline does, two point six million times

Per document, the work is unglamorous:

1. Normalise the text — Greek letters, dashes that are not hyphens, the four different Unicode characters that a typesetter will use for a prime.
2. Tokenise it, with rules that know a chemical name is not English.
3. Classify each token — word, numeral, formula-shaped, punctuation.
4. Match token runs against a dictionary of a few million names and synonyms.
5. Emit annotations: start offset, length, the identifier matched, a confidence.

None of that is expensive. A four kilobyte abstract goes through in tens of microseconds. The problem is entirely that there are 2.6 million documents in the corpus and the full pass has to finish inside a nightly window it shares with the index rebuild. At that scale the cost of the pipeline is not the arithmetic. It is how much garbage the arithmetic makes.

Half of "C# is fast" was actually "stop running on .NET Framework"

I should get this out of the way, because it is the least interesting reason and the largest single win.

The annotator started on .NET Framework 4.7.2 with a WCF service in front of it, because that is what the product was built on. Moving the annotation library to .NET Core, and eventually to .NET 5, took it from runs on the Windows build agents to runs anywhere and made it roughly a third faster without a line of logic changing. Some of that is six years of JIT and collection improvements. Some of it is that server GC and concurrent collection are things you actually configure now:

<PropertyGroup>
  <TargetFramework>net6.0</TargetFramework>
  <ServerGarbageCollection>true</ServerGarbageCollection>
  <!-- A batch pass has no latency requirement. Let the collector use the
       whole machine instead of doing background work politely. -->
  <ConcurrentGarbageCollection>false</ConcurrentGarbageCollection>
  <InvariantGlobalization>true</InvariantGlobalization>
  <TieredPGO>true</TieredPGO>
</PropertyGroup>

TieredPGO is opt-in on .NET 6 and worth the one line: the JIT recompiles hot methods using counts it collected from the tier-0 version, so the guarded devirtualisation actually guesses the type our dictionary matcher sees rather than the type the signature admits. On the annotator it was about four percent. Not a headline, but free.

If you are comparing runtimes and one side is .NET Framework, you are not comparing runtimes. Fix that first or your benchmark is about 2018.

Where the speed actually comes from

Four things, and they are all the same thing: not allocating.

A slice is not an object

ReadOnlySpan<char> is a length and a pointer into memory that somebody else owns. Slicing a string is arithmetic — no copy, no allocation, no GC involvement:

public readonly record struct Token(int Start, int Length, TokenKind Kind);

public ref struct ChemTokenizer
{
    private readonly ReadOnlySpan<char> _text;
    private int _at;

    public ChemTokenizer(ReadOnlySpan<char> text) => (_text, _at) = (text, 0);

    public bool TryNext(out Token token)
    {
        while (_at < _text.Length && _text[_at] == ' ') _at++;
        if (_at == _text.Length) { token = default; return false; }

        var start = _at;
        var kind = Classify(_text[_at]);
        while (_at < _text.Length && Continues(kind, _text[_at])) _at++;

        token = new Token(start, _at - start, kind);
        return true;
    }
}

A Token is twelve bytes of value. A thousand of them live in a Token[1000], which is one allocation of twelve kilobytes, contiguous, and the loop that walks it touches one cache line per ten tokens. The token never carries the text — it carries where the text is, and the caller does text.Slice(token.Start, token.Length) when it actually needs the characters, which is less often than you would think.

The Java version of that type is a class. Token[1000] is an array of a thousand references to a thousand objects, each with a twelve-byte header padded to sixteen, each somewhere else in memory. And Token holding a String means substring, which has copied its characters since 7u6 in 2012. So the idiomatic port allocates a token object and a string per token, against zero.

Project Valhalla is the answer to this and I have been reading about it since 2014. It is not something I can put in a nightly job this quarter.

Generics over value types are real generics

Dictionary<long, int> in .NET is a hash table of long keys and int values, laid out inline, because the runtime compiles a distinct specialisation of the generic type for each value-type argument.

HashMap<Long, Integer> is a hash table of references, because Java erases generics and only reference types can be arguments. Each entry is a Node object holding a boxed Long and a boxed Integer. Three objects and two pointer hops for a mapping we needed forty million times a pass.

The workarounds exist and are good — we used fastutil's Long2IntOpenHashMap in the port, which is exactly the right library. It is also a dependency and a different API for every primitive pair, and this is the second time already that the fast Java version and the natural Java version are different programs.

Renting instead of allocating

Per-document buffers came out of a pool, not out of new:

var buffer = ArrayPool<char>.Shared.Rent(text.Length);
try
{
    var length = Normalise(text, buffer);
    Annotate(buffer.AsSpan(0, length), sink);
}
finally
{
    ArrayPool<char>.Shared.Return(buffer);
}

Short buffers skipped the pool entirely and went on the stack — stackalloc char[256] for the normalisation scratch space in the formula matcher, which is a bump of the stack pointer and nothing else.

There is no stackalloc in Java. There is escape analysis, and C2's escape analysis is genuinely clever, but it scalar-replaces objects it can prove never escape after inlining, and our buffers were passed into a method that was too big to inline and stored into a sink that outlived the frame. It never fired where we needed it.

One loop of SIMD, because one loop deserved it

Character classification is embarrassingly parallel across characters, so the hot classifier compares sixteen at a time:

if (Avx2.IsSupported && remaining >= Vector256<ushort>.Count)
{
    var chunk = Vector256.Create(source.Slice(offset));   // 16 chars
    var isDigit = Avx2.And(
        Avx2.CompareGreaterThan(chunk.AsInt16(), Zero),   // >= '0'
        Avx2.CompareGreaterThan(Nine, chunk.AsInt16()));  // <= '9'
    // ...one mask per class, then a single movemask per chunk.
}

Thirty lines, a scalar fallback behind Avx2.IsSupported, and about eleven percent of the pass. System.Runtime.Intrinsics has been in the box since .NET Core 3.0.

Java's Vector API is jdk.incubator.vector — second incubation in 18, which shipped last month. Adding --add-modules for an incubating module to a production build was not a conversation I was going to open.

The port, written the way Java is written

Two engineers, four weeks, a faithful translation. It passed the annotation regression suite on the first corpus slice, which was a genuinely good sign and the reason the next number was so annoying.

Same machine, same 40 GB corpus, sixteen cores, warm page cache:

                        full pass   docs/s/core   allocated/doc
C# on .NET 6              40 min          1,180          3.1 KB
Java 17, first port     2 h 16 min          340           61 KB
Java 17, after tuning   1 h 06 min          720            9 KB

The middle row is the honest first result and nobody was upset by it — a first port is allowed to be slow. The interesting row is the third one, and what it cost.

Two weeks of making the Java version not look like Java

The tuning was not clever. It was the same four moves, applied by hand:

• Tokens stopped being objects. A TokenRun held int[] starts, int[] lengths and byte[] kinds with a count, reused per document — a struct-of-arrays, written out longhand.
• Nothing called substring. Everything took (char[] buffer, int offset, int length), and every method signature grew by two parameters.
• fastutil everywhere a map had primitive keys.
• Buffers were pooled per worker thread and cleared, not reallocated.

That got us to 1.6x off the C# number, and there was probably another twenty percent available. But look at what the code had become: manual offset arithmetic, parallel arrays, three-parameter signatures where the C# version passes one ReadOnlySpan<char>. Every reviewer's first instinct on every pull request was going to be to tidy it back into objects.

Here is the part I want to be fair about. That style is not a Java failing — it is precisely how Lucene is written, and Lucene is faster than anything I have written in any language. BytesRef is a byte[], an offset and a length. CharTermAttribute hands you a reused char[] and a length rather than a String. Lucene's term dictionary packs integers into bit-widths by hand. The Java sitting next to our annotator, doing more work than our annotator, is fast because it made all four of those moves a decade earlier and built its whole API around them.

So the JVM was never the ceiling. What was true is narrower and it is the thing I would say to anyone planning this kind of port: the design that made the C# version fast had no cheap expression in Java, and the port was budgeted as a translation. A Span<T> and a twelve-byte struct are two lines of C#. Their Java equivalents are an architecture. We had money for a translation, and a translation of that design lands at three times the runtime.

What the JVM was plainly better at

Since I was making the case against the port, I owed the room the other list. None of it has stopped being true.

Profiling. async-profiler with a flame graph off a running production JVM, and JFR always on at one percent overhead, were better than anything I had on .NET at the time. dotnet-counters and dotnet-trace work, and PerfView is powerful in the way a nuclear reactor is powerful. I spent more time getting a usable .NET profile than a usable JVM one.

Steady-state code that does allocate. The JVM's young generation is very good, and for allocation-heavy long-lived services C2 plus G1 will often win. Our pipeline was the wrong shape to show that off; a request-scoped service with a lot of short-lived objects is the right shape.

Pause times, if you need them. ZGC in 17 gave sub-millisecond pauses on heaps where .NET 6 still has visible gen-2 work. Irrelevant for a batch pass, decisive for the search tier.

Benchmark culture. JMH is better than BenchmarkDotNet, and BenchmarkDotNet is very good. Blackhole, @Fork, the fact that the tool argues with you about your methodology — JMH assumes you are about to fool yourself and it is usually right.

How we measured, so the numbers meant something

Three rules, agreed before anyone wrote a benchmark, because I have seen this argument poisoned by bad measurement more than once:

The unit of comparison was the pass, not the microbenchmark. JMH and BenchmarkDotNet ran too, on tokenisation of a fixed 4 KB abstract — 38 µs and zero allocations against 141 µs and 34 KB. But those numbers only ever supported a claim about tokenisation. The decision was made on the wall clock of the full corpus, twice, on the same box.

Allocation rate was reported next to throughput, always. A pass that is ten percent slower and allocates twenty times more is not ten percent worse; it is a different program that has not met a full heap yet. GC.GetTotalAllocatedBytes on one side, JFR's allocation events on the other.

Warmup was explicit and generous. Sixty seconds of corpus before the timer started, on both. Comparing a cold JVM against a ReadyToRun .NET binary would have handed me the answer I already wanted, and I would not have believed it.

We also ran the tuned Java version against the untuned C# version, which is the comparison nobody asks for. It won. Whichever runtime you are on, most of the distance is in how the code treats memory, and only the last stretch is the runtime.

What actually happened

The proposal came back as a smaller one, and I supported it: the service around the annotator moved, and the annotator did not. The HTTP surface, the job orchestration and the queue plumbing are now on the JVM next to the search tier, and the annotation library stays a .NET process the service shells out to over a local socket, with a JSON contract that is fifty lines long. Two runtimes, one deployment story, and the nightly window stayed at forty minutes.

That is not the tidy answer. It is the one that survived the numbers.

I do not think this makes C# faster than Java. I think it made the program faster than the port, for a reason that gets flattened into "the speed of the language" by the time it reaches a status report. The accurate version is that C# let me write the fast design without leaving the language everybody on the team already reads, and today Java does not. Valhalla will eventually close most of that gap and I will be pleased when it does. It is not going to close it before the next release.