Skip to main content

JEP 537: Vector API (Twelfth Incubator)

Introduction​

Modern CPUs hide a capability Java developers have long been able to "see but not touch": vector computation. The hardware can process a whole batch of data in one instruction, yet Java has only ever been able to hope the JIT would use it for you. JEP 537's Vector API puts that control into developers' hands for the first time, letting you write SIMD code explicitly and portably.

Why a vector API​

Modern CPUs are equipped with wide vector registers that can process several values at once in a single instruction, this is SIMD (Single Instruction, Multiple Data). It is the key to performance in numeric-intensive code.

But for Java developers, this capability has always been indirect and hard to control:

  • Modern CPUs compute on wide registers, many "lanes" per instruction.
  • HotSpot has an auto-vectoriser that tries to recognize your loops automatically and compile them into vector instructions. It does help, but only when it recognizes the loop.
  • The trouble is that auto-vectorisation is fragile. A small, seemingly unrelated change to a loop, one extra condition, a different phrasing, can make vectorization silently fail, halving performance in an instant, and the compiler gives you no warning at all. You may not even know you lost the performance.
  • Ultimately, numeric code had no way to ask for SIMD explicitly and portably. You could only passively depend on the JIT's mood.

The programming model​

One of the Vector API's core design goals is portability, your code does not need to know the hardware's vector width.

  • Vector<E> values are grouped by species. A species is the combination of "element type + vector shape." Through this abstraction, the same code runs optimally on 128-bit ARM NEON and 512-bit AVX-512 alike, it automatically adapts to the current hardware's vector width.
  • The API provides a full set of lane-wise operations: arithmetic, comparison, shuffle, reduction, and mask operations.
  • The mask mechanism makes "partial vectors" and "loop tails" first-class. When the array length is not a multiple of the vector width, masks gracefully handle the leftover elements that do not fill a full vector, exactly the kind of edge case that always made hand-written SIMD tricky.
  • Finally, the runtime can compile this code to optimal vector instructions on supported CPUs.

Code example: vectorized array addition​

static final VectorSpecies<Float> SPECIES =
    FloatVector.SPECIES_PREFERRED;

void addArrays(float[] a, float[] b, float[] c) {
    int i = 0;
    for (; i < SPECIES.loopBound(a.length);
         i += SPECIES.length()) {
        var va = FloatVector.fromArray(SPECIES, a, i);
        var vb = FloatVector.fromArray(SPECIES, b, i);
        va.add(vb).intoArray(c, i);
    }
    // scalar tail
    for (; i < a.length; i++) c[i] = a[i] + b[i];
}

This example adds two arrays element by element. Notice that each step of the main loop processes a full vector of data (SPECIES.length() elements), not a single float. SPECIES.loopBound() computes the boundary divisible by the vector width, and the main loop runs up to there; the remaining elements that do not fill a full vector are mopped up by the scalar tail loop, so not one is missed.

This code can be compiled to suitable vector instructions on supported CPUs, and SPECIES_PREFERRED selects the vector width best suited to the current hardware.

Before and after​

Scalar loop:

  • One element per iteration in the source.
  • Performance depends entirely on the JIT recognizing the loop's shape.
  • One unrelated refactor can silently shift performance.

Vector API:

  • The intent to vectorize is written right in the code. You are not hoping the JIT catches your meaning; you tell it explicitly, "vectorize here."
  • The same code adapts to vector shapes across hardware.
  • Results are more predictable across hardware. Performance is no longer a gamble.

In a word: a scalar loop leaves the vectorization decision to the JIT, while the Vector API writes the intent into the source. You will not lose SIMD acceleration for no apparent reason after a seemingly unrelated refactor.

Where it earns its keep​

The Vector API's sweet spot is workloads that process large amounts of primitive data intensively:

  • Linear algebra, signal processing, and image pipelines.
  • Machine-learning inference kernels and embedding math.
  • Cryptography, compression, and checksums.
  • Columnar analytics scanning large primitive arrays.

These domains share one trait: their hot loops do heavy number crunching. The Vector API lets you express those computations directly in Java, while the runtime can map them to supported hardware instructions.

Still incubating​

You might wonder: it is the twelfth incubator, why still not finalized?

The answer is Project Valhalla.

  • This is the twelfth incubator, note that it is an incubator API, not a preview API. Using it requires --add-modules jdk.incubator.vector.
  • The team is deliberately holding it back until Project Valhalla's value types land. The current Vector API makes a series of compromises to work around boxing and object allocation; value types are expected to let the API drop those compromises entirely, achieving a clean, zero-overhead abstraction.
  • The team would rather wait until they can get it right than rush to finalize an API with built-in limitations. That patience is exactly why it has not been finalized.

Conclusion​

The Vector API gives Java developers explicit, portable control over SIMD for the first time, letting you write vector code instead of hoping the JIT understands. It is still incubating, quietly waiting on Valhalla's value types, but its programming model is usable today. For numeric workloads that genuinely need performance, this is a tool worth learning and trying now.