# JNI Zero
A zero-overhead (or better!) middleware for JNI.

## Overview
JNI (Java Native Interface) is the mechanism that enables Java code to call
native functions, and native code to call Java functions.

 * Native code calls into Java using apis from `<jni.h>`, which basically mirror
   Java's reflection APIs.
 * Java code calls native functions by declaring body-less functions with the
  `native` keyword, and then calling them as normal Java functions.

JNI Zero generates boiler-plate code with the goal of making our code:
 1. easier to write,
 2. typesafe,
 3. more optimizable.

JNI Zero uses regular expressions to parse .java files, so don't do
anything too fancy :).

### Exposing Native Methods

There are two ways to have native methods be found by Java:
1) Explicitly register the name -> function pointer mapping using JNI's
   `RegisterNatives()` function.
2) Export the symbols from the shared library, and let the runtime resolve them
   on-demand (using `dlsym()`) the first time a native method is called.

(2) Is generally preferred due to a smaller code size and less up-front work, but
(1) is sometimes required (e.g. when OS bugs prevent `dlsym()` from working).
Both ways are supported by this tool.

### Exposing Java Methods

Java methods just need to be annotated with `@CalledByNative`. By default the
generated method stubs on the native side are not namespaced. The generated
functions can be put into a namespace using `@JNINamespace("your_namespace")`.

## Usage

### Calling Java -> Native

- For each JNI method:
  - C++ stubs are generated that forward to C++ functions that you must write.
    By default the c++ functions you are expected to implement are not
    associated with a class.
  - If the first parameter is a C++ object (e.g.
    `long native${OriginalClassName}`), then the bindings will not call a static
    function but instead cast the variable into a cpp `${OriginalClassName}`
    pointer type and then call a member method with that name on said object.

To add JNI to a class:

1. Create a nested-interface annotated with `@NativeMethods` that contains
   the declaration of the corresponding static methods you wish to have
   implemented.
2. Call native functions using `${OriginalClassName}Jni.get().${method}()`
3. In C++ code, #include the header `${OriginalClassName}_jni.h`. (The path will
   depend on the location of the `generate_jni` BUILD rule that lists your Java
   source code.)

Note: Include this header from only a single `.cc` file as the header defines
functions. That `.cc` must implement your native code by defining non-member
functions named `JNI_${OriginalClassName}_${UpperCamelCaseMethod}` for static
methods and member functions named
`${OriginalClassName}::${UpperCamelCaseMethod}` for non-static methods. Member
functions need be declared in the header file as well.

#### Example:
**Java**
```java
class MyClass {
  // Cannot be private. Must be package or public.
  @NativeMethods
  /* package */ interface Natives {
    void foo();
    double bar(int a, int b);
    // Either the |MyClass| part of the |nativeMyClass| parameter name must
    // match the native class name exactly, or the method annotation
    // @NativeClassQualifiedName("MyClass") must be used.
    //
    // If the native class is nested, use
    // @NativeClassQualifiedName("FooClassName::BarClassName") and call the
    // parameter |nativePointer|.
    void nonStatic(long nativeMyClass);
  }

  void callNatives() {
    // MyClassJni is generated by the generate_jni rule.
    // Storing MyClassJni.get() in a field defeats some of the desired R8
    // optimizations, but local variables are fine.
    Natives jni = MyClassJni.get();
    jni.foo();
    jni.bar(1,2);
    jni.nonStatic(mNativePointer);
  }
}
```

**C++**
```c++
#include "third_party/jni_zero/jni_zero.h"
#include "<path to BUILD.gn>/<generate_jni target name>/MyClass_jni.h"

class MyClass {
public:
  void NonStatic(JNIEnv* env);
}

// Notice that unlike Java, function names are capitalized in C++.
// Static function names should follow this format and don't need to be declared.
void JNI_MyClass_Foo(JNIEnv* env) { ... }
void JNI_MyClass_Bar(JNIEnv* env, jint a, jint b) { ... }

// Member functions need to be declared.
void MyClass::NonStatic(JNIEnv* env) { ... }
```

### Calling Native -> Java

1. Annotate some methods with `@CalledByNative`, the generator will now generate
   stubs in `${OriginalClassName}_jni.h` header to call into those java methods
   from cpp.
   * Inner class methods must provide the inner class name explicitly
     (ex. `@CalledByNative("InnerClassName")`)

2. In C++ code, `#include` the header `${OriginalClassName}_jni.h`. (The path
   will depend on the location of the `generate_jni` build rule that lists your
   Java source code). That `.cc` can call the stubs with their generated name
   `Java_${OriginalClassName}_${UpperCamelCaseMethod}`.

Note: For test-only methods, use `@CalledByNativeForTesting` which will ensure
that it is stripped in our release binaries.

Note: Because the generated header files contain definitions as well as declarations,
they must not be `#included` by multiple sources. If there are Java functions
that need to be called by multiple sources, one source should be chosen to
expose the functions to the others via additional wrapper functions.

### Writing Build Rules
1. Find or add a `generate_jni` target with your .java file, then add its `_java`
   subtarget to your `deps`.

   ```python
   generate_jni("abcd_jni") {
     sources = [ "path/to/java/sources/with/jni/Annotations.java" ]
   }

   android_library("abcd_java") {
     ...
     # For the generated `${OriginalClassName}Jni` classes.
     deps = [ ":abcd_jni_java" ]
   }

   source_set("abcd") {
    ...
    # Allows the cpp files to include the generated `${OriginalClassName}_jni.h`
    # headers.
    deps = [ ":abcd_jni" ]
   }
   ```

### Automatic Type Conversions using @JniType

Normally, Java types map to C++ types from `<jni.h>` (`jobject` for
reference types, `jint` for `int`, etc). The first thing most people do is
convert the jni spec types into standard C++ types.

`@JniType` to the rescue. By annotating a parameter or a return type with
`@JniType("cpp_type_here")` the generated code will automatically convert from
the jni type to the type listed inside the annotation. See example:

#### Original Code:
```java
class MyClass {
  @NativeMethods
  interface Natives {
    void foo(
            String string,
            String[] strings,
            MyClass obj,
            MyClass[] objs)
  }
}
```

```c++
#include "third_party/jni_zero/jni_zero.h"
#include "<path to BUILD.gn>/<generate_jni target name>/MyClass_jni.h"

void JNI_MyClass_Foo(JNIEnv* env, const JavaRef&, const JavaRef&, const JavaRef&, JavaRef&) {...}
```

#### After using `@JniType`
```java
class MyClass {
  @NativeMethods
  interface Natives {
    void foo(
            @JniType("std::string") String convertedString,
            @JniType("std::vector<std::string>") String[] convertedStrings,
            @JniType("myModule::CPPClass") MyClass convertedObj,
            @JniType("std::vector<myModule::CPPClass>") MyClass[] convertedObjects);
  }
}
```
```c++
#include "third_party/jni_zero/jni_zero.h"
#include "<path to BUILD.gn>/<generate_jni target name>/MyClass_jni.h"

void JNI_MyClass_Foo(JNIEnv* env, std::string&, std::vector<std::string>>&, myModule::CPPClass&, std::vector<myModule::CPPClass>&) {...}
```

#### Implementing Conversion Functions

Conversion functions must be defined for all types that appear in `@JniType`.
Forgetting to `#include` the header that defines it will will result in a
compile error.

```c++
// The conversion function primary templates.
template <typename O>
O FromJniType(JNIEnv*, const JavaRef<jobject>&);
template <typename O>
ScopedJavaLocalRef<jobject> ToJniType(JNIEnv*, const O&);
```

An example conversion function can look like:

```c++
#include "third_party/jni_zero/jni_zero.h"

namespace jni_zero {
template <>
EXPORT std::string FromJniType<std::string>(
    JNIEnv* env,
    const JavaRef<jstring>& input) {
  // Do the actual conversion to std::string.
}

template <>
EXPORT ScopedJavaLocalRef<jstring> ToJniType<std::string>(
    JNIEnv* env,
    const std::string& input) {
  // Do the actual conversion from std::string.
}
}  // namespace jni_zero
```

#### Array Conversion Functions

Array conversion functions look different due to the partial specializations.
The `ToJniType` direction also takes a `jclass` parameter which is the class of the
array elements, because java requires it when creating a non-primitive array.

```c++
template <typename O>
struct ConvertArray {
  static O FromJniType(JNIEnv*, const JavaRef<jobjectArray>&);
  static ScopedJavaLocalRef<jobjectArray> ToJniType(JNIEnv*, const O&, jclass);
};
```

JniZero provides implementations for partial specializations to wrap and unwrap
`std::vector` for object arrays and some primitive arrays.

#### Nullability

All non-primitive default JNI C++ types (e.g. `jstring`, `jobject`) are pointer
types (i.e. nullable). Some C++ types (e.g. `std::string`) are not pointer types
and thus cannot be `nullptr`. This means some conversion functions that return
non-nullable types have to handle the situation where the passed in java type is
null.

JNI Zero defines conversions functions for `std::optional<T>` that will treat
`nullptr` as missing.

### Testing Mockable Natives

```java
/**
 * Tests for {@link AnimationFrameTimeHistogram}
 */
@RunWith(RobolectricTestRunner.class)
public class AnimationFrameTimeHistogramTest {
    // Optional: Resets test overrides during tearDown().
    // Not needed when using Chrome's test runners.
    @Rule public JniResetterRule jniResetterRule = new JniResetterRule();

    @Mock
    AnimationFrameTimeHistogram.Natives mNativeMock;

    @Before
    public void setUp() {
        MockitoAnnotations.initMocks(this);
        AnimationFrameTimeHistogramJni.setInstanceForTesting(mNativeMock);
    }

    @Test
    public void testNatives() {
        AnimationFrameTimeHistogram hist = new AnimationFrameTimeHistogram("histName");
        hist.startRecording();
        hist.endRecording();
        verify(mNativeMock).saveHistogram(eq("histName"), any(long[].class), anyInt());
    }
}
```

### Special case: APK Splits
Each APK split with its own native library has its own generated `GEN_JNI`, which is
`<module_name>_GEN_JNI`. In order to get your split's JNI to use the `<module_name>` prefix, you
must add your module name into the argument of the `@NativeMethods` annotation.

So, for example, say your module was named `test_module`. You would annotate
your `Natives` interface with `@NativeMethods("test_module")`, and this would
result in `test_module_GEN_JNI`.


### How to Know if Native is Loaded?

You must call `System.loadLibrary("libname")` before making JNI calls, and it
is up to each application to do so. Using an app's `Application` subclass is a
good place to do this.

#### Chrome-specific Guidance

 * Be careful of logic in early initialization where native has not yet been
   loaded.
 * For tests, JNI Zero's check to ensure that all C++ implementations of JNI
   methods are included is disabled (due to too many violations). If you hit a
   `UnsatisfiedLinkError`, it's likely that you are missing a `deps` entry onto
   the code that implements that C++ side of a JNI method.

When you hit a scenario leading to an exception, relocate (or defer) the appropriate call to
be made to a place where (or time when) you know the native libraries have been initialized (eg.
`onStartWithNative`, `onNativeInitialized` etc).

Avoid calling `LibraryLoader.isInitialized()` / `LibraryLoader.isLoaded()`,
because the tell you only whether `System.loadLibray()` has been called, and
often return "true" in Robolectric tests, which contain only a small number of
JNI methods.

One robust solution is to use your own "`sIsNativeReady`" flag that is set via
a `@CalledByNative` method.

### Java Objects and Garbage Collection

All pointers to Java objects must be registered with JNI in order to prevent
garbage collection from invalidating them.

For other objects - use smart pointers to store them:
 * `ScopedJavaLocalRef<>` - When lifetime is the current function's scope.
 * `ScopedJavaGlobalRef<>` - When lifetime is longer than the current function's
   scope.
 * `LeakedJavaGlobalRef<>` - For singletons (avoids having a destructor).
 * `JavaObjectWeakGlobalRef<>` - Weak reference (does not prevent garbage
   collection).
 * `JavaRef<>&` - Use to accept any of the above as a parameter to a
   function without creating a redundant registration.

### Additional Guidelines / Advice

Minimize the surface API between the two sides. Rather than calling multiple
functions across boundaries, call only one (and then on the other side, call as
many little functions as required).

If a Java object "owns" a native one, store the pointer via
`"long mNativeClassName"`. Ensure to eventually call a native method to delete
the object. For example, have a `close()` that deletes the native object.

## Build Rules

 * `generate_jni` - Given a set of Java files, generates a header file to call
   into Java for all `@CalledByNative` functions. If `@NativeMethods` is
   present, also generates a `.srcjar` containing `<ClassName>Jni.java`, which
   should be depended on via the generated GN target
   `<generate_jni's target name>_java`.
 * `generate_jar_jni` - Given a `.jar` file, generates a header file similar to
   `generate_jni`, if every method and public field were annotated by
   `@CalledByNative`.
 * `generate_jni_registration` - Generates a whole-program Java and native
   link - required for all Java that calls into native via `@NativeMethods`.
 * `shared_library_with_jni` - A wrapper around a native `shared_library`, which
   also inserts a `__jni_registration` target for the library.
 * `component_with_jni` - Same as `shared_library` but for a `component`.

Refer to [jni_zero.gni](https://source.chromium.org/chromium/chromium/src/+/main:third_party/jni_zero/jni_zero.gni)
for more about the GN templates.

## JNI Benchmarking
Refer to the [performance
README.](https://source.chromium.org/chromium/chromium/src/+/main:third_party/jni_zero/benchmarks/README.md)

## Under the Hood
For `@CalledByNative`, we directly call the `<jni.h>` methods, which are
basically just reflection APIs, and then add a proguard rule to ensure the
annotated method/field is kept in Java. The registration step does nothing for
this direction of JNI, since we do not do any sort of proxying. However, using
the registration step for `@CalledByNatives` has been discussed before:
[go/proxy-called-by-natives-proposal](http://go/proxy-called-by-natives-proposal).

JNI Zero has 2 primary modes for `@NativeMethods`. In each, we insert a "proxy"
class per annotated class which allows us to fake for tests and optimize better.
We insert a class with the name of `<EnclosingClass>Jni`, and this class is just
a testable shim into the "real" `GEN_JNI` class. This `GEN_JNI` class is
generated at the registration step, and how the registration works is different
in different modes.

For examples, we will imagine we have the following two classes:
```java
class org.foo.Foo {
  @NativeMethods
  interface Natives {
    int f();
  }
}
class org.bar.Bar {
  @NativeMethods
  interface Natives {
    int b();
  }
}
```
Which will have the 2 `generate_jni` steps output something like:
```java
// Java .srcjar outputs
class FooJni {
  public int f() {
    return GEN_JNI.org_foo_Foo_f();
  }
}
class BarJni {
  public int b() {
    return GEN_JNI.org_bar_Bar_b();
  }
}
```
```c++
// C++ header outputs
class FooJni {
int Java_GEN_JNI_org_foo_Foo_f() {
  return JNI_Foo_f(); // User implements this native function.
}
int Java_GEN_JNI_org_bar_Bar_b() {
  return JNI_Bar_b(); // User implements this native function.
}
```

### Debug Mode
In debug mode, the `GEN_JNI` is a file containing `native` methods that match
every single `@NativeMethods` from every `generate_jni` in our program.
```java
class GEN_JNI {
  public static native int org_foo_Foo_f();
  public static native int org_bar_Bar_b();
}
```

### Release Mode
In release mode, the `GEN_JNI.java` is just a callthrough shim to `N.java` (a
short name to reduce size), and `N` uses multiplexing by signature type to
reduce the number of JNI functions. Then, we generate a C++ file with matching
names to the smaller list of functions in `N`, which de-multiplexes back into
the original functions.
```java
class GEN_JNI {
  public static int org_foo_Foo_f() {
    return N._I(0);
  }
  public static int org_bar_Bar_b() {
    return N._I(1);
  }
}
class N {
  public static native int _I(int switchNum);
}
```
```C++
// Generated C++ to be compiled into the final binary.
int Java_N__1V(jint switch_num) {
  switch (switch_num) {
    case 0:
      return org_foo_Foo_f();
    case 1:
      return org_bar_Bar_b();
  }
}
```

We also have the concept of "priority" classes, which are classes which need to
be in the front of the multiplexing numbers. This is not a performance thing,
it's so that Chrome can support multiple ABIs with a single Java file - we put
the smaller (subset) ABI switch numbers first, and the superset ABI's unique
classes get the final switch numbers.

### Legacy Modes
These are modes which JNI provides currently, but we hope to remove. Please do
not add any new uses of these.

#### Using the "native" Keyword

E.g.:
```
class Foo {
    native someMethod();
}
```

This is still supported by default, but is less efficient than `@NativeMethods`
interfaces. We plan to delete support for this.

#### Hashed Names
This was our old release mode. `GEN_JNI` would call into `N`, just as it does
for our current release mode, but instead of multipelxing, we'd just take a
short hash of the name so we have shorter exported string literals. This would
also change the output of the headers made by `generate_jni`, as they needed to
likewise have a hashed name generated.
```java
class GEN_JNI {
  public static int org_foo_Foo_f() {
    return N.MaQxW612();
  }
  public static int org_bar_Bar_b() {
    return N.M2R2WaZb();
  }
}
class N {
  public static native int MaQxW612();
  public static native int M2R2WaZb();
}
```

#### Per-File Natives
This was added to make transitioning to JNI Zero easier. The idea is that this
allows you to partially onboard without needing to use a registration step, so
no `GEN_JNI` is generated at all, and the `generate_jni` step's outputs look
different than "normal" mode.
```java
class FooJni {
  public static int f() {
    nativeF();
  }
  public static native nativeF();
}
class BarJni {
  public static int b() {
    nativeB();
  }
  public static native nativeB();
}
```


## Changing JNI Zero

 * Python golden tests live in `test/integration_tests.py`
 * A working demo app exists as `sample:jni_zero_sample_apk` and this app is
   tested in `sample:jni_zero_sample_apk_test`.
 * Compile-only tests exist in `test:jni_zero_compile_check_apk`
 * We are a Chromium project developed in the Chromium repo, but we intend to
   have no dependencies on Chromium, to allow this project to be easily
   portable.
 * `jni_zero.py` contains our flags and is the entry point, `jni_generator.py`
   is the main file for the per-library generation step, and
   `jni_registration_generator.py` is the main file for the whole-program
   registration step.