We continue to get acquainted with not very famous JEPs. This time, we have another sub-project from Valhalla, which is called nestmates . This feature allows nested classes to have unlimited access to each other. How exactly is described below. In essence, this article is a formal translation of JEP 181 , for it quite clearly describes the essence of the question for a beginner.

(If you need not a novice look, but some kind of expert opinion, then it is better to contact the experts - Nikita Lipsky, Tagir Valeev, Vladimir Ivanov and others. Most likely, they can be caught on Joker 2017. In August, early prices are valid by the way).

Let me remind you that JEP is JDK Enchancement Proposal, the process of collecting suggestions and improvements for OpenJDK, allowing committers to work more informally, until the release of the official formal JSR. You can think of JEPs as an OpenJDK strategic plan.

For an application programmer (not a committer in OpenJDK), such knowledge is hardly useful for writing code directly - instead, they provide a general architectural understanding of the Java platform, teach you how to think strategically, and allow you to predict the near future.

A few words for those who read a recent article on Minimum Value Types . (The article on the practical aspects of MVT is being written right now ) It was told about vwithfield vwithfield , which allows you to take the selected type-value and replace the field in it. Such a method should obviously not be in the most naked "value", but in its object wrapper. Logically, only a class with private access to the language object can change its fields. Accordingly, a special trust relationship should arise between the value-type components (VCC and DVT), the implementation of which I really want to have directly in the JVM.

Therefore, it is assumed that some future versions of the JVM can implement explicit nestmates at the VM level, which will have access to each other’s private fields and methods. This will not be a compiler hack for generating wrappers, but quite a native feature. In these versions of the JVM, the vwithfield instruction will be available to all nestmates of each specific value type. In other words, vwithfield will be available inside a kind of “capsule” in which all private methods are available.

Okay, stop talking, go to the JEP itself!

JEP 181: Access Control for Nested Classes

Short description

You must associate access checks in the JVM with the Java language rules defined for methods, constructors, and fields in nested classes, dividing the classes into nests — that is, groups of related classes that have common access control context (and usually come from single source file). In particular, to allow the class file to access to the private names of another class file, if all of them are compiled in the context of the same top-level type. This is necessary so that the compiler does not have to set up adapter methods that forward an increased level of access.

Goals

Extend the functionality of a virtual machine so that compilers can group classes into sockets (nest) that have a common access control context. This will allow classes to compile into individual class files, logically continuing to be part of one common entity (such as an internal class in Java), which will allow access to members of each other without creating special adapter methods.

Add the ability to accurately describe the nesting of classes and interests within class files.

Perform preparatory work in a virtual machine that will form the basis of related functionality, such as a safe and well-supported alternative to Unsafe.defineAnonymousClass() and sealed classes.

Restrictions

This JEP does not deal with improving other large-scale access control projects, such as modules.

Motivation

Many JVM languages ​​can declare several classes in one file (for example, Java has nested classes), or translate other sources (not classes!) Into class files. Meanwhile, from the user's point of view, all these things seem to be parts of the “same class”, and therefore the user intuitively assumes that the same security mode is applied to them. Trying to meet expectations, compilers often have to reduce the severity of checks of private members of a class to a package , using methods of adapters. Unfortunately, such adapters destroy all encapsulation, can lead to errors in various tools, or simply cause misunderstanding of the user. If we introduce a formal definition for a group of classfiles that form a nest in which neighbors ( nestmate ) are united by a common access control mechanism, this will achieve the same result more quickly, safely and transparently for all.

Description

The Java language specification allows classes and interfaces to nest within each other. JLS 7.6 introduces the notion of top-level type declaration. You can nest any number of nested types. About the top level type and all types inside it, it can be said that they "form a nest ". Two members of the nest are called neighbors in this nest ( nestmates ). Neighbors have unrestricted access to each other ( JLS 6.6.1 ), including private fields, methods, and constructors. This access extends completely to everything inside a top-level type declaration that contains all other types. (You can think of this top-level type as a “mini-package”, within which everyone is given extended access — broader than what is provided to members of this Java package, where they all lie).

The Java compiler compiles a group of nested types into its corresponding file class group. To materialize this nesting ( JVMS 4.7.6 , JVMS 4.7.7 ), it uses the InnerClasses and EnclosingMethod . These attributes are enough for the JVM to identify the neighborhood, and so that we can give our neighbors a broader and more general definition than if we considered them simply nested types. In order to increase efficiency, it is proposed to change the class file format by adding a couple of new attributes used by both neighbors and the upper-level class (which is called the nest top , nest top ). Each neighbor will have an attribute pointing to the vertex, and each vertex will have an attribute pointing to the known neighbors.

We will slightly change the access rules in the JVM by adding something like this in JVMS 5.4.4 :

A field or method R is accessible from a class or interface D if and only if one of the following conditions is met:

• ...
• R has private access, defined in another class or interface C , such that C and D are neighbors

For C and D become neighbors, they must have the same vertex. Type C declares itself to be a member of nest D , adding D to its MemberOfNest attribute. This neighborhood is checked when D also added C its NestMembers attribute. The check is started only when trying to access a private member of a neighbor. Because of this, some types will have to be loaded before this could have happened in other cases.

After the introduction of these new rules, and the corresponding changes in bytecode, no more javac wrappers will be needed, since javac will be able to generate instructions for directly calling private members of the neighbors. Relevant bytecodes for a call:

• invokespecial for private designers
• invokevirtual for private (non-interface) instance methods
• invokeinterface for private interface methods
• invokestatic for private static methods

After implementing the concept of neighbors and the corresponding rules in the VM code, we will not only let javac discard this role and improve existing access checks, but also help other projects:

• With the specialization of generics , any specialized type can be created as a neighbor for a generic
• A safe and well-supported replacement for the Unsafe.defineAnonymousClass() API will be able to create new classes, making them neighbors of existing ones
• The concept of "sealed classes" can be implemented as an opportunity to create only those subclasses that are neighbors

Softer access rules will affect checks in the following things:

• disclosure of fields and methods ( JVMS 5.4.3.2 , etc.)
• expansion method handle constants ( JVMS 5.4.3.5 )
• disclosure of call site specifiers ( JVMS 5.4.3.6 )
• checking access at the Java language level using java.lang.reflect.AccessibleObject instances
• access check when making requests to java.lang.invoke.MethodHandles.Lookup

You may have to consider interacting with other constraints that apply to calling bytecodes, for example:

• private interface methods must be invoked via invokespecial ( JVMS 6.5 )

and on the semantics of calling MethodHandle (which mirrors bytecode restrictions). Something will have to change in the general descriptions, such as JVMS 3.7 .

By starting to control access to neighbors at the virtual machine level, we can limit all other adapter methods generated using javac to the private level of access, rather than package-private, as is being done now. Many where they are not required at all.

Actual problems:

1. The suggestions made here extend the boundaries of access to neighbors, right at the VM level. If we are already engaged in this, should we not restrict access to classes declared as protected and private , thereby more precisely following the rules of the Java language? This will require the JVM to perform additional checks based on the value of Class.getModifiers . (Most likely, they should not, because it can break such code using reflection, which suggests that private access has been extended to package-private. In addition, new checks on protected classes can lead to global effects, since they are presented to the JVM as public).
2. Should the “neighborhood” relationship be seen with reflection? The basic reflection methods, of course, will have to be corrected so that they take into account the rules of access to the neighbors. But whether it is worth making a check for neighborhood in a public API - you will have to deal with this. The simplest query you can think of is: Class#getNestTop , which returns either null if the class is not part of the socket, or the top of the socket. (If the class itself is the top, the query will return itself).

Alternatives

We can continue to generate wrappers in the Java compiler. It is difficult to predict something. For example, in Project Lambda it was difficult to uncover references to methods if internal classes were involved in this disclosure, and this led to the creation of a new adapter method. Since compilers-generated wrappers are tricky and unpredictable, there are many bugs in them, and it is very difficult to analyze them with the help of tools, including decompilers and debuggers.

At the beginning, to define the neighborhood, we suggested using only the existing InnerClasses and EnclosingMethod . But the creation of special attributes of the neighborhood brought the question to a more general level than simply reflecting nested types at the Java language level, and allowed us to come up with a more efficient implementation.

Testing

We will have to develop a large set of tests for the JVM, which will check the access rules and changes to the semantics of bytecodes, which were introduced specifically to implement the neighborhood.

Similarly, you will have to write additional tests for reflection, references to methods, var-handles, and external access to standard APIs such as JDWP, JVM TI and JNI.

Since we do not offer any tests in the language, then tests for conformity to the language, too, do not have to write.

Adequate functional tests may themselves appear from language tests as soon as the Java compiler starts using the features described above.

Risks and assumptions

All these innovations will have to be associated with the new version number of the class file, since the rules by which the Java compiler works are changed.

To maintain backward compatibility with older versions of the JVM, the Java compiler will need to support the outdated wrapper generation logic.

Softer access rights should not break anything. As an exception, in theory, negative compliance tests may break.

The risk of losing compatibility is very small or absent altogether, since it is proposed not to tighten the access rules, but on the contrary, to make them softer. If users “discovered” the presence of magic wrapper methods and figured out how to use their existence, then after the introduction of our changes, they will no longer be able to do this. This risk is very small, because, first of all, such wrappers do not have stable names.

The risk of breaking the integrity of the platform is very small or absent, since the proposed rules assign new access rights only within a specific runtime package. By eliminating the need for adapter methods, we systematically reduce the chance of access between individual upper-level classes.

Neighborhood validation requires a high-level class, even if it is not in itself used (except as a container for nested members). This may have an impact on such tests or applications that throw unused classes from their distributions.

Impact on other systems

New descriptions will be required in the JVM specification, as well as changes to the JVM implementation. In addition, changes will be required in the specification and implementation of reflexion, references to methods, var-handles, and possibly JVM TI, JDWP and JNI (however, native interfaces usually ignore access rights - it may turn out that there is almost nothing to do here).

It is necessary to investigate how performance changes when performing additional access checks.

The current Java compiler generates adapter methods for accessing between sockets. Despite the fact that in this document we do not demand to stop generating them, nevertheless, it is better to throw them away as soon as they become useless.

Matching rules between a Java source and a classfile will be easier. And this is very timely, because Project Lambda complicates the same rules. However, some effects also occur at the intersection of products (for example, JDK-8005122 ), so the recent increase in complexity cannot be considered ordinary growth.

Throwing out adapters may slightly reduce the size of some applications.

You may have to correct the Pack200 specification.

The authors

John Rose is a JVM engineer and architect at Oracle. Lead Engineer Da Vinci Machine Project (part of OpenJDK). Lead Engineer JSR 292 (Supporting Dynamically Typed Languages ​​on the Java Platform), deals with the specification of dynamic calls and related issues such as type profiling and advanced compiler optimizations. Previously, he worked on inner classes, did the original HotSpot port on SPARC, the Unsafe API, and also developed many dynamic, parallel and hybrid languages, including Common Lisp, Scheme ("esh"), dynamic binding for C ++.

Translators

Oleg Chirukhin - at the time of writing this text, he is working as an architect in the company Sberbank-Technologies, developing architecture for automated business process management systems. Before moving to Sberbank-Technologies, he participated in the development of several state information systems, including state services and the electronic medical map, as well as in the development of online games. Speaker at JUG.ru conferences (JPoint, JBreak). Current research interests include virtual machines, compilers, and programming languages.