Java interacts with the operating system through methods marked with the native keyword using the system libraries loaded by the System.loadLibrary () procedure.
Download the system library is very simple, but to unload it, as it turned out, you need to put a lot of effort. How exactly the system libraries are unloaded, and why this is necessary, I will try to tell.
Suppose we want to make a small utility that users will run on their computers in the local network. We would like to save users from problems with installing and configuring the program, but there are no resources to deploy and support a centralized infrastructure. In such cases, the program is usually assembled along with all dependencies into a single jar file. This is easy to do with a maven-assembly-plugin or simply export from an IDE Runnable jar. The program will be launched by the command:
java -jar my-program.jar
Unfortunately, this does not work if one of the libraries requires a system dynamic library for its work, in other words, dll. Usually in one of the classes of such a library in a static initializer a call to System.loadLibrary () is made. To load the dll, you need to put it in a directory accessible through the jVM jli.library.path system property. How can this limitation be circumvented?
')
Pack the dll inside the jar file. Before using classes that require loading dlls, create a temporary directory, extract the library there and add the directory to java.library.path. It will look something like this:
prepareLibrary private void addLibraryPath(String pathToAdd) throws ReflectiveOperationException { Field usrPathsField = ClassLoader.class.getDeclaredField("usr_paths"); usrPathsField.setAccessible(true); String[] paths = (String[]) usrPathsField.get(null); String[] newPaths = Arrays.copyOf(paths, paths.length + 1); newPaths[newPaths.length - 1] = pathToAdd; usrPathsField.set(null, newPaths); } private Path prepareLibrary() throws IOException, ReflectiveOperationException { Path dir = Files.createTempDirectory("lib"); try (InputStream input = ExampleClass.class.getResourceAsStream("custom.dll")) { if (input == null) { throw new FileNotFoundException("Can't load resource custom.dll"); } Files.copy(input, dir.resolve("custom.dll")); } addLibraryPath(dir.toAbsolutePath().toString()); return dir; }
Unfortunately, you have to mix it with reflection, because Java
does not provide standard methods to extend java.library.path.
Now the library is loaded for the user transparently, and he does not have to worry about copying files or setting environment variables. To work, it is still enough just to run a regular script. However, after each launch of the program there is a temporary directory with files. This is not very good, so cleaning should be done at the exit.
try { ... } finally { delete(dir); }
But on Windows it does not work. The library loaded in the JVM blocks the dll-file and the directory in which it lies. Thus, to solve the problem of a neat program termination, you need to unload the system dynamic library from the JVM.
Attempt to solve
First of all, it is reasonable to add diagnostics to the code. If the files could be deleted, for example. when the library is not used, then nothing needs to be done, and if the files are locked, then take additional measures.
if (!delete(dir)) { forceDelete(dir); }
As a quick, but not the most beautiful solution, I used the scheduler. At the output, I create an xml file with the task to execute the cmd / c rd / s / q temp-dir command after 1 minute and load the task into the scheduler with the command “schtasks -create taskName -xml taskFile.xml”. By the time the task is completed, the program is already completed, and no one is holding the files
The most correct decision is to provide unloading of the library by means of a Java-machine. The documentation says that the system library will be unloaded when a class is deleted, and the class is deleted by the garbage collector along with the classcadder when there is not a single instance left of its classes. In my opinion, it is better to always write such code that completely clears all memory and other resources after it. Because if the code does something useful, sooner or later you will want to reuse it and install it on some server where other components are installed. So I decided to take the time to figure out how to programmatically unload the dll.
Using a classeloader
In my program, problems originated from the JDBC driver, so I will continue to look at the JDBC example. But with other libraries you can work in a similar way.
If the dll is loaded from the system class loader, then it will no longer be able to unload it, so you need to create your own classifier in such a way that the class that pulls the library is loaded from it. The new classifier must be connected to the system classifier via the parent property, otherwise the String, Object and other classes necessary for the household will not be available in the item.
Let's try:
Loading class from new loader (1) ClassLoader parentCl = ExampleClass.class.getClassLoader(); classLoader = new URLClassLoader(new URL[0], parentCl); Class.forName("org.jdbc.CustomDriver", classLoader, true); try (Connection connection = DriverManager.getConnection(dbUrl, dbProperties)) { if (connection.getClass().getClassLoader() != classLoader) { System.out.printf("- %n"); } ... }
Does not work. When a class is loaded, an attempt is first made to pick it up from the parent loader, so our driver did not load as we needed. To use the new classifier, you need to delete the JDBC driver from the jar file of the program so that it is not accessible to the system loader. So, we pack the library as an attached jar-file, and before using it we deploy it in the temporary directory (in the same place where we have the dll).
Loading class from new loader (2) ClassLoader cl = ExampleClass.class.getClassLoader(); URL url = UnloadableDriver.class.getResource("CustomJDBCDriver.jar"); if (url == null) { throw new FileNotFoundException("Can't load resource CustomJDBCDriver.jar"); } Path dir = prepareLibrary(); try (InputStream stream = url.openStream()) { Path target = dir.resolve("CustromJDBCDriver.jar"); Files.copy(stream, target); url = target.toUri().toURL(); } ClassLoader classLoader = new URLClassLoader(new URL[] {url}, cl); Class.forName("org.jdbc.CustomDriver", true, classLoader); try (Connection connection = DriverManager.getConnection(dbUrl, dbProperties)) { if (connection.getClass().getClassLoader() != classLoader) { System.out.printf("- %n"); } else { System.out.printf(", %n"); } ... }
We received an object loaded from our new loader, after finishing work we need to close everything we opened, clean all our variables, and, apparently, call System.gc (), then try to clean the files. In this place, it makes sense to encapsulate all the logic of working with class loaders in a separate class with explicit initialization methods.
Core class skeleton public class ExampleClass implements AutoCloseable { private final Path dir; private URLClassLoader classLoader; public ExampleClass() { ... } public void doWork() { ... } @Override public void close() { ... this.classLoader.close(); this.classloader = null; System.gc();
Experiments with the garbage collector
Despite the fact that everything seems to be formally necessary for unloading the library is done, in fact, unloading does not occur. Reading the sources from the java.lang package allowed us to determine that the removal of the native libraries is done in the finalize () method in one of the internal classes. This is distressing and alarming, because the documentation does not give any precise definition of when this method will be executed and whether it will be executed at all. That is, success depends on some factors that may vary in different environments, in different versions of the JVM, or in different garbage collectors. However, there is a System.runFinalization () method that gives some hope.
We try:
Run finalization ... @Override public void close() { ... this.classLoader.close(); this.classloader = null; System.gc(); System.runFinalization();
Does not work. The directory is locked by the java process. From this point on, I used this technique:
- I put on the output System.in.read ()
- When the program stops in this place, I make a memory dump from jvisualvm
- Watching a dump with the Eclipse Memory Analysis Tool or jhat
- Looking for instances of objects whose classes were loaded by my loader
Found 5 sources of leakage:
- Local variables
- Drivermanager
- ResourceBundle
- ThreadLocals
- Exceptions
Local variablesLocal variables
It turned out that the garbage collector does not consider a local variable to be unreachable until the function containing this variable is completed, even if the variable goes out of scope.
if (needConnection) { try (Connection connection = DriverManager.connect()) { ... } }
Therefore, to solve the unloading problem of the classifier, it is necessary to quit all functions that use the unloaded classes before calling gc.
DrivermanagerDrivermanager
JDBC drivers when loading their class are registered in the DriverManager class using the registerDriver () method. Apparently, before unloading, you must call the deregisterDriver () method. We try.
Enumeration<Driver> drivers = driverManager.getDrivers(); while (drivers.hasMoreElements()) { Driver driver = drivers.nextElement(); if (driver.getClass().getClassLoader() == classLoader) { DriverManager.deregisterDriver(driver); break; } }
Does not work. Heapdump has not changed. We look into the source code of the DriverManager class and find that in the deregisterDriver () method there is a check that the call must be from a class that belongs to the same class as the class that previously called registerDriver (). And registerDriver () is called by the driver itself from a static initializer. Unexpected turn.
It turns out, we can not directly unregister the driver. Instead, we have to ask some class from the new classifier to do it on its own behalf. The solution is to create a special class DriverManagerProxy, more precisely, even two, a class and an interface.
public interface DriverManagerProxy { void deregisterDriver(Driver driver) throws SQLException; } public class DriverManagerProxyImpl implements DriverManagerProxy { @Override public void deregisterDriver(Driver driver) throws SQLException { DriverManager.deregisterDriver(driver); } }
The interface will be in the main classpath, and the release will be loaded by the new loader from the auxiliary jar file along with the JDBC driver. Theoretically, one could do without an interface, but then to call the function, one would have to use reflection. Proxy is used as follows:
Using DriverManagerProxy public class ExampleClass implements AutoCloseable { private final Path dir; private URLClassLoader classLoader; private DriverManagerProxy driverManager; public ExampleClass() { ... this.classLoader = ...; Class.forName("org.jdbc.CustomDriver", true, classLoader); Class<?> dmClass = Class.forName("ru.example.DriverManagerProxyImpl", true, classLoader); this.driverManager = (DriverManagerProxy) dmClass.newInstance(); } public void doWork() { ... } @Override public void close() { ... Enumeration<Driver> drivers = driverManager.getDrivers(); while (drivers.hasMoreElements()) { Driver driver = drivers.nextElement(); if (driver.getClass().getClassLoader() == classLoader) { driverManager.deregisterDriver(driver); break; } } this.driverManager = null; this.classLoader.close(); this.classloader = null; System.gc(); System.runFinalization();
ResourceBundleResourceBundle
The next hook on the classifier that I was trying to unload was found in the depths of the ResourceBundle class. Fortunately, unlike DriverManager, ResourceBundle provides a special function, clearCache (), to which the classloader is passed as a parameter.
ResourceBundle.clearCache(classLoader);
It should be noted that, judging from the source, the ResourceBundle uses weak links that should not interfere with garbage collection. Perhaps, if you clear all other references to our objects, then you do not need to clear this cache.
ThreadLocalsThreadLocals
The last place where the unused driver tails were found was ThreadLocals. After the DriverManager story, clearing the local flow variables seems like a trifle. Although it could not do without reflection.
private static void cleanupThreadLocals(ClassLoader cl) throws ReflectiveOperationException { int length = 1; Thread threads[] = new Thread[length]; int cnt = Thread.enumerate(threads); while (cnt >= length) { length *= 2; threads = new Thread[length]; cnt = Thread.enumerate(threads); } for (int i = 0; i < cnt; i++) { Thread thread = threads[i]; if (thread == null) { continue; } cleanupThreadLocals(thread, cl); } } private static void cleanupThreadLocals(Thread thread, ClassLoader cl) throws ReflectiveOperationException { Field threadLocalsField = Thread.class.getDeclaredField("threadLocals"); threadLocalsField.setAccessible(true); Object threadLocals = threadLocalsField.get(thread); if (threadLocals == null) { return; } Class<?> threadLocalsClass = threadLocals.getClass(); Field tableField = threadLocalsClass.getDeclaredField("table"); tableField.setAccessible(true); Object table = tableField.get(threadLocals); Object entries[] = (Object[]) table; Class<?> entryClass = table.getClass().getComponentType(); Field valueField = entryClass.getDeclaredField("value"); valueField.setAccessible(true); Method expungeStaleEntry = threadLocalsClass.getDeclaredMethod("expungeStaleEntry", Integer.TYPE); expungeStaleEntry.setAccessible(true); for (int i = 0; i < entries.length; i++) { Object entry = entries[i]; if (entry == null) { continue; } Object value = valueField.get(entry); if (value != null) { ClassLoader valueClassLoader = value.getClass().getClassLoader(); if (valueClassLoader == cl) { ((java.lang.ref.Reference<?>) entry).clear(); expungeStaleEntry.invoke(threadLocals, i); } } } }
ExceptionsExceptions
We expect the cleanup code to be placed in a finally block. At the entrance to this block, we should already have everything closed automatically using the try-with-resources mechanism. However, our classifier will still not be deleted from memory in this place if an exception is thrown from the try block, the class of which is loaded by this classifier.
To remove an unwanted exception from memory, it is necessary to catch and process it, and if you need an error all the same, throw it up, then copy the exception to another class. Here's how I did it in my program:
try { ... } catch (RuntimeException e) { if (e.getClass().getClassLoader() == this.getClass().getClassLoader()) { throw e; } RuntimeException exception = new RuntimeException(String.format("%s: %s", e.getClass(), e.getMessage())); exception.setStackTrace(e.getStackTrace()); throw exception; } catch (SQLException e) { if (e.getClass().getClassLoader() == this.getClass().getClassLoader()) { throw e; } SQLException exception = new SQLException(String.format("%s: %s", e.getClass(), e.getMessage())); exception.setStackTrace(e.getStackTrace()); throw exception; }
Java Strikes Back
After clearing all detected references to the unloaded classes, a slightly paradoxical situation has turned out. There are no objects in memory, judging by the memory dump, the number of instances in all classes is 0. But the classes themselves and their loader have not gone away, and the native library has not been deleted accordingly.
To solve the problem it turned out like this:
System.gc(); System.runFinalization(); System.gc(); System.runFinalization();
Probably, in Java 1.7, which I used, there was some peculiarity of cleaning objects that are in PermGen. I did not experiment with the garbage collection settings, because I tried to write code that would work equally in different environments, including application servers.
After this reception, the code worked as it should, the library was unloaded, the directories were deleted. However, after the transition to Java 8, the problem returned. There was no time to figure out what was happening, but apparently, something has changed in the behavior of the garbage collector.
Therefore it was necessary to use heavy artillery, namely JMX:
How to make java collect garbage private static void dumpHeap() { try { Class<?> clazz = Class.forName("com.sun.management.HotSpotDiagnosticMXBean"); MBeanServer server = ManagementFactory.getPlatformMBeanServer(); Object hotspotMBean = ManagementFactory.newPlatformMXBeanProxy( server, "com.sun.management:type=HotSpotDiagnostic", clazz); Method m = clazz.getMethod("dumpHeap", String.class, boolean.class); m.invoke(hotspotMBean, "nul", true); } catch (@SuppressWarnings("unused") RuntimeException e) { return; } catch (@SuppressWarnings("unused") ReflectiveOperationException e) { return; } catch (@SuppressWarnings("unused") IOException e) { return; } }
Via HotSpotDiagnosticMXBean, we save the memory dump. We specify nul as the file name, which in Windows means the same as / dev / null in Unix. The second parameter indicates that only live objects should be uploaded to the dump. It is this parameter that causes the JVM to do a full garbage collection.
After this lifehack, the problem of deleting the library from the temporary directory no longer arose. The final file cleanup code looks like this:
this.classLoader = null; System.gc(); System.runFinalization(); System.gc(); System.runFinalization(); if (!delete(this.dir)) { dumpHeap(); if (!delete(this.dir)) { scheduleRemovalToTaskschd(this.dir); } }
OSGI Validation
To check the quality of the code, I wrote my JDBC driver, which completely cleans up after itself. It works like a wrapper around any other driver loaded from a separate classpath.
UnloadableDriver public class UnloadableDriver implements Driver, AutoCloseable { private final Path dir;
I inserted this driver into the OSGI service on Apache Felix.
JDBCService public interface JDBCService { Connection getConnection(String url, Properties properties) throws SQLException; } @Service(JDBCService.class) public class JDBCServiceImpl implements JDBCService { private UnloadableDriver driver; @Activate public void activate(ComponentContext ctx) throws SQLException { this.driver = new UnloadableDriver(); } @Deactivate public void deactivate() { this.driver.close(); this.driver = null; } @Override public Connection getConnection(String url, Properties info) throws SQLException { return this.driver.connect(url, properties); } }
When the module is started via the Apache Felix system console running on Java 1.8.0_102, a temporary directory with a dll file appears. The file is locked by the java process. As soon as the module stops, the directory is deleted automatically. If instead of UnloadableDriver you use DriverManager and the usual library from Embedded-Artifacts, then after updating the module, the error java.lang.UnsatisfiedLinkError occurs: Native Library already loaded in another classloader.
findings
There is no universal way to unload the system dynamic library from a Java machine, but this task is solvable.
In Java, there are quite a few places where you can accidentally leave links to your classes, and this is a prerequisite for memory leaks.
Even if your code does everything correctly, a leak can be introduced by some library that you are using.
Special attention should be paid to cases when the program loads something using the new class loader created during the execution. If at least one link to one of the loaded classes remains, then the classifier and all its classes will remain in memory.
To detect a memory leak, you need to make a dump and analyze with the help of special tools, such as Eclipse MAT.
If a memory leak is detected in a third-party library, you can try to fix it with one of the recipes described in the article.