Generating main-dex-list file

[Update - 10/31/2014] Gradle plugin v0.14.0 for Android adds support for Multi-Dex. Everything that is written below is now done automatically. All you need to do is to add this line in your build.gradle file:
android {
   defaultConfig {
      ...
      multiDexEnabled = true
}

In my previous post I described the 65536 methods issue and how to solve it using multi-dex technique. In overall, this solution is pretty straightforward - you tell dx to operate in multi-dex mode, and add the multi-dex support library to patch the ClassLoader on application start-up.

The only tricky part is the main-dex-list file that specifies all the classes that you want to be included in the main dex file. dx takes this file as a parameter and generates appropriate dex files.

Failing to keep required classes in the main dex file will produce the following runtime exception:

java.lang.RuntimeException: Unable to instantiate application info.osom.multidex2.Application: java.lang.ClassNotFoundException: Didn't find class "info.osom.multidex2.Application" on path: DexPathList[[zip file "/data/app/info.osom.multidex2-1.apk"],nativeLibraryDirectories=[/data/app-lib/info.osom.multidex2-1, /vendor/lib, /system/lib]]
            ..
     Caused by: java.lang.ClassNotFoundException: Didn't find class "info.osom.multidex2.Application" on path: DexPathList[[zip file "/data/app/info.osom.multidex2-1.apk"],nativeLibraryDirectories=[/data/app-lib/info.osom.multidex2-1, /vendor/lib, /system/lib]]
            at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:497)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:457)

Now, you may ask, which exact classes should be included in main dex file? And if I have many classes, do I need to write all those class names by hand?

I had no clear answer to those questions, so I took a further look on dx source code. Thanks God (and Google) it's open source :)

After searching a bit, I found this commit, whose descriptions says:

Introduction of a tool to find main dex classes.
This is for legacy application wanting to allow --multi-dex on dx and load the multiple files using the com.android.multidex.installer library. The mainDexClasses script will provide the content of the file to give to dx in --main-dex-list.

Seems like there's a script (called mainDexClasses) that generates the exact class names that should be included in main-dex-list file!

Important note: use it at your own risk. This script is not officially released, and I haven't found any references in the Internet. It worked for me well, but if you encountered any bugs - I would love to hear!

The script (comes in two versions - bash script and Windows's batch) is located in /dx/etc folder. Download two files from there - mainDexClasses (or mainDexClasses.bat for Windows) and mainDexClasses.rules. Place both files in your /sdk/build-tools/[build-tools-version] folder. The minimum supported build-tools version is 20 (I will explain later why).

Invoking the script without parameters echoes its usage:

Usage : mainDexClasses [--output <output file>] <application path>

Output file is obviously your main-dex-list. The second parameter (application path), requires to specify all the zip files and directories (separated by path separator) that dx receives as input. To find the paths for your project, run gradlew with --info parameter:

gradlew clean assembleDebug --info

and find the command for your dexDebug task. For example (relevant jars and directories coloured):

All input files are considered out-of-date for incremental task ':app:dexDebug'.
command: dx.bat --dex --num-threads=4 --multi-dex --main-dex-list=D:\multidex2\app/multidex.keep --output D:\multidex2\app\build\intermediates\dex\debug D:\multidex2\app\build\intermediates\classes\debug D:\multidex2\app\build\intermediates\dependency-cache\debug D:\multidex2\app\libs\android-support-multidex.jar

Note that the script expects class files within jar. By default, preDexDebug task is enabled, and it turns all the dependencies into dex files. Therefore, you have to disable library pre-dexing, by specifying dexOptions.preDexLibraries = false.

The mainDexClasses script does two things.

First, it invokes proguard to shrink all the classes according to rules defined in mainDexClasses.rules file (we downloaded it previously, remember?):

call "%proguard%" -injars %params% -dontwarn -forceprocessing  -outjars "%tmpJar%" -libraryjars "%shrinkedAndroidJar%" -dontoptimize -dontobfuscate -dontpreverify -include "%baserules%" 1>nul

The %params% (-injars parameter) is the second parameter that the script receives. The script expects to find shrinkedAndroid.jar in /sdk/build-tools/[build-tools-version]/lib folder. There's no file with such name there, so to make it work I copied the android.jar from /sdk/platforms/[api-version] and renamed it to shrinkedAndroid.jar.

Secondly, the script invokes dx.jar's com.android.multidex.ClassReferenceListBuilder class. This is a command-line utility that described as following:

This is a command line tool used by mainDexClasses script to find direct class references to other classes. First argument of the command line is an archive, each class file contained in this archive is used to identify a class whose references are to be searched, those class files are not opened by this tool only their names matter. Other arguments must be zip files or directories, they constitute in a classpath in with the classes named by the first argument will be searched. Each searched class must be found. On each of this classes are searched for their dependencies to other classes. Finally the tools prints on standard output a list of class files names suitable as content of the file argument --main-dex-list of dx.

The description is self-explanatory, so I have nothing to add here. This class was added to dx.jar in build-tools 20 (and yes, this is why it is a minimum supported version).

Last thing is to invoke the script:

mainDexClasses.bat --output mutlidex.keep "D:\multidex2\app\build\intermediates\classes\debug;D:\multidex2\app\build\intermediates\dependency-cache\debug;D:\multidex2\app\libs\android-support-multidex.jar"

.. and this is the output:


As expected!

Last notes:

• As this script relies on Proguard, it naturally won't keep your classes that are being invoked via reflection API - you will have to add them manually
• The generation of course can be simply extracted into a gradle task that runs before dexDebug task. I will post example project in the near future, so stay tuned!