Try-with-resources desugaring support in Android's build tools

During one of the code reviews with my colleague I've noticed he used try-with-resources Java 7's feature. To my then latest knowledge, try-with-resources required minSdkVersion 19 (while our app's minSdkVersion is 16) - which is based on the following statement (see under SDK Tools, Revision 22.6 (March 2014)):

Added support for Java 7 language features like multi-catch, try-with-resources, and the diamond operator. These features require version 19 or higher of the Build Tools. Try-with-resources requires minSdkVersion 19; the rest of the new language features require minSdkVersion 8 or higher.

My colleague told me that neither Lint nor compiler haven't warned about that, which was strange - since I vividly remember they both did in the past. Bug in Lint could explain that, but how did it compile? We decided to review our assumptions.

Verifying 64-bit libraries are being loaded

If you're bundling native libraries inside your Android application, then you've probably already know that starting August 1, 2019, your apps published on Google Play will need to support 64-bit architectures. In this post I'll show how we can explicitly choose the 64-bit ABIs to run, and verify that the 64-bit library was indeed loaded at runtime.

The support 64-bit architectures guide provides a command for selecting which ABI libraries to use during the APK installation process:

# A successful install:
$ adb install --abi armeabi-v7a APK_FILE.apk
Success

This command can be handy when your APK includes several ABI libraries, your CPU able to run multiple ABIs and you want to explicitly specify which ABI to use.

How to manually create Android emulator's internal storage disk image

Built-in AVD Manager incorrectly generates (internal storage) disk images when you're creating new virtual device targeting API 19 or greater.

On API 19 - no matter what size you specify during the virtual device configuration, the maximum that you're going to get is 200mb. On API 21 and 22, the maximum size increased to 550mb. This issue already has appropriate question on StackOverflow and AOSP bug report.

For average application this will be sufficient, but for large application (or when you need to install several application that talk to each other) it can be problematic.

Commons Codec on Android

If your application depends on Commons Codec library (version 1.4 and above), and you are not aware that Android framework already includes this library, then there's a chance that your application will crash with a runtime exception.


Why does the Android framework includes the Commons Codec library?

Android framework shipped with HttpClient (version 4.0 pre-BETA, an ancient 2008 release). HttpClient depends on Commons Codec. Therefore, AOSP had to include Commons Codec as well.


OK, so can I use framework's Commons Codec library in my application?

Unfortunately no. This API is hidden and not exposed in Android SDK for application developers. You can invoke the library's classes using reflection of course, but by doing so you take the risk of relying on undocumented APIs, which is not recommended at all.

Too many methods in main-dex?

Notes:
1. The content of this post is relevant only if your minSdk < 21. On Lollipop, no ClassLoader patching required.
2. This solution worked for me well, but it might not work for you in the same way. So please use it on your own risk. Nevertheless, I would be happy to know of your problems - so please post in comments!


Right after the release of multidex support we removed our custom solutions (for 64k methods problem) in favor of multidex support library. Gradle for Android plugin v0.14.0+ simplified the process even more - it implemented all the manual tasks (I wrote about some of them in my previous post) that we had to do previously.

Lately we encountered another issue: we have reached the 64k methods limit in main dex as well. It comes in form of the following error when we're building our project:

UNEXPECTED TOP-LEVEL EXCEPTION:
com.android.dex.DexException: Too many classes in --main-dex-list, main dex capacity exceeded
  at com.android.dx.command.dexer.Main.processAllFiles(Main.java:494)
  at com.android.dx.command.dexer.Main.runMultiDex(Main.java:332)
  at com.android.dx.command.dexer.Main.run(Main.java:243)
  at com.android.dx.command.dexer.Main.main(Main.java:214)
  at com.android.dx.command.Main.main(Main.java:106)

You can say: man, something is wrong with your design if you're having such a problem!
Well, you are quite right, and we're thinking hard on how to split various features of our application into separate apps. But for short-term, we had to find a solution for this specific issue.

Generating main-dex-list file

[Update - 10/31/2014] Gradle plugin v0.14.0 for Android adds support for Multi-Dex. Everything that is written below is now done automatically. All you need to do is to add this line in your build.gradle file:
android {
   defaultConfig {
      ...
      multiDexEnabled = true
}

In my previous post I described the 65536 methods issue and how to solve it using multi-dex technique. In overall, this solution is pretty straightforward - you tell dx to operate in multi-dex mode, and add the multi-dex support library to patch the ClassLoader on application start-up.

The only tricky part is the main-dex-list file that specifies all the classes that you want to be included in the main dex file. dx takes this file as a parameter and generates appropriate dex files.

Failing to keep required classes in the main dex file will produce the following runtime exception:

java.lang.RuntimeException: Unable to instantiate application info.osom.multidex2.Application: java.lang.ClassNotFoundException: Didn't find class "info.osom.multidex2.Application" on path: DexPathList[[zip file "/data/app/info.osom.multidex2-1.apk"],nativeLibraryDirectories=[/data/app-lib/info.osom.multidex2-1, /vendor/lib, /system/lib]]
            ..
     Caused by: java.lang.ClassNotFoundException: Didn't find class "info.osom.multidex2.Application" on path: DexPathList[[zip file "/data/app/info.osom.multidex2-1.apk"],nativeLibraryDirectories=[/data/app-lib/info.osom.multidex2-1, /vendor/lib, /system/lib]]
            at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:497)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:457)

Now, you may ask, which exact classes should be included in main dex file? And if I have many classes, do I need to write all those class names by hand?

Multi-dex to rescue from the infamous 65536 methods limit

You landed on this page because you've probably received the following stack trace when you tried to build your Android project:

UNEXPECTED TOP-LEVEL EXCEPTION:
java.lang.IllegalArgumentException: method ID not in [0, 0xffff]: 65536
at com.android.dx.merge.DexMerger$6.updateIndex(DexMerger.java:501)
at com.android.dx.merge.DexMerger$IdMerger.mergeSorted(DexMerger.java:282)
at com.android.dx.merge.DexMerger.mergeMethodIds(DexMerger.java:490)
at com.android.dx.merge.DexMerger.mergeDexes(DexMerger.java:167)
at com.android.dx.merge.DexMerger.merge(DexMerger.java:188)
at com.android.dx.command.dexer.Main.mergeLibraryDexBuffers(Main.java:439)
at com.android.dx.command.dexer.Main.runMonoDex(Main.java:287)
at com.android.dx.command.dexer.Main.run(Main.java:230)
at com.android.dx.command.dexer.Main.main(Main.java:199)
at com.android.dx.command.Main.main(Main.java:103)

or this stack trace if you're using dx version 1.8 (shipped with SDK build tools 19.0 and above):

trouble writing output: Too many method references: [num of methods]; max is 65536.
You may try using --multi-dex option.

If this is the first time you see this message, then you'll be surprised to know that Dalvik bytecode has a fundamental limitation which allows to invoke a maximum of 2^16 (65536) methods (per dex file). One glance on a list of StackOverflow questions on this topic is enough to realize how painful this limitation for us.

Managing started service's lifecycle efficiently

So you decided to run your time-consuming operation in service component because you want to ensure that it will continue to run even if the user interface is no longer visible. You chose to extend Service, rather than IntentService because you want your service to handle multiple requests in parallel. In this post I will share a few tips for implementing your service slightly more efficiently.

Symmetric encryption issue in Android 4.3

About a year ago, a bit after 4.3 release, I had to fix an error in one of my applications. I patched it then, but wished to return and understand better the underlying reason. The other day I came back to this code and dug a little deeper. This post is about what I found.

In one of my applications I use symmetric-key algorithm for encryption and decryption of data. Nothing fancy here, just a standard Java APIs everyone uses for such things.

Migrating to SQLCipher for Android 3.x

In my last post I wrote why you have seriously consider upgrading to SQLCipher for Android 3.0.2. Version 3.x introduced new default configuration that is not out-of-the-box compatible with databases created with previous versions (1.x/2.x). Particularly, the default key derivation function's (KDF) iteration count increased from 4000 to 64000 (yes, x16), and therefore some kind of migration of the current database required.

You won't be able to open the existing database file if you'll try to upgrade the SQLCipher libraries without migrating it (the database file) first. The following error will be thrown in attempt to open existing database:

03-29 16:14:04.199  18107-18107/info.osom.sqlciphertest I/Database? sqlite returned: error code = 26, msg = file is encrypted or is not a database
03-29 16:14:04.199  18107-18107/info.osom.sqlciphertest E/Database? CREATE TABLE android_metadata failed
03-29 16:14:04.199  18107-18107/info.osom.sqlciphertest E/Database? Failed to setLocale() when constructing, closing the database
    net.sqlcipher.database.SQLiteException: file is encrypted or is not a database
            at net.sqlcipher.database.SQLiteDatabase.native_setLocale(Native Method)
            at net.sqlcipher.database.SQLiteDatabase.setLocale(SQLiteDatabase.java:2092)
            at net.sqlcipher.database.SQLiteDatabase.(SQLiteDatabase.java:1958)
            at net.sqlcipher.database.SQLiteDatabase.openDatabase(SQLiteDatabase.java:875)
            at net.sqlcipher.database.SQLiteDatabase.openOrCreateDatabase(SQLiteDatabase.java:907)
            at net.sqlcipher.database.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:132)
            at net.sqlcipher.database.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:99)
            at info.osom.sqlciphertest.MainActivity$PlaceholderFragment$1.onClick(MainActivity.java:69)
            at android.view.View.performClick(View.java:4202)
            at android.view.View$PerformClick.run(View.java:17340)
            at android.os.Handler.handleCallback(Handler.java:725)
            at android.os.Handler.dispatchMessage(Handler.java:92)
            at android.os.Looper.loop(Looper.java:137)
            at android.app.ActivityThread.main(ActivityThread.java:5039)
            at java.lang.reflect.Method.invokeNative(Native Method)
            at java.lang.reflect.Method.invoke(Method.java:511)
            at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:560)
            at dalvik.system.NativeStart.main(Native Method)

Why you should upgrade SQLCipher to latest (3.0.2) version

If you're using SQLCipher (for Android) version <= 3.0.1, then your application may potentially leak sensitive data into the log system. That's because in case of error in its underlying native library, the (SQLCipher) Android wrapper library was printing the original query values into the log system.
This issue was fixed on February 1st, 2014 and integrated in version 3.0.2.

I'm including here a short demo that demonstrates the problem.

I've created a new basic project using Android Studio. Then downloaded SQLCipher for Android version 2.2.1, unzipped and added all the content to the project. This is the structure:

Guarding enumeration classes from ProGuard

During my application tests I received exception with the following call stack:

02-09 15:18:06.478 26166 26403 E TAG: Caused by: java.lang.NullPointerException
02-09 15:18:06.478 26166 26403 E TAG:   at java.lang.Enum$1.create(Enum.java:43)
02-09 15:18:06.478 26166 26403 E TAG:   at java.lang.Enum$1.create(Enum.java:35)
02-09 15:18:06.478 26166 26403 E TAG:   at libcore.util.BasicLruCache.get(BasicLruCache.java:54)
02-09 15:18:06.478 26166 26403 E TAG:   at java.lang.Enum.getSharedConstants(Enum.java:209)
02-09 15:18:06.478 26166 26403 E TAG:   at java.util.EnumSet.noneOf(EnumSet.java:48)
02-09 15:18:06.478 26166 26403 E TAG:   at af.a(SourceFile:115)

The exception occurred only in release version, while in debug everything worked fine. So I immediately suspected that ProGuard, whose processing is part of a release build, is somehow involved here.

From ProGuard's documentation:

If your application, applet, servlet, library, etc., contains enumeration classes, you'll have to preserve some special methods. Enumerations were introduced in Java 5. The java compiler translates enumerations into classes with a special structure. Notably, the classes contain implementations of some static methods that the run-time environment accesses by introspection (Isn't that just grand? Introspection is the self-modifying code of a new generation). You have to specify these explicitly, to make sure they aren't removed or obfuscated: 

-keepclassmembers,allowoptimization enum * {
      public static **[] values();
      public static ** valueOf(java.lang.String);
}

OK, problem found: I just need to add this 'keepclassmembers' exception in ProGuard project configuration file.

But what are values() and valueof(java.lang.String) needed for? I still was curious how exactly I received NullPointerException in java.lang.Enum entrails. Let's better understand it.

Programmatically filtering your application's logs from Logcat output

Note: this post discusses filtering Logcat output on devices running Android prior to version 4.1. Starting from Android 4.1 applications are allowed to read only their own logs.

Logcat isn't much convenient when it comes to filtering your application logs. In fact, if you look at the logcat documentation, you will find that you can filter logcat output only by priority and/or tag fields:

A filter expression follows this format tag:priority ..., where tag indicates the tag of interest and priority indicates the minimum level of priority to report for that tag. Messages for that tag at or above the specified priority are written to the log. You can supply any number of tag:priority specifications in a single filter expression. The series of specifications is whitespace-delimited.

The android.util.Log class is your interface for printing your application logs to the Logcat system. Common print statement looks like this:

Log.e(TAG, "something bad happened");

Invoked method specifies the priority of the log record - in this example e stands for error, and TAG specifies well.. the tag.
 I commonly set the class name as a TAG using this statement:

private static final String TAG = ClassName.class.getCanonicalName();

Resolving the class name at run-time allows me to stay flexible and be adjusted to all the obfuscations made by Proguard processing.

But here's a problem: given a project with a hundred classes, how would you filter Logcat's output to print just your application's logs?