Commons Codec on Android

If your application depends on Commons Codec library (version 1.4 and above), and you are not aware that Android framework already includes this library, then there's a chance that your application will crash with a runtime exception.


Why does the Android framework includes the Commons Codec library?

Android framework shipped with HttpClient (version 4.0 pre-BETA, an ancient 2008 release). HttpClient depends on Commons Codec. Therefore, AOSP had to include Commons Codec as well.


OK, so can I use framework's Commons Codec library in my application?

Unfortunately no. This API is hidden and not exposed in Android SDK for application developers. You can invoke the library's classes using reflection of course, but by doing so you take the risk of relying on undocumented APIs, which is not recommended at all.


What is the version of Commons Codec library that shipped with Android framework?

Version 1.3 (released in July, 2004), but without org.apache.commons.codec.digest.DigestUtils class.


What's wrong with including more advanced version (i.e. v1.9) as a library in my application?

Your application's Commons Codec classes will collide with those that are loaded as part of the framework. The dexopt process will identify conflicts and won't allow loading your classes.

Fortunately, for each such conflict dexopt will print a warning to logcat (when using Dalvik runtime; on ART those logs are omitted for some reason):

D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/Decoder;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/DecoderException;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/Encoder;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/EncoderException;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/StringEncoderComparator;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/language/DoubleMetaphone$DoubleMetaphoneResult;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/language/SoundexUtils;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/net/RFC1522Codec;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/BinaryDecoder;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/BinaryEncoder;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/StringDecoder;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/StringEncoder;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/binary/BinaryCodec;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/binary/Hex;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/language/DoubleMetaphone;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/language/Metaphone;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/language/RefinedSoundex;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/language/Soundex;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/net/BCodec;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/net/QCodec;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/net/QuotedPrintableCodec;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/net/URLCodec;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: 'Lorg/apache/commons/codec/binary/Base64;' has an earlier definition; blocking out
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/Decoder;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/DecoderException;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/Encoder;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/EncoderException;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/StringEncoderComparator;': multiple definitions
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/binary/Hex;'
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/language/DoubleMetaphone$DoubleMetaphoneResult;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/language/SoundexUtils;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/net/RFC1522Codec;': multiple definitions
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/DecoderException;'
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/BinaryDecoder;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/BinaryEncoder;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/StringDecoder;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/StringEncoder;': multiple definitions
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/binary/Base64;'
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/binary/Base64;'
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/binary/Base64;'
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/DecoderException;'
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/binary/BinaryCodec;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/binary/Hex;': multiple definitions
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/EncoderException;'
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/EncoderException;'
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/EncoderException;'
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/language/DoubleMetaphone;': multiple definitions
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/EncoderException;'
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/language/Metaphone;': multiple definitions
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/EncoderException;'
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/language/RefinedSoundex;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/language/Soundex;': multiple definitions
I/dalvikvm? DexOpt: not resolving ambiguous class 'Lorg/apache/commons/codec/EncoderException;'
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/net/BCodec;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/net/QCodec;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/net/QuotedPrintableCodec;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/net/URLCodec;': multiple definitions
D/dalvikvm? DexOpt: not verifying/optimizing 'Lorg/apache/commons/codec/binary/Base64;': multiple definitions

Even if you're compiling against newer Commons Codec version, your application will use framework's version 1.3 classes at runtime. This happens because version 1.3 classes exist in boot classpath, and therefore classloader will always load them in preference to the version you've packaged in your application.

That said, everything may work well if you don't use functionality that was introduced in version 1.4 or later. But what if you do use? For example, version 1.4 introduced new method -Base64#encodeBase64String. If I'll add dependency to Commons Codec 1.9 and call this method in my application, the compiler won't complain. But executing this code on the device will produce the following runtime exception:

18917-18917/info.osom.sandbox E/AndroidRuntime? FATAL EXCEPTION: main
    Process: info.osom.sandbox, PID: 18917
    java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.encodeBase64String
            at info.osom.sandbox.MainActivity.onCreate(MainActivity.java:48)
            at android.app.Activity.performCreate(Activity.java:5231)
            ...
            at dalvik.system.NativeStart.main(Native Method)

I copied the above error log from a device running KitKat with Dalvik. Lollipop's (with ART) error is a bit more verbose:

898-898/info.osom.sandbox E/AndroidRuntime? FATAL EXCEPTION: main
    Process: info.osom.sandbox, PID: 898
    java.lang.NoSuchMethodError: No static method encodeBase64String([B)Ljava/lang/String; in class Lorg/apache/commons/codec/binary/Base64; or its super classes (declaration of 'org.apache.commons.codec.binary.Base64' appears in /system/framework/ext.jar)
            at info.osom.sandbox.MainActivity.onCreate(MainActivity.java:48)
            at android.app.Activity.performCreate(Activity.java:5990)
            ...
            at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698)

So what shall I do?

You can avoid the aforementioned conflicts by changing the classes namespace, that is, the Commons Codec package. Either:

• do it manually - grab the Commons Codec (version 1.9 for example) sources and then refactor the code by changing the package name (you can find an example for someone who took this approach here); 
• or repackage the official release jar file using jarjar tool

jarjar basically allows to alter the bytecode according to specified rules. One of the available rules provides the ability to rename class names. Here's an example for repackaging version 1.9 of Commons Codec:

Get the latest jarjar jar from here, and run (I'm using jarjar-1.4) this command in terminal:

java -jar jarjar-1.4.jar process commons-codec.rules commons-codec-1.9.jar commons-codec-1.9-rep.jar

where commons-codec.rules is a plain text file with the following single line:

rule org.apache.** android.org.apache.@1

This rule (see more on rules here) tells the jarjar tool to add android prefix to all the class names within org.apache package, and put those classes in a new jar called commons-codec-1.9-rep.jar. For example, org.apache.commons.codec.binary.Base64 class will be renamed to android.org.apache.commons.codec.binary.Base64.

Add the repackaged jar to your application instead of original Commons Codec jar. This will allow both framework's Base64 class and your (newer version) Base64 class to coexist within application.

You can even create automated task and use it as part of your build process: check out the jarjar-gradle plugin for Gradle or even this task if you still use ant.