Sunday, July 12, 2015

How to manually create Android emulator's internal storage disk image

Built-in AVD Manager incorrectly generates (internal storage) disk images when you're creating new virtual device targeting API 19 or greater.

On API 19 - no matter what size you specify during the virtual device configuration, the maximum that you're going to get is 200mb. On API 21 and 22, the maximum size increased to 550mb. This issue already has appropriate question on StackOverflow and AOSP bug report.

For average application this will be sufficient, but for large application (or when you need to install several application that talk to each other) it can be problematic.


200mb on KitKat

550mb on Lollipop

To solve it you have two options. Use other (non built-in) emulators, the highly recommended Genymotion for example. Or, manually create the internal storage disk image, and replace the auto-generated one.

In this post I will show step-by-step how to manually generate disk image. It is based on Zsolt Muller's post on similar topic.

The tool that generates the disk image is called make_ext4fs and it is part of Android Open Source Project. I will download the sources, compile and run the tool.

The prerequisites are a Linux machine and network connection. I'm using 32-bit Ubuntu 14.04.

Create the folder structure first:
alex@ubuntu2:~$ mkdir platform
alex@ubuntu2:~$ cd platform/
alex@ubuntu2:~/platform$ mkdir system
alex@ubuntu2:~/platform$ cd system/

Clone platform/system/core repository:
alex@ubuntu2:~/platform/system$ git clone https://android.googlesource.com/platform/system/core
Cloning into 'core'...
remote: Sending approximately 13.42 MiB ...
remote: Counting objects: 152, done
remote: Finding sources: 100% (152/152)
remote: Total 45103 (delta 27513), reused 45103 (delta 27513)
Receiving objects: 100% (45103/45103), 13.46 MiB | 1.89 MiB/s, done.
Resolving deltas: 100% (27515/27515), done.
Checking connectivity... done.

Checkout android-4.2.1_r1 branch:
alex@ubuntu2:~/platform/system/core$ git checkout android-4.2.1_r1
Note: checking out 'android-4.2.1_r1'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
  git checkout -b new_branch_name
HEAD is now at 31da9db... merge in jb-mr1-release history after reset to jb-mr1-dev

Clone platform/system/extras repository and checkout the same android-4.2.1_r1 branch:
alex@ubuntu2:~/platform/system/core$ cd ..
alex@ubuntu2:~/platform/system$ git clone https://android.googlesource.com/platform/system/extras
Cloning into 'extras'...
remote: Counting objects: 152, done
remote: Finding sources: 100% (152/152)
remote: Total 6452 (delta 2897), reused 6452 (delta 2897)
Receiving objects: 100% (6452/6452), 3.15 MiB | 2.56 MiB/s, done.
Resolving deltas: 100% (2897/2897), done.
Checking connectivity... done.
alex@ubuntu2:~/platform/system$ cd extras
alex@ubuntu2:~/platform/system/extras$ git checkout android-4.2.1_r1

Note: checking out 'android-4.2.1_r1'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
  git checkout -b new_branch_name
HEAD is now at 80d69a7... merge in jb-mr1-release history after reset to jb-mr1-dev

Let's try to compile:
alex@ubuntu2:~/platform/system/extras$ cd ..
alex@ubuntu2:~/platform/system$ gcc -Wl,--as-needed -Icore/libsparse/include -Icore/include -DANDROID  extras/ext4_utils/make_ext4fs_main.c extras/ext4_utils/make_ext4fs.c extras/ext4_utils/ext4fixup.c extras/ext4_utils/ext4_utils.c extras/ext4_utils/allocate.c core/libsparse/backed_block.c core/libsparse/output_file.c extras/ext4_utils/contents.c extras/ext4_utils/extent.c extras/ext4_utils/indirect.c extras/ext4_utils/uuid.c extras/ext4_utils/sha1.c core/libsparse/sparse_crc32.c core/libsparse/sparse.c core/libsparse/sparse_err.c core/libsparse/sparse_read.c extras/ext4_utils/wipe.c -lz -o make_ext4fs

extras/ext4_utils/make_ext4fs.c: In function make_ext4fs:
extras/ext4_utils/make_ext4fs.c:319:2: warning: passing argument 3 of 'make_ext4fs_internal' discards 'const' qualifier from pointer target type [enabled by default]
  status = make_ext4fs_internal(fd, NULL, mountpoint, NULL, 0, 0, 0, 1, 0, sehnd);
In file included from extras/ext4_utils/make_ext4fs.c:17:0:
extras/ext4_utils/make_ext4fs.h:40:5: note: expected 'char *' but argument is of type const 'char *'
 int make_ext4fs_internal(int fd, const char *directory,
core/libsparse/output_file.c:29:18: fatal error: zlib.h: No such file or directory
 #include
compilation terminated.

The compilation fails - seems like we have to install zlib1g library:
alex@ubuntu2:~/platform/system$ sudo apt-get install zlib1g-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  zlib1g-dev
0 upgraded, 1 newly installed, 0 to remove and 319 not upgraded.
Need to get 181 kB of archives.
After this operation, 434 kB of additional disk space will be used.
Get:1 http://il.archive.ubuntu.com/ubuntu/ trusty/main zlib1g-dev i386 1:1.2.8.dfsg-1ubuntu1 [181 kB]
Fetched 181 kB in 0s (904 kB/s)
Selecting previously unselected package zlib1g-dev:i386.
(Reading database ... 167712 files and directories currently installed.)
Preparing to unpack .../zlib1g-dev_1%3a1.2.8.dfsg-1ubuntu1_i386.deb ...
Unpacking zlib1g-dev:i386 (1:1.2.8.dfsg-1ubuntu1) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up zlib1g-dev:i386 (1:1.2.8.dfsg-1ubuntu1) ...

Let's try to compile again:
alex@ubuntu2:~/platform/system$ gcc -Wl,--as-needed -Icore/libsparse/include -Icore/include -DANDROID  extras/ext4_utils/make_ext4fs_main.c extras/ext4_utils/make_ext4fs.c extras/ext4_utils/ext4fixup.c extras/ext4_utils/ext4_utils.c extras/ext4_utils/allocate.c core/libsparse/backed_block.c core/libsparse/output_file.c extras/ext4_utils/contents.c extras/ext4_utils/extent.c extras/ext4_utils/indirect.c extras/ext4_utils/uuid.c extras/ext4_utils/sha1.c core/libsparse/sparse_crc32.c core/libsparse/sparse.c core/libsparse/sparse_err.c core/libsparse/sparse_read.c extras/ext4_utils/wipe.c -lz -o make_ext4fs
extras/ext4_utils/make_ext4fs.c: In function 'make_ext4fs':
extras/ext4_utils/make_ext4fs.c:319:2: warning: passing argument 3 of 'make_ext4fs_internal' discards 'const' qualifier from pointer target type [enabled by default]
  status = make_ext4fs_internal(fd, NULL, mountpoint, NULL, 0, 0, 0, 1, 0, sehnd);
In file included from extras/ext4_utils/make_ext4fs.c:17:0:
extras/ext4_utils/make_ext4fs.h:40:5: note: expected 'char *' but argument is of type 'const char *'
 int make_ext4fs_internal(int fd, const char *directory,

This time it succeeds, and make_ext4fs executable was created:
alex@ubuntu2:~/platform/system$ ls -la
total 120
drwxrwxr-x  4 alex alex   4096 Jul 11 16:37 ./
drwxrwxr-x  3 alex alex   4096 Jul 11 16:29 ../
drwxrwxr-x 38 alex alex   4096 Jul 11 16:31 core/
drwxrwxr-x 22 alex alex   4096 Jul 11 16:33 extras/
-rwxrwxr-x  1 alex alex 104611 Jul 11 16:37 make_ext4fs*

It takes the following parameters:
alex@ubuntu2:~/platform/system$ ./make_ext4fs
Expected filename after options
make_ext4fs [ -l ] [ -j ] [ -b ]
    [ -g ] [ -i ] [ -I ]
    [ -L
I'll set file system length as 1 gigabyte, the android mountpoint should be data and the filename is userdata.img:
alex@ubuntu2:~/platform/system$ ./make_ext4fs -l 1G -a data userdata.img
Creating filesystem with parameters:
    Size: 1073741824
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 8192
    Inode size: 256
    Journal blocks: 4096
    Label:
    Blocks: 262144
    Block groups: 8
    Reserved block group size: 63
Created filesystem with 11/65536 inodes and 8536/262144 blocks
Go to your virtual device's location on disk (i.e. .android/avd/Nexus_4_API_19.avd/) and delete existing userdata.img and userdata-qemu.img (if exists; note that this will delete all the existing internal storage data) files.

Launch your virtual device, and check the internal storage size. It should display 1 gigabyte as expected:

1gb on KitKat

1 comment: