What you'll need:

- JDK for your OS :  HERE

- Smali and baksmali scripts & .jars located : HERE

- Your ADB environment set up ready to pull files if you're doing it on a live phone!

What is an ODEX file?

To summarise, an ODEX file is a version of a classes.dex file, with optimisations that are device specific. ODEX files have dependencies on every BOOTCLASSPATH file that is loaded when it is generated. The ODEX file is only valid when used with these exact BOOTCLASSPATH files.

The BOOTCLASSPATH is simply a list of jar/apk files from which classes can be loaded – in addition to the apk that you're looking to de-odex. A typical Android build has 5 jars in its base BOOTCLASSPATH... these are:

Core.jar, ext.jar, framework.jar, services.jar and android.policy.jar.

The above BOOTCLASSPATH files can be found in /system/framework.  Some apk files will have dependencies on BOOTCLASSPATH files beyond those of the base files. For example, Google Maps will need com.google.android.maps.jar, and Samsung apps will require twframework-res.jar.

The difficulty that these ODEX files creates, is that you cannot simply take an APK and ODEX file from a ROM and run it on another, unless the other ROM uses the exact same framework files. Another problem, is that is you make ANY changes to any of the BOOTCLASSPATH files, it will invalidate every ODEX file that depends on that file..... ie, every apk/jar on the device. For that reason, if you're de-odexing the framework, you'll have to de-odex the /system/app directory too. Luckily, there are tools out there to make it a bit easier. 

Manually De-Odexing with baksmali

First of all, you'll need to have the latest JDK installed. It is also important to note, that if you are calling the jar directly (ie, in windows), you can replace "baksmali" with "java –jar baksmali.jar".

In order for baksmali to de-odex an app, it has to load every BOOTCLASSPATH file that the ODEX file depends on. By default, it will try to find the 5 core BOOTCLASSPATH files (as mentioned in the 'What is an ODEX file' section above), in the current directory.

So... pull the contents of your framework directory from the phone, as well as the APK and ODEX files you want to de-odex, and place them in the same directory as the baksmali script.

If your build only uses the 5 core BOOTCLASSPATH files, then you can simply use the –x option to tell baksmali that it should de-odex the input file. For example:

baksmali –x Settings.odex

If, however, you need to to specify additional BOOTCLASSPATH files (additional jar dependencies), then you can do so with the –c Option. Let's say that you're de-odexing a Samsung Galaxy S ROM, and the Services.odex relies on twframework.jar. The colon (: ) will append the file to the BOOTCLASSPATH. See the below example

baksmali –c :twframework.jar –x Settings.odex

If you wish to specify all BOOTCLASSPATH files yourself in the command, then this is how you'd do it:

baksmali –c core.jar:framework.jar:android.policy.jar:services.jar:twframework.jar –x Settings.odex

Additional useful settings:

Using -d, allows you to set the dir where baksmali looks for the BOOTCLASSPATH files

In order for baksmali to de-odex an app, it has to load every

BOOTCLASSPATH file that the ODEX file depends on. By default, it will
try to find the 5 core BOOTCLASSPATH files (as mentioned in the 'What
is an ODEX file' section above), in the current directory. You can
also use the -d option to tell baksmali where to look for the
BOOTCLASSPATH files"

-I, ignores non-fatal errors

To fix the issue, simply find the jar file and add it to the
BOOTCLASSPATH using the –c option. You'll more than likely be able to
guess which jar is required from the error message!

You might also come across missing classes that don't actually exist
(usually containing Testing, or Test). As these are classes that can
safely be ignored you can add the baksmali option -I (capital i) which
ignores non-fatal errors (use with caution!)."

Problems with De-odexing

The most common problem when de-odexing, is an error which looks like this:

"org.jf.dexlib.Code.Analysis.ValidationException: class Lcom/google/android/gtalkservice/IChatListener; cannot be resolved."

The cause of this is as it sounds... the ODEX file has an additional dependency that isn't specified in the BOOTCLASSPATH.

To fix the issue, simply find the jar file and add it to the BOOTCLASSPATH using the –c option. You'll more than likely be able to guess which jar is required from the error message!

I hope that gives you a better understanding of the de-odexing process.