Raw AAC Player for Android 2

Unbelievable !!!

Something pushed to me to open blogger today and write a comment about the big improvement (in my opinion) of the aacdecoder-android, which I finished also today.

Now looking at my old post Raw AAC Player for Android and see that today it is exactly 2 years since I wrote my first post about the AAC streaming problems!

HE-AAC+ Mono / Stereo

Today I released version 0.6.1 of the aacdecoder-android.

Why is it a minor change (0.6 to 0.6.1) ?
Because it does not affect the Java interface to the library - everybody can use it in his/her existing projects without any changes.

And why is it a big improvement ?
Because it is able to play HE-AAC+ streams in stereo!
Yes, before that, the library was not able to play them. The reason was, that HE-AAC+ streams use a feature called Parametric Stereo (PS), which had to be unfortunately disabled due to some errors in the underlying OpenCORE library. So the decoder was able to play HE-AAC+ streams, but only in mono, not stereo.

Fortunately the problem has been solved by patching the OpenCORE library. But it was not as easy as it looks like (or the patch itself looks like).

Which is the Right Source of OpenCORE ?

First of all: there exist several versions of the OpenCORE aacdec library across the internet:

1. http://code.google.com/p/opencore-aacdec/ - this is the original (version 2009) from the PacketVideo team. Unfortunately there are problems with AAC+ (SBR) and HE-AAC+ (SBR+PS) streams. I used this library until mid of 2011, when I found:
2. http://android.git.kernel.org - GIT repository of the Android's source code including the OpenCORE aacdec code. Unfortunately this was not accessible during last months (spring 2012), so I had then to look for other possibilities. Unbelievable - today there is again (a new) web front-end for browsing. But the code in platform/external/opencore was better - it was able to play AAC+ (SBR), but still not the HE-AAC+ (SBR+PS).
3. http://crepedroid.git.sourceforge.net/ I understand this is a clone of the Android. I used this as a workaround to get the newer sources during spring 2012 when the official Android's repository was not accessible for normal browsing (only using repo command). Same quality as in (2).
4.  http://source.android.com/ - using repo command one is able to retrieve the newest version of OpenCORE. It is in platform/frameworks/base/media/libstagefright . Unfortunately there was still the problem with HE-AAC+ streams (2012/05) - playing mono only. But there is one improvement - when enabling PS feature, the library is no more crashing. This was the starting point to patch the OpenCORE library !

Patching of OpenCORE aacdec

So there was a way how to at least make the library stable. The problem was, that if PS was enabled sometimes the produced sound was very noisy. And sometimes ok, but mono.

I have been trying to find a patch - but there were no patches available anywhere. Even in the Android's source tree there was nothing usable.

It made me to start learning about PS encoding and decoding. Nice mathematics - I can recommend this paper: Parametric Coding of Stereo Audio [PDF]. I spent several evenings reading it. But it made me an overview of the algorithm and pushed me forward.

In order to be able to debug the code I created a tool for converting AAC to WAV - of course reusing the existing native code from the project. I was able to run it directly in my computer (not the Android emulator).

Then snapped some live HE-AAC+ streams and started debugging, tracing and comparing the results. The first hint which I found was the right one! Then I had to create the correct patch suitable for all streams. But this took me less time than to prepare the tools and environments before!

Summary

Here are the summary of versions of the aacdecoder-android library:

• 2011-05-25 Version 0.5 - initial version - separated from the aacplayer-android project
• 2011-07-26 Version 0.5.1 - PS crash detected and PS disabled, the first stable version
• 2012-05-25 Version 0.6 - AAC+ (SBR) is playing, but only mono, not stereo
• 2012-06-28 Version 0.6.1 - PS patched developed; HE-AAC+ streams can be played in stereo

The challenge for next months is to speed-up the library. Although it is still faster than FAAD2, it consumes a lot of CPU now (for HE-AAC+). It is even impossible to play the streams smoothly on emulator now. So what's next ? To learn ARM Architecture :-)

Java Crashing in libc.so.6

I must share this horrible experience although it is not directly related to programming. But I hope that article will be helpful for somebody.

I use Java in my browser for practical things like digital signing of requests when communicating with offices and banks. I use it very often and without any serious problems. Of course I had to configure my truststore and keystore and I also have two versions of extension (jre/lib/ext) directory, because different offices and banks use different versions of the IAIK library. But it is another story.

But one nice morning when I tried to connect to a page containing an applet, my Java console window (which I have configured to be always started) flashed and the entire browser crashed. I used Java JDK 1.6.0_04 (32bit on 64bit Linux machine - kernel 2.6.23).

I tried again and again, but with the same result - crash. After a while I found, that there are generated files with name "hs_err_pidXXXX.log" wher XXXX was a PID of the process which crashed. I looked into it - it began with these lines:

#
# An unexpected error has been detected by Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0094a8ac, pid=5587, tid=4108254096
#
# Java VM: Java HotSpot(TM) Client VM (10.0-b19 mixed mode, sharing linux-x86)
# Problematic frame:
# C  [libc.so.6+0x718ac]  memcpy+0x1c
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

And then a lot of stack traces, thread states and so on, which I was not interested.

So it looked like a problem in libc.so.6 ! It was strange, because I used Java in my browser a day before without problems.

I realized that was the time to upgrade my browser, so I tried these steps:

• updating from FireFox 3.0 to FireFox 3.6 - hmm, the Java plugin (libjavaplugin_oji.so) did not worked


• updating from JDK 1.6.0_04 to JDK 1.6.0_21 - hmm, the Java plugin (libjavaplugin_oji.so) still did not worked - using info at FireFox Java Plugin Doc


• Finally I found info about new Java plugin technology here, so Java was able to start in FireFox 3.6, but crashed again - only the Java process, not FireFox, but I needed Java...

I tried to clear all my FireFox user profile (including cache, cookies..) - but Java crashed again.

I tried to clear all my Java user profile data - but the result was the same - crash!

I even simulated crash with Java only - no browser plugin. So now it seemed that problem is not in the plugin or browser at all.

There was still the same info in the hserr_pidXXXX.log - "problematic frame in libc.so.6". Ok, it was the time to look at this.

I found that I use libc-2.7.so. It seemed that problem could be there. OK, I had to try to compile the newest version of it and try to start Java with it (certainly I could not change the core library for the whole system !!)

(Un)Fortunately I could not compile the libc sources.
It was horrible - only reinstalling the whole system was my last chance to bring Java to life again.

I looked into the log file again and tried to decompile the info there:

Stack: [0xf4d9f000,0xf4df0000],  sp=0xf4d6e0f4,  free space=-196k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [libc.so.6+0x718ac]  memcpy+0x1c
C  [libc.so.6+0x59c01]  _IO_getline+0x41
C  [libc.so.6+0x6236b]  fgets_unlocked+0x5b
C  [libnss_files.so.2+0x3045]
C  [libnss_files.so.2+0x382b]  _nss_files_gethostbyname2_r+0xdb

The last line was a little sun in dark. It looked like a problem when translating DNS names to IP addresses using file - /etc/hosts !

I looked into /etc/hosts and I was surprised - there were two lines (records) with duplicating host names:
127.0.0.1 localhost localhost localhost .....
The whole file had 3.3MB in size !! That was the reason that it crashed in the libc - it was an overflow error caught by the system and killing the process !

And why Java only was crashing ? Maybe other applications would crash soon as well. Java was crashing just after the start, because it tried to resolve DNS names stored in the cache - in the cache of loaded applets - that was the reason !!

Uff, I was very happy, that the solution was so simple at the end.

Raw AAC Player for Android

This blog (and project) was written for those developers spending hours of googling the internet to answer a simple question:

How to make Android to play raw AAC files (streams) ?

I've spent many hours by it and I want to save other's time to public what I found.

I tried to:

• invoke MediaPlayer on raw AAC http:// URL (error)


• invoke MediaPlayer on raw AAC file (error in emulator, OK on HTC Desire)


• pass a NetSocket FileHandle to MediaPlayer (error)


• filter the stream using local ServerSocket and:
  
  
  • discard HTTP header (error)
  
  
  • change content-type to several possibilities like audio/mpeg (error)
  
  
  • sync ADTS header (icecast does not do it) (error)
  
  
• split the AAC stream into small file chunks (OK, but not excellent)


• use 3rd party native library (FAAD2) to decode AAC (OK on HTC Desire, working on emulator - but slow)

Using FAAD2 library works on my HTC Desire phone without problems. I tried 32kbit AAC stream served by www.play.cz (the exact URL you can find in the sources). It failed when trying 128kbit AAC stream served by the same server - but it also failed when using mplayer or when I saved it to a file and tried the internal HTC Desire's AAC player.

On the emulator it works, but it is not smooth (performance problems).
I tried Android 1.5 (cupcake) and Android 2.1 (Eclair) versions - both with the same results (working, but not perfect).

So the result is that playing raw AAC on Android devices (er even on the emulator) is possible, but there are several "buts":

• although Android contains AAC decoder, you cannot use it / access it natively


• if you create your own AAC decoder, then as a developer you must obtain a license from the patent holders - and you must pay for it

The sources you can find here:
http://code.google.com/p/aacplayer-android/

---

Updated 2011-05-05
I've added OpenCORE aacdec library/decoder and rewrote both Java and C parts of the player. Now I am able to play streams smoothly even on the emulator (using no more than 60% of my host CPU)!
You can download the sources and compiled shared libraries from the Downloads section (aacplayer-android-r20.zip).

How to compile it ? Just unzip the archive, edit local.properties, copy sample.ant.properties to .ant.properties (oh, yes in rev20 you must change the name of the property "loglevel" to "jni.loglevel") and run Apache Ant (beginners please consult Android SDK and NDK documentation - you need to have installed Apache Ant and GNU Make - Windows users also Cygwin). Or read the README file :-)

---

Updated 2011-05-26
I've created a follow-up project "AACDecoder" which uses only OpenCORE aacdec library and allows using the working library in other 3rd party Android projects.

http://code.google.com/p/aacdecoder-android/

Thank you all for your feedback!

GAE/J - Memcache namespaces and validations

In GAE/J SDK 1.3.4 the method MemcacheService.setNamespace(String ns) was deprecated. Instead of that one should use MemcacheServiceFactory.getMemcacheService(String ns).

When using the new one, the name of the namespace is validated against pattern, but this validation is not done when calling the old (deprecated) method:

  String namespace = "TEST:One";

  MemcacheService msOld = MemcacheServiceFactory.getMemcacheService();
  // deprecation warning by compiler, but OK at runtime:
  msOld.setNamespace( namespace );

  // no warning by compiler, but throws exception at runtime:
  MemcacheService msNew = MemcacheServiceFactory.getMemcacheService( namespace );

The exception is:

  java.lang.IllegalArgumentException:
    Namespace 'TEST:One' does not match pattern '[0-9A-Za-z._-]{0,100}'.

I think that this behaviour should be documented in low-level API at least.

http://groups.google.com/group/google-appengine-java/browse_thread/thread/8b5792d38bf74f28#

Google App Engine Data Viewer - GQL Java

A simple data viewer based on the GQL dynamic parser. The parser needs ANTLR library - antlr-runtime-3.2.jar

Currently this code was tested on local devapp server running GAE SDK 1.3.1

<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<jsp:directive.page language="java" contentType="text/html; charset=UTF-8" isELIgnored="false"/>
<jsp:directive.page import="com.google.appengine.api.datastore.Cursor"/>
<jsp:directive.page import="com.google.appengine.api.datastore.DatastoreServiceFactory"/>
<jsp:directive.page import="com.google.appengine.api.datastore.Entity"/>
<jsp:directive.page import="com.google.appengine.api.datastore.FetchOptions"/>
<jsp:directive.page import="com.google.appengine.api.datastore.QueryResultIterator"/>
<jsp:directive.page import="com.google.appengine.api.datastore.PreparedQuery"/>
<jsp:directive.page import="com.spoledge.audao.parser.gql.GqlDynamic"/>
<jsp:directive.page import="java.util.ArrayList"/>
<jsp:directive.page import="java.util.HashSet"/>
<jsp:directive.page import="java.util.Iterator"/>
<html>
<head>
<title>GQL Console</title>
<style type="text/css">
body { background: #fff; color: #333; font-family: Arial, Verdana, Sans-Serif;}
table { border-collapse: collapse; border: 1px solid #888; width:100%}
td { border-top: 1px solid #888; padding: 2px 10px; font-size: 85%}
tr.header td { font-weigth: bold; background: #555; color: #fff}
tr.even td { background: #fff;}
tr.odd td { background: #ccc;}
</style>
</head>
<body>
<form action="gql.jsp" method="post">
    Enter a GQL query (e.g. "SELECT * FROM MyEntity WHERE propertyName='propertyValue'"):<br/>
    <input type="text" size="96" name="gql" value="<c:out value="${param.gql}" escapeXml="true"/>"/><br/>
    <input type="submit" value="Execute"/>
</form>
<%
    String gql = request.getParameter( "gql" );
    String encCursor = request.getParameter( "cursor" );
    int offset = 0;
    int maxRows = 20;

    try {
        offset = Integer.parseInt( request.getParameter("offset"));
    }
    catch (Exception e) {}

    if (gql != null && gql.length() != 0) {
        GqlDynamic gqld = new GqlDynamic();
        gqld.setDatastoreService( DatastoreServiceFactory.getDatastoreService());

        try {
            PreparedQuery pq = gqld.prepareQuery( gql );
            FetchOptions fo = gqld.getFetchOptions();

            if (encCursor != null) {
                // add +1 to detect more rows:
                FetchOptions foo = FetchOptions.Builder.withLimit( (fo.getLimit() != null ? fo.getLimit() : maxRows)+1);
                foo.cursor( Cursor.fromWebSafeString( encCursor ));

                fo = foo;
            }
            else {
                offset = fo.getOffset();

                if (fo.getLimit() == null) fo.limit( maxRows + 1 );
            }

            QueryResultIterator<Entity> result = pq.asQueryResultIterator( fo );
            Cursor cursor = null;

            // find all property names
            ArrayList<Entity> entities = new ArrayList<Entity>( maxRows );
            HashSet<String> propNameSet = new HashSet<String>();

            while (result.hasNext()) {
                Entity ent = result.next();
                propNameSet.addAll( ent.getProperties().keySet() );

                entities.add( ent );
                if (entities.size() >= maxRows) {
                    if (result.hasNext()) cursor = result.getCursor();
                    break;
                }
            }

            ArrayList<String> propNames = new ArrayList<String>( propNameSet );
            java.util.Collections.sort( propNames,
                new java.util.Comparator<String>() {
                    public int compare( String s1, String s2) {
                        return s1.compareTo( s2 );
                    }
                });

            // render header
            out.print("<table><tr class=\"summary\"><td colspan=\""+ (propNames.size()+1)+"\">");
            if (entities.size() == 0) {
                out.print("No records found");
            }
            else {
                out.print("<b>" + (offset+1) + '-' + (offset+entities.size()) + "</b> ");

                if (cursor != null) {
                    out.print("<a href=\"gql.jsp?gql=" + gql + "&cursor=" + cursor.toWebSafeString()
                        + "&offset=" + (offset+maxRows)
                        + "\">Next</a> " );
                }
                else {
                    out.print( "Next" );
                }
            }
            out.print("</td></tr>");

            if (entities.size() != 0) {
                out.print("<tr class=\"header\">");
                out.print("<td>Id</td>");
                for (String propName : propNames) {
                    out.print("<td>" + propName + "</td>");
                }
                out.print("</tr>");
            }

            // render values
            int i = 0;
            for (Entity ent : entities) {
                out.print("<tr class=\"" + ((++i % 2)==0 ? "odd" : "even" ) +"\">");
                out.print("<td>");
                out.print( ent.getKey().getName() != null ? ent.getKey().getName() : ent.getKey().getId());
                out.print("</td>");
                for (String propName : propNames) {
                    out.print( "<td>" );
                    if (ent.hasProperty( propName )) {
                        Object o = ent.getProperty( propName );
                        if (o == null) {
                            out.print( "null" );
                        }
                        else {
                            pageContext.setAttribute("val", o.toString());
%>
    <c:out value="${val}" escapeXml="true"/>
<%
                        }
                    }
                    out.print( "</td>" );
                }
                out.print("</tr>");
            }
            out.print("</table>");
        }
        catch (Exception e) {
            out.print( "Invalid GQL: " + e );
            e.printStackTrace();
        }
    }
    else {
%>
    Please enter a query.
<%
    }
%>
</body>
</html>

Lists and Nulls in Google App Engine Datastore

Do you know what happens when you store empty List into Google App Engine Datastore ? And do you know what happens if you store a List containing one null value ?

Let's look at a piece of code:

    DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
    ArrayList emptyList = new ArrayList();
    ArrayList hasNullList = new ArrayList();
    hasNullList.add( null );

    Entity ent = new Entity("Test");
    ent.setProperty("nullProp", null);
    ent.setProperty("emptyList", emptyList);
    ent.setProperty("hasNullList", hasNullList);

    System.out.println("ENTITY - before: " + ent);

    Key key = ds.put( ent );
    ent = ds.get( key );

    System.out.println("ENTITY - after: " + ent);

The output of the code is following:

 ENTITY - before: <Entity [Test(no-id-yet)]:
     nullProp = null
     hasNullList = [null]
     emptyList = []
 >
 
 ENTITY - after: <Entity [Test(1)]:
     nullProp = null
     hasNullList = [null]
     emptyList = null
 >

So it means, that GAE does not store an empty list at all ! Instead of an empty list, the property is null.

This is not so big surprise and is not against anything in the official documentation. But the problem is the following:

Can we filter 'null' and '[null]' values separately ?

We will answer quickly - just extend our first example by these lines:

    System.out.println( "count(emptyList==null) = "
        + ds.prepare( 
            new Query("Test").addFilter("emptyList",
                Query.FilterOperator.EQUAL, null)
         ).countEntities());

    System.out.println( "count(hasNullList==null) = "
        + ds.prepare(
            new Query("Test").addFilter("hasNullList",
                Query.FilterOperator.EQUAL, null)
         ).countEntities());

And the output is:

 count(emptyList==null) = 1
 count(hasNullList==null) = 1

This means that it is not possible (this way) to distinguish between null and [null] properties!

So the summary is: do not store nulls into Lists, otherwise you will get into a problem in future.

Large Response in AJAX XMLHttpRequest

Sometimes happens that you need to send large data in the response of the XMLHttpRequest (XHR). For example you need to fetch a sorce code and display it somewhere in the document - exactly as I needed.

The code was as follows:

    if (window.XMLHttpRequest) {
      xhr=new XMLHttpRequest();
    }
    else {
      xhr=new ActiveXObject("Microsoft.XMLHTTP");
    }
    xhr.open("GET","/code/MyCode.java",false);
    xhr.send("");

    xmlDoc=xhr.responseXML;

    // a bug is somewhere here:
    code=xmlDoc.getElementsByTagName('code').childNodes[0].nodeValue;

Unfortunately this worked fine in Internet Explorer (IE). In FireFox (FF) it worked if the size of the code was less than certain limit. The limit in my FF 3.0 was 4096 bytes.

So the solution which works ion both IE and FF was the following:

    cnt = xmlDoc.getElementsByTagName('code')[0].childNodes;

    code = cnt[0].nodeValue;
    for (i=1; i < cnt.length; i++) code += cnt[j].nodeValue;

So you do not need to create chunks on the server, just concatenate what FF split on the client.