Monday, June 28, 2010

Raw AAC Player for Android

This blog (and project) was written for those developers spending hours of googling the internet to answer a simple question:

How to make Android to play raw AAC files (streams) ?

I've spent many hours by it and I want to save other's time to public what I found.

I tried to:


  • invoke MediaPlayer on raw AAC http:// URL (error)

  • invoke MediaPlayer on raw AAC file (error in emulator, OK on HTC Desire)

  • pass a NetSocket FileHandle to MediaPlayer (error)

  • filter the stream using local ServerSocket and:

    • discard HTTP header (error)

    • change content-type to several possibilities like audio/mpeg (error)

    • sync ADTS header (icecast does not do it) (error)

  • split the AAC stream into small file chunks (OK, but not excellent)

  • use 3rd party native library (FAAD2) to decode AAC (OK on HTC Desire, working on emulator - but slow)



Using FAAD2 library works on my HTC Desire phone without problems. I tried 32kbit AAC stream served by www.play.cz (the exact URL you can find in the sources). It failed when trying 128kbit AAC stream served by the same server - but it also failed when using mplayer or when I saved it to a file and tried the internal HTC Desire's AAC player.

On the emulator it works, but it is not smooth (performance problems).
I tried Android 1.5 (cupcake) and Android 2.1 (Eclair) versions - both with the same results (working, but not perfect).

So the result is that playing raw AAC on Android devices (er even on the emulator) is possible, but there are several "buts":

  • although Android contains AAC decoder, you cannot use it / access it natively

  • if you create your own AAC decoder, then as a developer you must obtain a license from the patent holders - and you must pay for it



The sources you can find here:
http://code.google.com/p/aacplayer-android/



Updated 2011-05-05
I've added OpenCORE aacdec library/decoder and rewrote both Java and C parts of the player. Now I am able to play streams smoothly even on the emulator (using no more than 60% of my host CPU)!
You can download the sources and compiled shared libraries from the Downloads section (aacplayer-android-r20.zip).

How to compile it ? Just unzip the archive, edit local.properties, copy sample.ant.properties to .ant.properties (oh, yes in rev20 you must change the name of the property "loglevel" to "jni.loglevel") and run Apache Ant (beginners please consult Android SDK and NDK documentation - you need to have installed Apache Ant and GNU Make - Windows users also Cygwin). Or read the README file :-)



Updated 2011-05-26
I've created a follow-up project "AACDecoder" which uses only OpenCORE aacdec library and allows using the working library in other 3rd party Android projects.

http://code.google.com/p/aacdecoder-android/

Thank you all for your feedback!
Posted by at 6:28 PM

41 comments:

  1. Hi Vaclav -- thanks for posting this... Have you tried playing an aac file directly from the SD card?
    I tested on two samsung moments one having android 1.5 (cupcake) and the second having 2.0 (donut) both yield the same "choppy" results using the FAAD2 decoder (the other option doesn't work at all). hmmmm the moment has 1GHZ processor
    shouldn't that be enough to handle a decoder?

    ReplyDelete

  2. Hi, thanks for sharing your insights. Q: when you split the stream into chunks, what container format did you use?
    I read that it should be able to support the ADIF format as well, which is basically just a header in front of the stream, and not in front of every aac frame.

    So perhaps it is possible to put an ADIF header in front of the raw stream and make the player believe it's an ADIF file...?

    Just an idea....

    ReplyDelete

  3. Hello, Vaclav!

    Thank you for this post.
    You wrote "split the AAC stream into small file chunks (OK, but not excellent)".

    Please tell/mail me (silvanovich.michael@gmail.com) how you did it (split AAC stream into small file chunks, and how you played these chunks).

    Thank you.

    ReplyDelete

  4. Hi Vaclav,

    I've been trying to use the aacPlayer project put up in the google code. I must say that it was a big relief to find a working solution :)

    One question, the streams do not appear to play continuously. I assume that the delay is because of the need to decode and play. But is there any strategy to refine this?

    Regards,
    Hari

    ReplyDelete

  5. Hi,

    Thanks for sharing your project.

    I am working on a similar where i should use ffmpeg to encode the audio file into G726 format. Can I use directly your ffmpeg compiled code or should I have to compile once again for Android using Linux. Since I am developing application in windows. Please help me if you have some suggestions for my question.

    Thanks
    KARUMANCHI

    ReplyDelete

  6. Hi,

    I'm new on Android development (I do iPnoe dev).

    Can you please provide an working exaple code with latest build (r14), that would help me a lot.

    Thanks!

    ReplyDelete

  7. Hi all !

    I've added OpenCORE aacdec library/decoder and rewrote both Java and C parts of the player. Now I am able to play streams smoothly even on the emulator (using no more than 60% of my host CPU)!
    You can download the sources and compiled shared libraries from the Downloads section (aacplayer-android-r20.zip).

    How to compile it ? Just unzip the archive, edit local.properties, copy sample.ant.properties to .ant.properties (oh, yes in rev20 you must change the name of the property "loglevel" to "jni.loglevel") and run Apache Ant (beginners please consult Android SDK and NDK documentation - you need to have installed Apache Ant and GNU Make - Windows users also Cygwin).

    Thanks all for feedback !

    Vaclav

    ReplyDelete

  8. Hi KARUMANCHI,

    no, you should compile it yourself, because I restrict the compilation process only to files related to AAC.
    See jni/ffmpeg/Android.mk

    Vaclav

    ReplyDelete

  9. Hi Hari,

    there were three problems there: first - FAAD2/FFmpeg are quite slow (but only on older devices/emulator).

    Second - the original algorithm of decoding was wrong. The input buffer was passed to the decode() function and the ouput buffer was supposed to be filled in - like 1:1. But this did not work unless you were very careful about the sizes of the buffers. Now the decode() function is called only with the output buffer and the decode() function requests (callback to Java) input data on-demand.

    Third - the AudioTrack object was badly initialized. AudioTrack object is able to buffer audio data - just call method write() first and the the method play(). Also the size of the audio buffer should be initialized according to some rules.

    Now the last two problems were solved (Revision 20).

    Generally speaking we have 3 buffers - IN (input), DEC (decoded data), AU (buffer in AudioTrack). I recommend the following:
    1. Set AU to some value to overcome networking problems (e.g. 2 seconds)
    2. Must: DEC < AU, should: 2*DEC < AU (e.g. 0.75 sec)
    3. Should: IN [bytes] = bitrate / 8 * DEC

    I'll try to explain this later as a separate post or on project's Wiki.

    Vaclav

    ReplyDelete

  10. Hi Vaclav,

    Must say its a great work !!

    Are you planning to do something similar for WMA as well? Meaning support the playing out of WMA streams on Android device?

    ReplyDelete

  11. Hi "Mobile",

    thanks.
    Regarding WMA - yes, I just committed initial version into the SVN trunk of (rev23 - see http://code.google.com/p/aacplayer-android/issues/detail?id=3#c4)
    It is not a final version yet. I will continue working on it after getting back from my vacation (end of June).

    Vaclav

    ReplyDelete

  12. Looks great !
    Can you upload .apk file ?
    I have problems installing apache Ant , unable to compile the project

    ReplyDelete

  13. Hi Amit,
    unfortunately I cannot upload APK. As for the AAC decoder one must pay license fees. So I only published the source code / libraries, but not the final audio player.
    Several people were able to compile it, so I hope it would not be a big issue for you.
    Vaclav

    ReplyDelete

  14. This comment has been removed by the author.

    ReplyDelete

  15. Hi, can i use
    http://code.google.com/p/aacplayer-android/
    in commercial applications ? (to ask fee for application, to put banner service in it)

    ReplyDelete

  16. Hi Vaclav,

    I just looked at the WMA support and tried using the code. This is awesome, it plays well for 90 % of the streams. However, for 10 %, I observed app crashes. Have also raised this issue at the google code project. Can you please address this

    Thanks in Advance

    ReplyDelete

  17. Hi Alex,

    the project http://code.google.com/p/aacplayer-android/ is licensed under GPL, so you can create commercial apps on top of it, but you need to fullfill the GPL - mainly it means to publish your code as well.

    If you use the second project http://code.google.com/p/aacdecoder-android/ , then you do not need to publish your code (the library is licensed under LGPL).

    Vaclav

    ReplyDelete

  18. Hi Mobile,

    yes, I know - the problem is somewhere inside the libmms library.

    Vaclav

    ReplyDelete

  19. Hi Vaclav,

    Thanks for the reply

    Any recent efforts are you making to address this?

    Thanks

    ReplyDelete

  20. Hi Mobile,

    I am not going to fix it soon. The main reason is that it is located in the libmms - 3rd party library. Also I don't have much time for this project now. Sorry.

    Vaclav

    ReplyDelete

  21. hi Vaclav!
    first of all,deep appreciate for your project,i am newbie of android app develop.after finished to build AACplayer project,i have a question
    how to make a universal player that can be autodectect the format and play the audio. is this possible to do that just use the FFMpeg open source? can you tell me the procedure or guideline.thanks a lot.

    ReplyDelete

  22. Hi firebird,
    unfortunately there exists no simple answer/solution for doing it. You need to port all audio related parts of FFMpeg to Android. Then to connect it with Android's audiotrack for playing. In Android NDK r5 you can use native audiotrack functions, so the whole code could be probably written in pure C/C++.
    I expect a lot of obstacles when porting such a big piece of code. Good luck !
    Vaclav

    ReplyDelete

  23. Hi Vaclav,

    Big thanks for your efforts!

    We've been playing with similar stuff in FFMPEG at https://github.com/mikebevz/AFPlayer - mainly to support HLS.

    Recently I've been investigating the built in OpenCode AAC decoder that all phones are bundled with from the Android system.

    It seems it is actually possible to load the shared libraries on the phone (in /system/lib/), and so also load libomx_aacdec_sharedlibrary.so containing the AAC decoder.

    The a hazzle is that the .so file isnt always callled the same thing.

    SE_XPeria_mini libomx_aacdec_sharedlibrary.so
    HTC_Wildfire libomx_aacdec_sharedlibrary.so

    Samsung_Galaxy_SII libsaaceomxoc.so

    Android4 libstagefright_soft_aacdec.so


    My questions is, did you look into this possibility?

    Regards,
    Jacob Nordfalk

    ReplyDelete

  24. Hi Vaclav,

    My name is Duško, and i'm trying to build Internet radio player, for devices with Froyo as minimal platform. As all of the people here, i'm stuck with aac decoding.
    I'm using your library with OpenCORE (0.5.1 version), and it's working of course. The problem is, when using ACCPlayer CPU is heavy utilized, and application gets unresponsive for any interface action (although it plays radio stream in background). It does not happening with MPEG stream when using MediaPlayer.

    For testing purposes i've created one activity with one button (play/stop) and two methods (play() and stop()). AACPlayer is intialized by following your instructions
    AACPlayer aac=new AACPlayer();
    aac.play(aacStream);

    When presing button to stop streaming ..debug window comes with info " Key dispatching timed out sending to ..." and after few seconds, application gives "Force close / Wait" dialog.

    Is there something that i'am doing wrong, because there is no difference between running app. in emulator or real device (i suppose device performanse is not the reason for this issue).

    Thank you for your work.
    Best regards!

    ReplyDelete

  25. Hi Duško,

    is the same problem hapenning also when using the example application ? On emulator or on device (or both?).

    I suppose that you need to use the "playAsync" method instead "play" method. Maybe this is your problem ?

    Vaclav

    ReplyDelete

  26. Hi Vaclav,

    I just checked your latest OpenCore AAC Libraries and they are awesome.

    The only question I have is. What should I do if I include them in my project. Meaning, I wanted to know about the licensing etc.

    -Hari

    ReplyDelete
    Replies

    1. Hi Hari,

      if you include the libraries as is, then you don't need to do anything. LGPL license allows you to use it as-is. You can use the compiled binaries or recompile the source yourself if you wish. Does not matter if your final application will be free or payed.

      Vaclav

      Delete

    2. Thanks Vaclav :)

      What about the AAC Licensing. Do I have to worry about that?

      Delete

    3. Yes, you should conform to the AAC licenses as I wrote in the README/Project wiki. But I am not a layer, so I cannot tell you what exactly you may and must.
      You should probably ask AAC License holders if you have any doubts.

      V.

      Delete

  27. I deleted my first comment as I've made it a little further :)

    I now have the app opening and running, but when I put in a URL that works (via the command line build of the your aacplayer code) I'm getting the following three lines before getting a socketException


    01-21 16:05:59.859: D/dalvikvm(392): Trying to load lib /data/data/com.XXX/lib/libaacdecoder.so 0x4817af40
    01-21 16:05:59.859: D/dalvikvm(392): Added shared lib /data/data/com.XXX/lib/libaacdecoder.so 0x4817af40
    01-21 16:05:59.859: D/dalvikvm(392): No JNI_OnLoad found in /data/data/com.XXX/lib/libaacdecoder.so 0x4817af40, skipping init

    Any help would be greatly appreciated :-)

    ReplyDelete

  28. OK... I have once again made a newbie mistake. Helps if I actually let the app use the internet :)

    Thanks for this great code. Now I need to figure out the rest of my app :)

    ReplyDelete

  29. Vaclav

    Do you have any idea where to look to start pulling streaming metadata along with the AACP audio?

    ReplyDelete

  30. Hello Vaclav, thank you again for your awesome work on this. I'm no expert in decoders and such so this definitely is a great learning experience for me.

    I do have a question though. I'm studying your aacdecoder-int and was wondering if there's a way to tinker this a bit so it can parse m4a-contained streams. I know your code only plays raw AAC streams with an ADTS header present, but was wondering if its possible to leverage this to play m4a streams as well. Again, I'm no expert on codecs and decoders, but how would I do so on m4a streams? I've been googling to see what how to strip the m4a container programmatically but haven't had much luck .... most searches simply redirects me to aac streams with ADTS header present (unless I'm searching for the wrong keywords)

    If you can provide some pointers that will surely help!

    ReplyDelete
    Replies

    1. Hi,

      unfortunately I don't think it could be as easy as adding few lines of code. In the aacplayer project I added possibility to play also WMA streams, but I had to use the ffmpeg library in deep. So in that case it would not have much in common with this aacdec code.
      But of course it would be possible. The question is if it is really needed - probably standard Android's MediaPlayer should play it, or shouldn't ?

      Vaclav

      Delete

  31. HI -

    I'm also curious about how to pull metadata from a AAC stream.

    Thanks

    Rich

    ReplyDelete

  32. |HI, Vaclav

    It was playAsync trouble..it's a newbie mistake, but now i got a little further and it's out of my knowledge. aacdecoder works great on Froyo platform. I tested it on emulator and on real device, and it's playing perfectly. The trouble is with Gingerbread. This is the stacktrace i'm getting from users with real devices, and dont know what to do (it has been tested on gingerbread inside emulator, and its working).

    Caused by: java.lang.UnsatisfiedLinkError: Couldn't load aacdecoder: findLibrary returned null
    at java.lang.Runtime.loadLibrary(Runtime.java:429)
    at java.lang.System.loadLibrary(System.java:554)
    at com.spoledge.aacdecoder.Decoder.loadLibrary(Decoder.java:167).

    library is included (armeabi and armeabiv7) in libs folder and aacdecoder.jar is imported via build path. I must say, the same version, works on Froyo, but not on Gingerbread.

    In your guide, you said only to import armeabi (only with aacdecoder.so) folder in libs and include jar file. In player example, i've found armeabi (but there are extra files included, besides aacdecoder.so) and armeabiv7 folder without aacdecoder.so. And i must admit, i've tried all variants of importing libraries and Gingerbread wont load them.

    Best regards.

    ReplyDelete

  33. This is one of the nice post.I like your blog features.Try to get more this kind of information.supper.
    Android app developer

    ReplyDelete

  34. Hi Vaclav,

    Another question to you. Does your decoder support aacPlus v1 encoded streams? I have an audio file encoded to aacPlus v1 (64kbps @ 44100 KHz, contained in an ADTS container) and when I tried to play this file to your player/decoder, it is throwing an exception because the decoder is reporting a sampling rate of 96000Hz, even though I've explicitly encoded it at 44100Hz (and other players like VLC and Quicktime are reporting an output sample rate of 44100Hz). I'm trying to determine whether the fault is the transcoder tool that I'm using, but thought I'd ask you in case you know this offhand and can provide a quicker response. I'm also not sure if this is a bug in your decoder or the tool itself. When I encoded the file using AAC codec (NOT aacPlus v1), it works fine.

    Can you help? Would really appreciate that. Thanks.

    JL

    ReplyDelete
    Replies

    1. Hi,

      I'm quite sure AAC+ v1 (SBR) is supported as far as the underlying OpenCORE library supports it. My decoder is only a kind of wrapper on top of it.

      I've found that AAC+ v2 (resp. Parametric Stereo enhacement) was not working even though I've been using the latest OpenCORE code (directly from GIT repos).

      Anyway, the latest OpenCORE code was better (producing better quality sound for some streams) than the one I'm using in this project. The problem is that one must have checked-out all GIT tree for the OpenCORE so it is not so easy to compile it as it is now. And I had no time to put it (including how-to) to the SVN yet.

      Vaclav

      Delete

    2. Hmm, really? I think I was able to play an AACv2 @40kbps stream (on ADTS container) with your encoder just fine. Its the AACv1 that I continue to have issues with. I'm using aacPlusEnc tool for Windows to encode some of my test wav files. I think the decoder is able to parse the ADTS header, but for some reason its receiving a 96KHz sampling rate even though I've encoded the wav at 44.1KHz. If I manually change the sampling rate to 44.1KHz when I instantiate the AudioTrack, the decoder throws an exception during playback.

      What version of OpenCore are you using in your project?

      Thanks again for all your help.

      JL

      Delete

Subscribe to: Post Comments (Atom)