In Praise of ARC

It’s not all the fault of the garbage collector…but I’m growing to love ARC

When I started developing on Windows in the 1990′s, software was fast even though computers were pretty slow. The word ‘Native’ was used to describe an indigenous person and Java was still just a type of coffee.

Somehow this all changed. You could call it progress I guess.

Managed languages, virtual machines, JIT and byte code took over. This seemed to go hand in hand with garbage collection. “Horray” we all shouted. No more memory leaks.

Wrong. They just became harder to find.

There were lots of other advantages though….weren’t there? Well maybe. .Net allowed us to write in a slew of different languages that could talk to each other without having to use shared C libraries. A managed environment protected all the applications from one another and looked after resources more intelligently. So it was all still good. Right?

Frustration. Managed

I’m writing this post to vent a bit of frustration with the promises of byte code VM’s and garbage collection; I’ve fallen out of love with ‘managed’. iOS and Objective-C have shown me another way.

Android’s latest version, Jelly bean, has put the emphasis on delivering a ‘buttery smooth’ user experience. You know, the kind of experience the iPhone has enjoyed for 5 years! Well, now the Java based Android (running on the dalvik VM) has achieved the same thing. 5 years on. Thanks in no small part to huge leaps in its graphics acceleration hardware and a quad core processor!

On Windows, .Net and WPF are slow, hardware hungry beasts. If you want speed, you have to drop down to the native DirectX API’s..and until recently you could not combine these different graphics technologies very easily; Windows suffers from severe ‘air-space‘ issues.

When I started developing for iOS, I was pleasantly surprised by several things:

  • All the different API’s in the graphics stack played nice together.
  • Apps were lightning fast and beautifully smooth with low memory overhead.
  • I found the lack of garbage collection liberating.

[Garbage release]

I did not, and do not, miss the managed environment. Before ARC we had to do reference counting in Objective-C on iOS. I was used to this from my days with COM on Windows but reference counting on iOS made more sense somehow. The rules seemed clearer.

And then the compiler got clever. The compiler. Not a VM.

With the introduction of ARC we dont have to do reference counting. The compiler analyses the code and does it all for us. In the main, it does a fantastic job. The compiler and the tools for developing on iOS manage to produce native code, make it easy to consume C and C++, make reference counting almost invisible, produce sandboxed apps that can’t crash other apps and give me the good grace to use pointers where I see fit without having to declare my code “unsafe” (most of the time anyway)

I still love the Microsoft C# language and the BCL. But as for the whole managed thing? I am happy to leave it behind.

 

Understanding AurioTouch

I have been playing around with CoreAudio on iOS of late. The trouble with media API’s is that they are necessarily complex and CoreAudio is no exception.

While trying to figure out how to read data coming from the microphone and visually render the samples to the screen, I came across the aurioTouch example provided by Apple. It looked great..until I tried to work out what the code was doing!

There are so many aspects of the code that I struggled to make sense of, from arbitrary scaling factors to the odd bit of inline assembly, but here I will mention just one. In hindsight, it doesn’t seem so obscure now. But hindsight is a wonderful thing.

After having obtained a buffer full of PCM audio data, the following code is used to fill an array of values that is used to draw the data:

SInt8 *data_ptr = (SInt8 *)(ioData->mBuffers[0].mData);
for (int i=0; i<numFrames; i++)
{
    if ((i+drawBufferIdx) >= drawBufferLen)
    {
        cycleOscilloscopeLines();
        drawBufferIdx = -i;
    }

    drawBuffers[0][i + drawBufferIdx] = data_ptr[2];
    data_ptr += 4;
}

m_Buffers[0].mData contains an array of SInt32 values. These are PCM samples in 8.24 fixed-point format. This means that nominally, 8 bits of the 32 bits are used to contain the whole number part, and the remaining 24 bits are used to contain the fractional part.

I could not understand why the code was iterating through it using an SInt8 pointer and why, when the actual value was extracted, it was using data_ptr[2]. i.e It was using the third byte of the 32 bit (4 byte) 8.24 fixed point sample and chucking away the rest. I was so confused that I turned to stackoverflow for help. The answer given is spot on the money…but perhaps not all that clear if you are an idiot like me.

After printing out the binary contents of each sample I finally understood.

The code is using an SInt8 pointer because, at the end of the day, it is only interested in a single byte of data in each sample. Once this byte of data has been extracted, data_ptr is advanced by 4 bytes to move it to the beginning of the next complete sample (32 bit, 8.24 fixed point format)

The reason it extracts data_ptr[2] becomes apparent when you look at the binary. What I was failing to appreciate (a school boy error on my part) was that the samples are in little-endian format. This is what a typical sample might look like in memory:

data_ptr[0]     data_ptr[1]     data_ptr[2]    data_ptr[3]
----------------------------------------------------------
 01001100    |   01000000    |   11001111    |  11111111
----------------------------------------------------------

The data is little-endian, meaning the least significant byte has the lowest memory address, and conversely, the most significant byte has the highest memory address. In CoreAudio 8.24 fixed point LPCM data, the first (most significant) 8 bits is used to indicate the sign. They are either all set to zero or all set to one. The sample code ignores this and looks at the most significant byte of the remaining 24 bits…which is data_ptr[2]

It is safe to throw the rest away as it is of little consequence to the display of the signal; throwing the rest of the data away still gives you a ‘good enough’ representation of the sample.

Later on in the sample code (not shown above), this value is divided by 128 to give a value between -1 and 1. It is divided by 128 because an SInt8 can hold a value ranging from -128 to +127

Like I said, this is just one of many confusing bits of code in the sample app. CoreAudio is not for the feint hearted. If you are a novice, like me, then perhaps the aurioTouch sample is not the best place to start!

 

iBooks for Kids: Confusion

A while ago I blogged about a silly little iPad app I wrote for my three year old son, and how it led me to wonder about iBooks.

It struck me that an app that was essentially a book that played sounds, and perhaps had a few animations, would make more sense as an iBook. Especially as we now have access to the iBooks 2.0 platform which supports rich media content and has an easy to use authoring package.

So I wondered. And I looked. And this is what I found out.

Existing iBooks for kids are boring and overpriced.

  • Most stuff already out there uses iBooks pre version 2.0. This means you get a pretty basic eBook with pictures. A very few support sound and even some animation but it is a far from engaging experience.
  • The use of a standard eBook is a bad fit. The pages in kids books are often oddly sized and require lots of space for illustrations; you dont want simple text wrapping. As such, by default you will find that quite often the pages scroll from left to right as you flip the page, which is a pretty bad user experience. You can make both pages of a double page fold fit on the screen but you lose any sense of involvement and the book becomes letter-boxed and lost in the unused screen space.
  • Many publishes offer iBook versions of their paper counterparts. They are digital copies, plain and simple. Considering they offer such a bad experience its surprising that they are actually the same price as the printed media. One book in ‘real-life’ was a touchy-feely cloth book for 0-3 year olds with just 5 pages. Its iBook equivalent was still £4.99 yet its main function (texture) was obviously non existant!

The App Store is not the right place for books

  •  If you are looking for an engaging kids book in the real world you go to a book store, or at least the book section in a toy store. In the world of iPad it makes sense that people should go to the iBooks store but not necessarily the App Store.
  • But the iBooks store is newer than the App Store and its not installed by default. Yet.
  • Because the App Store is just there and because apps offer limitless possibilities for the user experience, many compelling kids books can be found on there. Except they are not books. They are apps. It just isn’t…quite…right.

iBooks 2.0 still isn’t a perfect fit

  • When you start to explore iBooks 2.0 and iBooks Author one thing becomes apparent: It was designed for text books. This should come as no surprise given the marketing push…but for me at least, it did. I was surprised to find that it so rigidly organises everything into chapters and sections. It does this to such an extent that the only kids books that I found that did use iBooks 2.0 have had to work around it.
  • Expecting kids to get the ‘pinch into and out of a chapter’ gesture is unrealistic. At least for small children. As a workaround the books I saw used a single iBooks chapter with a single section and titled various pages in that section “chapter 1″, “chapter 2″ etc.
  • There is no page flip animation. Sad. But probably more practical for tiny hands.
  • Landscape is where it’s at. Although iBooks Author supports both orientations, you are confined to landscape for navigation; portait is an optional extra.
  • There are still relatively few built in ‘widgets’. Because it is geared towards text books, if you want something a bit different you will need to write some code. Specifically you will need to create some HTML5 widgets in dashcode.

Conclusion

iBooks Author for iBooks 2.0 is the best there is. But it is still not as good as developing an app…and you will need to work around several issues. However, as iBooks continues to grow it seems like it should be the place to look for these kind of books that are heavy on rich media and…well, fun kid stuff.

I wonder then. If you want to create a compelling kids book that will be found by parents who are searching for such things: Do you create an app or a iBook? Do you need a developer or a publisher?

I think maybe its time to find out.

 

Playing a Sound from a Dashcode Widget

I’ve just begun to play around with Dashcode for the first time. I wanted to develop a widget that, when clicked, plays a sound. Figured it would be pretty simple. It nearly was.

The general steps are as follows:

  • Launch Dashcode and select ‘Custom’ from the Dashboard section.
  • Click on Library (top right) and click Parts
  • Drag a button onto the widget
  • Drag and drop a sound file (.m4a or any other file supported by QuickTime) from Finder onto the widget
  • For the sake of following these instructions, rename the sound file element from ‘video’ to ‘sfx’

Okay. Now we just have to write some code that will play the sound when you click the button. Fortunately or unfortunately, Apple provide some example code. If you go to the Library again and select Code instead of Parts and type ‘play quicktime’ in the search box you get some sample code. This is what you will see:

// Values you provide
var qtElement = document.getElementById("elementID");	// replace with the ID of a QuickTime element

// QuickTime code
qtElement.Play();

Now, at this point it should simply be a case of opening up the inspector, navigating to the events tab (far right), creating an onclick handler, pasting in the code and renaming ‘elementID’ to ‘sfx’.

Well it would be…If the sample code wasn’t completely wrong! The code that actually works is below:

function onButtonClick(event)
{
    var qtElement = document.getElementById("sfx");
    qtElement.firstChild.play();
}

There are two problems with the original sample code.

  1. The actual element is just a div container. To get to the media player element you have to access the first child of this container.
  2. The Play() method has the wrong casing and should be lower case, not upper case.

It kinda makes you wonder what the point of providing sample code is if it doesn’t actually work? Anyhow. Hope someone finds this useful.

I woud be interested to know if anyone knows of a better way of playing a sound effect? If you do, please post your suggestion in the comments.