Smooth Streaming Multi Resolution Support for Windows Phone

Disclaimer: The following article does only apply for Windows Phone 7.1 devices. Windows Phone 8 should have no constraints regarding playing multi-resolution streams.

I believe everyone who tried to create a video streaming app for Windows Phone, found rather frustrating that the Smooth Streaming Media Element did not support multiple resolutions. If you were using a manifest with variable resolutions on your desktop application, you could certainly be sure that this manifest will not work on your Windows Phone app out of the box. You had to handle the ManifestReady event and then use the RestrickTracks method to play only the tracks that had the same resolution.

However, I never really understood why the Multi-Resolution support was not provided in the Smooth Streaming Media Element Client for Windows Phone.

Well, not until I recently read the Video playback guidance for Nokia Lumia 610 on the Nokia Developer wiki.

The reason why first generation devices could not stream multi-resolution video was the processor.
So, basically Multiple Resolution support is available on Qualcomm 8x55 processors or higher; 7x27a and 8x50 do not support it.

There is even a property that was added to the SDK since version 7.1: MediaCapabilities.IsMultiResolutionVideoSupported which indicates whether the current device supports multi-resolution video.

Also, in MSDN you can find all the information regarding supported media codecs by each processors with specifications about Smooth Streaming support.

I just do not know, how could I miss it!

Below you will find a table specifically for Smooth Streaming Video Support according to different processors. Notice that I personally work with h.264, therefore the table references only h.264 codec.

Also, I did a small research of all available devices on the market to find out whether they support multi-resolution smooth streaming video or not.

*Information is not complete or no official data was found.

So, what can you do to start supporting multi-resolution? Well there are a couple of possible solutions:

• You could create different manifests files containing only the required resolutions for different kind of devices. Which you could request from the server by passing a custom parameter.
• Also, you could add a CustomAttribute tag to each stream and perform the selection based on it’s value when handling ManifestReady event. For example:

<QualityLevel Index="5" Bitrate="307200" FourCC="WVC1" MaxWidth="720" MaxHeight="480" CodecPrivateData = "1207F840">  
    <CustomAttributes>  
        <Attribute Name = "hardwareProfile" Value = "0" />  
    </CustomAttributes>  
</QualityLevel>  

• Adding a custom attribute requires you to mess up with the manifest. If you cannot or do not want to do this, you can manually parse the CodecPrivateData attribute for each track and based on the retrieved information select different tracks for different devices.

Personally I prefer the last option: parse the CodecPrivateData attribute.

The CodecPrivateData attribute is a base 16 text string and it contains information about h264 profile and level, height and width plus a bunch of different other encoding params.

CodecPrivateData has the following structure:
0x00000001 SequenceParameterSet 0x00000001 PictureParameterSet

Please, see ISO/IEC-14496-10 for further details on Start Codes (0x00000001), Sequence Parameter Set (SPS) and Picture Parameter Set (PPS) formats.

So, basically you will need to know the following information for a safe choice of tracks:

• Is Multi-Resolution Video Supported
• H.264 Profile
• H.264 Level
• Height
• Width
• Was CABAC used or not
• Was CAVLC used or not

By combining all these params you will be able to filter specific tracks based on supported media codecs information for each type of device.

In the end, I believe this to be the most universal solution. Since you do not need to mess up with the manifest structure in any way. You also do not need to set up proxies to different manifest files. Which is great, specially if you are using 3rd party streaming services that cannot be modified in any way.

I strongly recommend to take into account this information for your next application update and add Multi-Resolution support to your app. This way, you could get rid of reviews like ‘bad video quality’ once and for all. However, you may start getting more hateful reviews like ‘battery drainer’, be ready :)