Background Audio and Cross Platform Development with Xamarin (App Dev on Xbox series)
We are back with yet another app, Backdrop, an open source sample media app developed to showcase a cross-device music experience. In this blog post, we will dive deep into the new background model in the Universal Windows Platform, and specifically focus on how to enable your application to continue playing audio in the background across Windows 10 devices, including Xbox One, delivering a stellar customer experience. In addition, we will show you how we built Backdrop to be a cross platform application by using Xamarin to share code between Windows 10 and AppleTV. The source code for the application is available on GitHub right now so make sure to check it out.
If you missed the previous blog post from last week on Unity interop and app extensibility, make sure to check it out. We covered how to get started building great 2D and 3D experiences with Unity and XAML as well as how to make your apps work great with other apps on the platform. To read the other blog posts and watch the recordings from the App Dev on Xbox live event that started it all, visit the App Dev on Xbox landing page.
Backdrop
Figure 1. Xbox One view
Backdrop is a sample music application that lets a group of friends collaborate on the music selection process and share their experience and music choices. A device is first chosen to be the host device where the music will be played. Then anyone is able to add and vote on tracks on their own device. Each friend can see the current playlist in real time and the progress of the current track on their own device. Each can vote on different tracks to set the order in which they will be played as well as suggest other tracks to be played.
The application has been written for the Universal Windows Platform (UWP) and tvOS by using Xamarin to share the majority of business logic, view models, playlist management, cloud and device communication. The UI, in turn, is written using the native controls and affordances of each platform. Using the shared project, additional platforms such as Android and iOS can easily be added.
Figure 2. architectural diagram
Background audio
With the Windows Anniversary Update, a new single process model for playing background audio was introduced to simplify UWP development. Using MediaPlayer or MediaPlayerElement with the new model should make implementing background audio much easier than it was before.
Previously, your app was required to manage a background process in addition to your foreground app and then manually communicate state changes between the two processes. Under the new model, you simply add the background audio capability to your app manifest and your app will automatically continue playing audio when it moves to the background. You will use the two new application life cycle events, EnteredBackground and LeavingBackground, to find out when your app is entering and leaving the background.
Background audio in Backdrop
Figure 3. Playing audio in background and System Media Transport Controls on Xbox One
Here is the step-by-step process for implementing background audio:
- Add the Background Media Playback capability to your app manifest.
- If your app disables the automatic integration of MediaPlayer with the System Media Transport Controls (SMTC), such as by setting the IsEnabled property to false, then you must implement manual integration with the SMTC in order to enable background media playback. You must also manually integrate with SMTC if you are using an API other than MediaPlayer, such as AudioGraph, to play audio if you want to have the audio continue to play when your app moves to the background.
- While your app is in the background, you must stay under the memory usage limits set by the system for background apps.
Modifying the app manifest
To enable background audio, you must add the background media playback capability to the app manifest file, Package.appxmanifest. You can modify the app manifest file either by using the designer or manually.
To do this through the Microsoft Visual Studio designer, in Solution Explorer, open the designer for the application manifest by double-clicking the package.appxmanifest item.
- Select the Capabilities
- Select the Background Media Playback check box.
To set the capability by manually editing the app manifest xml, first make sure that the uap3 namespace prefix is defined in the Package element. If not, add it as shown below. [see code on GitHub]
<Package xmlns="http://schemas.microsoft.com/appx/manifest/foundation/windows10"
xmlns:mp="http://schemas.microsoft.com/appx/2014/phone/manifest"
xmlns:uap="http://schemas.microsoft.com/appx/manifest/uap/windows10"
xmlns:uap3="http://schemas.microsoft.com/appx/manifest/uap/windows10/3"
IgnorableNamespaces="uap mp uap3"
>
Then add the backgroundMediaPlayback capability to the Capabilities element:
<Capabilities>
<Capability Name="internetClient" />
<uap3:Capability Name="backgroundMediaPlayback" />
</Capabilities>
System Media Transport Controls
Figure 4. System Media Transport Controls on mobile
The Backdrop client uses a MediaPlayer to play audio in the PlaybackService class [see code on GitHub].
public PlaybackService()
{
// Create the player instance
_player = new MediaPlayer();
_player.PlaybackSession.PositionChanged += PlaybackSession_PositionChanged;
_player.PlaybackSession.PlaybackStateChanged += PlaybackSession_PlaybackStateChanged;
_playlist = new MediaPlaybackList();
_playlist.CurrentItemChanged += _playlist_CurrentItemChanged;
_player.Source = _playlist;
_player.Play();
Dispatcher = new DispatcherWrapper(CoreApplication.MainView.CoreWindow.Dispatcher);
When using a MediaPlayer, it is necessary to implement MediaTransportControls. The media transport controls let users interact with their media by providing a default playback experience comprised of various buttons including play, pause, closed captions and others.
When adding audio tracks to the player, you also need to create a MediaPlaybackItem from the audio source and set its display properties in order to play it. Here is the code from Backdrop [see code on GitHub]:
var source = MediaSource.CreateFromUri(new Uri(song.StreamUrl));
var playbackItem = new MediaPlaybackItem(source);
var displayProperties = playbackItem.GetDisplayProperties();
displayProperties.Type = Windows.Media.MediaPlaybackType.Music;
displayProperties.MusicProperties.Title = song.Title;
displayProperties.MusicProperties.Artist = song.Artist;
displayProperties.Thumbnail = RandomAccessStreamReference.CreateFromUri(new Uri(song.AlbumArt));
playbackItem.ApplyDisplayProperties(displayProperties);
You can also override the default SMTC controls and even use manual control if you are using your own media playback. MediaTransportControls is ultimately just a composite control made up of several other XAML controls, which are all contained within a root Grid element. Because of this, you can re-template the control to change its appearance and functionality. For more info, see Create custom transport controls and the Media transport controls sample.
Managing resources in background
When your app moves from the foreground to the background, the EnteredBackground event is raised. And when your app returns to the foreground, the LeavingBackground event is raised. Because these are app life cycle events, you should register handlers for these events when your app is created. In the default project template, this means adding it to the App class constructor in App.xaml.cs.
Because running in the background will reduce the memory resources your app is allowed to retain by the system, you should also register for the AppMemoryUsageIncreased and AppMemoryUsageLimitChanging events, which will be used to check your app’s current memory usage and the current limit. The handlers for these events are shown in the following examples. For more information about the application lifecycle for UWP apps, see App life cycle. You can see the code from Backdrop here [see code on GitHub]:
private void App_LeavingBackground(object sender, LeavingBackgroundEventArgs e)
{
_isInBackgroundMode = false;
// Reastore view content if it was previously unloaded.
if (Window.Current.Content == null)
{
CreateRootFrame(ApplicationExecutionState.Running, string.Empty);
}
}
private void App_EnteredBackground(object sender, EnteredBackgroundEventArgs e)
{
_isInBackgroundMode = true;
ReduceMemoryUsage(0);
}
When your app transitions to the background, the memory limit for the app is reduced by the system in order to ensure that the current foreground app has sufficient resources to provide a responsive user experience. The AppMemoryUsageLimitChanging event handler lets your app know that its allotted memory has been reduced and provides the new limit in the event args passed into the handler. [see code on GitHub]
private void MemoryManager_AppMemoryUsageIncreased(object sender, object e)
{
var level = MemoryManager.AppMemoryUsageLevel;
if (level == AppMemoryUsageLevel.OverLimit || level == AppMemoryUsageLevel.High)
{
ReduceMemoryUsage(MemoryManager.AppMemoryUsageLimit);
}
}
private void MemoryManager_AppMemoryUsageLimitChanging(object sender, AppMemoryUsageLimitChangingEventArgs e)
{
if (MemoryManager.AppMemoryUsage >= e.NewLimit)
{
ReduceMemoryUsage(e.NewLimit);
}
}
Xamarin
Xamarin is a free Microsoft tool that allows developers to write fully native Android and iOS apps using C# and programming models already familiar to .NET developers. With Xamarin, you can build native Android and iOS apps without needing to know Java or Objective-C and best of all it is included with all editions of Visual Studio 2015. For Android development, Visual Studio has one of the most performant Android emulators built in so you can test your Xamarin.Android apps from your Windows development environment. For iOS development, Xamarin provides a remote iOS simulator for Visual Studio to test your applications without having to switch to a Mac. It even supports touch and multi-touch; something that is missing from testing iOS apps on the iOS simulator on a Mac. You will still need to connect to a physical Mac in order to build the application, but the entire development process is completely done in Visual Studio, including editing storyboards.
Apps built using Xamarin compile to native code on their respective platforms. Xamarin wraps low level native APIs so you are able to access sensors and other platform specific features. Xamarin.iOS projects compile to .app files that can be deployed to Apple mobile devices. Xamarin.Android projects compile to .apk files, the application package used for deploying to Android devices.
Because Xamarin lets you use Visual Studio projects to organize your Xamarin.Android and Xamarin.iOS code, you can include different platform versions of your app side-by-side under a common solution. When you do this, it is important to clearly separate your UI code from your business layer code. It is also important to separate out any code you intend to share between platforms from code that is platform-specific.
Sharing code with Xamarin
Xamarin provides two ways of sharing business layer code between your cross-platform projects. You can use either a Shared Project or a Portable Class Library (PCL). Each has its advantages and disadvantages.
A Shared Project is generally the easier way to create common code. The .cs files in a Shared Project simply get included as part of any project that references it. On build, the Shared Project is not compiled into a DLL. Instead, the Shared Project’s code files are compiled into the output for the referencing project. Shared Projects become tricky when they include platform specific code. You will need to use conditional compilation directives like #if and #endif in these situations to test for the appropriate compiler version.
Unlike Shared Projects, Portable Class Library projects are compiled into DLLs. You decide which platforms your PCL can run on by setting its Profile identifier. For instance, you might have your PCL target all of the following: .NET Framework 4.5, ASP.NET Core 1.0, Xamarin.Android, Xamarin.iOS and Windows 8. PCLs use conditional compilation directives in order to fork platform specific code. Instead, you may want to use a Dependency Injection pattern with multiple PCLs for this.
The Backdrop reference app uses a PCL for sharing common code between a UWP app, a tvOS app for the Apple TV, and music_appService—a RESTful web service consumed by both.
Building the UI
Each platform needs to have its own UI project. You can choose, however, to share common visual elements using the Xamarin.Forms cross-platform UI library, or create custom UIs for each platform. Both options will create fully-native UIs that will be completely familiar to end-users, but Xamarin.Forms is more restrictive. Developers seeking the greatest amount of code-sharing and efficiency will gravitate toward Xamarin.Forms, while those who require a more granular level of control should choose Xamarin.Android and Xamarin.iOS for a custom UI. The Backdrop reference app uses custom UIs.
You are probably already familiar with how to create a native UWP UI, either in Xaml or with the Visual Studio designer. You will be glad to learn that Visual Studio comes with a Xamarin Designer for iOS that provides a similarly easy way for you to work with native iOS controls. The iOS Designer maintains full compatibility with the Storyboard and .xib formats, so that files can be edited in either Visual Studio or Xcode’s Interface Builder.
Wrap up
Adding other platforms is straightforward and can be an exercise for the reader. We implemented tvOS and UWP apps, but the same shared code can be used in iOS and Android projects just as easily. The cross-platform architecture demonstrated in the Backdrop source code provides a great way to achieve the greatest reach with a single, maintainable cross-platform code base. It also shows us how the cross-platform story for UWP and the cross-platform story for Xamarin actually dovetail to create apps with much greater reach than we have ever seen before. Consider that Backdrop’s architectural design ultimately allows friends on an android phone, an iPhone, a Windows 10 phone, Apple TV and Xbox One to all work together on a common music playlist. How’s that for a build-once solution?
Until next time…
…check out the app source on our official GitHub repository, read through some of the resources provided, watch the event if you missed it, and let us know what you think through the comments below or on twitter.
Next week we will release another app experience and go in-depth on how to build hosted web experiences that take advantage of native platform functionality and different input modalities across UWP and other native platforms.
Until then, happy coding!
Resources
In the meantime, below are some additional resources to help you understand background audio as well as the architectural underpinnings of modern cross-platform apps build with Xamarin.
- https://msdn.microsoft.com/en-us/windows/uwp/audio-video-camera/background-audio
- https://msdn.microsoft.com/en-us/windows/uwp/audio-video-camera/integrate-with-systemmediatransportcontrols
- https://msdn.microsoft.com/en-us/windows/uwp/audio-video-camera/system-media-transport-controls
- https://developer.xamarin.com/guides/cross-platform/application_fundamentals/building_cross_platform_applications/
Download Visual Studio to get started!
The Windows team would love to hear your feedback. Please keep the feedback coming using our Windows Developer UserVoice site. If you have a direct bug, please use the Windows Feedback tool built directly into Windows 10.
Source: Background Audio and Cross Platform Development with Xamarin (App Dev on Xbox series)
