Sergiy Baydachnyy

Blog about technologies

Posts Tagged ‘Universal Applications

UWP: How to implement Drag and Drop functionality

leave a comment »

One more new feature, which is available for Windows 10 developers, is Drag and Drop support. Starting with Windows 10 you can implement Drag and Drop functionality between UI parts of your application or use external sources/targets including Win 32 applications.

Let’s start with Drag operation. In order to show how Drag operation works, I simply added an image from the application package to the main page.

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"> <Image Source="Assets\drone.jpg" Name="image" CanDrag="True" DragStarting="Image_DragStarting" Margin="100" VerticalAlignment="Top" HorizontalAlignment="Left"></Image> </Grid>

You can see two important attributes there that activate Drag operation: CanDrag and DragStarting. CanDrag attribute is just a flag which enables the feature for all UIElement controls but DragStaring attribute contains name for event handler. Thanks to this event handler you can define any content to drag. In my case I implemented the following handler:

private async void Image_DragStarting(UIElement sender, DragStartingEventArgs args) { List<IStorageItem> files = new List<IStorageItem>(); StorageFile file = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Assets/drone.jpg")); files.Add(file); args.DragUI.SetContentFromDataPackage(); args.Data.RequestedOperation = DataPackageOperation.Copy; args.Data.SetStorageItems(files); }

In this event handler I used StorageFile class in order to pass my image like a file and thanks to Data property of DragStartingEventArgs parameter I packaged the file to the object of DataPackage class. DataPackage class is very popular among different features in Universal Windows Platform and usually you need to pass it to the operation system and OS allows to select the target application. But in case of Drag functionality user selects the target directly. So, we just need to prepare the DataPackage and that’s all.

Additionally I used two important properties there: DragUI and RequestedOperation. Thanks to the RequestedOperation I can assign the right operation and user should not be able to select anything from system menu – just drag and drop. Thanks to DragUI I can apply the content which will be shown during Drag operation. If you don’t use DragUI property, user will see the same image with the same width and height like in your application. It’s not very cozy to drag a huge image especially if you don’t use RequestedOperation – the system menu will be behind the image. That’s why you can assign any other content using DragUI or use SetContentFromDataPackage method in order to ask API to prepare appropriate icon for you based on content in DataPackage.

Just run the application and drag and drop the image to the file explorer – image will be copied to the selected folder.

Let’s see how to implement an opposite task – Drop functionality. I want to accept several images. So, I am going to use ListView in order to show my items.

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}" AllowDrop="True" Drop="Grid_Drop" DragEnter="Grid_DragEnter"> <ListView Margin="50" Name="listView"> <ListView.ItemTemplate> <DataTemplate> <Grid> <Image Source="{Binding Source}" Width="200" Margin="10"></Image> </Grid> </DataTemplate> </ListView.ItemTemplate> </ListView> </Grid>

You can see that I am using AllowDrop to activate the Drop functionality, DragEnter – to setup allowed operations (Copy) and Drop – to get content from DataPackage and show it using ListView.

In order to create items source for images I created BitmapItem class:

class BitmapItem { public ImageSource Source { get; set; } }

In the next step I implemented DragEnter event handler in order to notify the system about supported operations.

private void Grid_DragEnter(object sender, DragEventArgs e) { e.AcceptedOperation = DataPackageOperation.Copy; }

Finally, I am using DataPackageView in order to get reference to the content. DataPackageView can contain anything but I want to work with files only, so I call GetStorageItemsAsync in order to get references to files there and use BitmapImage to prepare the image files for Image objects.

private async void Grid_Drop(object sender, DragEventArgs e) { var files=await e.DataView.GetStorageItemsAsync(); List<BitmapItem> items = new List<BitmapItem>(); foreach(StorageFile file in files) { try { BitmapImage bi = new BitmapImage(); bi.SetSource(await file.OpenAsync(FileAccessMode.Read)); items.Add(new BitmapItem() { Source = bi }); } catch { } } listView.ItemsSource = items; }

I am too lazy, so I decided to avoid any checking – I simply use empty catch block if user passed non-image file(s).

That’s all. You can see that it’s easy to implement Drag and Drop functionality and you can make experiments with different content types there or implement the same functionality inside the same application (drag and drop content from one part of application to another).

Written by Sergiy Baydachnyy

08/25/2015 at 10:33 PM

Live Visual Tree in Visual Studio

leave a comment »

In this post I am going to talk about a cool tool in Visual Studio – Live Visual Tree. This tool is available for Store and WPF applications including Store applications for Windows 8.x.

Live Visual Tree allows to observe XAML visual tree in runtime, review properties of elements there and even change properties in runtime in order to check how changes will affect interface without restarting application itself. There are lots of scenarios when you need to use a tool like this and I have already adopted this tool for the following tasks:

· If you have a huge number of visual states in your interface it’s really hard to understand while a state is not fired. Thanks to Live Visual Tree you can check properties of all controls and see what happens applying changes to these properties – check if a state is applied and how the result looks like;

· Visual designer in Visual Studio works fine but if you have lots of bindings you can use Live Visual Tree to change something in runtime and see how it affects the design of the running application with active bindings and data there;

· Live Visual Tree allows to visualize layout in order to show alignments and space for UI elements. It allows to find some mistakes which you cannot find in design mode due to absence of real data;

· Checking count of elements in each container you can find potential problem with performance;

· Live Visual Tree allows to check XAML tree not just for your applications – you can easily attach debugger to any running “XAML application” and see XAML visual tree there and apply any changes in the same way;

In order to open the Live Visual Tree window for your application you need to run your application in the Debug Mode and you can find the Live Visual Tree using Debug->Windows menu item. The second window, which is associated with the Live Visual Tree, is Live Property Explorer. Using the Live Visual Tree you can navigate between XAML elements and using Live Property Explorer you can check, change and create properties there.

clip_image002

You can see that Live Visual Tree provides information about number of XAML elements inside each container. Pay special attention that you can see elements from XAML visual tree only. So, Live Visual Tree contains visible elements only, and if interface changes from one state to another you can see that Live Visual Tree is changing in runtime.

Live Property Explorer shows default values for properties, values which were inherited from other controls and local values of control properties itself. Of course, you can modify local values only. If a value has been assigned to a property previously you can modify it but you can add any other available properties and assign values to them as well.

In Live Visual Tree window you can find two useful buttons. The first one allows to select any element in the running application in order to find it in the XAML tree. It’s very useful when you want to locate a place of a button and other controls that have tap event handler. The second one allows to visualize layout. Once you select any element you will be able to see layout there.

As I mentioned earlier you can attach Visual Studio to any existing “XAML window”. For example, you can run the settings window and select Attach to Process in Debug menu of Visual Studio 2015. In the Attach to Process dialog select SystemSettings process and you will be able to see the structure of the window. You even can change current settings and check layout of the window.

clip_image004

So, Live Visual Tree is a very powerful tool which can help to make your application better.

Written by Sergiy Baydachnyy

08/11/2015 at 9:41 PM

Win2D: How to use graphics without knowledge in DirectX

with one comment

Direct2D is a very good technology but using of it requires some knowledge in DirectX. Yes, DirectX is cool but lots of developers who build business applications have never used DirectX in their entire life. That’s why many of them ask about Direct2D features like about a separate Windows Runtime API, which is ready for non-game developers. Win2D is such an API.

Win2D is a Windows Runtime API which is available for C++ and C# developers and brings GPU optimized 2D graphic to Windows 8.x/Windows 10 XAML applications.

The best way to start working with Win2D is to visit Win2D Team Blog where you can find links to the documentation, sample code, source code (Win2D is an open source project) etc. But if you want to understand the full power of Win2D it is better to start develop something using this API. So, let’s look at some useful classes there and learn how to start developing with Win2D.

Since Win2D is an open source project and is not a part of Universal Windows Platform by default, you need to use NuGet package manager to add the latest version of Win2D libraries (use Win2D.uwp).

clip_image002

NuGet will add a reference to Microsoft.Graphics.Canvas assembly what contains all Win2D classes. So, it’s easy to open this assembly and find all namespaces and classes at the same place.

clip_image003

Just open Microsoft.Graphics.Canvas.UI.Xaml namespace and find all XAML controls there. You can see that there are two classes based on UserControl: CanvasAnimatedControl and CanvasControl. Additionally, there is CanvasSwapChainPanel class which is derived from Grid class. Let’s see these classes in more details.

First of all you need to add Microsoft.Graphics.Canvas.UI.Xaml namespace to your XAML file:

xmlns:canvas="using:Microsoft.Graphics.Canvas.UI.Xaml"

Right after that you can use the controls there and we will start with the simplest one, CanvasControl.

<canvas:CanvasControl Draw="CanvasControl_Draw"></canvas:CanvasControl>

CanvasControl is a place for drawing but we need some places to implement drawing algorithms. That’s why CanvasControl has two important events: Draw and CreateResources. Let’s start with Draw event and look at the code below:

int i = 0; private void CanvasControl_Draw(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasDrawEventArgs args) { i++; CanvasTextFormat format = new CanvasTextFormat() { FontSize = 96, FontWeight = FontWeights.Bold }; args.DrawingSession.DrawText($"Hello {i}", 200, 100, Colors.Green, format); }

In this event handler we used DrawText method to print text. You can find lots of different methods inside CanvasDrawingSession class and it’s easy to use. Pay special attention that I injected integer variable to the output string. Thanks to this variable we can see when our event handler works. Just run the application and try to work with the window for some time. You will see that Draw event fires when it’s needed for Windows to redraw the window. So, if you change window size your counter will increase but if you don’t touch widow at all the counter will remain the same.

clip_image005

That’s why CanvasControl works fine for “static” content.

The second important event for CanvasControl is CreateResources event. Let’s look at our code again and you can see that the application creates CanvasTextFormat object each time when Draw event handler is invoked. Of course, this object is not very complex but in case of real scenarios developers need to make lots of objects first before they can start drawing and additionally developers should guarantee that all objects will be created before event handler is invoked. There are two ways to initialize all needed objects: assign Draw event handler dynamically (right after initialization methods) or use CreateResources event handler. The second method is more straightforward and implements some stuff for asynchronous programming.

Let’s modify our XAML page:

<canvas:CanvasControl Draw="CanvasControl_Draw" CreateResources="CanvasControl_CreateResources"></canvas:CanvasControl>

And here is code:

CanvasLinearGradientBrush brush; CanvasTextFormat format; private void CanvasControl_CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.CanvasCreateResourcesEventArgs args) { brush = new CanvasLinearGradientBrush(sender, Colors.Red, Colors.Green); brush.StartPoint = new Vector2(50, 50); brush.EndPoint = new Vector2(300, 300); format = new CanvasTextFormat() { FontSize = 96, FontWeight = FontWeights.Bold }; } int i = 0; private void CanvasControl_Draw(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasDrawEventArgs args) { i++; args.DrawingSession.DrawText($"Hello {i}", 50, 50, brush, format); }

You can see that I used CanvasControl_CreateResources method to make a brush for my text.

clip_image007

Using CreateResource event handler you can guarantee that Draw event will not be fired up to the event handler is completed.

In the previous example we used synchronous event handler for CreateResource method but even for simple image loading action you need to call asynchronous APIs. Of course advanced C# developers will add async keyword before the event handler signature but this approach doesn’t work for CreateResources event handler. Instead of async keyword you should use the following approach:

CanvasImageBrush brush; CanvasTextFormat format; private void CanvasControl_CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.CanvasCreateResourcesEventArgs args) { args.TrackAsyncAction(CreateResources(sender).AsAsyncAction()); } async Task CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender) { brush = new CanvasImageBrush(sender); brush.Image= await CanvasBitmap.LoadAsync(sender, "Assets/drone.jpg"); format = new CanvasTextFormat() { FontSize = 96, FontWeight = FontWeights.Bold }; } int i = 0; private void CanvasControl_Draw(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasDrawEventArgs args) { i++; args.DrawingSession.DrawText($"Hello {i}", 50, 50, brush, format); }

You can see that we used parameter of event handler to call TrackAsyncAction method and this method can get async method as a parameter.

Ok. Thanks to CanvasControl we can draw “static” content and it’s fine for many scenarios. For example, you can use this control to apply effects to images, you can create graphs or high performance text rendering application. But you cannot use CanvasControl for developing simple 2D games or similar applications which should present “dynamic” content. In order to make it, Win2D provides another control: CanvasAnimatedControl.

We can use almost the same code, slightly modifying it:

<canvas:CanvasAnimatedControl Draw="CanvasAnimatedControl_Draw" CreateResources="CanvasAnimatedControl_CreateResources"></canvas:CanvasAnimatedControl> CanvasImageBrush brush; CanvasTextFormat format; async Task CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasAnimatedControl sender) { brush = new CanvasImageBrush(sender); brush.Image= await CanvasBitmap.LoadAsync(sender, "Assets/drone.jpg"); format = new CanvasTextFormat() { FontSize = 96, FontWeight = FontWeights.Bold }; } int i = 0; private void CanvasAnimatedControl_Draw(Microsoft.Graphics.Canvas.UI.Xaml.ICanvasAnimatedControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasAnimatedDrawEventArgs args) { i++; args.DrawingSession.DrawText($"Hello {i}", 50, 50, brush, format); } private void CanvasAnimatedControl_CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasAnimatedControl sender, Microsoft.Graphics.Canvas.UI.CanvasCreateResourcesEventArgs args) { args.TrackAsyncAction(CreateResources(sender).AsAsyncAction()); }

You can see the same text but counter will increase very fast (every 16.6 ms or 60 steps per second). So, CanvasAnimatedControl is better to create a large number of dynamic objects that are flying, firing, jumping etc. Of course, it’s not enough to have just Draw method because in real games you need to guarantee timing and have the same speed on all devices. So, a game loop is more complex than a simple Draw method. But developers of Win2D know about it and you can find some more events and useful properties in CanvasAnimatedControl class.

Let’s review those properties and events slightly modifying the previous Draw even handler:

private void CanvasAnimatedControl_Draw(Microsoft.Graphics.Canvas.UI.Xaml.ICanvasAnimatedControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasAnimatedDrawEventArgs args) { i++; args.DrawingSession.DrawText($"Hello {i}", 50, 50, brush, format); sender.Paused = true; }

You can see that we used Paused property to make pause in our game loop. And if you don’t touch the application you can see that the counter is frozen but once you resize the window the counter will grow fast. It happens because in the pause mode draw method is called once Windows needs to redraw the window. So, it’s better to use Draw event handler only for drawing. If you need to change any data in the game loop then it is better to use Update event. Update event handler will be called before Draw method and in case of pause Update event handler will be frozen.

private void CanvasAnimatedControl_Update(Microsoft.Graphics.Canvas.UI.Xaml.ICanvasAnimatedControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasAnimatedUpdateEventArgs args) { i++; }

In general, Draw+Update event handlers should be called every 16.6 ms but if you have a slow device and running timeframe is greater, Update call may be passed. So, you can see several Draw calls before Update call. It allows saving same speed of the game but you can see some drawing problems. If it happens not very often it’s not very critical but in some cases you can decrease the elapsed time for each step (for example, 30 frames per second). You can easily do it using TargetElapsedTime property.

Additionally, you can use events like GameLoopStarting and GameLoopStopped. These events fire before and after the game loop and can be used for scene initialization and for destroying all objects from memory respectively.

Pay special attention that all event handlers fire in separate gaming loop. So, any actions there will not block interface thread. But developers should think how to pass the data to the gaming thread from the interface one. The best way is calling RunOnGameLoopThreadAsync method from the interface thread:

await myCanvas.RunOnGameLoopThreadAsync(()=> {/*call something here*/});

The last control in Microsoft.Graphics.Canvas.UI.Xaml namespace is CanvasSwapChainPanel. If you know something about game development you should have heard about the swap chains. The main idea is have two or more buffers (pages) for your game. The first page you use for presenting an updated scene on the screen while updating the second one behind the scene. You should not use CanvasSwapChainPanel class or CanvasSwapChain with CanvasAnimatedControl because the last one uses swap chains internally. But if you want to implement your own CanvasAnimatedControl or a similar control you can use both classes.

var swapChain = new CanvasSwapChain(device, width, height, dpi); swapChainPanel.SwapChain = swapChain; //draw swapChain.Present();

Ok, right now we have some knowledge how to draw something and what are the differences between controls there. So, it’s time to talk about other useful classes and I am going to start with image effects.

Image effect is the most growing category in Win2D and just several days ago they added 10 more effects but we are still waiting for some classes, which help to create our own custom effects. In any case Win2D API already contains more than 50 effects, so, you definitely can find some useful filters there. Here is an example how to apply two effects to the same image:

GrayscaleEffect effect; GaussianBlurEffect blurEffect; async Task CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender) { effect = new GrayscaleEffect(); var bitmap=await CanvasBitmap.LoadAsync(sender, "Assets/drone.jpg"); effect.Source = bitmap; blurEffect = new GaussianBlurEffect(); blurEffect.BlurAmount = 5; blurEffect.Source = effect; } private void myCanvas_Draw(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasDrawEventArgs args) { args.DrawingSession.DrawImage(blurEffect); } private void myCanvas_CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.CanvasCreateResourcesEventArgs args) { args.TrackAsyncAction(CreateResources(sender).AsAsyncAction()); }

In this example I used blur and grayscale filters. Since I don’t have any other logic I used CreateResources event handler.

clip_image009

If you use the same effect for the same image you can simply apply this effect in advance and reuse it later. But in many cases you need to build more complex objects which should contain not just effect but some drawing etc. In this case you can use CanvasCommandList class in order to prepare and preserve your object for future use. I just slightly modified the previous example:

GrayscaleEffect effect; GaussianBlurEffect blurEffect; CanvasCommandList cl; async Task CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender) { cl = new CanvasCommandList(sender); using (CanvasDrawingSession clds = cl.CreateDrawingSession()) { effect = new GrayscaleEffect(); var bitmap = await CanvasBitmap.LoadAsync(sender, "Assets/drone.jpg"); effect.Source = bitmap; blurEffect = new GaussianBlurEffect(); blurEffect.BlurAmount = 5; blurEffect.Source = effect; clds.DrawImage(blurEffect); } } private void myCanvas_Draw(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasDrawEventArgs args) { args.DrawingSession.DrawImage(cl); } private void myCanvas_CreateResources(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.CanvasCreateResourcesEventArgs args) { args.TrackAsyncAction(CreateResources(sender).AsAsyncAction()); }

In this example we created our own CanvasDrawingSession and used it to draw all needed objects and to apply filters there. Right after that we destroyed CanvasDrawingSession and preserved CanvasCommandList for future use (making global reference on it). Thanks to this approach you can create all complex objects in advance and later draw them as images.

When I told about image effects I didn’t mention geometry and text but you can apply effects to these entities as well. In order to do in you need to use CanvasRenderTarget class for converting vector data to pixels:

private void myCanvas_Draw(Microsoft.Graphics.Canvas.UI.Xaml.CanvasControl sender, Microsoft.Graphics.Canvas.UI.Xaml.CanvasDrawEventArgs args) { var myBitmap = new CanvasRenderTarget(sender, 300, 300); using (var ds = myBitmap.CreateDrawingSession()) { ds.DrawText("Hello", 0, 0, Colors.Green, new CanvasTextFormat() {FontSize=96, FontWeight=FontWeights.Bold }); } var blur = new GaussianBlurEffect { BlurAmount = 10, Source = myBitmap }; args.DrawingSession.DrawImage(blur); }

Running this code you will see the following window:

clip_image011

I think this is enough for the first look at Win2D. I missed lots of classes that help to work outside XAML or allow working with images pixel by pixel but I hope that it’s easy to understand using documentation because Win2D is easy.

Written by Sergiy Baydachnyy

08/04/2015 at 1:40 AM

UWP: New Controls (part 5 – Media)

leave a comment »

Of course, the existing Media controls are not new but I cannot overlook them due to number of new features there. I have developed lots of applications which work with video and I know about disadvantages of Media controls in Windows 8.x like:

· Problem with adaptive streaming – in order to start working with adaptive streaming you needed to use external libraries like Player Framework and Smooth Streaming SDK because Windows 8.x SDK doesn’t support embedded features for adaptive streaming. Even if you use these libraries you will get support for Smooth Streaming only;

· No support of existing formats for closed captions – Windows 8.x SDK doesn’t support closed captions at all. SMPTE-TT and TTML were introduced in Player Framework SDK only;

· No way to change template for existing media player control – Windows 8.x supports SystemMediaTransportControls but there were not many ways to change something. So, you had to design your own player from scratch or use existing player from Player Framework with less ability to change anything there;

Let’s see how Microsoft fixed all these disadvantages in Universal Windows Platform.

Adaptive Streaming

HTTP Live Streaming (HLS), Dynamic Adaptive Streaming over HTTP (DASH) and Microsoft Smooth Streaming are the most popular technologies for adaptive streaming. New MediaElement control supports all of them.

If you want to test how it works it’s better to use Azure account (you can use trial) and create Media Service instance there. Media Service supports dynamic packaging, so you can upload your video there and encode it using one of profiles which are ready for adaptive streaming.

clip_image001

Pay special attention that you need to create at least one streaming unit. Once video is encoded, thanks to dynamic packaging, you can stream it using any format that I mentioned previously (HLS, Dash, Smooth).

Let’s test, how to work DASH. In order to do it you need just add MediaElement to XAML page:

<MediaElement Name="media" AreTransportControlsEnabled="True" />

In order to create adaptive media source you can use AdaptiveMediaSource class like this:

AdaptiveMediaSourceCreationResult result = await AdaptiveMediaSource.CreateFromUriAsync(new Uri("http://testadaptive.streaming.mediaservices.windows.net/e1f03724-c228-4f86-9570-7321f1767fc5/Module%202.1_H264_4500kbps_AAC_und_ch2_128kbps.ism/Manifest(format=mpd-time-csf)", UriKind.Absolute)); if (result.Status == AdaptiveMediaSourceCreationStatus.Success) { var astream = result.MediaSource; media.SetMediaStreamSource(astream); }

My video works fine and thanks to MediaTransportControls I have great interface for my media player:

clip_image003

Captions

If you want to add closed captions it’s easy to do with new controls as well. Just use the following code to associate TTM file with existing adaptive stream:

AdaptiveMediaSourceCreationResult result = await AdaptiveMediaSource.CreateFromUriAsync(new Uri("http://testadaptive.streaming.mediaservices.windows.net/e1f03724-c228-4f86-9570-7321f1767fc5/Module%202.1_H264_4500kbps_AAC_und_ch2_128kbps.ism/Manifest(format=mpd-time-csf)", UriKind.Absolute)); if (result.Status == AdaptiveMediaSourceCreationStatus.Success) { var astream = result.MediaSource; var ttmSource = TimedTextSource.CreateFromUri(new Uri("ms-appx:///assets/captions.ttm")); var mediaSource=MediaSource.CreateFromAdaptiveMediaSource(astream); mediaSource.ExternalTimedTextSources.Add(ttmSource); var mediaElement = new MediaPlaybackItem(mediaSource); media.SetPlaybackSource(mediaElement); }

You can see that we use TimedTextSource class to create source based on file with captions. In order to associate the captions with media source we used MediaSource class and MediaPlaybackItem class to prepare source for MediaElement.

Template for Media

Starting with Windows 10 the MediaTransportControls class has its own states and a template according to modern design. It allows to change anything there by modifying the existing XAML template. So you should not create buttons from scratch, implement logic etc. There are three ways to get existing template for MediaTransportControl: visit MSDN, find generic.xaml file on your computer or use Blend.

Because I have written this post before release of Windows 10 SDK there was no information on MSDN but I hope that you can find it right after release.

Generic.xaml file you can find using the following path C:\Program Files (x86)\Windows Kits\10\DesignTime\CommonConfiguration\Neutral\UAP\10.0.10069.0\Generic. Just open generic.xaml and find template for MediaTransportControls there.

Finally, you can extract template using Blend tool. I would like recommend this way because it’s easy not just to create copy of template but to modify it as well. In order to extract template for MediaTransportControls you need to create a new project (or open existing) in Blend and add MediaTransportControls to a page. Using context menu select Edit Template -> Edit a Copy menu item.

clip_image004

Blend will propose to select location and name for new style. So, select it and click OK.

clip_image006

Right after it you can open XAML and modify new template there or use powerful editor in Blend to see and modify everything there

clip_image007

Written by Sergiy Baydachnyy

07/31/2015 at 8:28 PM

UWP: Extensions and ApiInformation

leave a comment »

Universal Windows Platform allows to build universal applications for all devices which run Windows 10. I have already published several posts about new controls which allow to create universal interfaces for different screen sizes and resolutions. But what about device-specific features? For example: Raspberry Pi has set of pins which are not available for phone and tablets; you can use desktops and laptops in order to print something directly from your application but this functionality is not available for phones; lots of phones support vibration what is very uncommon for laptops etc. That’s why all contracts with device-specific functionality were moved to special libraries called extensions.

You can include existing extensions to your project using Add Reference dialog and open Extensions tabs. Today you can find three most important extensions there like Mobile Extension, Desktop Extension and IoT Extension.

clip_image002

Of course you might see more extensions soon. For example, Microsoft might publish some extensions for Xbox and Hololens but you will be able to create your own for own Windows 10 devices.

If you want to see all contracts which were included to the extension, you can open the manifest file for the selected extension and check all contracts there. For example, you can find the IoT manifest using the following folder C:\Program Files (x86)\Windows Kits\10\Extension SDKs\WindowsIoT\10.0.10069.0 and manifest looks like

<?xml version="1.0" encoding="utf-8"?> <FileList TargetPlatform="UAP" TargetPlatformMinVersion="10.0.0.1" TargetPlatformVersion="10.0.10069.0" SDKType="Platform" DisplayName="Windows IoT Extension SDK" AppliesTo="WindowsAppContainer" MinVSVersion="14.0" ProductFamilyName="Windows.IoT" SupportsMultipleVersion="Error" TargetFramework=".NETCore, version=v4.5.3;" SupportPrefer32Bit="True" MoreInfo="http://www.microsoft.com/en-us/server-cloud/internet-of-things.aspx"> <ContainedApiContracts> <ApiContract name="Windows.Devices.DevicesLowLevelContract" version="1.0.0.0"/> <ApiContract name="Windows.System.SystemManagementContract" version="1.0.0.0"/> </ContainedApiContracts> </FileList>

So, right now there are two contracts only and you can find all classes there using Object Browser in the Visual Studio.

I already published how to use IoT extensions for Raspberry Pi 2. So, in this article I want to talk more about how to implement universal approach even if you add an extension.

The main idea of Universal Windows Platform is to give a way to create just one application and one binary for all devices. But what happens when we add an extension? Should we recompile our application for different platforms with this extension and without the extension? The answer is No.

Extensions are designed in the way when you can continue to run your code on a platform which doesn’t support extensions which you have included to your project. But if you try to call methods and create objects based on classes from the extensions, these calls will generate runtime exception.

That’s why you need to check in runtime if selected extension is available on the platform. Of course, you should not do it in exception handler. It’s better to adopt the interface of your application for the specific device rather than notify users about some unsupported feature when the application is running. So, developers need to have better tool to check availability of contract and Microsoft implemented all needed features in ApiInformation class.

ApiInformation is located in Windows.Foundation.Metadata namespace and contains several static methods like IsApiContractPresent, IsEventPresent, IsMethodPresent etc.

So, if you want to check, if GPIO is available, you can use the following code:

if (ApiInformation.IsApiContractPresent("Windows.Devices.DevicesLowLevelContract", 1)) { //doing something with GPIO }

Pay special attention that IsApiContractPresent requires two parameters. The second one is major version (you can use minor version as well – third parameter). This parameter is needed because Microsoft can update some of contracts and you can target specific version(s).

The code above doesn’t mean that you need to create hundreds of if blocks if you include one or several contracts. It’s better to create a custom trigger based on this information and implement several visual states.

Written by Sergiy Baydachnyy

07/27/2015 at 6:50 PM

UWP: New Controls (part 4 – Inking)

leave a comment »

One more powerful control in Universal Windows Platform is InkCanvas. Thanks to this control you can enable inking anywhere in your application and you can use not just stylus but fingers and mouse as well. So, thanks to this feature you can bring inking functionality to any device based on Windows 10.

To enable inking you need to start with InkCanvas element and place it inside any container like StackPanel, Grid, etc. Like other UI controls based on FrameworkElement, InkCanvas contains a bunch of properties but in the simplest case you do not have to declare anything:

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"> <InkCanvas Name="ink"></InkCanvas> </Grid>

In this case InkCanvas will fill all space inside the container and you can start make your notes or paint something. Pay special attention that by default InkCanvas accepts input from stylus (pen) only. So, if you want to use fingers or mouse you need to implement the following code:

ink.InkPresenter.InputDeviceTypes = CoreInputDeviceTypes.Mouse|CoreInputDeviceTypes.Touch;

In this code we are using the second important object – InkPresenter. InkCanvas is just a control which contains just one inking property – reference to InkPresenter but InkPresenter contains all information about input methods and lots of different settings there including collection of strokes. You cannot create InkPresenter directly but you can get reference to the object of this class using InkPresenter property of InkCanvas. Frankly speaking InkCanvas doesn’t contain any other properties or methods related to inking. But we still have something to discuss about InkCanvas and the most important question there is how to enable inking anywhere because usually you need to enable inking for images, videos, text rather than using inking inside a blank container. Let’s add several controls to the container and see what happens there.

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"> <InkCanvas Name="ink"></InkCanvas> <StackPanel> <TextBlock Text="Hello. Here is some text." Margin="20"></TextBlock> <Button Content="Click me" Margin="20"></Button> </StackPanel> </Grid>

If you run this code you will see that all controls work fine and you can make any notes there.

clip_image002

In our example we placed InkCanvas behind other controls but if you swap StackPanel and InkCanvas you can see that InkCanvas is placed above StackPanel and controls like Button don’t work at all. You can resolve it thanks to Canvas.ZIndex property.

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}"> <StackPanel Canvas.ZIndex="1"> <TextBlock Text="Hello. Here is some text." Margin="20"></TextBlock> <Button Content="Click me" Margin="20"></Button> </StackPanel> <InkCanvas Name="ink" ></InkCanvas> </Grid>

Let’s talk about settings of InkPresenter and how to make different effects there. It’s all around InkDrawingAttributes class. You simply need to create an object of this class and set different properties like Color, PenTip, Size etc.

InkDrawingAttributes attr = new InkDrawingAttributes(); attr.Color = Colors.Red; attr.IgnorePressure = true; attr.PenTip = PenTipShape.Circle; attr.Size = new Size(4, 10); attr.PenTipTransform = Matrix3x2.CreateRotation((float)(70 * Math.PI / 180)); ink.InkPresenter.UpdateDefaultDrawingAttributes(attr);

Code below shows how to use InkDrawingAttributes. In order to update attributes you need to call UpdateDefaultDrawingAttributes and pass InkDrawingAttributes object there. In code below I used PenTipTransform property as well. This is a very interesting property which allows to apply transformation to pin shape and get more natural look for strokes thanks to different height of strokes based on direction of pen (moving direction/angle).

clip_image003

You can check the image below where I tried to paint symbol “f” and you can see that the image contains strokes with different height. In fact I used the same settings but this effect was applied thanks to PenTipTransform.

Finally, you can paint anything but how can you erase some strokes if you make anything wrong? In order to do it you need to change mode of InkPresenter:

ink.InkPresenter.InputProcessingConfiguration.Mode = InkInputProcessingMode.Erasing;

The next important class which is related to inking is InkStroke. Anything what you paint on InkCanvas consists of InkStroke. In order to get access to functionality around InkStroke objects you can use StrokeContainer property of InkPresenter. Thanks to this property you can save existing strokes to file, add new strokes, get access to selected strokes etc. So, if you want to implement any functionality around strokes then StrokeContainer is your friend. Of course the most popular methods there are LoadAsync and SaveAsync.

Lots of applications require recognition functionality as well. In order to recognize text based on hand writing you can use InkRecognizerContainer class. In the simplest way you can use this class like this:

InkRecognizerContainer container = new InkRecognizerContainer(); var result=await container.RecognizeAsync(ink.InkPresenter.StrokeContainer, InkRecognitionTarget.All); string s=result[0].GetTextCandidates()[0];

This code will use default recognizer and convert the first text candidate from the list to text. Of course it’s not enough for more complex scenarios. For example, I spend 10 years at school to train my handwriting for Russian and Ukrainian but not for English. So, when I am trying to write something in English usually I can find the right result on the second or third place in the list of text candidates. Additionally, I can have several recognizers on my computer.

So, it’s better to allow to select the right recognizer. You can simply use SetDefaultRecognizer and GetRecognizers of InkRecognizerContainer object to do it. In case of non-native English writers I am not sure that you need to present full list of possible results. It’s better to ask users to setup recognizer using system settings. For example, I can setup the default recognizer to adopt to my writing style if I select “write each character separately” and make some “teach” settings:

clip_image005

At the end of the post I want to draw your attention to Ink Toolbar control. It’s not a default UWP control and you need to install this control separately. Once you install the control it will be available in Extension tab of Add Reference dialog (close and open Visual Studio after installation) and it’s ready to use. You just need to add reference to the library:

xmlns:ink="using:Microsoft.Labs.InkToolbarControl"

Find a place for the control and make reference to the existing InkCanvas:

<InkCanvas Name="ink" ></InkCanvas> <ink:InkToolbar TargetInkCanvas="{x:Bind ink}" VerticalAlignment="Top" HorizontalAlignment="Right"></ink:InkToolbar>

Below you can see the control itself which generates our code:

clip_image006

Written by Sergiy Baydachnyy

07/27/2015 at 6:45 PM

UWP: New Controls (part 3 – Maps)

leave a comment »

In this post I am going to talk about maps in Universal Windows Platform applications. And it’s better to start with the answer to the question that developers asked me many times: when will offline maps be available for Windows applications?

Starting with Windows 10 offline maps are available not just for phones but for desktops, laptops, tablets etc. User can visit Settings window and download all needed maps there. Once maps are downloaded the existing map control will use them by default.

clip_image002

It’s really cool because I can use my Windows 10 devices anywhere in order find addresses, the right direction etc. and you can access? all these things directly from your application.

But offline maps are not the single feature of new Windows 10 maps. Let’s look at other features there:

· Unified platform – since Windows 10 you should not use several services like Here or Bing Maps for different tasks (or different services on different devices). All services are available under the same umbrella and can be used for any UWP applications;

· Adaptive interface – existing Maps application works fine on all devices. You can use touch, pen or mouse as well as use different screen sizes. The same we can say about MapControl which is available for developers;

· 2D view with business and traffic information – existing maps support all common features for 2D maps like different views, traffic information (you can on or off it), provide information about business locations and transit;

· Location – existing Maps application allows to find and display user location. Using Windows.Devices.Geolocation API you can find user location in your application in order to provide better user experience and implement lots of different tasks;

· External elements – developers can place own icons, rectangles, polygons and even XAML controls on maps. It allows to customize existing maps and extend the number of possible scenarios;

· Geofence – Geofencing API allows to notify application if user arrives to the defined area;

· Routing – developers can use Maps services that can calculate route to selected destination using different methods there (driving, walking);

· StreetSide – new functionality which allows to get images of the selected area to display it inside the interface. It’s very useful if user wants to understand what the selected area looks like;

· 3D views – one more new functionality which is available for developers starting with Windows 10: right now you can show maps not just in 2D but in 3D view modes. If 3D view is available for the selected area you can implement the same set of features like in standard Maps application;

It’s time to talk how to use maps in your applications. You have two ways there: you can redirect some user requests to the existing Maps application or you can integrate maps directly to your application.

If you are going to utilize existing Maps application you can redirect user to it using Launcher class. This class contains LaunchUriAsync static method which allows to open an external application using information from Uri. For example, if you use “http://….” in your uri then Launcher will open web browser but in case of Maps you need to start your uri with bingmaps:. Of course, it’s not enough and you need to pass some parameters using your uri. The full list of possible parameters you can find using the following link. For example, this code will open Maps application in move center to North Vancouver and zoom the map to level 14:

await Launcher.LaunchUriAsync(new Uri("bingmaps:?lvl=14&rpt=adr.North%20Vancouver%20BC"));

In order to start working with maps in your application you need to get “access key” visiting Maps Dev Center. You can select Trial key or Basic key based on your needs. Right after Windows 10 is released you will be able to find Universal Application in the list of possible applications but right now there is still no information about Windows 10. In any case you cannot publish Windows 10 applications right now (several days left), so you can use maps without key. It works fine but you will see the message that MapServiceToken is not specified and you cannot publish your application without MapServiceToken.

Once Windows 10 is available you can create Basic key for your universal application using the portal:

clip_image003

In order to start working with maps in your application you need to use three namespaces – Windows.UI.Xaml.Controls.Maps, Windows.Devices.Geolocation and Windows.Services.Maps and one control – MapControl. The first namespace contains MapControl itself and several classes which allows to place something on maps, setup camera and styles, prepare street view. The second namespace allows to get user location and contains Geofencing API as well. And the last one supports several utility classes which allows to find location by address, calculate route etc.

Because MapControl is not in default XAML namespaces you need to add a new namespace to XAML file:

xmlns:maps="using:Windows.UI.Xaml.Controls.Maps"

And you are ready to use MapControl:

<maps:MapControl MapServiceToken="..."/>

Right now we know how to add maps to your application and it’s time to see some tasks there.

Let’s start with basic functionality like how to center map, zoom map etc. Let’s have a look at the following code:

maps.ZoomLevel = 14; maps.Center = (await MapLocationFinder.FindLocationsAsync("North Vancouver, BC", null)).Locations[0].Point;

In this code we use reference to MapControl in order to assign two properties ZoomLevel and Center. Of course, in case of ZoomLevel you should not do anything special but if you want to center your map based on address you need to use map services – MapLocationFinder in this case. In my example I used FindLocationsAsync method and assigned the first location from the list to the Center property.

In order to apply 3D view to the maps you simply need to use Style attribute. It’s somewhat confusing because Style attributes exists for all UI controls and that can have different meaning.

<maps:MapControl Name="maps" Style="Aerial3D"/>

Of course, in case of the 3D View, it’s better to change default angle for map camera. You can do it usingthe static method of MapScene class called CreateFromLocationAndRadius. Thanks to this method you can create a scene using radius (meters), zoom level and radius (degrees). Once you have MapScene object you need to use TrySetSceneAsync method to apply the scene:

await maps.TrySetSceneAsync(MapScene.CreateFromLocationAndRadius((await MapLocationFinder.FindLocationsAsync("Vancouver Downtown, BC", null)).Locations[0].Point,500,90,60));

If you run my code you will see center of Vancouver in 3D:

clip_image005

So, applying 3D view is easy. The same works for the StreetView functionality. If you want to use the street view you need just two lines of code:

StreetsidePanorama panorama=await StreetsidePanorama.FindNearbyAsync((await MapLocationFinder.FindLocationsAsync("Vancouver Downtown, BC", null)).Locations[0].Point); maps.CustomExperience = new StreetsideExperience(panorama);

If you run this code you can see something like this:

clip_image007

Of course, existing maps show lots of different information but if you integrate maps to your application, probably, you want to add your own objects to maps.

The simplest objects which you can add to the maps are MapIcon, MapPolygon and MapPolyline. For example, if you want to add an icon to the maps you can run the following code:

MapIcon mapIcon = new MapIcon(); mapIcon.Location = maps.Center; mapIcon.NormalizedAnchorPoint = new Point(0.5, 1.0); mapIcon.Image = RandomAccessStreamReference.CreateFromUri(new Uri("ms-appx:///Assets/weather.png")); mapIcon.ZIndex = 0; maps.MapElements.Add(mapIcon);

You can use this code to place weather icons in the area, points of interest etc.

Additionally you can add XAML controls to the maps. In this case you need to use Children collection instead of MapElements as before. But it’s not enough to add XAML UI control, you need to place the control to the right position on the maps. In order to do it you can use the SetLocation method of the MapControl class. This method has two parameters: reference to added UI control and location.

Therefore you can see that the current version of MapControl is very powerful and supports lots of features. It’s even possible to write a separate book just about this control. If you want to know more, the best starting point for you is Bing portal for developers. The portal contains lots of examples and information how to use Bing maps from JavaScript or in applications for alternative platforms.

Written by Sergiy Baydachnyy

07/20/2015 at 7:21 PM