The Azure DataMarket is a stellar resource for application data. At the current moment, it features over 100 datasets (from baseball to census to mortgage) in OData format, much of which is free, and the recently announced Self-Service Publishing Wizard should help to manifest even more hidden data into a common format. Mobile devices, being disconnected from pretty much everything, are ideal consumers of cloud-based data. In this post, we'll look at one way of querying data from the Azure DataMarket in a Windows Phone 7 application. Before we get rolling, a quick note about the free datasets. They are reg-ware, which means a Windows Live ID is required, and you'll need to generate an application key to use when making requests. I show you how to find your key below. Finding and Registering For A Dataset Time to get started--let's find a free dataset we can play with. Go to http://datamarket.azure.com/, click the Data button on the top menu, then on the left menu, click Free under the Price category. We're now browsing the list of all free datasets--56 of them at this time. If we use the search feature, the Data and Free filters will be honored. All the good BI demos involve despair of some kind, so let's see if there is crime data. Searching for crime brings us back two datasets--London Borough Profiles, and 2006-2008 Crime in the United States. Bingo on the second one! Clicking on the set's title takes us to a page with more information about the dataset. On the right is a large green "Sign Up" button, with subscription details right above it. This dataset is free for unlimited transactions per month. Fun! When you click the Sign Up button, you'll be asked to either log in with an existing Windows Live ID, or create a new one. Do what you need to here--remember that if you're signing up for a service that may incur charges, this Live ID will be used for the billing information. After you've logged in (and created a Live ID, if necessary), you'll be presented with another registration screen. This one creates your Azure DataMarket account, and and only needs to be completed the first time. You won't see this when you sign up for additional datasets. Next up is the usual gigantic page of legalese. These are the DataMarket Terms and Conditions, which you'll have to agree to if you want to consume data. For our sample apps, we're unlikely to run afoul of these, but if you'e an ISV, you might want to read these over. This is also a first-time-only requirement, until they change. Almost there! Now that we've signed up for DataMarket, we need to agree to the T&C of our dataset. Again, for our sample app using a free dataset, we're not likely to run afoul of these, but pay attention if there is money involved, or if the dataset is possibly copyrighted (e.g., non-government, like baseball stats or Zillow). You will see one of these forms for every dataset you sign up for. Finally, we're done! We can now start consuming our dataset. The Thank You page has links to some useful resources for us, notably the DataSet explorer, and using the dataset in Visual Studio. You can access a similar page from Account >> My Data, too. The dataset explorer is a really cool tool--here we can interactively build queries, chart and export our data. For some purposes, this may be all the tool you need. When building a Windows Phone Application, it's a great tool to debug your queries. Note that any query you make in the explorer counts against your allotment, if there is one. Groovy. At this point, it's time for a little architectural discussion regarding how we consume Azure DataMarket data. System.Data.Services.Client and DataMarket Queries DataMarket data is provided via the Open Data Protocol (OData). It's an incredibly rich system for querying online datasets, and data can be returned in both Atom and JSON. The workings of OData are beyond the scope of what we'll be doing here, but check out the Developer section on their site for more information. Data can be queried from the DataMarket using REST, but Windows Phone 7 provides us a library which wraps DataMarket calls into a more familiar method using Linq or traditional service operation. This library is System.Data.Services.Client.aspx), and is pretty impressive. Return types are DataServiceCollection
Now for a little code behind. This is also available in the sample project. At the top of the code behind, add the following import/using statements:
For the sake of simplicity, we're not going to implement MVVM here; we'll simply execute our code when the page loads, and container the data on this page. It's easy to change this later if you want.
Inside the class, but above the constructor, add the following declarations:
Private _context As datagovCrimesContainer
Private ReadOnly _crimesUri = New Uri("https://api.datamarket.azure.com/data.gov/Crimes/") Private _crimes As DataServiceCollection(Of CityCrime) Private _userId As String = "fake email address" Private _key As String = "real account key here"
Where it says "fake email address", put any fake email address. At the current time, the DataMarket authentication ignores this field. However, to make a proper credential, you need to enter any gibberish text which looks like an email address. Where it says "real account key here", put your correct account key. This is required by DataMarket authentication. Inside the page constructor, add the following code. You can see where to customize your query, you can even add parameters based on GPS or text inputs if you want. For asynchronous requests (which is the only wayWP7 communicates with services), we need to tell the application what to do once the data is returned, so we use AddHandler to point to another method which does something with the data. The LoadAsync method then executes our query.
_context = New datagovCrimesContainer(_crimesUri) _crimes = New DataServiceCollection(Of CityCrime)(_context) Dim _query = From c In _context.CityCrime Where c.State = "Pennsylvania" And c.City = "Pittsburgh" Select c Order By c.Year _context.Credentials = New NetworkCredential(_userId, _key) AddHandler _crimes.LoadCompleted, AddressOf crimes_loadcompleted _crimes.LoadAsync(_query)
Finally, add the handler method to our class. All we're doing here is binding our returned data to the listbox. If we had any additional post-query processing, this is where we'd do it.
Private Sub crimes_loadcompleted() Me.LayoutRoot.DataContext = _crimes End Sub
At this point, you should be able to build and run your application! From here, practice changing your Linq query to filter or shape the data some more. You can grab the sample code below if you need to, and be sure to read the notes below. Download VB.NET Sample Code: Flexible Query VB Download C# Sample Code: Flexible Query CS A Note About MVVM
By executing the query and containering the data on this page, all the controls on this page will have access to the date, but other pages we add will not. We use MVVM to serve as an application-wide bucket of data (if you'd like a little more information about MVVM, check out Greg Lutz's video at http://vimeo.com/20347067). You can also read Implementing the Model-View-ViewModel Pattern in a Windows Phone Application.aspx) on MSDN. Using Your Key, Or Not? In this sample, we used our own API key in our application. Were we to distribute this application, everyone who uses it would count against our allotment of API calls. For the service we chose, not a big deal, but this will end up costing you a lot of money on a pay service. You can require your users to create their own DataMarket account, with their own billing information, and then enter their credentials into your application. OAuth is probably a better choice than our API key in this sample (see below). I also suggest adding a new key to your account, and using that in your application. If you have more than one application, use more than one key. This helps you keep your keys a little more private, and track usage of particular apps against your account should you need to. Additional References We've used HTTP Basic Authentication in the sample; OAuth 2.0 is also available. A discussion on which one to choose and how to implement can be found in the Marketplace Authentication on MSDN. How to: Execute Asynchronous Data Service Queries (WCF Data Services) http://msdn.microsoft.com/en-us/library/dd756367.aspx Open Data Protocol (OData) Client for Windows Phone http://msdn.microsoft.com/en-us/library/gg521146(v=VS.92).aspx.aspx) WCF Data Services Learning Center http://msdn.microsoft.com/en-us/data/bb931106 Consuming a Windows Azure Data Service by using the OData Client http://create.msdn.com/en-US/education/quickstarts/odataservice Studio for Windows Phone 7 Want to add some awesome charts, gauges and other controls to your application? Check out ComponentOne Studio for Windows Phone 7.