Recap: "An introduction to Windows Azure"

In December I did a presentation titled "An introduction to Windows Azure" for some Microsoft Student Partners in Brussels. A lot of MSPs could not attend and my not-so-well-rehearsed presentation was a bit chaotic, so I have decided to publish most of it in the form of a blog post. It has been a while, I have been quite busy (exams, thesis, work...), so sorry for the delay in posting this.

The Azure poster displayed above is a nice overview of what Azure has to offer (you can find a full size version over here). I have marked what I will discuss in this blog post with green rectangles: cloud services (a web role and a worker role, with a queue between them) and blob storage. This blog post is by no means meant to be a comprehensive overview of what these services have to offer, but rather a fun example of a common use case. By the way, if anyone knows where to get a printed version of that poster, I want one!

So, what could we make with Azure Cloud Services? Well, a cloud service can contain web roles and worker roles. What a web role can do is rather obvious: it serves a website. A worker role is basically an application running on a virtual server in the cloud. It can communicate with external services and/or end users (you can define endpoints: ports that will be available to your application), but it does not have to. A worker role could be a WebSocket server, an image processing server (for example to create thumbnails), a video trans-coding service... A web role is basically a worker role with IIS pre-installed and an endpoint for TCP port 80. If for some reason you prefer to use Apache instead of IIS, you could do so using a worker role.

Azure Cloud Services are pretty open: you can use the technology of your choice to write applications. On the Azure website (pictured above) you can find pretty good documentation for .net, node.js, java, php and python. All APIs are RESTful, so using another language or technology should not be a problem either. In this post I will only discuss .net technologies. Of course, the integration of Azure with .net and Visual Studio is pretty good.

Okay, so let's build a fun application. In this example we will build a web app that generates images like this one. A picture is shown on a black background in white frame with two sentences underneath it (let us call them "title" and "description" for clarity).

Let's call our application "Memeify".

This is our cloud architecture. We will need a website where users can upload an image and enter the text they want to add to it. We use a web role to serve this website. After a user has uploaded an image we need to make the composition with the black background and the text and then return the finished image to the user. Generating this composition and saving it to an image file may take a couple of seconds, so we probably should not be doing this on our web server. We will create a worker role that does the image generation out of band.

It is perfectly possible to have direct communication between a web role and a worker role, however this kind of tight coupling is probably not the best way to go in our use case. It is easy to imagine that when our website gets popular, we may want to have multiple instances of the worker role processing multiple images at the same time. It would be great if we could have some kind of scalable queue somewhere in the cloud, wouldn't it? The web role could then put tasks on the queue and the worker role could monitor the queue and complete the tasks. This kind of loose coupling is very common in cloud applications.

Azure offers two queuing mechanisms: Storage Queues and Service Bus Queues. For a comprehensive comparison between the two I refer to this msdn page. Besides the technical comparison it is also a good idea to take a look at the pricing differences between the two systems, depending on the intended/predicted load (use the online pricing calculator).

We will use Service Bus Queues, but it would be trivial to do the same with Storage Queues.

Let's get started! We will use Visual Studio 2012, with all the right tools installed. I would advise you to use the Web Platform Installer to find and install the latest Azure tools and SDKs. We will also be using ASP.NET MVC 4 to build the website, you will find that in WPI as well.

I always like to start with a Blank solution and then add projects to that empty solution. (File » New » Project...)

The first project we add to this solution is a an MVC web application. (Right click on solution name in the Solution Explorer » Add » New Project...). We could call this project "Memeify.Web".

As this is about Azure, I won't go into to much details about ASP.NET MVC 4 (watch this or start here). I'll start with a basic ASP.NET MVC 4 project and will not bother with unit tests (but you should!). I will use the Razor view engine, because it is awesome.

At this point we can add an Azure project to the solution. This is nothing more than some configuration files that define a cloud service. We could call this project "Memeify.Azure".

We then add the existing web project as a web role to the cloud service.

By right clicking the role we can get to its settings. These settings are stored in XML files (with a .cscfg extension) in the project which, alternatively, we can edit manually as well.

Pro tip: while experimenting and testing or even deploying to a limited number of users, set the VM size to "Extra small". Extra small instances are charged at 1/6th of the rate of a small instance. See this page for more information.

Let's now do some work on the website.

We will need a model to represent a meme. Models/MemeModel.cs Let's keep it simple: a POCO that has properties for the image, the title and the description. The image is a file and title and description are strings.

We also create a view. Views/Home/Index.cshtml This is an extremely simple HTML form that has three fields (a file upload and two textboxes) and a submit button. We use the HTML helper provided by the web framework to link this form to our model at the server side. One thing that is easy to forget is to set the encoding to "multipart/form-data", in order to support file upload.

Next, we will need a controller. Controllers/HomeController.cs On a GET, we simply want to show the form, so we return the view. When the form is POSTed, we need to create the image and somehow return it to the user.

At this point we could just write some code to create the image right in this action of the controller (the line in the above marked with a TODO comment, line 11). However, we do not want our web server(s) to handle this work. Instead, in the controller we need to do two things: (1) store the image in blob storage so that the worker role can access it and (2) put a task on a queue to let the worker role know that there is an image waiting to be processed (or "memeified" in our example).

The Azure documentation provides very clear step-by-step instructions on how to use all these services. Here is the how-to on using blob storage in .net. Because this documentation is very good, I will skip ahead to the code here. In summary, what you have to do is create a blob storage account using the online portal and add the "connection string" to the settings of the role.

We should first validate that the form input is correct (e.g. all required fields are there, the text is not to long, the image is of a valid type and not to big...). Again, because this is a post about Azure and not about ASP, I will skip that part and assume the input is valid. If you need some help adding validation: this is a good read.

In a blob storage account you can have several containers (directories) and these containers contain the blobs (files). There is no real hierarchy like the file organisation on your computer, but blob names can contain the slash sign, so you can emulate directories within containers. We connect to a blob storage account and get a container named "Images".

We can create this container first using the online portal or using a third party tool (e.g. CloudXplorer) or create it in code as demonstrated below. We should also set the permissions: blob containers can either be private or public. A private container would be fine here (as only our own worker role needs access), but the code below demonstrates how to make the container public. In our example, this would for example be useful to show the uploaded picture in the web interface.

Next, we will need to give the image a unique name and upload it to the container. Let's just take a random GUID and add a timestamp. The chance of a name like that not being unique is remote, but you should probably still check! Checking whether a file already exists in blob storage is not that simple in the .net implementation, but Steve Marx has written a nice extension method.

And then we can upload the blob.

Once the image is in blob storage, the only thing that is left to do in the web role is to add the task to a queue. Queues work in a very similar way: we first create a storage account (storage queues) or a namespace (service bus queues) in the online portal, we then add the connection string to the settings of the role and use the .net implementation of the REST API to access the queue. In the following we will use service bus queues.

On to the worker role! Let's add the project and call it "Memeify.ImageServer".

This project contains a template called "WorkerRole.cs", this file has a method "OnStart", this is were we will put our code.

We start a new thread that runs an infinite loop that pulls messages (tasks) from the queue and calls a method "CreateMeme" when a new task has been pulled from the queue. In the case of service bus queues the "Receive" call is blocking: it does not return immediately if the queue is empty, so there is no need to add some sleep at the end of an iteration. This is an important difference between service bus queues and storage queues!

The CreateMeme method pulls the image from blob storage, adds the background, frame and text using GDI and saves the result back to blob storage. The bulk of it has little to do with Azure, so here is the code without further ado.

One thing that I am not addressing here is how to get the result back to the user after the worker role is done with it. We could for example use websockets or a library like SignalR to notify the web page. We could also let the worker role send the user an email with a link to the generated image. It all depends on the use case. For the purpose of this demo, lets just say that the worker role saves the result to blob storage and prepends "result-" to the name (see the code above). We can then show the user a page with an image tag that points to this URL in blob storage (http://{storageaccountname}{containername}/{result-blobname}) and tell them to wait a bit and refresh the page. Not very nice, but good enough for now. One can easily see that designing a good, loosely coupled, cloud architecture requires some thought.

I am making the source available on GitHub, so you can easily browse it and fork it to share your own changes. Please note that I have removed connection strings to my accounts, so you will have to dive into the settings to enter your own information to make it work. Also, in the view "Home/Show.cshtml" you will have to change the URL to point to your own storage account. No go play with it and make some great stuff!


Popular posts from this blog

Mobile Vikings App for Windows Mobile v2.0

Mobile Vikings App for Windows Mobile

The best hosting offer you've ever seen