Continuous Delivery with Visual Studio Team Services

In a series of posts by the Azure Development Community, some of the features available in Azure and Visual Studio Team Services are explored. VSTS Build is an excellent mechanism for enabling automation for a development team. It allow for teams to construct build pipelines to handle a variety of scenarios and technology. The service can run hosted or on-premises, and supports a large number of build steps ranging from Node.js and Gulp to MSBuild and Azure Deployment. Its flexibility extends to the ability to bring in tasks from the Marketplace and write custom build steps.

Visual Studio Team Services – Creating a build pipeline (Part 1)

Posted in Azure, Visual Studio Team Services | Tagged | Leave a comment

Azure Service Bus Queue – Sessions

In scenarios where there are multiple Azure Service Bus clients and you want to route particular messages to particular consumers, sessions are an ideal feature of the Azure Service Bus Queue. This post builds upon the sample project available on MSDN: Getting Started: Messaging with Queues.

Setup

Install either the MSDN sample project or create a new project using the SDK Template shown below:
GettingStartedTemplate
This gives us a great starting point to look at sessions.

Scenario

To illustrate sessions, three messages will be posted to the service bus. Two will have the same session id and the third will have a different session id. We will then use two clients to consume the messages. There are many scenarios where this technique could be useful. Here are three examples:

  • Priority – A low priority queue and a high priority queue could be created where the high priority items are worked before the low priority queue items.
  • Partitioning – The distribution of the queue items across multiple message brokers.
  • Aggregation – Routing messages to specific message brokers

Code

The first step is to set the queue to require sessions when being created:

private static void CreateQueue()
{
	NamespaceManager namespaceManager = NamespaceManager.Create();
 
	Console.WriteLine("\nCreating Queue '{0}'...", QueueName);
 
	// Delete if exists
	if (namespaceManager.QueueExists(QueueName))
	{
		namespaceManager.DeleteQueue(QueueName);
	}
 
	var description = new QueueDescription(QueueName)
	{
		RequiresSession = true
	};
 
	namespaceManager.CreateQueue(description);
}

When messages are sent to the service bus, a specific session is required to be specified.

private static BrokeredMessage CreateSampleMessage(string messageId, string messageBody, string sessionId)
{
	BrokeredMessage message = new BrokeredMessage(messageBody);
	message.MessageId = messageId;
	message.SessionId = sessionId;
	return message;
}

This post illustrates two ways to consume the messages. The first creates a queue client per client:

private static void ReceiveMessages()
{
	Console.WriteLine("\nReceiving message from Queue...");
	BrokeredMessage message = null;
 
	var sessions = queueClient.GetMessageSessions();
 
	foreach (var browser in sessions)
	{
		Console.WriteLine(string.Format("Session discovered: Id = {0}", browser.SessionId));
		var session = queueClient.AcceptMessageSession(browser.SessionId);                
 
		while (true)
		{
			try
			{
				message = session.Receive(TimeSpan.FromSeconds(5));
 
				if (message != null)
				{
					Console.WriteLine(string.Format("Message received: Id = {0}, Body = {1}", message.MessageId, message.GetBody<string>()));
 
					// Further custom message processing could go here...
					message.Complete();
				}
				else
				{
					//no more messages in the queue
					break;
				}
			}
			catch (MessagingException e)
			{
				if (!e.IsTransient)
				{
					Console.WriteLine(e.Message);
					throw;
				}
				else
				{
					HandleTransientErrors(e);
				}
			}
		}
	}
	queueClient.Close();
}

The second uses an implementation of IMessageSessionHandler:

public class MyMessageSessionHandler : IMessageSessionHandler
{
	private string WhoAmI = Guid.NewGuid().ToString().Substring(0, 4);
 
	public void OnCloseSession(MessageSession session)
	{
		Console.WriteLine(string.Format("MyMessageSessionHandler {1} close session Id = {0}", session.SessionId, WhoAmI));
	}
 
	public void OnMessage(MessageSession session, BrokeredMessage message)
	{
		Console.WriteLine(string.Format("MyMessageSessionHandler {3} received messaged on session Id = {0}, Id = {1}, Body = {2}", session.SessionId, message.MessageId, message.GetBody<string>(), WhoAmI));
 
		message.Complete();
	}
 
	public void OnSessionLost(Exception exception)
	{
		Console.WriteLine(string.Format("MyMessageSessionHandler {1} OnSessionLost {0}", exception.Message, WhoAmI));
	}
}

To register the message handler register using the queueclient:

queueClient.RegisterSessionHandler(typeof(MyMessageSessionHandler), new SessionHandlerOptions { AutoComplete = false });

The completed project has been uploaded onto MSDN here.

Posted in Azure, C#, Service Bus | Tagged , , , | Leave a comment

Publish-AzureServiceProject: Failed to generate package

Recently when creating a new service project in Azure I ran into this disturbing error when publishing from powershell:

Publish-AzureServiceProject : Failed to generate package. Error: Microsoft(R) Azure(TM) Packaging Tool version 2.5.0.0 for Microsoft(R) .NET Framework 4.0
Copyright ¸ Microsoft Corporation. All rights reserved.
F:\putti\AzurePHPPoc\ServiceDefinition.csdef: Warning  CloudServices040 : The ‘schemaVersion’ attribute is unspecified. Please set the attribute to avoid this warning.
cspack.exe: Warning   : CloudServices68 : No TargetFrameworkVersion specified for role AzurePHPPocWeb. Using .NET framework v4.0 for packaging.
cspack.exe: Error   : Could not load file or assembly ‘Microsoft.WindowsAzure.Packaging, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′ or one of its dependencies. The system cannot find the file specified.

It turns out this is an issue with the windows authoring tools

image

The solution is to remove the installations (if more than one version installed) and then reinstall the Azure SDK for .NET:

image

Note: In my case I had to install both the 2.5.1 and the 2.6 versions of the SDK.

Posted in Uncategorized | Leave a comment

Azure Deployment Failure

If you receive a deployment failure specifying the certificate was not found, here are some steps to hopefully resolve this.

image

I stumbled upon this issue for several reasons but all relate to an unfamiliarity with the tools.  My situation has multiple cloud services in multiple environments.  The issue above was caused when the remote access certificate was not located on the server.  This is specified in the service configuration properties:

image

The certificate selected must exist on the cloud service.  You can verify this by looking in the management portal:

image

If it is not missing then the certificate can be uploaded to the server.

Also, it is probably best to fix this via the publishing tools as updating this value directly in properties causes the publishing wizard to get upset: .

You can set the certificate here also:

image

By selecting the more options:

image

Posted in Azure, Deployment | Leave a comment

Managing Multiple Azure Environments with Visual Studio Online Builds

Carrying on from the Multiple Azure Cloud Environments post, I now want to manage the progression of my solution through the development, staging, and production environment. 

Staging in the context indicates an environment and not the staging deployment of a cloud service.  In hindsight, I probably should have named it Test to avoid confusion but the word Test could have be perceived as temporary.  UAT is another popular term for the environment before production.  Please post a comment for the terms that your organisation(s) uses.

My preferred approach is to use Visual Studio Online to manage the build process due to its tight integration with Visual Studio and Azure.  The image below illustrates this process:

image  

The development team develops and tests locally before checking in their changes. When a check-in occurs, the Visual Studio Online server then gets the latest changes, builds and then deploys to the Development cloud service.  Then once the changes have been reviewed, another build and deployment is then made to the Staging and production environments.

This post will illustrate this approach by using the SpikeCloud solution created in the earlier post.

Add to Visual Studio Online

The first step is to add the solution to Visual Studio Online:

image

You will then be asked to authorise the connection:

image

This step links your cloud service to Visual Studio Online

Add solution to source control

Now that the link is complete, the solution needs to be added to Visual Studio Online.  From within your solution, this can be done by adding the solution to source control:

image

You will be prompted for the type of source control system:

image

Once the solution has been added, Visual Studio will indicate this:

image

If we were only supporting a single environment, we could check-in and the solution would then be deployed to Azure using continuous integration.  Because of the multiple environments, we have a bit more configuration to do.

Go ahead and check-in the changes now as the build is disabled by default.

Supporting Multiple Environments

In the Team Explorer, select Builds.  If you do not see the Team Explorer, you can open it under the View menu.

image

Edit the Build definition.

image

The first page of properties allows us name and set the queue processing status.  Go ahead and enable the build:

image

For the development environment, leave the Trigger set to continuous as we want to run a build each time someone checks in:

image

For the purpose of this post, we will leave the Source Settings, Build Defaults and Retention Policy as the defaults.  In the Process tab we need to set some information about our solution.  First of all the solution to build needs to be specified by clicking the elipse in the Projects field:

image

The solution file needs to be added by first clicking the add button:

image

And then navigating to the solution file:

image

In order to specify the azure configuration we require, we need to indicate this as an msbuild parameter.  This is pretty straightforward as the target profile matches the service configuration we created in the previous post:

image

The following shows our completed build settings with the fields we were required to set indicated:

image

In order to test the changes, check-in a change or manually start a build by selecting Queue New Build:

image

Conclusion

By combining source control and build automation, linking your Azure cloud services to Visual Studio Online provides a number of benefits.  Besides providing a robust and feature rich collaboration tool, managed builds provides a team with a simple and efficient mechanism for deploying to Azure.  In my experience, I have seen my deployment time reduced up to 5 times by deploying from VSO instead of local workstations.

Posted in Azure, C#, Deployment | Leave a comment

Redeploy Previous Visual Studio Online Build

One feature to highlight with controlling your Azure deployments from a build server is the ability to track the current deployment and the ability to perform a redeploy of a build.

this feature only works well when the entire deployment process is controlled from Visual Studio Online.  If another deployment method is used, the Deployments in Azure will not recognise the change and not reflect the deployment accurately.

In the management portal, the deployments can be viewed in the Deployments tab.  The following illustrates a collection of deployments to the PRODUCTION environment.

image

Besides having convenient access to view the log of the deployment, it is also a powerful mechanism to perform a re-deploy of an existing package.  For example, if the deployment from October 29th was displayed then the option to redeploy can be used.

image

Posted in Azure, Deployment | Leave a comment

Multiple Azure Cloud Environments

Most real world Azure deployments will require more than one environment.  A typical topology is represented below:

image

There are many resources available that describe this concept so it will not be re-explained here.  Instead this post will present how to set this up in Azure. 

My scenario includes two cloud services: a web role and a worker role.  The creation of the two roles is illustrated below:

image

As the content of the roles is not significant to the post, I just created a basic MVC web app:

image

My first step in setting up publishing to multiple environments is to create the basic publishing profile.  First, select publish on the created cloud project and select the option to create a cloud service.  This is shown below:

image

After the basic settings have been selected the publishing settings are shown below:

image

And once the cloud service has been successfully published:

image

I can then browse my deployed service:

image

And in the server explorer, I can see the newly published service running.

image

And just to illustrate the worker role is running, I will update the diagnostic settings.

image

And set the Event log’s log level to information:

image

After a couple of minutes I can see an error was written.  Interesting error but should not affect the demo!

image

Multiple Environment Configurations

Now that we have a basic service established it is time to set up our different environments.  In my scenario I want to create a Development and a Production environment.

image

The first step is to create two configurations: one for the development environment and one for the production environment.  On the cloud service project, the Manage Configuration option is selected.

image

The first step is to rename the Cloud configuration to Development using the Rename button:

image

And changing the name to Development.

image

Next, the Development configuration is copied and the copy renamed to Production.

image

Below shows the Development and Production configuration files:

image

To illustrate the different configuration settings, let’s update the production configuration to be different than the development configuration.  To customise the configuration settings for a particular cloud service configuration, double-click the role in the Roles folder. 

In our example, the WebRole was selected.

There are several sections that allow you to customise the settings of the cloud role.  In this example the instance count will be updated to 2 VMs in production.  First the Production configuration is selected.

image

And then the instance count is set to 2.

image

Multiple Publishing Profiles

Because we now have both development and production environments to publish to we will need multiple publishing profiles.  To create a development and a production environment, first select the Publish option of the cloud project context menu:

image

In the Target profile drop down, select the manage option:

image

We will create a copy and rename the profiles by adding a suffix to identify the different environments:

image

Now there are two profiles available in the Target profile drop down:

image

Using the previous button we will go back to settings to update our production setting to use the Production service configuration:

image

Because we want separate development cloud roles, we will need to create a new service for the development role:

image

When making changes to multiple publishing profiles, do not forget to click save:

image

Below shows the final settings for production:

image

Below shows the final settings for development:

image

After deploying both profiles the effect of setting the instance count to 2 for the production web role is evident:

image

Conclusion

The above scenario is simple and does not include the complexity of other components.  In particular storage and database resources.  When dealing with the multiple environments managing the connection strings may become tricky as a combination of service configuration and application or web configuration (app.config/web.config) may be required. 

Posted in Azure, Deployment | Leave a comment

EF Database Migrations–No Connection String

Sometimes things don’t work the way you would expect them to or how they worked in the past.  Entity Framework is one of those things that frustrates me when it goes wrong…

I had created a simple database migration in a class library and could not figure out why I kept getting a

System.InvalidOperationException: No connection string named ‘OperationsDatabase’ could be found in the application config file.

I had the connection string defined in the app.config and built the library but no luck.  And I selected the default project in the package manager console.  Simple enough in the end after stumbling upon this post.

update-database –startupprojectname “myclasslibrary”

Posted in C#, Entity Framework | Leave a comment

DataTables.net–sorting on date column

DataTables.net plug-in for jQuery is a powerful javascript library for managing HTML tables.

The following is a note on how to solve a common issue: sorting on the value of a column when the displayed value is different.  An example of this is when a date is displayed as a friendly string like Month Year:

image

The default sorting behaviour would end up with less than desirable results:

image

There are more than one way to solve this, but a simple solution is to include an additional hidden column with the date value.  Then in the column defs the column to be sorted, in this case the first column or 0 index, can reference the hidden column when sorting:

“columnDefs”: [{                       
                       “targets”: [0],
                       “dataSort”: 4 },
                           {                       
                       “targets”: [4],
                       “bVisible”: false
                       }],

The end result is correct sorting on the hidden column:

image

Posted in DataTables.net, HTML, JavaScript, JQuery | Leave a comment

npm install fails with CERT_UNTRUSTED

I have not investigated why but during the install of the azure node.js package, I received a CERT_UNTRUSTED error as indicated below:

PS C:\> npm install azure
npm http GET https://registry.npmjs.org/azure
npm http GET https://registry.npmjs.org/azure
npm http GET https://registry.npmjs.org/azure
npm ERR! Error: SSL Error: CERT_UNTRUSTED
npm ERR! at ClientRequest.<anonymous> (C:\Program Files (x86)\nodejs\node_modules\npm\node_modules\request\main.js:4
40:26)
npm ERR! at ClientRequest.g (events.js:156:14)
npm ERR! at ClientRequest.emit (events.js:67:17)
npm ERR! at HTTPParser.parserOnIncomingClient [as onIncoming] (http.js:1256:7)
npm ERR! at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:91:29)
npm ERR! at CleartextStream.socketOnData [as ondata] (http.js:1288:20)
npm ERR! at CleartextStream._push (tls.js:375:27)
npm ERR! at SecurePair.cycle (tls.js:734:20)
npm ERR! at EncryptedStream.write (tls.js:130:13)
npm ERR! at Socket.ondata (stream.js:38:26)
npm ERR! [Error: SSL Error: CERT_UNTRUSTED]
npm ERR! You may report this log at:
npm ERR! <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR! <npm-@googlegroups.com>
npm ERR! System Windows_NT 6.2.9200
npm ERR! command “C:\\Program Files (x86)\\nodejs\\\\node.exe” “C:\\Program Files (x86)\\nodejs\\node_modules\\npm\\bin\
\npm-cli.js” “install” “azure”
npm ERR! cwd C:\
npm ERR! node -v v0.6.20
npm ERR! npm -v 1.1.37
npm ERR! message SSL Error: CERT_UNTRUSTED
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! C:\npm-debug.log
npm ERR! not ok code undefined
npm ERR! not ok code 1

To continue with the install, you can use the following:

npm config set strict-ssl false

Posted in Azure, JavaScript, Uncategorized | Leave a comment