Azure Resource Manager – a journey to understand basics

As of BUILD 2015 we have declarative or procedural deployment control of assets or group of assets for Network and Compute resources. In November/Oct 2014 excitement was fixed towards ability to download and apply gallery templates or work with website assets. But this release of ARM API (2015) has brought in lot of abilities.

This enables sophisticated scenarios. For e.g., Corey sanders has an example of using it to deploy containers on a virtual machine with famous three together (nginx,redis and mongo). Or look at chef integration or Network Security Groups, RBAC (role based access).

You can interact with assets via Portal UX. But it is easier to understand what is happening using PowerShell commands in Debug/Verbose mode.

ARM-Build2015

What are Resource Groups
Usually applications deployed in Microsoft Azure are composed of a combination of different cloud assets (e.g., VMs, Storage accounts, a SQL database, a Virtual Network etc). RGs are provided by RG providers to be provisioned/managed in a location

Ref – https://msdn.microsoft.com/en-us/library/azure/dn948464.aspx

Let us get started. Verify in your Azure PowerShell

$PSVersionTable 

is 3.0 or 4.0+. And

 Get-Module AzureResourceManager 

is 0.9 at least. Otherwise you need to get latest Azure Powershell

Let us start from simple command to deploy a resource group

Switch-AzureMode AzureResourceManager

#do not execute following, maybe do get-help on it to get an idea of what it requires
get-help New-AzureResourceGroup -detailed
get-help New-AzureResourceGroupDeployment -detailed

The command NewAzureResourceGroup implies we need a simplistic json file. How much simpler we can make it. If you look at github profile hosting all the examples they are pretty daunting if you are starting out first time. So let us start from basics.

What are Azure Resource Manager Templates?
Azure Resource Manager Templates allow us to deploy and manage these different resources together by using a JSON description of the resources and associated configuration and deployment parameters. After creating JSON-based resource template, you can pass it to the Powershell command to execute its directives which ensures that the resources defined within it deployed in Azure.

Create a simple file helloarm.json, this is the template file for describing resources, their properties and ways to populate them via parameters. We will go on simple journey to accomplish that. By the end of session we will have single parameter file which pushes data to the template file which in turn is used by ARM to create resources.

helloarm.json We will explore the possible content of each collection or array as required later.

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {},
    "variables": {},
    "resources": [],
    "outputs": {
        "result": {
            "value": "Hello World",
            "type": "string"
        }
    }
}

Let us see how we can test it out, execute the following

New-AzureResourceGroup -Name GSK-ARM-HelloARM -Location "South Central US" -Tag @{Name="GSK-ARM-RG";Value="TEST"} 

this will result in empty resource group which can later be populated by New-AzureResource or New-AzureResourceGroupDeployment cmdlets to add resources and deployments to this resource group.

What is with parameters of the powershell command?

  • Name is the name of the resource group
  • Location is the data center we want to deploy to.
  • Tag is the new feature which allows us to tag our resources with company approved labels to classify assets.bare-min-rg

    For now let us remove the newly created RG by

     Remove-AzureResourceGroup -Name "GSK-ARM-HelloARM" 

    – this command does not return any output by default. Use -Passthru parameter to get that information and to suppress the questions – use -Force parameter. At present you can’t immediately see Operational log populated with these actions in your portal. We will come to it in a minute.

    Now let us provision using our empty template

     
    New-AzureResourceGroup –Name "GSK-ARM-HelloARM" –Location "South Central US" -Tag @{Name="GSK-ARM-RG";Value="TEST"}
    ## This creates a RG for us to add resources to
    New-AzureResourceGroupDeployment -Name "GSK-ARM-DEP-HelloARM" -ResourceGroupName "GSK-ARM-HelloARM" -TemplateFile .\helloarm.json
    
    DeploymentName    : GSK-ARM-DEP-HelloARM
    ResourceGroupName : GSK-ARM-HelloARM
    ProvisioningState : Succeeded
    Timestamp         : 5/3/2015 7:58:52 PM
    Mode              : Incremental
    TemplateLink      :
    Parameters        :
    Outputs           :
                        Name             Type                       Value
                        ===============  =========================  ==========
                        result           String                     Hello World
    

    Can you execute the 1st command without location or Name? Or you wished Test-AzureResourceGroup existed?

    You can see the resource group and associated tags

     
    Get-AzureResourceGroup -Name GSK-ARM-HelloARM
    
    ResourceGroupName : GSK-ARM-HelloARM
    Location          : southcentralus
    ProvisioningState : Succeeded
    Tags              :
                        Name        Value
                        ==========  =====
                        GSK-ARM-RG  TEST
    
    Permissions       :
                        Actions  NotActions
                        =======  ==========
                        *
    
    ResourceId        : /subscriptions/XXXXXXXXXXXXXXXXX/resourceGroups/GSK-ARM-HelloARM2
    
    

    Let us get the tags

     
    (Get-AzureResourceGroup -Name GSK-ARM-HelloARM2).Tags 
    
    Name                           Value
    ----                           -----
    Value                          TEST
    Name                           GSK-ARM-RG
    

    What happened to the deployment

     
    Get-AzureResourceGroupLog -ResourceGroup GSK-ARM-HelloARM2 # docs says to provide -Name - but that is wrong.
    
     
    Get-AzureResourceGroupDeployment -ResourceGroupName GSK-ARM-HelloARM2
    
    DeploymentName    : hell-arm
    ResourceGroupName : GSK-ARM-HelloARM2
    ProvisioningState : Succeeded
    Timestamp         : 5/3/2015 4:37:14 AM
    Mode              : Incremental
    TemplateLink      :
    Parameters        :
    Outputs           :
                        Name             Type                       Value
                        ===============  =========================  ==========
                        result           String                     Hello World
    

    aha ! there you see the output information🙂

     
    Get-AzureResourceLog -ResourceId /subscriptions/XXXXXXXXXXX/resourceGroups/GSK-ARM-HelloARM 
    

    does not provide anything if you run this command after an hour of inactivity on that resource. You can add -StartTime 015-05-01T00:30 parameter by varying your date to get idea about what all has happened on that particular resource. In this case you should see a write operation like

     
    EventSource : Microsoft.Resources
    OperationName : Microsoft.Resources/subscriptions/resourcegroups/write
    

    Ok can we just enable tags for a subscription via the template file as is possible through this REST API https://msdn.microsoft.com/en-us/library/azure/dn848364.aspx ? Nope not that I am able to find something right now.

    So let us create storage account with a tag in a new Resource Group.

    Ok so what are these resources we keep talking about ?

     
    Get-AzureResource | Group-object ResourceType | Sort-Object count -descending | select Name 
    

    Above command will mostly have following resources in the top 10 as output resources if you have used Azure in classic way for sometime (it will not list all the possible resources, it is only listing used Resources)
    Microsoft.ClassicStorage/storageAccounts
    Microsoft.ClassicCompute/domainNames
    Microsoft.ClassicCompute/virtualMachines
    Microsoft.Web/sites
    Microsoft.ClassicNetwork/virtualNetworks
    Microsoft.Sql/servers/databases

    Let us explore creating a storage account resource in a resource group

     
    $RGName= "GSK-RG-STORAGE-TEST"
    $dName = "GSK-RG-DEP-STORAGE-TEST"
    $lName ="East US"
    $folderLocation = "YourDirectory"
    $templateFile= $folderLocation + "\azuredeploy.json"
    

    Create following .json file and save in folderLocation mentioned above.

    azuredeploy.json

     
    {
        "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
        "contentVersion": "1.0.0.0",
        "parameters": {},
        "variables": {},
        "resources": [{
            "type": "Microsoft.Storage/storageAccounts",
            "name": "gskteststoragesc",
            "location": "East US",
            "apiVersion": "2015-05-01-preview",
            "tags": {
                "dept": "test"
            },
            "properties": {
                "accountType": "Standard_LRS"
            }
        }],
        "outputs": {
            "result": {
                "value": "Hello World Tags & storage",
                "type": "string"
            }
        }
    }
    

    It will be good to look at azuredeploy.json file. Output directive is almost same, this can be retrievd in Get-AzureResourceGroupDeployment command .

  • Type represents the Resource provider of storageAccount (v2) – the new resource, compared to Microsoft.ClassicStorage/storageAccounts.
  • Properties belong to the asset/resource in this case – what kind of storage we want – local replica or geo dr etc. We have hardcoded it for now.
  • location represents the datacenter location, in this case we have hard coded to “East US”
  • Tags represent the tags we want to apply
  • apiVersion is used by ARM to ensure latest compliant bits are used to provision resource at right location.

Let us deploy this resource

 
New-AzureResourceGroup –Name $RGName –Location $lName -Tag @{Name="GSK-ARM-RG";Value="TEST"}
## This creates a RG for us to add resources to
 
New-AzureResourceGroupDeployment -Name $dname -ResourceGroupName $RGName -TemplateFile $templateFile
## dname provides deployment label for the resources specified in the template file to be deployed to RGName. 

Your new portal might show visualization like below
success-storage-test

 
Get-AzureResourceLog -ResourceId /subscriptions/#####################/resourceGroups/GSK-RG-STORAGE-TEST -StartTime 2015-05-02T00:30 #modify this for your time and subscription

Now let us modify this template to accept parameters from other file. Make following changes to azuredeploy.json and save it as azuredeploy2.json Notice evertyhting else remains same

 
{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {

        "newStorageAccountName": {
            "type": "string",
            "metadata": {
                "description": "Unique DNS Name for the Storage Account where the Virtual Machine's disks will be placed."
            }
        },
        "location": {
            "type": "string",
            "defaultValue": "West US",
            "allowedValues": [
                "West US",
                "East US",
                "West Europe",
                "East Asia",
                "Southeast Asia"
            ],
            "metadata": {
                "description": "Location of resources"
            }

        }
    },
    "variables": {
        "storageAccountType": "Standard_LRS"
    },
    "resources": [{
        "type": "Microsoft.Storage/storageAccounts",
        "name": "[parameters('newStorageAccountName')]",
        "location": "[parameters('location')]",
        "apiVersion": "2015-05-01-preview",
        "tags": {
            "dept": "test"
        },
        "properties": {
            "accountType": "[variables('storageAccountType')]"
        }
    }],
    "outputs": {
        "result": {
            "value": "Hello World Tags & storage",
            "type": "string"
        }
    }
}

What has changed ? Focus on the resources section 1st.

  • Look at the name – this name will be picked from parameters which is of type string.
  • Location is interesting in the sense it has a default value of US West, you will see the impact of same later on in case you do not provide location.
  • AccountType on other hand is picked from variables section and is of single type STANDARD_LRS – local replication.
  • Then we have allowedValues in Parameter section for location – this ensures only these values are accepted when cmdlet New-AzureResourceGroupDeployment executes using this template file. Takes away the pain of validating the inputs.

Let us execute the command earlier to provision this new Resource Group

After setting $templateFile properly to point to new template

$templateFile = $folderLocation + "\azuredeploy2.json"
New-AzureResourceGroupDeployment -Name $dname -ResourceGroupName $RGName -TemplateFile $templateFile

You will get prompted for the parameter – newStorageAccountName – in our case we gave it gskrgstparam and got following output


DeploymentName    : GSK-RG-DEP-STORAGE-TEST
ResourceGroupName : GSK-RG-STORAGE-TEST
ProvisioningState : Succeeded
Timestamp         : 5/3/2015 4:36:15 PM
Mode              : Incremental
TemplateLink      :
Parameters        :
                    Name             Type                       Value
                    ===============  =========================  ==========
                    newStorageAccountName  String                     gskrgstparam
                    location         String                     West US

Outputs           :
                    Name             Type                       Value
                    ===============  =========================  ==========
                    result           String                     Hello World Tags & storage

1st thing – notice the new parameters section getting populated and location parameter not providing the prompt and picking up default value “US West”. If you have access to the new portal , you will see something similar for the resource.
param-based-prompt-storage-acct

Let us see if we can automate this command line interaction alltogether.

Create a new file azure2deploy.parameters.json with following content

{
    "newStorageAccountName": {
        "value": "gskrgparamfiletest"
    },

    "location": {
        "value": "West US"
    }
}

Create new variable for holding template parameters file location

$templateParam = $folderLocation + "\azure2deploy.parameters.json"

Now execute the similar provisioning command now with template parameters file passed to it. You will not get prompted for the name of the storage account as you have already passed it in the parameters file in command line.

New-AzureResourceGroupDeployment -Name $dname -ResourceGroupName $RGName -TemplateFile $templateFile -TemplateParameterFile $templateParam

DeploymentName    : GSK-RG-DEP-STORAGE-TEST
ResourceGroupName : GSK-RG-STORAGE-TEST
ProvisioningState : Succeeded
Timestamp         : 5/3/2015 4:59:36 PM
Mode              : Incremental
TemplateLink      :
Parameters        :
                    Name             Type                       Value
                    ===============  =========================  ==========
                    newStorageAccountName  String                     gskrgparamfiletest
                    location         String                     West US

Outputs           :
                    Name             Type                       Value
                    ===============  =========================  ==========
                    result           String                     Hello World Tags & storage

param-based-fileinput-storage-acct
To wrap this piece

Get-AzureResourceGroup -Name $RGName
ResourceGroupName : GSK-RG-STORAGE-TEST
Location          : eastus
ProvisioningState : Succeeded
Tags              :
                    Name        Value
                    ==========  =====
                    GSK-ARM-RG  TEST

Permissions       :
                    Actions  NotActions
                    =======  ==========
                    *

Resources         :
                    Name                Type                               Location
                    ==================  =================================  ========
                    gskrgparamfiletest  Microsoft.Storage/storageAccounts  westus
                    gskrgstparam        Microsoft.Storage/storageAccounts  westus
                    gskteststoragesc    Microsoft.Storage/storageAccounts  eastus

ResourceId        : /subscriptions/XXXXXXXXXXXXXXX/resourceGroups/GSK-RG-STORAGE-TEST


Ok that is lot of stuff for the session. We started from bare-minimum resource group and gradutated to using parameters, variables and finally using parameters file to the provisioning engine. We found how we could get log of the resource or remove it. REST API allows you to check status after the post has been done.
WRT the .json file which has different sections like output to dump – messages, resources to hold resources to provision which in turn pick data from variables and parameter section.

Other questions could be

    How do you modify say a deployment? – Add a disk to VM? That is a different process for now.
    What is the relation between chef, puppet, ansible and ARM ? It is symbiotic as chef and puppet create infrastructure usng ARM and provision software install/check not possible through simple shell/install files. Look at

KundanaP’ and John Gossman’s s sample

    https://github.com/Azure/azure-quickstart-templates/tree/master/chef-json-parameters-ubuntu-vm
    What has ARM got to do with sharing and governance ? You can create common azuredeploy.json and folks can create different azuredeploy.parameters.json files to create their own environment. So you control the deployment.
    Where are more samples? https://github.com/Azure/azure-quickstart-templates You can even contribute more by following guidelines there.

What do I miss – ability to put monitoring hook, but looking at providers from OpsInsight and AppsInsight- I can guess they are coming.

For fun now let us create one VM which provisions Aerospike on DS series machine on ubuntu. We will need to execute shell command via customlinuxscript extension and learn dependencies so that we do not create individual assets.

Ref – http://blog.davidebbo.com/, Mahesh T and Tabrez.
REST APIhttps://msdn.microsoft.com/en-us/library/azure/dn790568.aspx
For folks who face issues while trying to provision assets at different data centers – Please remember this API is in preview. The combination with the apiVersion and ResourceProvder ensures how features are made available in Azure. As of 3rd May 2015 this combination does not work at some of the data centers. This is expected to roll out in the near future across all data centers.
Example Error message could be – DisallowedLocation”,”message”:”The provided location ‘South Central US’ is not permitted for subscription. List of permitted regions is ‘eastasia,southeastasia,eastus,westus,westeurope’. Please contact support to change your supported regions.”}}. As an example of which Resource Providers are active where with what API version is here – https://gist.github.com/govindk/10b2fa173620c2241d9e  (Thanks Shawn C for the Bearer Token tip)

Azure Resource Manager – a journey to understand basics

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s