As of BUILD 2015 we have declarative or procedural deployment control of assets or group of assets for Network and Compute resources. In November/Oct 2014 excitement was fixed towards ability to download and apply gallery templates or work with website assets. But this release of ARM API (2015) has brought in lot of abilities.
This enables sophisticated scenarios. For e.g., Corey sanders has an example of using it to deploy containers on a virtual machine with famous three together (nginx,redis and mongo). Or look at chef integration or Network Security Groups, RBAC (role based access).
You can interact with assets via Portal UX. But it is easier to understand what is happening using PowerShell commands in Debug/Verbose mode.
What are Resource Groups
Usually applications deployed in Microsoft Azure are composed of a combination of different cloud assets (e.g., VMs, Storage accounts, a SQL database, a Virtual Network etc). RGs are provided by RG providers to be provisioned/managed in a location
Ref – https://msdn.microsoft.com/en-us/library/azure/dn948464.aspx
Let us get started. Verify in your Azure PowerShell
$PSVersionTable
is 3.0 or 4.0+. And
Get-Module AzureResourceManager
is 0.9 at least. Otherwise you need to get latest Azure Powershell
Let us start from simple command to deploy a resource group
Switch-AzureMode AzureResourceManager #do not execute following, maybe do get-help on it to get an idea of what it requires get-help New-AzureResourceGroup -detailed get-help New-AzureResourceGroupDeployment -detailed
The command NewAzureResourceGroup implies we need a simplistic json file. How much simpler we can make it. If you look at github profile hosting all the examples they are pretty daunting if you are starting out first time. So let us start from basics.
What are Azure Resource Manager Templates?
Azure Resource Manager Templates allow us to deploy and manage these different resources together by using a JSON description of the resources and associated configuration and deployment parameters. After creating JSON-based resource template, you can pass it to the Powershell command to execute its directives which ensures that the resources defined within it deployed in Azure.
Create a simple file helloarm.json, this is the template file for describing resources, their properties and ways to populate them via parameters. We will go on simple journey to accomplish that. By the end of session we will have single parameter file which pushes data to the template file which in turn is used by ARM to create resources.
helloarm.json We will explore the possible content of each collection or array as required later.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [],
"outputs": {
"result": {
"value": "Hello World",
"type": "string"
}
}
}
Let us see how we can test it out, execute the following
New-AzureResourceGroup -Name GSK-ARM-HelloARM -Location "South Central US" -Tag @{Name="GSK-ARM-RG";Value="TEST"}
this will result in empty resource group which can later be populated by New-AzureResource or New-AzureResourceGroupDeployment cmdlets to add resources and deployments to this resource group.
What is with parameters of the powershell command?
- Name is the name of the resource group
- Location is the data center we want to deploy to.
- Tag is the new feature which allows us to tag our resources with company approved labels to classify assets.
For now let us remove the newly created RG by
Remove-AzureResourceGroup -Name "GSK-ARM-HelloARM"
– this command does not return any output by default. Use -Passthru parameter to get that information and to suppress the questions – use -Force parameter. At present you can’t immediately see Operational log populated with these actions in your portal. We will come to it in a minute.
Now let us provision using our empty template
New-AzureResourceGroup –Name "GSK-ARM-HelloARM" –Location "South Central US" -Tag @{Name="GSK-ARM-RG";Value="TEST"} ## This creates a RG for us to add resources to New-AzureResourceGroupDeployment -Name "GSK-ARM-DEP-HelloARM" -ResourceGroupName "GSK-ARM-HelloARM" -TemplateFile .\helloarm.json DeploymentName : GSK-ARM-DEP-HelloARM ResourceGroupName : GSK-ARM-HelloARM ProvisioningState : Succeeded Timestamp : 5/3/2015 7:58:52 PM Mode : Incremental TemplateLink : Parameters : Outputs : Name Type Value =============== ========================= ========== result String Hello WorldCan you execute the 1st command without location or Name? Or you wished Test-AzureResourceGroup existed?
You can see the resource group and associated tags
Get-AzureResourceGroup -Name GSK-ARM-HelloARM ResourceGroupName : GSK-ARM-HelloARM Location : southcentralus ProvisioningState : Succeeded Tags : Name Value ========== ===== GSK-ARM-RG TEST Permissions : Actions NotActions ======= ========== * ResourceId : /subscriptions/XXXXXXXXXXXXXXXXX/resourceGroups/GSK-ARM-HelloARM2Let us get the tags
(Get-AzureResourceGroup -Name GSK-ARM-HelloARM2).Tags Name Value ---- ----- Value TEST Name GSK-ARM-RG
What happened to the deployment
Get-AzureResourceGroupLog -ResourceGroup GSK-ARM-HelloARM2 # docs says to provide -Name - but that is wrong.
Get-AzureResourceGroupDeployment -ResourceGroupName GSK-ARM-HelloARM2 DeploymentName : hell-arm ResourceGroupName : GSK-ARM-HelloARM2 ProvisioningState : Succeeded Timestamp : 5/3/2015 4:37:14 AM Mode : Incremental TemplateLink : Parameters : Outputs : Name Type Value =============== ========================= ========== result String Hello Worldaha ! there you see the output information 🙂
Get-AzureResourceLog -ResourceId /subscriptions/XXXXXXXXXXX/resourceGroups/GSK-ARM-HelloARM
does not provide anything if you run this command after an hour of inactivity on that resource. You can add -StartTime 015-05-01T00:30 parameter by varying your date to get idea about what all has happened on that particular resource. In this case you should see a write operation like
EventSource : Microsoft.Resources OperationName : Microsoft.Resources/subscriptions/resourcegroups/write
Ok can we just enable tags for a subscription via the template file as is possible through this REST API https://msdn.microsoft.com/en-us/library/azure/dn848364.aspx ? Nope not that I am able to find something right now.
So let us create storage account with a tag in a new Resource Group.
Ok so what are these resources we keep talking about ?
Get-AzureResource | Group-object ResourceType | Sort-Object count -descending | select Name
Above command will mostly have following resources in the top 10 as output resources if you have used Azure in classic way for sometime (it will not list all the possible resources, it is only listing used Resources)
Microsoft.ClassicStorage/storageAccounts
Microsoft.ClassicCompute/domainNames
Microsoft.ClassicCompute/virtualMachines
Microsoft.Web/sites
Microsoft.ClassicNetwork/virtualNetworks
Microsoft.Sql/servers/databasesLet us explore creating a storage account resource in a resource group
$RGName= "GSK-RG-STORAGE-TEST" $dName = "GSK-RG-DEP-STORAGE-TEST" $lName ="East US" $folderLocation = "YourDirectory" $templateFile= $folderLocation + "\azuredeploy.json"
Create following .json file and save in folderLocation mentioned above.
azuredeploy.json
{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": {}, "variables": {}, "resources": [{ "type": "Microsoft.Storage/storageAccounts", "name": "gskteststoragesc", "location": "East US", "apiVersion": "2015-05-01-preview", "tags": { "dept": "test" }, "properties": { "accountType": "Standard_LRS" } }], "outputs": { "result": { "value": "Hello World Tags & storage", "type": "string" } } }It will be good to look at azuredeploy.json file. Output directive is almost same, this can be retrievd in Get-AzureResourceGroupDeployment command .
- Type represents the Resource provider of storageAccount (v2) – the new resource, compared to Microsoft.ClassicStorage/storageAccounts.
- Properties belong to the asset/resource in this case – what kind of storage we want – local replica or geo dr etc. We have hardcoded it for now.
- location represents the datacenter location, in this case we have hard coded to “East US”
- Tags represent the tags we want to apply
- apiVersion is used by ARM to ensure latest compliant bits are used to provision resource at right location.
Let us deploy this resource
New-AzureResourceGroup –Name $RGName –Location $lName -Tag @{Name="GSK-ARM-RG";Value="TEST"}
## This creates a RG for us to add resources to
New-AzureResourceGroupDeployment -Name $dname -ResourceGroupName $RGName -TemplateFile $templateFile ## dname provides deployment label for the resources specified in the template file to be deployed to RGName.
Your new portal might show visualization like below
Get-AzureResourceLog -ResourceId /subscriptions/#####################/resourceGroups/GSK-RG-STORAGE-TEST -StartTime 2015-05-02T00:30 #modify this for your time and subscription
Now let us modify this template to accept parameters from other file. Make following changes to azuredeploy.json and save it as azuredeploy2.json Notice evertyhting else remains same
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"newStorageAccountName": {
"type": "string",
"metadata": {
"description": "Unique DNS Name for the Storage Account where the Virtual Machine's disks will be placed."
}
},
"location": {
"type": "string",
"defaultValue": "West US",
"allowedValues": [
"West US",
"East US",
"West Europe",
"East Asia",
"Southeast Asia"
],
"metadata": {
"description": "Location of resources"
}
}
},
"variables": {
"storageAccountType": "Standard_LRS"
},
"resources": [{
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('newStorageAccountName')]",
"location": "[parameters('location')]",
"apiVersion": "2015-05-01-preview",
"tags": {
"dept": "test"
},
"properties": {
"accountType": "[variables('storageAccountType')]"
}
}],
"outputs": {
"result": {
"value": "Hello World Tags & storage",
"type": "string"
}
}
}
What has changed ? Focus on the resources section 1st.
- Look at the name – this name will be picked from parameters which is of type string.
- Location is interesting in the sense it has a default value of US West, you will see the impact of same later on in case you do not provide location.
- AccountType on other hand is picked from variables section and is of single type STANDARD_LRS – local replication.
- Then we have allowedValues in Parameter section for location – this ensures only these values are accepted when cmdlet New-AzureResourceGroupDeployment executes using this template file. Takes away the pain of validating the inputs.
Let us execute the command earlier to provision this new Resource Group
After setting $templateFile properly to point to new template
$templateFile = $folderLocation + "\azuredeploy2.json" New-AzureResourceGroupDeployment -Name $dname -ResourceGroupName $RGName -TemplateFile $templateFile
You will get prompted for the parameter – newStorageAccountName – in our case we gave it gskrgstparam and got following output
DeploymentName : GSK-RG-DEP-STORAGE-TEST
ResourceGroupName : GSK-RG-STORAGE-TEST
ProvisioningState : Succeeded
Timestamp : 5/3/2015 4:36:15 PM
Mode : Incremental
TemplateLink :
Parameters :
Name Type Value
=============== ========================= ==========
newStorageAccountName String gskrgstparam
location String West US
Outputs :
Name Type Value
=============== ========================= ==========
result String Hello World Tags & storage
1st thing – notice the new parameters section getting populated and location parameter not providing the prompt and picking up default value “US West”. If you have access to the new portal , you will see something similar for the resource.
Let us see if we can automate this command line interaction alltogether.
Create a new file azure2deploy.parameters.json with following content
{
"newStorageAccountName": {
"value": "gskrgparamfiletest"
},
"location": {
"value": "West US"
}
}
Create new variable for holding template parameters file location
$templateParam = $folderLocation + "\azure2deploy.parameters.json"
Now execute the similar provisioning command now with template parameters file passed to it. You will not get prompted for the name of the storage account as you have already passed it in the parameters file in command line.
New-AzureResourceGroupDeployment -Name $dname -ResourceGroupName $RGName -TemplateFile $templateFile -TemplateParameterFile $templateParam
DeploymentName : GSK-RG-DEP-STORAGE-TEST
ResourceGroupName : GSK-RG-STORAGE-TEST
ProvisioningState : Succeeded
Timestamp : 5/3/2015 4:59:36 PM
Mode : Incremental
TemplateLink :
Parameters :
Name Type Value
=============== ========================= ==========
newStorageAccountName String gskrgparamfiletest
location String West US
Outputs :
Name Type Value
=============== ========================= ==========
result String Hello World Tags & storage
To wrap this piece
Get-AzureResourceGroup -Name $RGName
ResourceGroupName : GSK-RG-STORAGE-TEST
Location : eastus
ProvisioningState : Succeeded
Tags :
Name Value
========== =====
GSK-ARM-RG TEST
Permissions :
Actions NotActions
======= ==========
*
Resources :
Name Type Location
================== ================================= ========
gskrgparamfiletest Microsoft.Storage/storageAccounts westus
gskrgstparam Microsoft.Storage/storageAccounts westus
gskteststoragesc Microsoft.Storage/storageAccounts eastus
ResourceId : /subscriptions/XXXXXXXXXXXXXXX/resourceGroups/GSK-RG-STORAGE-TEST
Ok that is lot of stuff for the session. We started from bare-minimum resource group and gradutated to using parameters, variables and finally using parameters file to the provisioning engine. We found how we could get log of the resource or remove it. REST API allows you to check status after the post has been done.
WRT the .json file which has different sections like output to dump – messages, resources to hold resources to provision which in turn pick data from variables and parameter section.
Other questions could be
- How do you modify say a deployment? – Add a disk to VM? That is a different process for now.
- What is the relation between chef, puppet, ansible and ARM ? It is symbiotic as chef and puppet create infrastructure usng ARM and provision software install/check not possible through simple shell/install files. Look at
KundanaP’ and John Gossman’s s sample
- What has ARM got to do with sharing and governance ? You can create common azuredeploy.json and folks can create different azuredeploy.parameters.json files to create their own environment. So you control the deployment.
- Where are more samples? https://github.com/Azure/azure-quickstart-templates You can even contribute more by following guidelines there.
What do I miss – ability to put monitoring hook, but looking at providers from OpsInsight and AppsInsight- I can guess they are coming.
For fun now let us create one VM which provisions Aerospike on DS series machine on ubuntu. We will need to execute shell command via customlinuxscript extension and learn dependencies so that we do not create individual assets.
Ref – http://blog.davidebbo.com/, Mahesh T and Tabrez.
REST API – https://msdn.microsoft.com/en-us/library/azure/dn790568.aspx
For folks who face issues while trying to provision assets at different data centers – Please remember this API is in preview. The combination with the apiVersion and ResourceProvder ensures how features are made available in Azure. As of 3rd May 2015 this combination does not work at some of the data centers. This is expected to roll out in the near future across all data centers.
Example Error message could be – DisallowedLocation”,”message”:”The provided location ‘South Central US’ is not permitted for subscription. List of permitted regions is ‘eastasia,southeastasia,eastus,westus,westeurope’. Please contact support to change your supported regions.”}}. As an example of which Resource Providers are active where with what API version is here – https://gist.github.com/govindk/10b2fa173620c2241d9e (Thanks Shawn C for the Bearer Token tip)
Some place else 