Export all data from Azure Cosmos DB for MongoDB to file for archiving

azure

If you want to export all data from a MongoDB in Azure, then you have come to the right place.

Maybe there is a better way, but I could not find it. Azure data studio allows me to connect, and in a query to the collection get data, but it only gets me about 30 items at the time, so copy paste from the results in not realisitc.

This procedure, is done on a WIndows environment, with internet connectivity, the MongoDB does not have any IP restrictions, if it does, you would have to fix connectivity first. I guess a good test can be to connect using the Azure Data Studio.

Here it goes!

mongo2

mongoexport /h mymongodbaccount.mongo.cosmos.azure.com:10255 /u mymongodbaccount /p "RANDOMCHARACTERSRANDOMCHARACTERSRANDOMCHARACTERSRANDOMCHARACTERSRANDOMCHARACTERS==" /ssl /d myDatabase /c myDataCollection /o c:\Temp\myDataCollection.json --authenticationDatabase "admin"

Broken down, this command has the following components:

mongoexport

command line .exe tool that is part of the mongodb database tools package that is a free download from mongodb.com, see link in the reference section

/h mymongodbaccount.mongo.cosmos.azure.com:10255

host, the complete path including the port to your MongoDB in Azure. You will find this string in the connection string section in the Azure portal

/u mymongodbaccount

User, this is the name of the Cosmos DB for MongoDB account, also the first section of the host string.

/p “RANDOMCHARACTERSRANDOMCHARACTERSRANDOMCHARACTERSRANDOMCHARACTERS==”

Password, this is also find in the connection string section (I could be wrong here, I deleted all my DB’s before creating this post…if not here, then its in a different section in the portal, it IS there though)

/ssl

LTS/SSL/Https, secure encrypted connection

/d myDatabase

Database, the database name where you are exporting from

/c myDataCollection

Collection, the name of the collection you are exporting from. The file will contain all data from this collection in json format

/o c:\Temp\myDataCollection.json

Output/target file, put it anywhere locally on your computer

–authenticationDatabase “admin”

This is required for the auth using the account + password to work

The result, when executed in a CMD prompt looks like this:

mongo

Important!
If you have more than one collection, in more than one mongoDB, then you will have to run the command again and again, remember though to change the output filename.

Happy exporting!

References

https://www.mongodb.com/try/download/database-tools

https://www.mongodb.com/community/forums/t/unable-to-authenticate-using-mechanism-scram-sha-1/86151/4

___________________________________________________________________________________________________

Enjoy!

Regards

Thomas Odell Balkeståhl on LinkedIn

List all changes made in your Azure environment in one query using Azure Resource Graph

azure

The kusto query below is using the new ResourceChanges resource, and will give you a list of all changes made in your Azure environment, create, update, delete, automatic or manual, it will all be there. (Where you have access).
This is a great way to keep track of what is happening in and to your environment.

You will get the following info from each change made:

Subcription Name
ChangeTime
ResourceName
ResourceType
ResourceGroup
ChangeType
SubscriptionName
SubscriptionId
TargetResourceId
CorrelationId
ChangeCount
ChangedProperties

This list can get pretty long, so it can be a good idea to filer and sort a bit. Especially on the timeframe (in the sample query, it is set to 7 days).
Filter can be applies to subscription, dates, resource groups, resource name, type, etc.
Example:

| where resourceGroup contains “production”
Will give you only the changes made to RG’s with production in the name.

| where resourceType = “virtualMachines”
Will give you only the changes made to resources of type Virtual Machine.

Open the Azure portal, find ‘Azure Resource Graph Explorer’, then paste the following query in the query windows and hit run.

ResourceChanges
| join kind=inner
   (resourcecontainers
   | where type == 'microsoft.resources/subscriptions'
   | project subscriptionId, subscriptionName = name)
   on subscriptionId
| extend changeTime = todatetime(properties.changeAttributes.timestamp), targetResourceId = tostring(properties.targetResourceId),
changeType = tostring(properties.changeType), correlationId = properties.changeAttributes.correlationId,
changedProperties = properties.changes, changeCount = properties.changeAttributes.changesCount
| extend resourceName = tostring(split(targetResourceId, '/')[-1])
| extend resourceType = tostring(split(targetResourceId, '/')[-2])
| where changeTime > ago(7d)
// Change the time span as preferred, 1d(1 day/24h), 7d, 30d...
| where subscriptionName contains "DevTest" // "" for all subscriptions
| order by changeType asc, changeTime desc
// Change what you sort by as prefered, type, time, subscriptionName, etc.
| project changeTime, resourceName, resourceType, resourceGroup, changeType, subscriptionName, subscriptionId, targetResourceId, 
correlationId, changeCount, changedProperties

As this is a functionality in public preview at the time of posting, the following applies:

Important!
Resource configuration changes is in Public Preview and only supports changes to resource types from the Resources table in Resource Graph. This does not yet include changes to the resource container resources, such as Management groups, Subscriptions, and Resource groups.

Happy resource mining!

References

https://docs.microsoft.com/en-us/azure/governance/resource-graph/how-to/get-resource-changes

___________________________________________________________________________________________________

Enjoy!

Regards

Thomas Odell Balkeståhl on LinkedIn

List all Linux VM extensions in one query using Azure Resource Graph

azure

The kusto query below will give you a list of all Linux VMs and the extensions they have…
It was handy when the OMI vulnerability was out.

You will get the following info:

VM Name
Operating System
Extensions
Add additional columns to get more info.

Resources
| where type == 'microsoft.compute/virtualmachines'
| extend
    JoinID = toupper(id),
    OSName = tostring(properties.osProfile.computerName),
    OSType = tostring(properties.storageProfile.osDisk.osType),
    VMSize = tostring(properties.hardwareProfile.vmSize)
| join kind=leftouter(
    Resources
    | where type == 'microsoft.compute/virtualmachines/extensions'
    | extend
        VMId = toupper(substring(id, 0, indexof(id, '/extensions'))),
        ExtensionName = name
) on $left.JoinID == $right.VMId
| where OSType == 'Linux'
| summarize Extensions = make_list(ExtensionName) by OSName, OSType
| order by tolower(OSName) asc

The result looks something like this:

Happy resource mining!

References
https://docs.microsoft.com/en-us/azure/governance/resource-graph/samples/advanced?tabs=azure-cli

___________________________________________________________________________________________________

Enjoy!

Regards

Thomas Odell Balkeståhl on LinkedIn

List all UDR User Defined Routes in one query using Azure Resource Graph

azure

The kusto query below will give you a list of all manually added routes/UDRs in all of your Route Tables(RT) in all subnets in all subscriptions. (Where you have access).
This is a great way to keep track of your UDRs…lest they get out of hand…

You will get the following info from each UDR:

Subcription Name
Resource Group Name
Subnet Name
RT Name
UDR Name
Target Prefix
Next Hop Type
Next Hop IP Address
Provisioning State
Has BGP Override

In my current Azure network, the total UDR count is around 413, in 39 different RTs. Its not easy to keep track and t-shoot the network, not easy at all.

Use different sort or where clauses to filter and sort on what you are currently looking for, if you for example filter on
| where addressPrefix == ‘0.0.0.0/0’
you will see only the routes for ‘unrouted’ traffic.
| where nextHopType == ‘VirtualAppliance’
will list all routes to your(?) Virtual appliance Firewall, and so on.
(You can add the where clauses at the end below the last line)

resources
| where type =~ "Microsoft.Network/routeTables"
| mv-expand rules = properties.routes
| join kind=leftouter (resourcecontainers 
| where type=='microsoft.resources/subscriptions' 
| project SubcriptionName=name, subscriptionId) on subscriptionId
| extend subnet_name = split((split(tostring(properties.subnets), '/'))[10], '"')[0]
| extend addressPrefix = tostring(rules.properties.addressPrefix)
| extend nextHopType = tostring(rules.properties.nextHopType)
| extend nextHopIpAddress = tostring(rules.properties.nextHopIpAddress)
| extend hasBgpOverride = tostring(rules.properties.hasBgpOverride)
| extend provisioningState = tostring(rules.properties.provisioningState)
| extend udrname = rules.name
| extend rtname = name
| project SubcriptionName, resourceGroup, subnet_name, rtname, udrname, addressPrefix, nextHopType, nextHopIpAddress, provisioningState, hasBgpOverride
| sort by SubcriptionName, resourceGroup asc, rtname asc, addressPrefix asc

The result looks something like this:

Happy resource mining!

References
https://docs.microsoft.com/en-us/azure/governance/resource-graph/samples/advanced?tabs=azure-cli

___________________________________________________________________________________________________

Enjoy!

Regards

Thomas Odell Balkeståhl on LinkedIn

List all NSG security rules in one query using Azure Resource Graph

azure

The kusto query below will give you a list of all manually added security rules on all of your NSGs in all of your subnets. (Where you have access).
This is a great way to keep track of your vNets and subnets, what is allowed where…

You will get the following info from each NSG security rule:

Subcription Name
Resource Group Name
Subnet Name
NSG Name
Direction
Priority
Destination IP Prefix
Destination Port
Source IP Prefix
Source Port
Description
(Optional: SubscriptionId, extended.properties)

In my current Azure network, the count is around 200, in 75 different NSGs. Its not easy to keep track and find the ‘holes’ if you cannot get a good overview.

Use different sort or where clauses to filter and sort on what you are currently looking for, if you for example filter on
| where destport == ‘*’
you will see only the rules allowing traffic to any port.
| where destprefix == ‘*’
will list all rules allowing traffic to any ip address on the subnet, and so on.

Resources
| where type =~ "microsoft.network/networksecuritygroups"
| join kind=leftouter (ResourceContainers | where type=='microsoft.resources/subscriptions' | project SubcriptionName=name, subscriptionId) on subscriptionId
| where resourceGroup == 'production' or resourceGroup == 'testing'
// Only if you don't want to see all, add more resourceGroups as needed: or resourceGroup == 'xxx'
| mv-expand rules=properties.securityRules
| extend direction = tostring(rules.properties.direction)
| extend priority = toint(rules.properties.priority)
| extend description = rules.properties.description
| extend destprefix = rules.properties.destinationAddressPrefix
| extend destport = rules.properties.destinationPortRange
| extend sourceprefix = rules.properties.sourceAddressPrefix
| extend sourceport = rules.properties.sourcePortRange
| extend subnet_name = split((split(tostring(properties.subnets), '/'))[10], '"')[0]
//| where destprefix == '*'
| project SubcriptionName, resourceGroup, subnet_name, name, direction, priority, destprefix, destport, sourceprefix, sourceport, description //, subscriptionId, rules.properties
| sort by SubcriptionName, resourceGroup asc, name, direction asc, priority asc

Happy resource mining!

References
https://docs.microsoft.com/en-us/azure/governance/resource-graph/samples/advanced?tabs=azure-cli

___________________________________________________________________________________________________

Enjoy!

Regards

Thomas Odell Balkeståhl on LinkedIn