Skip to content

Latest commit

 

History

History
652 lines (510 loc) · 72.8 KB

README.md

File metadata and controls

652 lines (510 loc) · 72.8 KB
Raw

InSpec for Azure

  • Project State: Maintained

For more information on project states and SLAs, see this documentation.

Build Status

This InSpec resource pack uses the Azure REST API and provides the required resources to write tests for resources in Azure.

Table of Contents

Prerequisites

  • Ruby
  • Bundler installed

Authentication

Azure CLI Authentication:

-The Azure CLI provides a command-line interface for interacting with Azure services. To enable authentication, you will need to install the Azure CLI.- https://learn.microsoft.com/en-us/cli/azure/install-azure-cli

  az login --tenant AZURE_TENANT_ID

a. Use the az login --tenant AZURE_TENANT_ID command to log in with a specific Azure tenant: If you have a specific Azure tenant ID, you can provide it as a parameter to the az login command. If you don't specify the tenant ID, the CLI will provide a list of available tenants.

b. If the CLI can open your default browser: If the CLI can open your default browser, it will initiate the authorization code flow and open the Azure sign-in page in the browser for authentication.

c. If no web browser is available or fails to open: In case a web browser is not available or fails to open, the CLI will initiate the device code flow. It will provide you with a code and instruct you to open a browser page at https://aka.ms/devicelogin. You need to enter the code displayed in your terminal on that page for authentication.

d. Storing retrieved credentials: The documentation suggests storing the retrieved credentials, such as tenant_id and subscription_id, in environment variables prefaced with AZURE_. It provides an example of using a .envrc file or creating environment variables using the preferred method.

AZURE_TENANT_ID=<your-azure-tenant-id-here>
AZURE_SUBSCRIPTION_ID=<your-azure-subscription-id-here>

Azure Service Principal Account Authentication:

Service Principal

Your Azure Service Principal Account must have a minimum of reader role of the Azure roles to any subscription that you'd like to use this resource pack.

You must have the following pieces of information:

  • TENANT_ID
  • CLIENT_ID
  • CLIENT_SECRET
  • SUBSCRIPTION_ID

To create your account Service Principal Account:

  1. Log in to the Azure portal.
  2. Click Azure Active Directory.
  3. Click APP registrations.
  4. Click New application registration.
  5. Enter name and select Web from the Application Type drop-down.
  6. Save your application.
  7. Note your Application ID. This is your client_id.
  8. Click Certificates & secrets.
  9. Click New client secret.
  10. Create a new password. This value is your client_secret above.
  11. Go to your subscription (click All Services then subscriptions).
  12. Choose your subscription from that list.
  13. Note your Subscription ID.
  14. Click Access control (IAM).
  15. Click Add.
  16. Select the reader role.
  17. Select the application you created and click save.

These must be stored in an environment variables prefaced with AZURE_. If you use Dotenv, then you can save these values in your own .envrc file. Either source it or run direnv allow. If you do not use Dotenv, then you can create environment variables in the way that you prefer.

Use the Resources

Since this is an InSpec resource pack, it only defines InSpec resources. To use these resources in your controls, you should create your profile:

Create a new profile

inspec init profile --platform azure my-profile

Example inspec.yml:

name: my-profile
title: My own Azure profile
version: 0.1.0
inspec_version: '>= 4.23.15'
depends:
  - name: inspec-azure
    url: https://github.com/inspec/inspec-azure/archive/x.tar.gz
supports:
  - platform: azure

(For available inspec-azure versions, see this list of inspec-azure versions.)

Resource Documentation

List of generic resources:

With the generic resources:

  • Azure cloud resources pack, which does not include a static InSpec resource and can be tested.
  • Azure resources from different resource providers and resource groups can be tested at the same time.
  • Server-side filtering can be used for more efficient tests.

List of static resources

Singular Resource Plural Resource
azure_active_directory_domain_service azure_active_directory_domain_services
azure_active_directory_object azure_active_directory_objects
azure_aks_cluster azure_aks_clusters
azure_api_management azure_api_management
azure_application_gateway azure_application_gateways
azure_bastion_hosts_resource azure_bastion_hosts_resources
azure_blob_service azure_blob_services
azure_cdn_profile azure_cdn_profiles
azure_container_group azure_container_groups
azure_container_registry azure_container_registries
azure_cosmosdb_database_account No Plural Resource
azure_data_factory azure_data_factories
azure_data_factory_dataset azure_data_factory_datasets
azure_data_factory_linked_service azure_data_factory_linked_services
azure_data_factory_pipeline azure_data_factory_pipelines
azure_data_factory_pipeline_run_resource azure_data_factory_pipeline_run_resources
azure_data_lake_storage_gen2_filesystem azure_data_lake_storage_gen2_filesystems
azure_data_lake_storage_gen2_path azure_data_lake_storage_gen2_paths
azure_db_migration_service azure_db_migration_services
azure_ddos_protection_resource azure_ddos_protection_resources
azure_dns_zones_resource azure_dns_zones_resources
azure_event_hub_authorization_rule No Plural Resource
azure_event_hub_event_hub No Plural Resource
azure_event_hub_namespace No Plural Resource
azure_express_route_circuit azure_express_route_circuits
No Singular Resource azure_express_route_providers
azure_graph_user azure_graph_users
azure_hdinsight_cluster No Plural Resource
azure_hpc_asc_operation No Plural Resource
azure_hpc_cache azure_hpc_caches
azure_hpc_storage_target azure_hpc_storage_targets
azure_iothub No Plural Resource
azure_iothub_event_hub_consumer_group azure_iothub_event_hub_consumer_groups
azure_key_vault azure_key_vaults
azure_key_vault_key azure_key_vault_keys
azure_key_vault_secret azure_key_vault_secrets
azure_load_balancer azure_load_balancers
azure_lock azure_locks
azure_managed_application azure_managed_applications
azure_management_group azure_management_groups
azure_mariadb_server azure_mariadb_servers
azure_microsoft_defender_pricing azure_microsoft_defender_pricings
azure_microsoft_defender_security_contact No Plural Resource
azure_microsoft_defender_setting azure_microsoft_defender_settings
azure_migrate_assessment azure_migrate_assessments
azure_migrate_assessment_group azure_migrate_assessment_groups
azure_migrate_project No Plural Resource
azure_migrate_project_database azure_migrate_project_databases
azure_migrate_project_database_instance azure_migrate_project_database_instances
azure_migrate_project_event azure_migrate_project_events
azure_migrate_project_machine azure_migrate_project_machines
azure_migrate_assessment_project azure_migrate_assessment_projects
azure_migrate_project_solution azure_migrate_project_solutions
azure_monitor_activity_log_alert azure_monitor_activity_log_alerts
azure_monitor_log_profile azure_monitor_log_profiles
azure_mysql_database azure_mysql_databases
azure_mysql_server azure_mysql_servers
azure_mysql_server_configuration azure_mysql_server_configurations
azure_network_interface azure_network_interfaces
azure_network_security_group azure_network_security_groups
azure_network_watcher azure_network_watchers
No Singular Resource azure_policy_assignments
azure_policy_definition azure_policy_definitions
azure_policy_exemption azure_policy_exemptions
azure_policy_insights_query_result azure_policy_insights_query_results
azure_postgresql_database azure_postgresql_databases
azure_postgresql_server azure_postgresql_servers
azure_power_bi_app azure_power_bi_apps
azure_power_bi_app_dashboard azure_power_bi_app_dashboards
azure_power_bi_app_dashboard_tile azure_power_bi_app_dashboard_tiles
azure_power_bi_app_report azure_power_bi_app_reports
No Singular Resource azure_power_bi_capacities
azure_power_bi_capacity_refreshable azure_power_bi_capacity_refreshables
azure_power_bi_capacity_workload azure_power_bi_capacity_workloads
azure_power_bi_dataflow azure_power_bi_dataflows
No Singular Resource azure_power_bi_dataflow_storage_accounts
azure_power_bi_dataset azure_power_bi_datasets
No Singular Resource azure_power_bi_dataset_datasources
azure_power_bi_embedded_capacity azure_power_bi_embedded_capacities
azure_power_bi_gateway azure_power_bi_gateways
No Singular Resource azure_power_bi_generic_resources
azure_public_ip No Plural Resource
azure_redis_cache azure_redis_caches
azure_resource_group azure_resource_groups
azure_resource_health_availability_status azure_resource_health_availability_statuses
azure_resource_health_emerging_issue azure_resource_health_emerging_issues
No Singular Resource azure_resource_health_events
azure_role_definition azure_role_definitions
azure_security_center_policy azure_security_center_policies
azure_service_bus_namespace azure_service_bus_namespaces
No Singular Resource azure_service_bus_regions
azure_service_bus_subscription azure_service_bus_subscriptions
azure_service_bus_subscription_rule azure_service_bus_subscription_rules
azure_service_bus_topic azure_service_bus_topics
azure_service_fabric_mesh_application azure_service_fabric_mesh_applications
azure_service_fabric_mesh_network azure_service_fabric_mesh_networks
azure_service_fabric_mesh_replica azure_service_fabric_mesh_replicas
azure_service_fabric_mesh_service azure_service_fabric_mesh_services
azure_service_fabric_mesh_volume azure_service_fabric_mesh_volumes
azure_snapshot azure_snapshots
azure_sql_database azure_sql_databases
azure_sql_database_server_vulnerability_assessment azure_sql_database_server_vulnerability_assessments
azure_sql_managed_instance azure_sql_managed_instances
azure_sql_server azure_sql_servers
azure_sql_virtual_machine azure_sql_virtual_machines
azure_sql_virtual_machine_group azure_sql_virtual_machine_groups
azure_sql_virtual_machine_group_availability_listener azure_sql_virtual_machine_group_availability_listeners
azure_storage_account_blob_container azure_storage_account_blob_containers
azure_storage_account azure_storage_accounts
azure_streaming_analytics_function azure_streaming_analytics_functions
azure_subnet azure_subnets
azure_subscription azure_subscriptions
azure_synapse_notebook azure_synapse_notebooks
azure_synapse_workspace azure_synapse_workspaces
azure_virtual_machine azure_virtual_machines
azure_virtual_machine_disk azure_virtual_machine_disks
azure_virtual_network azure_virtual_networks
azure_virtual_network_gateway azure_virtual_network_gateways
azure_virtual_network_gateway_connection azure_virtual_network_gateway_connections
azure_virtual_network_peering azure_virtual_network_peerings
azure_virtual_wan azure_virtual_wans
azure_web_app_function azure_web_app_functions
azure_webapp azure_webapps

Please refer to the specific resource pages for more details and different use cases.

Examples

Ensure that all resources have specified names within the subscription regardless of type and resource Group

azure_generic_resources(substring_of_name: 'NAME').ids.each do |id|
  describe azure_generic_resource(resource_id: id) do
    its('location') { should eq 'eastus' }
  end
end

Ensure all resources has a specified tag defined regardless of the value

azure_generic_resources(tag_name: 'NAME').ids.each do |id|
  describe azure_generic_resource(resource_id: id) do
    its('location') { should eq 'eastus' }
  end
end

Verify Properties of an Azure Virtual Machine

describe azure_virtual_machine(resource_group: 'RESOURCE_GROUP', name: 'NAME-WEB-01') do
  it { should exist }
  it { should have_monitoring_agent_installed }
  it { should_not have_endpoint_protection_installed([]) }
  it { should have_only_approved_extensions(['MicrosoftMonitoringAgent']) }
  its('type') { should eq 'Microsoft.Compute/virtualMachines' }
  its('installed_extensions_types') { should include('MicrosoftMonitoringAgent') }
  its('installed_extensions_names') { should include('LogAnalytics') }
end

Verify Properties of a Network Security Group

describe azure_network_security_group(resource_group: 'RESOURCE_GROUP', name: 'NAME-SERVER') do
  it { should exist }
  its('type') { should eq 'Microsoft.Network/networkSecurityGroups' }
  its('security_rules') { should_not be_empty }
  its('default_security_rules') { should_not be_empty }
  it { should_not allow_rdp_from_internet }
  it { should_not allow_ssh_from_internet }
  it { should allow(source_ip_range: '0.0.0.0', destination_port: '22', direction: 'inbound') }
  it { should allow_in(service_tag: 'Internet', port: %w{1433-1434 1521 4300-4350 5000-6000}) }
end

Parameters Applicable To All Resources

The generic resources and their derivations support the following parameters unless stated otherwise on their specific resource page.

api_version

As an Azure resource provider enables new features, it releases a new version of the REST API. They are generally in the format of 2020-01-01. InSpec Azure resources can be forced to use a specific version of the API to eliminate the behavioral changes between the tests using different API versions. The latest version is used unless a specific version is provided.

User-Provided API Version

describe azure_virtual_machine(resource_group: 'RESOURCE_GROUP', name: 'NAME', api_version: '2020-01-01') do
  its('api_version_used_for_query_state') { should eq 'user_provided' }
  its('api_version_used_for_query') { should eq '2020-01-01' }
end

Pre-defined Default Api Version

DEFAULT api version can be used, if it is supported by the resource provider.

describe azure_generic_resource(resource_provider: 'Microsoft.Compute/virtualMachines', name: 'NAME', api_version: 'DEFAULT') do
  its('api_version_used_for_query_state') { should eq 'DEFAULT' }
end

Latest API Version

LATEST version is determined by this resource pack within the supported API versions. If the latest version is a preview, than an older, but a stable version might be used. Explicitly forcing to use the LATEST version.

describe azure_virtual_networks(api_version: 'LATEST') do
  its('api_version_used_for_query_state') { should eq 'LATEST' }
end

LATEST version is used unless provided (Implicit).

describe azure_network_security_groups(resource_group: 'RESOURCE_GROUP') do
  its('api_version_used_for_query_state') { should eq 'LATEST' }
end

LATEST version is used if the provided is invalid.

describe azure_network_security_groups(resource_group: 'RESOURCE_GROUP', api_version: 'invalid_api_version') do
  its('api_version_used_for_query_state') { should eq 'LATEST' }
end

endpoint

Microsoft Azure cloud services are available through a global and three national networks of the datacenter as described here. The preferred data center can be defined via endpoint parameter. Azure Global Cloud is used if not provided.

  • azure_cloud (default)
  • azure_china_cloud
  • azure_us_government_L4
  • azure_us_government_L5
  • azure_german_cloud
describe azure_virtual_machines(endpoint: 'azure_german_cloud') do
  it { should exist }
end

It can be defined as an environment variable or a resource parameter (has priority).

The pre-defined environment variables for each cloud deployment can be found here.

http_client parameters

The behavior of the HTTP client can be defined with the following parameters:

  • azure_retry_limit: Maximum number of retries (default - 2, Integer).
  • azure_retry_backoff: Pause in seconds between retries (default - 0, Integer).
  • azure_retry_backoff_factor: The amount to multiply each successive retries interval amount by (default - 1, Integer).

They can be defined as environment variables or resource parameters (has priority).

WARNING The following resources are using their azure_ counterparts under the hood, and they will be deprecated in the InSpec Azure version 2. Their API versions are fixed (see below) for full backward compatibility. It is strongly advised to start using the resources with azure_ prefix for an up-to-date testing experience.

Legacy Resource Name Fixed API version Replaced by
azurerm_ad_user, azurerm_ad_users v1.0 azure_graph_user, azure_graph_users
azurerm_aks_cluster, azurerm_aks_clusters 2018-03-31 azure_aks_cluster, azure_aks_cluster
azurerm_api_management, azurerm_api_managements 2019-12-01 azure_api_management, azure_api_managements
azurerm_application_gateway, azurerm_application_gateways 2019-12-01 azure_application_gateway, azure_application_gateways
azurerm_cosmosdb_database_account 2015-04-08 azure_cosmosdb_database_account
azurerm_event_hub_authorization_rule 2017-04-01 azure_event_hub_authorization_rule
azurerm_event_hub_event_hub 2017-04-01 azure_event_hub_event_hub
azurerm_event_hub_namespace 2017-04-01 azure_event_hub_namespace
azurerm_hdinsight_cluster 2015-03-01-preview azure_hdinsight_cluster
azurerm_iothub 2018-04-01 azure_iothub
azurerm_iothub_event_hub_consumer_group, azurerm_iothub_event_hub_consumer_groups 2018-04-01 azure_iothub_event_hub_consumer_group, azure_iothub_event_hub_consumer_groups
azurerm_key_vault, azurerm_key_vaults 2016-10-01 azure_key_vault, azure_key_vaults
azurerm_key_vault_key, azurerm_key_vault_keys 2016-10-01 azure_key_vault_key, azure_key_vault_keys
azurerm_key_vault_secret, azurerm_key_vault_secrets 2016-10-01 azure_key_vault_secret, azure_key_vault_secrets
azurerm_load_balancer, azurerm_load_balancers 2018-11-01 azure_load_balancer, azure_load_balancers
azurerm_locks 2016-09-01 azure_locks
azurerm_management_group, azurerm_management_groups 2018-03-01-preview azure_management_group, azure_management_groups
azurerm_mariadb_server, azurerm_mariadb_servers 2018-06-01-preview azure_mariadb_server, azure_mariadb_servers
azurerm_monitor_activity_log_alert, azurerm_monitor_activity_log_alerts 2017-04-01 azure_monitor_activity_log_alert, azure_monitor_activity_log_alerts
azurerm_monitor_log_profile, azurerm_monitor_log_profiles 2016-03-01 azure_monitor_log_profile, azure_monitor_log_profiles
azurerm_mysql_database, azurerm_mysql_databases 2017-12-01 azure_mysql_database, azure_mysql_databases
azurerm_mysql_server, azurerm_mysql_servers 2017-12-01 azure_mysql_server, azure_mysql_servers
azurerm_network_interface, azurerm_network_interfaces 2018-11-01 azure_network_interface, azure_network_interfaces
azurerm_network_security_group, azurerm_network_security_groups 2018-02-01 azure_network_security_group, azure_network_security_groups
azurerm_network_watcher, azurerm_network_watchers 2018-02-01 azure_network_watcher, azure_network_watchers
azurerm_postgresql_database, azurerm_postgresql_databases 2017-12-01 azure_postgresql_database, azure_postgresql_databases
azurerm_postgresql_server, azurerm_postgresql_servers 2017-12-01 azure_postgresql_server, azure_postgresql_servers
azurerm_public_ip 2020-05-01 azure_public_ip
azurerm_resource_groups 2018-02-01 azure_resource_groups
azurerm_role_definition, azurerm_role_definitions 2015-07-01 azure_role_definition, azure_role_definitions
azurerm_security_center_policy, azurerm_security_center_policies 2015-06-01-Preview azure_security_center_policy, azure_security_center_policies
azurerm_sql_database, azurerm_sql_databases 2017-10-01-preview azure_sql_database, azure_sql_databases
azurerm_sql_server, azurerm_sql_servers 2018-06-01-preview azure_sql_server, azure_sql_servers
azurerm_storage_account, azurerm_storage_accounts 2017-06-01 azure_storage_account, azure_storage_accounts
azurerm_storage_account_blob_container, azurerm_storage_account_blob_containers 2018-07-01 azure_storage_account_blob_container, azure_storage_account_blob_containers
azurerm_subnet, azurerm_subnets 2018-02-01 azure_subnet, azure_subnets
azurerm_subscription 2019-10-01 azure_subscription
azurerm_virtual_machine, azurerm_virtual_machines 2017-12-01 azure_virtual_machine, azure_virtual_machines
azurerm_virtual_machine_disk, azurerm_virtual_machine_disks 2017-03-30 azure_virtual_machine_disk, azure_virtual_machine_disks
azurerm_virtual_network, azurerm_virtual_networks 2018-02-01 azure_virtual_network, azure_virtual_networks
azurerm_webapp, azurerm_webapps 2016-08-01 azure_webapp, azure_webapps

Development

If you would like to contribute to this project, please see Contributing Rules.

For a detailed walk-through of resource creation, see the Resource Creation Guide.

Developing a Static Resource

The static resource is an InSpec Azure resource that is used to interrogate a specific Azure resource, such as, azure_virtual_machine, azure_key_vaults. As opposed to the generic resources, they might have some static properties created by processing the dynamic properties of a resource, such as azure_virtual_machine.admin_username.

The easiest way to start by checking the existing static resources. They have detailed information on leveraging the backend class within their comments.

The common parameters are:

  • resource_provider: Such as Microsoft.Compute/virtualMachines. It has to be hardcoded in the code by the resource author via the specific_resource_constraint method, and it should be the first parameter defined in the resource. This method includes user-supplied input validation.
  • display_name: A generic one will be created unless defined.
  • required_parameters: Define mandatory parameters. The resource_group and resource name in the singular resources are default mandatory in the base class.
  • allowed_parameters: Define optional parameters. The resource_group is optional in plural resources, but this can be made mandatory in the static resource.
  • resource_uri: Azure REST API URI of a resource. This parameter should be used when a resource does not reside in a resource group. It requires add_subscription_id to be set to either true or false. See azure_policy_definition and azure_policy_definitions.
  • add_subscription_id: It indicates whether the subscription ID should be included in the resource_uri or not.

Singular Resources

The singular resource is used to test a specific resource of a specific type and should include all of the properties available, such as azure_virtual_machine.

  • In most cases, resource_group and resource name should be required from the users, and a single API call would be enough for creating methods on the resource. See azure_virtual_machine for a standard singular resource and how to create static methods from resource properties.
  • If it is beneficial to accept the resource name with a more specific keyword, such as server_name, see azure_mysql_server.
  • If a resource exists in another resource, such as a subnet on a virtual network, see azure_subnet.
  • If it is necessary to make an additional API call within a static method, the create_additional_properties should be used. See azure_key_vault.

Plural Resources

A plural resource is used to test the collection of resources of a specific type, such as, azure_virtual_machines. This allows for tests to be written based on the group of resources.

  • A standard plural resource does not require a parameter, except optional resource_group. See azure_mysql_servers.
  • All plural resources use FilterTable to be able to provide filtering within returned resources. The filter criteria must be defined table_schema Hash variable.
  • If the properties of the resource are to be manipulated before populating the FilterTable, a populate_table method has to be defined. See azure_virtual_machines.
  • If the resources exist in another resource, such as subnets of a virtual network, a resource_path has to be created. For that, the identifiers of the parent resource, resource_group and virtual network name vnet, must be required from the users. See azure_subnets.

Setting the Environment Variables

The following instructions helps you get your development environment setup to run integration tests.

Copy .envrc-example to .envrc and fill in the fields with the values from your account.

export AZURE_SUBSCRIPTION_ID=<subscription id>
export AZURE_CLIENT_ID=<client id>
export AZURE_TENANT_ID=<tenant id>
export AZURE_CLIENT_SECRET=<client secret>

For PowerShell, set the following environment variables.

$env:AZURE_SUBSCRIPTION_ID="<subscription id>"
$env:AZURE_CLIENT_ID="<client id>"
$env:AZURE_CLIENT_SECRET="<client secret>"
$env:AZURE_TENANT_ID="<tenant id>"

in order to run tests along with mock train URI

export RAKE_ENV=test

Setup Azure CLI

  • Follow the instructions for your platform here
    • macOS: brew update && brew install azure-cli
  • Login with the azure-cli
    • rake azure:login
  • Verify azure-cli is logged in:
    • az account show

Starting an Environment

First, ensure your system has Terraform installed.

This environment may be used to run your profile against or to run integration tests on it. We are using Terraform workspaces to allow teams to have unique environments without affecting each other.

Direnv

Direnv is used to initialize an environment variable WORKSPACE to your username. We recommend using direnv and allowing it to run in your environment. However, if you prefer to not use direnv you may also source .envrc.

Rake Commands

Creating a new environment:

rake azure:login
rake tf:apply

Updating a running environment (For example, when you change the .tf file):

rake tf:apply

Checking if your state has diverged from your plan:

rake tf:plan

Destroying your environment:

rake tf:destroy

To run Rubocop and Syntax, check for Ruby and InSpec:

rake test:lint

To run unit tests:

rake test:unit

To run integration tests:

rake test:integration

Please note that Graph API resource requires specific privileges granted to your service principal.

Please refer to the Microsoft Documentation for information on how to grant these permissions to your application.

To run a control called azure_virtual_machine only:

rake test:integration[azurerm_virtual_machine]

Note that in zsh you need to escape the [, ] characters.

You may run selected multiple controls only:

rake test:integration[azure_aks_cluster,azure_virtual_machine]

To run lint and unit tests:

rake

Optional Components

The creation of the following resources can be skipped if there are any resource constraints.

  • Network Watcher
rake tf:apply[network_watcher]
  • HDinsight Interactive Query Cluster
rake tf:apply[hdinsight_cluster]
  • Public IP
rake tf:apply[public_ip]
  • API Management
rake tf:apply[api_management]
  • Management Group
rake tf:apply[management_group]

A combination of the above can be provided.

rake tf:apply[management_group,public_ip,network_watcher]