Introduction Into Azure Machine Learning

Azure Machine Learning is a platform for operating machine learning workloads in the cloud.

Azure Machine Learning
microsoft

Built on the Microsoft Azure cloud platform, Azure Machine Learning enables you to manage:

  • Scalable on-demand compute for machine learning workloads.
  • Data storage and connectivity to ingest data from a wide range sources.
  • Machine learning workflow orchestration to automate model training, deployment, and management processes.
  • Model registration and management, so you can track multiple versions of models and the data on which they were trained.
  • Metrics and monitoring for training experiments, datasets, and published services.
  • Model deployment for real-time and batch inferencing.

Azure Machine Learning workspaces

A workspace defines the boundary for a set of related machine learning assets. You can use workspaces to group machine learning assets based on projects, deployment environments (for example, test and production), teams, or some other organizing principle. The assets in a workspace include:

  • Compute targets for development, training, and deployment.
  • Data for experimentation and model training.
  • Notebooks containing shared code and documentation.
  • Experiments, including run history with logged metrics and outputs.
  • Pipelines that define orchestrated multi-step processes.
  • Models that you have trained.
An Azure Machine Learning workspace

Role-Based Access Control

You can assign role-based authorization policies to a workspace, enabling you to manage permissions that restrict what actions specific Azure Active Directory (AAD) principals can perform. For example, you could create a policy that allows only users in the IT Operations group to create compute targets and datastores, while allowing users in the Data Scientists group to create and run experiments and register models.

Creating a Workspace

You can create a workspace in any of the following ways:

  • In the Microsoft Azure portal, create a new Machine Learning resource, specifying the subscription, resource group, workspace name, and workspace edition.
  • Use the Azure Machine Learning Python SDK to run code that creates a workspace. For example:

PythonCopy

    from azureml.core import Workspace
    
    ws = Workspace.create(name='aml-workspace', 
                      subscription_id='123456-abc-123...',
                      resource_group='aml-resources',
                      create_resource_group=True,
                      location='eastus',
                      sku='enterprise'
                     )


  • Use the Azure Command Line Interface (CLI) with the Azure Machine Learning CLI extension. For example, you could use the following command (which assumes a resource group named aml-resources has already been created):

bashCopy

    az ml workspace create -w 'aml-workspace' -g 'aml-resources'

Considerations for Creating a Workspace

When planning to create an Azure Machine Learning workspace, there are some options you should consider.

Region

Your workspace will be created in an Azure region, which determines the data center in which the workspace resources will be hosted. You should ensure that the region where you create the workspace will support the resources you will need.

For example, you might need to create virtual machines in your workspace to support notebook development or model training. If you intend to leverage graphical processing unit (GPU) processing, which are commonly used to perform training of deep neural network (DNN) models, then you should ensure that the region where your workspace is hosted supports the appropriate series of virtual machines (for example, NC-series virtual machines support vGPUs, but are not available in all regions).

Edition

Azure Machine Learning workspaces are available in two editions:

  • Enterprise – includes all features.
  • Basic – includes core features, but does not include Designer or graphical tools for automated machine learning or data drift monitoring.

Azure Machine Learning tools and interfaces

Azure Machine Learning studio

You can manage the assets in your Azure Machine Learning workspace in the Azure portal, but as this is a general interface for managing all kinds of resources in Azure, data scientists and other users involved in machine learning operations may prefer to use a more focused, dedicated interface.

Azure Machine Learning studio

Azure Machine Learning studio is a web-based tool for managing an Azure Machine Learning workspace. It enables you to create, manage, and view all of the assets in your workspace and provides the following graphical tools:

  • Designer: A drag and drop interface for “no code” machine learning model development.
  • Automated Machine Learning: A wizard interface that enables you to train a model using a combination of algorithms and data preprocessing techniques to find the best model for your data.

The Azure Machine Learning SDK

While graphical interfaces like Azure Machine Learning studio make it easy to create and manage machine learning assets, it is often advantageous to use a code-based approach to managing resources. By writing scripts to create and manage resources, you can:

  • Run machine learning operations from your preferred development environment.
  • Automate asset creation and configuration to make it repeatable.
  • Ensure consistency for resources that must be replicated in multiple environments (for example, development, test, and production)
  • Incorporate machine learning asset configuration into developer operations (DevOps) workflows, such as continuous integration / continuous deployment (CI/CD) pipelines.

Installing the Azure Machine Learning SDK for Python

You can install the Azure Machine Learning SDK for Python by using the pip package management utility, as shown in the following code sample:bashCopy

pip install azureml-sdk

The SDK includes optional extras that aren’t required for core operations, but can be useful in some scenarios. For example, the **notebooks extra include widgets for displaying detailed output in Jupyter Notebooks, the automl extra includes packages for automated machine learning training, and the explain extra includes packages for generating model explanations. To install extras, specify them in brackets as shown here:bashCopy

pip install azureml-sdk[notebooks,automl,explain]

More Information: For more information about installing the Azure Machine Learning SDK for Python, see the SDK documentation. Also, you should be aware that the SDK is updated on a regular basis, and review the release notes for the latest release.

Connecting to a Workspace

After installing the SDK package in your Python environment, you can write code to connect to your workspace and perform machine learning operations. The easiest way to connect to a workspace is to use a workspace configuration file, which includes the Azure subscription, resource group, and workspace details as shown here:JSONCopy

{
    "subscription_id": "1234567-abcde-890-fgh...",
    "resource_group": "aml-resources",
    "workspace_name": "aml-workspace"
}

 Tip

You can download a configuration file for a workspace from the Overview page of its blade in the Azure portal.

To connect to the workspace using the configuration file, you can use the from_config method of the Workspace class in the SDK, as shown here:PythonCopy

from azureml.core import Workspace

ws = Workspace.from_config()

By default, the from_config method looks for a file named config.json in the folder containing the Python code file, but you can specify another path if necessary.

As an alternative to using a configuration file, you can use the get method of the Workspace class with explicitly specified subscription, resource group, and workspace details as shown here – though the configuration file technique is generally preferred due to its greater flexibility when using multiple scripts:PythonCopy

from azureml.core import Workspace

ws = Workspace.get(name='aml-workspace',
                   subscription_id='1234567-abcde-890-fgh...',
                   resource_group='aml-resources')

Whichever technique you use, if there is no current active session with your Azure subscription, you will be prompted to authenticate.

Working with the Workspace Class

The Workspace class is the starting point for most code operations. For example, you can use its compute_targets attribute to retrieve a dictionary object containing the compute targets defined in the workspace, like this:PythonCopy

for compute_name in ws.compute_targets:
    compute = ws.compute_targets[compute_name]
    print(compute.name, ":", compute.type)

The SDK contains a rich library of classes that you can use to create, manage, and use many kinds of asset in an Azure Machine Learning workspace.

More Information: For more information about the Azure Machine Learning SDK, see the SDK documentation.

The Azure Machine Learning CLI Extension

The Azure command-line interface (CLI) is a cross-platform command-line tool for managing Azure resources. The Azure Machine Learning CLI extension is an additional package that provides commands for working with Azure Machine Learning.

To install the Azure Machine Learning CLI extension, you must first install the Azure CLI. See the full installation instructions for all supported platforms for more details.

After installing the Azure CLI, you can add the Azure Machine Learning CLI extension by running the following command:bashCopy

az extension add -n azure-cli-ml

To use the Azure Machine Learning CLI extension, run the az ml command with the appropriate parameters for the action you want to perform. For example, to list the compute targets in a workspace, run the following command:bashCopy

az ml computetarget list -g 'aml-resources' -w 'aml-workspace'

 Note

In the code sample above, the -g parameter specifies the name of the resource group in which the Azure Machine Learning workspace specified in the -w parameter is defined. These parameters are shortened aliases for –resource-group and –workspace-name.

More Information: For more information about the Azure Machine Learning CLI Extension, see the documentation.

Compute Instances

Azure Machine Learning includes the ability to create Compute Instances in a workspace to provide a development environment that is managed with all of the other assets in the workspace.

Notebook VM

Compute Instances include Jupyter Notebook and JupyterLab installations that you can use to write and run code that uses the Azure Machine Learning SDK to work with assets in your workspace.

You can choose a VM image that provides the compute specification you need, from small CPU-only VMs to large GPU-enabled workstations. Because the VMs are hosted in Azure, you only pay for the compute resources when they are running; so you can create a VM to suit your needs, and stop it when your workload has completed to minimize costs.

You can store notebooks independently in workspace storage, and open them in any VM.

Visual Studio Code

Visual Studio Code (VS Code) is a lightweight code editing environment for Microsoft Windows, Apple macOS, and Linux. It provides a visual interface for many kinds of code, including Microsoft C#, Javascript, Python and others; as well as intellisense and syntax formatting for common data formats such as JSON and XML.

VS Code’s flexibility is based on the ability to install modular extensions that add syntax checking, debugging, and visual management interfaces for specific workloads. For example, the Microsoft Python extension for VS Code adds support for writing and running Python code in scripts or notebooks within the VS Code interface.

VS Code

The Azure Machine Learning Extension for VS Code provides a graphical interface for working with assets in an Azure Machine Learning workspace. You can combine the capabilities of the Azure Machine Learning and Python extensions to manage a complete end-to-end machine learning workload in Azure Machine Learning from the VS Code environment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s