Azure Machine Learning on Databricks Seth

Azure Machine Learning on Databricks Seth Mottaghinejad [email protected] Wolfgang Pauli, PhD [email protected]

Agenda Azure Databricks Azure Machine Learning AML Integration with Databricks AutoML

Azure Databricks

What is Azure Databricks? A fast, easy and collaborative Apache Spark based analytics platform optimized for Azure Best of Databricks Best of Microsoft Designed in collaboration with the founders of Apache Spark One-click set up; streamlined workflows Interactive workspace that enables collaboration between data scientists, data engineers, and business analysts. Native integration with Azure services (Power BI, SQL DW, Cosmos DB, Blob Storage) Enterprise-grade Azure security (Active Directory integration, compliance, enterprise -grade SLAs)

Azure Databricks Enhance Productivity Build on secure & trusted cloud Scale without limits

AML Integration with Databricks

AML SDK Databricks End-to-end custom Machine Learning Easily install the SDK in Azure Databricks clusters Use the SDK for: - Logging training run metrics - Containerize Spark ML models - Deploy them into ACI or AKS, just like any other models. Automated ML (public preview) of Azure ML SDK with Azure Databricks

AutoML

Machine Learning Process SQL DB Cosmos DB Datawarehouse Data lake Blob storage Prepare Data Build & Train Deploy

Machine Learning Complexity Source: http://scikit-learn.org/stable/tutorial/machine learning map/index.html