Changes coming to Azure Synapse

At the 2019 Ignite Conference, Microsoft announced Azure Synapse Analytics was replacing Azure Data Warehouse. As a standard with Azure-based products, Microsoft is continually adding and changing the functionality. Here is what’s new with Azure Synapse and what’s coming down the pipe.

 What’s new:

• COPY Statement Functionality in Azure Data Factory
  • A new COPY statement is now available in public preview, which provides the most flexibility for high-throughput data ingestion. With this copy, you can do tasks like load data from a public storage account, load authenticating via Share Access Signature (SAS), load with a column list with default values authenticating via Storage Account Key, load Parquet or ORC using existing file format object, or load specifying wildcards and multiple files. The performance of the COPY command is currently about the same as Polybase but will be faster once it hits general availability (estimated by end of the year 2020).
• What’s coming down the pipe:
  • We’ve seen many cool demos of what Azure Synapse will be able to do once in general availability. So far, most of these are still in preview. See the following list for items that are in general availability as of today:
    • Provisioned compute
    • Materialized views
    • Workload importance
    • Column and row-level security
    • Dynamic data masking
  • Here is the list of items that are in the preview. There currently is no timeline for when these items will be ready for general availability:
    • Workload isolation
    • On-demand query
    • Power BI integration
    • Azure Machine Learning integration
    • Data Lake exploration
    • Streaming analytics
    • Apache Spark integration
    • Hybrid data ingestion
    • Azure Synapse studio
    • Private endpoints.

Diving further into the Azure Synapse Analytics, Microsoft refers to the main interface as Azure Synapse Studio. It is essentially a culmination of many different Azure-based products, rolled into a centralized system. See below for the interface breakout (Icons on the left toolbar are numbered 1-6 from top to bottom):

1. Home splash page – which includes Quickstart bar, Resources, and Useful links
2. Data – Integrated SQL development for querying relational and non-relational data at petabyte-scale, with quick visualizations and query optimization functionality
3. Insights – Develop insights using integrated development windows including SQL scripts, Notebooks, Data flows, Spark job definitions, and Power BI.
4. Orchestrate – Similar to Data Factory, design pipelines for moving and transforming data
5. Monitor – Centralized view allowing monitoring of all Azure related resources
6. Manage – Configure the workspace, pool, and access to artifacts

These updates will continue to be released and evolve as Synapse does. We will continue to keep on top of these updates as they roll out.

Ryan Brossart