Azure Synapse Analytics

Important

This guide is only relevant if you are using MetaKraftwerk's Azure Synapse Analytics (ASA) integration. If you are only developing ELT patterns, you can skip this page.

Azure Synapse Analytics combines big data and data warehousing.
MetaKraftwerk patterns can create Synapse pipelines, SQL scripts, and Spark notebooks.

Uploading Templates from Synapse

Develop your pipelines, notebooks, and SQL scripts directly in Synapse Studio.
Then export and upload them to MetaKraftwerk as pattern templates.

Exporting Pipelines and Scripts

• In Synapse Studio, export the pipeline JSON definitions from the Integrate section.
• Export notebooks as .ipynb and SQL scripts as .sql.

Importing into MetaKraftwerk

• Upload the pipeline JSON, .sql, and .ipynb files to the pattern.
• Use placeholders like ${DATABASE}, ${SCHEMA}, ${SPARK_POOL}, ${DATASET_PATH} to parameterize your templates.