An ETL developer in Azure is responsible for designing, implementing, and maintaining data pipelines that securely move, transform, and store data while ensuring performance, quality, and traceability, utilizing services like Azure Data Factory, Azure Data Warehouse, Key Vault, Azure SQL, Azure DevOps, and Git Flow.

Core Responsibilities

Data Extraction and Loading

ETL developers extract data from multiple sources, including relational databases, APIs, flat files, or event streaming platforms (Event Hubs, Kafka, IoT Hub). They ensure secure connections via Azure Key Vault for credentials and secrets and perform incremental or full loads into centralized storage solutions like Azure Data Lake Storage (ADLS Gen2) or Azure SQL / Synapse Data WarehouseSQL Server

Data Transformation

During transformation, data is cleansed, normalized, deduplicated, and enriched to meet business logic.



ETL developers implement these transformations using Azure Data Factory mapping data flows, Azure Databricks (PySpark/Scala/Python/SQL notebooks), T-SQL, or custom scripts. Transformations include calculations, aggregations, joining multiple datasets, and handling Slowly Changing Dimensions (SCDs) for historical tracking SQL Server

Pipeline Orchestration and Automation

ETL developers design robust, modular, and parameterized ETL/ELT pipelines using Azure Data Factory, leveraging control flow, triggers, and integration runtimes. They implement retry mechanisms, error handling, and incremental updates to maintain idempotency and pipeline reliability. Pipelines can be batch-based or real-time streaming, incorporating Databricks Structured Streaming.

📌 Etl Developer (Bengaluru)
🏢 Tata Consultancy Services
📍 Bengaluru