Roles & Responsibilities
Design, develop, and deliver scalable end-to-end data pipelines using Azure Data Factory, ensuring robust integration
of enterprise-wide data from diverse sources
• Build and optimize data engineering workflows using Databricks and PySpark
• Write efficient, high-performance SQL for data transformation and analysis
• Work with the Azure Cloud platform and associated services, applying strong understanding of data warehousing,
data models, and pipelines
• Provide technical leadership to a team of developers, including code reviews and enforcing best practices across the
development lifecycle
• Oversee CI/CD implementation using Azure DevOps, managing deployments across development, QA, and production
environments with proper change control processes
• Collaborate with cross-functional teams to translate business requirements into scalable data solutions
• Ensure data quality, reliability, and performance across all pipelines and platforms
– Must have minimum 8+ years of overall experience in Data Engineering, Data Development, or related data technology roles, with strong hands-on experience in enterprise data pipeline development.
Mandatory (Experience 2) – Must have strong hands-on experience with Azure Databricks, including development and optimization of scalable data engineering workflows using Databricks and PySpark.
Mandatory (Experience 3) – Must have strong hands-on proficiency in PySpark/Python and SQL, with proven experience developing complex data transformations, processing workflows, and performance-optimized queries.
Mandatory (Experience 4) – Must have hands-on experience with Azure Data Factory (ADF) for designing, developing, and orchestrating end-to-end data pipelines and integrating data from multiple sources.
Mandatory (Experience 5) – Must have strong experience working on the Azure Cloud platform and associated data