Key Responsibilities
Design, build, and optimize scalable data pipelines for batch and real-time data processing.
Architect enterprise-grade data platforms using cloud-native technologies.
Develop and maintain robust ETL/ELT frameworks for integrating data from multiple sources.
Lead data migration and modernization initiatives from legacy systems to cloud platforms.
Ensure data quality, governance, security, and compliance across enterprise data ecosystems.
Implement data models, data marts, and data warehouse solutions to support analytics and reporting.
Collaborate with Data Scientists, BI teams, Product Owners, and business stakeholders.
Optimize data processing performance, scalability, and cost efficiency.
Establish best practices for data engineering, CI/CD, automation, monitoring, and observability.
Mentor junior and senior engineers while driving technical excellence within the team.
Required SkillsData Engineering & Big Data
Python
SQL
Apache Spark
Hadoop Ecosystem
Kafka
Airflow
Databricks
Cloud Platforms
Azure (Preferred)
Azure Data Factory (ADF)
Azure Synapse Analytics
Azure Data Lake Storage (ADLS)
Azure Event Hub
AWS
Glue
Redshift
EMR
S3
GCP (Valuable to have)
BigQuery
Dataflow
Composer
Database & Data Warehousing
SQL Server
PostgreSQL
Oracle
Snowflake
Azure Synapse
Redshift
DevOps & Automation
Git
Azure DevOps
Jenkins
Terraform
Docker
Kubernetes
Data Governance
Data Quality Frameworks
Metadata Management
Data Lineage
Data Security & Compliance
Master Data Management (MDM)
Leadership Responsibilities
Lead and guide a team of Data Engineers.
Participate in architecture review boards and technical governance discussions.
Define enterprise data engineering standards and best practices.
Drive stakeholder management and business engagement.
Support project estimation, planning, and execution.
Qualifications
Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field.
14+ years of experience in Data Engineering