Roles and Responsibilities:

- Design, develop, test, deploy, and maintain large-scale, fault-tolerant data pipelines using Python, PostgreSQL, and Airflow on GCP Cloud.

- Triage, debug, and optimize production pipeline failures, handling memory management, data quality, and edge cases.

- Collaborate with cross-functional teams to gather requirements and design scalable dimensional data models (Facts, Dimensions, SCD Type 2).

- Implement automated CI/CD pipelines (GitHub Actions/Jenkins) for productive code promotion across DEV, UAT, and PROD environments.

- Lead and mentor a team of engineers to ensure timely project delivery while enforcing code quality, unit testing, and best practices.

Job Requirements:

- 8–12 years of hands-on experience in Data Engineering with core expertise in Python, PostgreSQL, GCP (BigQuery/Cloud Storage), and Airflow.

- Strong understanding of advanced SQL, query performance tuning (EXPLAIN ANALYZE), composite indexing, and partitioning.

- Hands-on experience with CI/CD pipeline development (Jenkins, GitHub Actions, or similar tools).

- Proven track record of managing small engineering teams, conducting code reviews, and coordinating with project management and clients.

📌 Lead Data Engineer - Python, PostgreSQL, GCP Cloud (8+ Yrs) (Chennai)
🏢 OneMagnify
📍 Chennai