Role & responsibilities

- Design, build, and optimize scalable and reusable data pipelines on Databricks, SQL, and Spark
- Orchestrate automated ETL workflows using Airflow, ADF, and CI/CD pipelines
- Implement and manage scalable data processing systemsensuring performance, reliability, and cost-effectiveness
- Ensure data quality, consistency, and compliance with governance/security standards
- Collaborate with stakeholders to clarify requirements and translate them into technical solutions
- Continuously evaluate and implement new tools/technologies to improve pipeline automation, monitoring, and efficiency

Preferred candidate profile

- Bachelors or Master’s degree in Computer Science, Engineering, or related field
- 8+ years of experience in Data Engineering with solid big data expertise
- Deep expertise in Databricks, Spark, and SQL (including Spark SQL)
- Strong data modeling and ETL experience with large-scale datasets




- Hands-on experience orchestrating data pipelines with Airflow and CI/CD
- Proven experience with Delta Live Tables, Unity Catalog, Delta Sharing, and SQL
- Warehouse (server & serverless)
- Proficiency with Python for data engineering workflows
- Familiarity with analytics engineering and DBT
- Experience with data governance, compliance, and security in a cloud environment
- Strong problem-solving ability, requirement clarification, and communication skills
- Experience adopting software engineering best practices (testing, modularization, refactoring, automation)
- Ability to optimize data systems for performance and cost efficiency in cloud environments
- Exposure to AI/ML frameworks for development efficiency
- Demonstrated ability to lead projects independently and mentor junior engineers

📌 Senior Data Engineer (Uttar Pradesh)
🏢 Instinctools India
📍 Uttar Pradesh