LOCATION: NOIDA (OFFICE)

*instinctools is a software development company that provides custom software solutions for businesses of all sizes. Our team works closely with clients to understand their specific needs and provide personalized solutions that meet their business requirements.

*instinctools is looking for a Senior Data Engineer for one of our clients.

Our Client is one of the TOP-5 global management consulting firms considered to be among the most prestigious ones in the world. Hundreds of customers from Fortune-500, including the largest global financial institutions, the worlds top media companies, technology companies and federal government agencies rely on our Clients proven platform and services.

Project is a dynamic solution empowering companies to optimize promotional activities for maximum impact. It collects and validates data, analyzes promotion effectiveness, plans calendars, and integrates seamlessly with existing systems. The tool enhances vendor collaboration, negotiates better deals, and employs machine learning to optimize promotional plans, enabling companies to make informed decisions and maximize return on investment.

Stack on the project: Databricks, SQL, Spark, Python Delta Lake (Delta Live Tables, Unity Catalog, Delta Sharing) Airflow, Azure Data Factory CI/CD pipelines (Azure DevOps / GitHub Actions or equivalent) Cloud Platforms (Azure preferred) BI and analytics frameworks (DBT)

Project duration: 12 months (with possible extension)

Tasks:

- Design, build, and optimize scalable and reusable data pipelines on Databricks, SQL, and Spark
- Orchestrate automated ETL workflows using Airflow, ADF, and CI/CD pipelines
- Implement and manage scalable data processing systemsensuring performance, reliability, and cost-effectiveness
- Ensure data quality, consistency, and compliance with governance/security standards




- Collaborate with stakeholders to clarify requirements and translate them into technical solutions
- Continuously evaluate and implement new tools/technologies to improve pipeline automation, monitoring, and efficiency

Our expectations of the ideal candidate:

- Bachelors or Masters degree in Computer Science, Engineering, or related field
- 6+ years of experience in Data Engineering with solid big data expertise
- Deep expertise in Databricks, Spark, and SQL (including Spark SQL)
- Strong data modeling and ETL experience with large-scale datasets
- Hands-on experience orchestrating data pipelines with Airflow and CI/CD
- Proven experience with Delta Live Tables, Unity Catalog, Delta Sharing, and SQL
- Warehouse (server & serverless)
- Proficiency with Python for data engineering workflows
- Familiarity with analytics engineering and DBT
- Experience with data governance, compliance, and security in a cloud environment
- Strong problem-solving ability, requirement clarification, and communication skills
- Experience adopting software engineering best practices (testing, modularization, refactoring, automation)
- Ability to optimize data systems for performance and cost efficiency in cloud environments
- Exposure to AI/ML frameworks for development efficiency
- Demonstrated ability to lead projects independently and mentor junior engineers

Soft Skills:

- Strong self-management: ability to work independently and meet deadlines
- Proactive and ownership-driven mindset
- Effective and clear communication, including in distributed teams
- Collaborative, open to feedback, and focused on shared goals
- Strong problem-solving skills, especially in ambiguous contexts
- High attention to detail and code quality
- Adaptability to shifting priorities and new technologies

English: Upper-Intermediate or higher Working time zone: IST+EU overlap

📌 Senior Data Engineer (Noida)
🏢 Instinctools India
📍 Noida