Responsibilities

- Actively participate in Agile ceremonies, including requirements refinement, sprint planning, effort estimation, sprint reviews, and retrospectives, to support successful delivery of platform capabilities.
- Participate in technical design discussions, analyze requirements, and translate complex business needs into scalable data platform solutions.
- Design, develop, and optimize core enterprise data platform capabilities, including data acquisition, ingestion, transformation, workflow orchestration, and reusable processing frameworks, leveraging Python, SQL, cloud-native technologies, and cloud data warehouse platforms to support large-scale, high-performance data processing.
- Build and optimize large-scale data pipelines supporting data ingestion, transformation, validation, and distribution across enterprise cloud environments.
- Implement metadata-driven processing frameworks and reusable engineering patterns that improve automation, standardization, scalability, and reuse.
- Build and enhance platform capabilities related to metadata management, lineage, dependency tracking, data quality, governance, and workflow control.
- Incorporate Agentic AI capabilities into platform services to enhance automation, intelligent decision-making, developer productivity, and operational efficiency.
- Partner with cross-functional teams to support collaborative development, system integration, and end-to-end testing across platform components and dependent systems.
- Develop and maintain system, integration, regression, and test automation frameworks to ensure platform quality, reliability, and production readiness.
- Support deployment and release activities, ensuring successful rollout of platform enhancements and seamless integration with dependent systems and downstream consumers.
- Contribute to engineering standards, architectural patterns, operational best practices, and framework conventions that improve platform maintainability, consistency, scalability, and supportability.
- Provide L2/L3 production support for data platforms and products, perform root cause analysis, and implement corrective actions to ensure platform stability and operational excellence.
- Drive continuous improvements in platform performance, scalability, resilience, observability, and operational efficiency across data workflows and framework components.
- Lead Proof of Concepts (POCs)



and technology evaluations to assess emerging technologies, validate architectural approaches, and identify opportunities for platform modernization and innovation.
- Support product documentation by providing technical expertise, implementation details, and content validation to ensure accuracy, completeness, and alignment with platform capabilities.
- Conduct live product demonstrations and participate in user workshops to showcase platform capabilities, gather feedback, validate solutions, and support platform adoption

Required Qualifications

- Bachelors degree in Computer Science, Information Systems, or a related technical field.
- 8+ years of experience in Data Engineering, Data Platform Engineering, or a related software engineering role.
- Hands-on experience with Apache Airflow for orchestrating complex, dependency-driven data pipelines and workflows.
- Strong proficiency in Python, with hands-on experience developing scalable, production-grade data pipelines and frameworks.
- Hands-on experience designing and building ETL/ELT frameworks, reusable data pipeline components, and workflow-driven data systems.
- Hands-on experience with Snowflake or equivalent cloud-native analytical data platforms.
- Strong hands-on experience writing and optimizing complex SQL, including stored procedures, UDFs, and performance-critical queries.
- Proven experience in query optimization, performance tuning, and workload management for large-scale data environments.
- Extensive hands-on experience with dbt, including data modeling, transformation, testing, documentation, and deployment within contemporary data platforms.
- Proven track record of improving the performance, reliability, scalability, and maintainability of enterprise data platforms and pipelines.
- Hands-on experience with streaming technologies (e.g., Snowpipe, Kafka) and messaging platforms for building scalable, real-time data processing solutions.
- Hands-on experience building solutions in cloud-native and distributed environments, leveraging object storage,



cloud data services, and major cloud platforms such as Azure, AWS, or GCP.
- Exposure to Apache Iceberg, leveraging schema evolution, partition evolution, and time-travel capabilities to build scalable and reliable data lake solutions.
- Experience with SDK and API-based development, enabling seamless integration of data platforms, cloud services, and enterprise applications through secure and scalable interfaces.
- Hands-on experience designing and developing cloud-native data platforms, including Data Lake and Data Warehouse architectures, using modern data engineering best practices.
- Exposure to AI-native development tools such as Windsurf, Cursor, and Antigravity, leveraging AI-assisted coding, testing, and automation to improve engineering productivity and software delivery.
- Understanding of Agentic AI architectures, including Skills, Model Context Protocol (MCP), Vector Databases, Retrieval-Augmented Generation (RAG), and AI Agent orchestration concepts.
- Experience working with Docker, Kubernetes, and containerized deployment architectures in modern engineering environments.
- Strong understanding of software engineering principles, scalable system design, and modern development practices, including Agile methodologies, Git-based source control, CI/CD pipelines, code reviews, and production-grade development standards.
- Experience implementing and managing metadata, lineage, dependency management, data quality controls, observability, and operational governance within enterprise data ecosystems.
- Experience driving technical design decisions, decomposing complex initiatives, mentoring engineers, and collaborating effectively with cross-functional engineering and business stakeholders.
- Excellent verbal and written communication skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.
- Strong analytical, critical-thinking, and problem-solving skills, with a proactive, solution-oriented mindset and the ability to apply innovative, out-of-the-box thinking to solve complex challenges.
- Adaptable and collaborative team player with a continuous learning mindset and passion for emerging technologies.
- Proven ability to mentor and guide engineers, fostering technical excellence, knowledge sharing, and engineering best practices across teams.

📌 Data Engineer (Pune)
🏢 hackajob
📍 Pune