Job Description:

We are looking for Senior Data Engineer with 8+Yrs of experience in Hyderabad, Bangalore, Navi Mumbai & Gurugram locations. Candidates Should have experience in Pyspark, Python, SQL & Databricks. Should be proficient in Python Programming.

Location : Navi Mumbai , Hyderabad , Gurugram , Bangalore

Work Mode : All 5 days work from office

The impact you will have:

- Guide Big Data Transformations: Implementation of comprehensive big data projects, including the development and deployment of innovative big data and AI applications.
- Ensure Best Practices: Guarantee that Databricks best practices are applied throughout all projects to maintain high-quality service and successful implementation.
- Support Project Management: Assist the Professional Services leader and project managers with estimating efforts and managing risks within customer proposals and statements of work.
- Architect Complex Solutions: Design, develop, deploy, and document complex customer engagements, either independently or as part of a technical team, serving as the technical lead and authority.
- Enable Knowledge Transfer: Facilitate the transfer of knowledge and provide training to team members, customers, and partners, including the creation of reusable project documentation.
- Contribute to Consulting Excellence: Share expertise with the consulting team and offer best practices for client engagement, enhancing the effectiveness and efficiency of other teams.

Minimum qualifications:

- Educational Background: Bachelors degree in Computer Science, Information Technology, or a related field (or equivalent experience).
- Experience: 8+ years of experience as a Data Engineer, with proficiency in at least two major cloud platforms (AWS, Azure, GCP).
- Proven experience in designing, developing,



and implementing comprehensive data engineering solutions using Databricks, specifically for large-scale data processing and integration projects
- Develop scalable streaming and batch solutions using cloud-native components.
- Perform data transformation tasks, including cleansing, aggregation, enrichment,and normalization, utilizing Databricks and related technologies.
- Experience in applying DataOps principles and implementing CI/CD and DevOps practices within data environments to optimize development and deployment workflows.

Technical Skills:

- Expert-level proficiency in Spark Scala, Python, and PySpark.
- In-depth knowledge of data architecture, including Spark Streaming, Spark Core, Spark SQL, and data modeling.
- Hands-on experience with various data management technologies and tools,such as Kafka, StreamSets, and MapReduce.
- Proficient in using advanced analytics and machine learning frameworks, including Apache Spark MLlib, TensorFlow, and PyTorch, to drive data insights and solutions.

Databricks Specific Skills:

- Extensive experience in data migration from on-premises to cloud environments and in implementing data solutions on Databricks across cloud platforms (AWS,Azure, GCP).
- Skilled in designing and executing end-to-end data engineering solutions using Databricks, focusing on large-scale data processing and integration.
- Proven hands-on experience with Databricks administration and operations, including notebooks, clusters, jobs, and data pipelines.
- Experience integrating Databricks with other data tools and platforms to enhance overall data management and analytics capabilities.

Certifications:

- Certification in Databricks Engineering (Professional)
- Microsoft Certified: Azure Data Engineer Associate
- GCP Certified: Qualified Google Cloud Certified.
- AWS Certified Solutions Architect Professional

📌 Senior Data Engineer (Hyderabad)
🏢 Koantek
📍 Hyderabad