You are detailed reviewing and analyzing structured, semi-structured and unstructured data sources for quality, completeness, and business value.

· You design, architect, implement and test rapid prototypes that demonstrate value of the data and present them to diverse audiences.

· You participate in early state design and feature definition activities.

· Responsible for implementing robust data pipeline using Microsoft, Databricks Stack

· Responsible for creating reusable and scalable data pipelines.

· You are a Team-Player, collaborating with team members across multiple engineering teams to support the integration of proven prototypes into core intelligence products.

· You have strong communication skills to effectively convey complex data insights to non-technical stakeholders.

· You have an experience working collaboratively in cross-functional teams and manage multiple projects simultaneously

Requirements

Skills: · Advanced working knowledge and experience with relational and non-relational databases.

· Experience building and optimizing Big Data pipelines, architectures, and datasets.





· Strong analytic skills related to working with structured and unstructured datasets.

· Hands-on experience in Azure Databricks utilizing Spark to develop ETL pipelines.

· Solid proficiency in data analysis, manipulation, and statistical modeling using tools like Spark, Python, Scala, SQL, or similar languages.

· Strong experience in Azure Data Lake Storage Gen2, Azure Data Factory, Databricks, Event Hub, Azure Synapse.

· Familiarity with several of the following technologies: Event Hub, Docker, Azure Kubernetes Service, Azure DWH, API Azure, Azure Function, Power BI, Azure Cognitive Services.

· Azure DevOps experience to deploy the data pipelines through CI/CD.

Qualifications and Experience:

· Minimum 5-7 years of practical experience as Data Engineer.

· Bachelor’s degree in computer science, software engineering, information technology, or a related field.

· Azure cloud stack in-production experience.

📌 Data Engineer (Hyderabad)
🏢 Techcarrot Fz
📍 Hyderabad