Description
We are seeking a highly skilled Data Engineer with expertise in Python, PySpark, SQL, Azure, and Microsoft Fabric. The successful candidate will be responsible for designing, developing, and maintaining scalable data pipelines and data architectures across cloud-based platforms.
Responsibilities:
• Collaborate with cross-functional teams to design, develop, and implement scalable data solutions using Python, PySpark, SQL, and Azure services.
• Design, develop, and maintain robust ETL/ELT pipelines for efficient data ingestion, transformation, and integration.
• Leverage Azure Data Factory, Databricks, Microsoft Fabric to build and maintain modern data architectures such as medallion architecture.
• Work closely with Power BI developers and data analysts to enable seamless data integration for reporting and analytics.
• Participate in all phases of the Software Development Lifecycle (SDLC) including requirement gathering, development, testing, and deployment.
• Write effective, scalable, and optimized code to handle large volumes of structured and unstructured data.
• Ensure data quality, consistency, and reliability across data pipelines and storage systems.
• Collaborate with data scientists, analysts, and product teams to solve business and technical challenges.
• Conduct code reviews and provide constructive feedback to team members.
• Troubleshoot, debug, and resolve data pipeline and performance issues.
• Stay up to date with industry trends and best practices in data engineering and cloud technologies.
Requirements:
• Strong experience in data engineering using Python, PySpark, and SQL.
• Proven expertise in designing and implementing ETL/ELT pipelines for data ingestion, transformation, and integration.
• Solid understanding of relational databases with proficiency in SQL, including creating views and stored procedures.
• Hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
• Experienc
📌 Associate Consultant (Bengaluru)
🏢 BSR
📍 Bengaluru