What This Job Is
You will be designing and building data pipelines that are both scalable and effective, using tools and technologies like Python, PySpark, and SQL. Your role will include collaborating closely with data scientists and analysts to understand their needs for high-quality datasets. You'll also be responsible for leveraging Azure services such as Databricks and Data Factory to implement solutions that ensure seamless data integration.

What You Will Own
Develop and enhance data pipelines using Python and PySpark to facilitate swift data processing and transformation.
Work with data stakeholders to gather requirements and produce quality datasets tailored to analytics needs.
Utilize Azure technologies, including Azure Data Factory and Databricks, for creating effective data integration solutions.
Optimize processes for data storage and retrieval, focusing on performance efficiency and cost-effectiveness.




Monitor data pipeline performance, performing troubleshooting as needed to maintain data quality.
Keep abreast of industry trends in data engineering for continuous improvement of practices.

Skills That Matter
Robust proficiency in Python and PySpark for handling data processing tasks.
Solid SQL skills for querying databases and managing datasets.
Experience with Azure cloud services, particularly in the context of Databricks and Data Factory.
Familiarity with big data technologies like Spark is essential.
Excellent analytical and problem-solving skills with a keen eye for detail.
Capacity to communicate effectively with different stakeholders while collaborating within a team setting.

📌 Data Engineer Python, Pyspark, Sql Bengaluru
🏢 MEEDEN LABS PRIVATE
📍 Bengaluru