11 Nov
|
Propelis
|
India
Experience Required: 8 to 12 Years
Location:
Remote Job Summary: We are looking for a highly skilled and experienced Data Engineer with robust expertise in Databricks, PySpark, and big data technologies. The ideal candidate will play a key role in designing, building, and optimizing scalable data pipelines and analytics solutions to support business decision-making.
Key Responsibilities
• Design, develop, and maintain scalable data pipelines using Databricks and PySpark.
• Work with large-scale structured and unstructured data from various sources.
• Optimize data workflows for performance and reliability.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
• Implement data quality checks, monitoring, and alerting mechanisms.
• Ensure data security and compliance with governance policies.
• Participate in code reviews and contribute to best practices in data engineering.
Required Skills &
Qualifications:
• 8–12 years of experience in data engineering or related roles.
• Strong hands-on experience with Databricks and Apache Spark (PySpark).
• Proficiency in Python and SQL.
• Experience with cloud platforms such as Azure, AWS, or GCP (preferably Azure).
• Solid understanding of ETL/ELT processes, data warehousing, and data modeling.
• Familiarity with Delta Lake and Parquet formats.
• Experience with CI/CD pipelines and version control (e.g., Git).
• Excellent problem-solving and communication skills. Preferred
Qualifications:
• Knowledge of data governance, security, and compliance standards.
• Exposure to real-time data processing using tools like Kafka or Spark Streaming.
📌 Senior Data Engineer – Databricks (India)
🏢 Propelis
📍 India