09 Sep
|
VARITE
|
Bengaluru
Company Name: VARITE India Private Limited
About The Client:
Client is one of the world’s leading professional services firms and the fastest growing Big Four accounting firm
About The Job:
- We are looking for an experienced Data Engineer with strong Databricks expertise to join our team in a secure Clean Room environment. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and data processing solutions while ensuring compliance with data privacy, security, and governance standards.
Essential Job Functions:
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks.
- Build and optimize data ingestion frameworks for structured, semi-structured, and unstructured data sources.
- Develop data transformation and processing workflows using PySpark and Spark SQL.
- Work with Delta Lake, Unity Catalog, and Databricks ecosystem tools to manage large-scale datasets.
- Collaborate with business stakeholders, data architects, and analytics teams to deliver data solutions.
- Implement data quality checks, monitoring,
and performance optimization techniques.
- Ensure adherence to data governance, security, and privacy requirements in a Clean Room environment.
- Troubleshoot and resolve issues related to data pipelines and platform performance.
- Maintain technical documentation, data lineage, and process workflows.
- Support cloud-based data engineering initiatives and modernization programs.
Qualifications:
- 4-5 years of overall experience in Data Engineering.
- Solid hands-on experience with Databricks.
- Proficiency in PySpark, Python, and Spark SQL.
- Experience working with Delta Lake and Databricks architecture.
- Strong understanding of ETL/ELT processes and Data Warehousing concepts.
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Strong SQL skills and experience with relational and distributed databases.
- Familiarity with workflow orchestration tools such as Databricks Workfl
📌 Data Engineer (Bengaluru)
🏢 VARITE
📍 Bengaluru