09 Sep
|
VARITE
|
Bengaluru
Company Name: VARITE India Private Limited
About The Client
Client is one of the world’s leading professional services firms and the fastest growing Big Four accounting firm
About The Job:
- We are looking for an experienced Data Engineer with strong Databricks expertise to join our team in a secure Clean Room environment. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and data processing solutions while ensuring compliance with data privacy, security, and governance standards.
Essential Job Functions:
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks.
- Build and optimize data ingestion frameworks for structured, semi-structured, and unstructured data sources.
- Develop data transformation and processing workflows using PySpark and Spark SQL.
- Work with Delta Lake, Unity Catalog, and Databricks ecosystem tools to manage large-scale datasets.
- Collaborate with business stakeholders, data architects, and analytics teams to deliver data solutions.
- Implement data quality checks, monitoring, and performance optimization techniques.
- Ensure adherence to data governance, security, and privacy requirements in a Clean Room environment.
- Troubleshoot and resolve issues related to data pipelines and platform performance.
- Maintain technical documentation, data lineage, and process workflows.
- Support cloud-based data engineering initiatives and modernization programs.
Qualifications
- 4-5 years of overall experience in Data Engineering.
- Strong hands-on experience with Databricks.
- Proficiency in PySpark, Python, and Spark SQL.
- Experience working with Delta Lake and Databricks architecture.
- Strong understanding of ETL/ELT processes and Data Warehousing concepts.
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Strong SQL skills and experience with relational and distributed databases.
- Familiarity with workflow orchestration tools such as Databricks Workflows, Airflow, etc.
- Experience with Git, CI/CD practices, and deployment automation.
- Good understanding of data security, governance, and compliance requirements.
Preferred Skills
- Experience working in Data Clean Room environments.
- Knowledge of privacy-preserving analytics and secure data-sharing frameworks.
- Experience with Azure Data Factory, Azure Synapse, Snowflake, or similar technologies.
- Databricks Certification (Associate/Professional) is a plus.
- Exposure to DevOps and Infrastructure-as-Code tools.
Education
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related discipline.
Key Competencies
- Robust analytical and problem-solving skills.
- Excellent communication and stakeholder management abilities.
- Ability to work independently and collaboratively in a fast-paced environment.
- Strong attention to detail and commitment to data quality.
- Adaptability and eagerness to learn new technologies.
- Mandatory Skills: Databricks, PySpark, Python, Spark SQL, Cloud (Azure/AWS/GCP), ETL/ELT, Delta Lake.
How to Apply: Interested candidates are encouraged to respond/submit their updated resumes, and for additional job opportunities, please visit Jobs In India – VARITE.
Unlock Rewards: Refer Candidates and Earn.
If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions. VARITE offers a Candidate Referral program, where you'll receive a one-time referral bonus based on the following scale if the referred candidate completes a three-month assignment with VARITE.
Experience Level Bonus Referral:
0-2 years
INR 5,000
2-6 years
INR 7,500
6+ years
INR 10,000
About VARITE: VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA. VARITE is currently a primary and direct vendor to the leading corporations in the verticals of Networking, Cloud Infrastructure, Hardware and Software, Digital Marketing and Media Solutions, Clinical Diagnostics, Utilities, Gaming and Entertainment, and Financial Services.
Equal Opportunity Employer:
VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.
📌 Data Engineer - Databricks (Bengaluru)
🏢 VARITE
📍 Bengaluru