17 Sep
|
VARITE
|
Bengaluru
Company Name: VARITE India Private Limited
About The Client
Client is one of the world’s leading professional services firms and the fastest growing Big Four accounting firm
About The Job:
- We are seeking a skilled Data Engineer (Databricks) to join our team and build scalable, high-performance data solutions.
- The ideal candidate will have hands-on experience designing and maintaining ETL/ELT pipelines on Databricks using PySpark, Spark SQL, and Delta Lake, with a strong foundation in modern data warehousing and cloud-based data platforms.
Essential Job Functions:
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Spark SQL, and Delta Lake.
- Implement and optimize data ingestion, transformation, and data quality processes across large-scale datasets.
- Build and maintain data models following fact/dimension modeling, Slowly Changing Dimensions (SCDs), and incremental loading strategies.
- Develop solutions aligned with Medallion Architecture (Bronze, Silver, Gold layers).
- Orchestrate, monitor, troubleshoot, and performance-tune Databricks Workflows to ensure reliable production execution.
- Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
- Manage data governance, security, and access controls using Unity Catalog.
- Integrate Databricks with cloud storage services on AWS or Azure.
- Collaborate with business stakeholders, data analysts, and engineering teams to translate requirements into robust data solutions.
- Ensure adherence to best practices around data quality, testing, monitoring, and documentation.
Qualifications
Must-Have
- 2-4 years of experience in Data Engineering with strong exposure to Databricks.
- Hands-on expertise in:
- PySpark
- Spark SQL
- Delta Lake
- Databricks Workflows
- Strong knowledge of
- Data Warehousing Concepts
- Fact & Dimension Modeling
- Slowly Changing Dimensions (SCD Type 1 & 2)
- Incremental Data Processing
- Medallion Architecture
- Advanced SQL programming skills.
- Experience with Unity Catalog for governance and access management.
- Experience integrating Databricks with AWS S3, ADLS Gen2, Azure Blob Storage, or similar cloud storage services.
- Good understanding of data pipeline monitoring, troubleshooting, and optimization techniques.
Good-to-Have
- Experience with CI/CD implementation for Databricks deployments.
- Exposure to Airflow, ADF, or other orchestration tools.
- Knowledge of data quality frameworks and monitoring solutions.
- Databricks Certification is a plus.
Preferred Candidate Profile
- Strong problem-solving and analytical skills.
- Ability to work in a fast-paced client-facing workplace.
- Excellent communication and stakeholder management skills.
- Self-driven professional capable of independently delivering data engineering solutions.
- Experience Range: 2-4 Years
How to Apply: Interested candidates are encouraged to respond/submit their updated resumes, and for additional job opportunities, please visit Jobs In India – VARITE.
Unlock Rewards: Refer Candidates and Earn.
If you're not available or interested in this opportunity, please pass this along to anyone in your network who might be a good fit and interested in our open positions. VARITE offers a Candidate Referral program, where you'll receive a one-time referral bonus based on the following scale if the referred candidate completes a three-month assignment with VARITE.
Experience Level Bonus Referral:
0-2 years
INR 5,000
2-6 years
INR 7,500
6+ years
INR 10,000
About VARITE: VARITE is a global staffing and IT consulting company providing technical consulting and team augmentation services to Fortune 500 Companies in USA, UK, CANADA and INDIA. VARITE is currently a primary and direct vendor to the leading corporations in the verticals of Networking, Cloud Infrastructure, Hardware and Software, Digital Marketing and Media Solutions, Clinical Diagnostics, Utilities, Gaming and Entertainment, and Financial Services.
Equal Opportunity Employer:
VARITE is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, marital status, veteran status, or disability status.
📌 Data Engineer (Databricks) (Bengaluru)
🏢 VARITE
📍 Bengaluru