Company Name: VARITE India Private Limited
About The Client:
Client is a pioneering Software Engineering and IT consultancy company, transforming and executing at the intersection of Domain and Technology to create digital leaders for our people, clients, partners, and communities.
About The Job:
- We are looking for a highly skilled Senior Big Data Engineer with 8+ years of overall IT
- experience and strong expertise in Azure, Databricks, DevOps, and large-scale data
- engineering. The ideal candidate will have deep experience building scalable data pipelines,
- optimizing Spark workloads, managing cloud infrastructure, and contributing to robust DevOps
- practices.
Essential Job Functions:
Big Data Engineering
- Design, develop, and optimize Spark/Scala jobs to process large-scale structured and
- unstructured datasets.
- Perform data profiling, data mining, and exploratory data analysis to support business
- decisions.
- Build and maintain complex ETL/ELT pipelines involving formats such as CSV, TSV,
- XML, JSON, fixed-width, delimited, and multi-record file structures.
- Create scalable data processing solutions for integrating new data sources and enabling
- recent product features.
Qualifications:
- Azure Cloud & Data Services
Work extensively with Azure services such as:
- Azure Data Lake
- Azure Databricks
- Azure Functions
- Azure Blob Storage
- Azure VMs
- Azure SQL & PaaS services
- Strong understanding of Azure networking, identity management, monitoring, and alerting.
Databricks & Unity Catalog
- Manage and configure Databricks infrastructure, including workspace setup, cluster
- governance, CI/CD integration, and environment segregation.
Expert-level experience with Unity Catalog, including:
- Data lineage
- Access control
- Metastores
- Table management and auditing
DevOps & Automation
Hands-on experience with Azure DevOps:
- Creating and maintaining YAML-based ADO pipelines
- Working within automated release pipelines
- Strong working knowle
📌 Senior Big Data Engineer (India)
🏢 VARITE
📍 India