19 Nov
|
Witqualis Technologies
|
Noida
19 Nov
Witqualis Technologies
Noida
Job Summary
Architect and build enterprise Lakehouse solutions, high-volume ETL/ELT pipelines, and automated DataOps workflows utilizing Databricks, AWS, and GitLab CI/CD.
Responsibilities
- Design, develop, and maintain scalable data pipelines and Delta Lake architectures on Databricks.
- Implement automated DataOps CI/CD workflows using GitLab for continuous testing, validation, and multi-workplace deployment.
- Integrate Databricks seamlessly with AWS cloud storage, data lakes, and enterprise analytical warehouses.
- Optimize Databricks cluster configurations, query performance, and compute costs.
- Ensure enterprise-grade data security, data lineage, quality checks, and regulatory compliance across data pipelines.
Requirements
- 5-10+ years of specialized experience in Data Engineering with deep expertise in Databricks, Apache Spark, and Delta Lake.
- Strong proficiency with AWS data ecosystem (S3, Glue, EMR,
Redshift, Athena, IAM) and cloud data warehousing.
- Hands-on experience implementing DataOps and CI/CD pipelines using GitLab CI, Git, and automated testing suites.
- Proficiency in PySpark, Python, advanced SQL, and data orchestration tools (Airflow / Databricks Workflows).
- Expertise in data lakehouse design, data governance, performance tuning, and cluster cost optimization.
Skills
- Databricks
- AWS
- DevOps
- Git
- CI/CD
- GitLab
- PySpark
- Delta Lake
- SQL
Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.
📌 Databricks Data Engineer (Noida)
🏢 Witqualis Technologies
📍 Noida