Coherent Pixel Systems is Hiring!

Position: Consultant – Data Engineer

Experience: 6–9 Years

Location: Fully Remote

Work Mode: Remote

Shift: 1:00 PM – 10:00 PM IST

We are looking for an experienced Data Engineer with robust hands-on expertise in Databricks, Apache Spark/PySpark, Data Mesh, Unity Catalog, Python, and SQL.

The role will focus on building scalable Data Domains and Data Products, along with data discovery, governance, platform administration, and CI/CD.

Responsibilities

- Design and build Data Domains and Data Products using Data Mesh principles.
- Develop scalable data pipelines using Databricks, Spark/PySpark, Python, and SQL.
- Implement data governance, access controls, metadata, and lineage using Unity Catalog.
- Perform independent data discovery, profiling, and source analysis.
- Optimize Spark/Databricks workloads for performance and scalability.




- Build and maintain GitLab CI/CD pipelines for data engineering deployments.
- Support Databricks platform administration, including compute, jobs, permissions, and monitoring.
- Collaborate with business and technical teams to translate requirements into data solutions.

Requirements

- Strong hands-on experience with Databricks and Apache Spark/PySpark.
- Strong knowledge of Data Mesh, Data Domains, and Data Products.
- Hands-on experience with Databricks Unity Catalog.
- Strong Python and SQL skills.
- Experience with data discovery, data profiling, and data mapping.
- Experience with GitLab and CI/CD.
- Experience with Databricks Platform Administration.
- Good understanding of Delta Lake/Lakehouse architecture, data governance, security, and performance optimization.

Work Location: In person

📌 Consultant – Data Engineer (Chennai)
🏢 Coherent Pixel Systems
📍 Chennai