Job Summary:

We are looking for a skilled Data Engineer to design, develop, and optimize enterprise-grade data pipelines within an on-premises, containerized data platform environment. The ideal candidate should have strong expertise in Apache Spark, Python, SQL/PL-SQL, and container orchestration technologies to support scalable data ingestion, transformation, and analytics workloads.

The role involves building and maintaining robust data solutions that enable Business Intelligence, Analytics, AI/ML initiatives, and regulatory reporting across the organization.

Key Responsibilities:

- Design, develop, and maintain scalable ETL/ELT pipelines using Apache Spark (PySpark/Scala).
- Develop data ingestion frameworks for structured, semi-structured, and unstructured data sources.
- Deploy and manage containerized data workloads within OpenShift or Kubernetes environments.
- Build and maintain Lakehouse architectures using Medallion Architecture (Bronze, Silver, Gold layers).
- Implement data quality validation, monitoring, and alerting mechanisms.
- Optimize Spark workloads for performance, scalability, and resource efficiency.
- Develop and manage workflow orchestration using Apache Airflow.
- Collaborate with Business, Analytics, DevOps, and Data Governance teams to deliver enterprise data solutions.
- Ensure adherence to data governance, compliance, and security standards.
- Participate in code reviews, troubleshooting,



and performance tuning activities.

Required Skills

Technical Skills:

- Robust experience in Python programming.
- Hands-on experience with Apache Spark (PySpark preferred; Scala is an advantage).
- Expertise in SQL, PL/SQL, and relational databases such as Oracle and PostgreSQL.
- Experience with Red Hat OpenShift (OCP) or Kubernetes.
- Knowledge of Apache Airflow for workflow orchestration.
- Experience with Git and CI/CD pipelines.
- Understanding of Data Warehousing concepts and Lakehouse architecture.
- Familiarity with Delta Lake or equivalent open-source storage formats.
- Experience with Data Quality frameworks and monitoring solutions.

Preferred Skills:

- Exposure to Medallion Architecture (Bronze, Silver, Gold).
- Experience working in large-scale enterprise data environments.
- Certifications in Spark, Kubernetes/OpenShift, Python, or Data Engineering technologies will be an added advantage.

Qualifications:

- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- Relevant certifications in Data Engineering technologies are preferred.

Soft Skills:

- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management abilities.
- Ability to work independently as well as in cross-functional teams.
- Strong ownership mindset with a focus on quality and timely delivery.

📌 Data Engineer (Ahmedabad)
🏢 Team Computers
📍 Ahmedabad