Role & responsibilities

- Design, build, and maintain scalable ETL/ELT pipelines using Apache Spark, PySpark, and Spark SQL.

- Develop and optimize complex SQL queries, with a focus on performance tuning and cost efficiency.

- Orchestrate and schedule workflows using Airflow, Dagster, or similar platforms.

- Build and manage data workloads on Databricks or comparable Spark-based platforms.

- Design data models, warehouse layers, and lakehouse structures that support analytics and reporting.

- Work with Parquet, Delta Lake, and/or Apache Iceberg table formats.

- Implement data quality checks, metadata management, lineage tracking, and governance practices.

- Own production workloads: monitor jobs, troubleshoot failures, and perform root-cause analysis to prevent recurrence.

- Collaborate with analysts, engineers, and business stakeholders to understand requirements and deliver trusted data.

- Document pipelines, data models, and operating procedures.

Preferred candidate profile

- Strong experience with Apache Spark,



PySpark, and Spark SQL.

- Proficiency in Python, Java, or Scala.

- Solid SQL skills, including query optimization and performance tuning.

- Proven experience building and operating production-grade ETL/ELT pipelines.

- Experience with Airflow, Dagster, or similar orchestration tools.

- Hands-on experience with Databricks or a comparable Spark-based platform.

- Solid understanding of data modeling, data warehousing, and lakehouse concepts.

- Experience with Parquet, Delta Lake, and/or Apache Iceberg.

- Familiarity with data quality, metadata, lineage, and governance practices.

- Experience owning production workloads, troubleshooting issues, and performing root-cause analysis.

- Strong analytical, problem-solving, and communication skills.

- Ability to work independently while collaborating effectively across teams.

📌 Data Analyst (Bengaluru)
🏢 Moolya Software Testing
📍 Bengaluru