- Design, develop, build, and maintain scalable data ingestion and transformation pipelines using Python, SQL, PySpark, and ETL/ELT frameworks.

- Develop and manage batch and streaming data processing workflows to support business and analytical requirements.

- Work with contemporary data platforms such as Databricks, Snowflake, Microsoft Fabric, AWS, or Google Cloud.

- Build and maintain data lakes, lakehouses, and data warehouse solutions using Delta Lake, Parquet, and other modern storage formats.

- Implement data modelling, data validation, data quality checks, and data reconciliation processes.

- Develop and optimise SQL queries, Spark jobs, and data processing workflows for improved performance.

- Implement workflow orchestration using Apache Airflow or equivalent orchestration tools.

- Work with dbt for data transformation, testing, and documentation.

- Understand Kafka fundamentals and support event-driven and real-time data ingestion requirements.

- Implement incremental data loading, partitioning strategies, and schema evolution.

- Monitor pipeline execution, troubleshoot failures, identify root causes, and ensure data pipeline reliability.

📌 Job Opening For Data Platform Engineer (Bengaluru)
🏢 Qapitol QA
📍 Bengaluru