We are looking for a Data Engineer with hands-on experience in Python, PySpark, SQL, and Spark. Experience with Unix, Shell Scripting, and Airflow is preferred.
Key Responsibilities:
- Develop and maintain scalable data pipelines using Python and PySpark.
- Build and optimize data processing solutions using Apache Spark.
- Write and optimize SQL queries for data processing and analysis.
- Troubleshoot data pipeline and production issues.
- Work with Unix/Shell scripting and Airflow for data workflows.