Responsibilities:
Design and develop ETL solutions using PySpark and Spark SQL.
Understand business requirements and convert them into technical specifications.
Develop complex data transformation modules and reusable processes.
Perform performance tuning and ensure coding quality standards.
Handle error logging, monitoring, and scheduling activities.
Provide technical solutions and support the team in resolving technical issues.
Perform effort estimation, execution planning, task assignment, and activity tracking.
Work on development and maintenance/support activities as required.