- Design, build, and optimize scalable data pipelines using Azure Databricks and PySpark.
- Develop effective data models for large-scale datasets.
- Build batch and real-time data processing workflows.
- Work with Azure Data Lake and Azure Data Factory.
- Perform PySpark performance tuning and optimization.
- Develop data-processing solutions using Python and SQL.
- Optimize complex SQL queries for better performance.
- Follow strong coding, testing, and software engineering practices.
- Support CI/CD and automation for data applications.
- Good to Have: GitLab, CI/CD, PowerShell/Bash/Perl, Power BI, Agile/Scrum, and financial domain experience.