- Strong experience in programming languages such as Python, PySpark, or Scala for data engineering tasks.
- Design, develop, and maintain scalable and robust data pipelines on Databricks
- Hands-on experience with cloud technologies, expertise in AWS/Azure data engineering services to handle large-scale data processing and real-time data streams.
- In-depth knowledge of data warehousing solutions such as Amazon Redshift, Google BigQuery, and Snowflake for building and managing data warehouses.
- Proficiency in designing, developing, and maintaining ETL (Extract, Transform, Load) processes using cloud services, ADF, Glue etc
Valuable-to-Have
- Strong understanding of data modeling concepts and techniques to create efficient and scalable data models.
- Experience with version control systems such as Git for code management and collaboration.
- Knowledge of data governance, data quality standards, and data security practices to ensure compliance and protection of sensitive information