s:
Implementing data ingestion pipelines from different types of data sources i.e Databases, S3, Files et
c..Experience in building ETL/ Data Warehouse transformation proce
ss.Experience working with structured and unstructured da
ta.Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/librari
es,Developing scalable and re-usable, self-service frameworks for data ingestion and processi
ng,Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of da
ta,Processing performance analysis and optimizati
on,Bringing best practices in following areas: Design & Analysis, Automation (Pipelining, IaC), Testing, Monitoring, Documentati
on.
Valuable to have (Knowled
ge):
Experience in cloud-based solut
ions,Knowledge of data management princi