- 4-6 years of experience in data engineering tools and technologies.
- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure/AWS Databricks.
- Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained.
- Willing to work in 2nd shift.
- Willing to work in Hybrid mode.
- Strong knowledge of Data Management principles.
- Good Experience in SQL or any native SQL like query languages.
- Experience in ETL / data warehouse transformation processes.
- Direct experience of building data pipelines using Apache Spark (preferably Databricks).
- Good to have experience in writing code in python.
- Positive to have experience in Power BI.
- Working with other members of the project team to support delivery of additional project components.
- Evaluating the performance and applicability of multiple tools against customer requirements.
- Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.
- Able to work independently.
- Good oral and written communication skill.
Key Responsibilities
- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure/AWS Databricks.
- Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained.
- Support Biz for adhoc issues.
- Create and maintain optimal data pipeline architecture.
- Assemble large, complex data sets that meet business requirements.
- Identify, design, and implement internal process improvements.
- Optimize data delivery and re-design infrastructure for greater scalability.
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS technologies.
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
- Work with internal and external stakeholders to assist with data-related technical issues and support data infrastructure needs.
Skill Requirements
- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure/AWS Databricks.
- Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained.
Other Requirements
📌 Senior Technical Lead (India)
🏢 HCL Technologies
📍 India
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.