· Develop and maintain data pipelines using Apache Spark and Scala to process large volumes of data efficiently.
· Implement data transformation logic to cleanse, normalize, and aggregate data from various sources, ensuring high data quality and consistency.
· Integrate data from different systems and sources, including structured and unstructured data, databases, and Mainframe systems, ensuring seamless data flow and compatibility.
· Implement data validation and quality checks within data pipelines to ensure data accuracy, consistency, and integrity, and handle exception handling and error logging.
· Develop and execute unit tests, integration tests, and end-to-end tests to ensure the reliability and correctness of Spark applications.
· Debug and troubleshoot issues during development and production phases.
Mandatory Skills
Spark, Databricks
📌 Spark, Databricks (India)
🏢 Infosys
📍 India
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.