What you will be doing...
• Leverage AWS services to ingest data from source systems within their business domain
• Leverage databricks platform to process, clean and transform data
• Ensure and measure data quality according to business requirements and tracked KPIs
• Monitor and maintain existing data product such as data ingestion pipeline and ETLs
• Alert and act on data quality issues and system incidents within their business domain
• Ensure that data is available, accurate and fit for purpose following governance guidelines and compliance policies
• Ensure that knowledge of data mesh framework is shared
• Write and publish clear documentation of approaches and functionalities
• Take part in all agile ceremonies (planning, backlog refinement, retrospective, stand ups…) and necessary team meetings
• Make all artifacts available through version control tool such as git
• Be part of the review process both as a reviewer as well as a submitter (PRs, architecture decisions, approaches….) Skill Set :
• 5 to 9 years of Strong experience developing software in an Agile setting following best practices
• Experience with AWS cloud platform
• Ability to write production grade python and Scala code
• Ability to leverage Spark through Databricks platform (PySpark or Scala)
• Ability to create and automate data ingestion pipelines for both batch and streaming data
• Experience with monitoring and alerting tooling such as AWS Cloudwatch, Datadogs…
• Experience deploying software using orchestration tools such as Jenkins
📌 Data Engineer (India)
🏢 CoffeeBeans
📍 India
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.