- Leverage AWS services to ingest data from source systems within their business domain
- Leverage databricks platform to process, clean and transform data
- Ensure and measure data quality according to business requirements and tracked KPIs
- Monitor and maintain existing data product such as data ingestion pipeline and ETLs
- Alert and act on data quality issues and system incidents within their business domain
- Ensure that data is available, accurate and fit for purpose following governance guidelines and compliance policies
- Ensure that knowledge of data mesh framework is shared
- Write and publish clear documentation of approaches and functionalities
- Take part in all agile ceremonies (planning, backlog refinement, retrospective, stand ups…) and necessary team meetings
- Make all artifacts available through version control tool such as git
- Be part of the review process both as a reviewer as well as a submitter (PRs, architecture decisions, approaches….)
Skill Set
- 5 to 9 years of Strong experience developing software in an Agile setting following best practices
- Experience with AWS cloud platform
- Ability to write production grade python and Scala code
- Ability to leverage Spark through Databricks platform (PySpark or Scala)
- Ability to create and automate data ingestion pipelines for both batch and streaming data
- Experience with monitoring and alerting tooling such as AWS Cloudwatch, Datadogs…
- Experience deploying software using orchestration tools such as Jenkins
📌 Data Engineer (Bangalore Urban)
🏢 CoffeeBeans
📍 Bangalore Urban
Reply to this offer
Impress this employer describing Your skills and abilities, fill out the form below and leave Your personal touch in the presentation letter.