Data Engineer – AWS & Databricks

nRole Overview

nThe Client Data Engineer will be responsible for the design, development, validation, optimization, and ongoing ownership of data solutions primarily using AWS and Databricks . The role will work closely with internal teams, business SMEs, architects, governance teams, Power BI/analytics teams, and implementation partners.

nThe ideal candidate should have 8+ years of experience in data engineering, ETL/ELT, and data integration , with solid hands-on expertise in Databricks, AWS, SQL, Python, PySpark, and Spark .

nKey Responsibilities

n

n
- Design, develop, and support AWS and Databricks data pipelines using SQL, Python, PySpark, and Spark.
n
- Develop and maintain Bronze, Silver, and Gold data layers using Databricks and Delta Lake.
n
- Work with source-system data, business rules, mappings, and integration requirements.
n
- Review and validate data solutions delivered by implementation partners.
n
- Perform data profiling, reconciliation, testing, and troubleshooting.
n
- Support data quality, metadata, lineage, and Unity Catalog activities.
n
- Optimize Databricks pipelines and queries for performance, scalability, and cost efficiency.
n
- Participate in code reviews, CI/CD, deployment, and production support.
n




- Maintain technical documentation and support knowledge transfer to internal teams.
n
- Collaborate closely with BI and analytics teams on downstream reporting and data requirements.
n
- Support integration with Tableau and Power BI for reporting and analytics use cases.
n

nRequired Skills

n

n
- 8+ years of experience in data engineering, ETL/ELT, or data integration.
n
- Strong hands-on experience with Databricks .
n
- Strong experience with AWS data services and cloud-based data platforms .
n
- Strong SQL and Python skills.
n
- Strong experience with PySpark / Apache Spark .
n
- Experience designing and implementing data pipelines and data transformation frameworks.
n
- Strong understanding of data integration, transformation, reconciliation, and data quality.
n
- Strong analytical, troubleshooting, and communication skills.
n

nPreferred Skills

n

n
- Databricks, Delta Lake, and Unity Catalog .
n
- AWS Databricks / Databricks on AWS experience.
n
- Medallion/Lakehouse architecture.
n
- Experience with Tableau reporting and Power BI .
n
- Git, CI/CD, and DevOps practices.
n
- Experience with data governance, metadata, lineage, and data quality frameworks.
n
- Databricks certification is a plus.
n

n

n

📌 Data Engineer (Madurai)
🏢 SoftEdge
📍 Madurai