Job Description

Data Engineer – AWS & Databricks

n

Role Overview

n

The Client Data Engineer will be responsible for the design, development, validation, optimization, and ongoing ownership of data solutions primarily using AWS and Databricks . The role will work closely with internal teams, business SMEs, architects, governance teams, Power BI/analytics teams, and implementation partners.

n

The ideal candidate should have 8+ years of experience in data engineering, ETL/ELT, and data integration , with strong hands-on expertise in Databricks, AWS, SQL, Python, PySpark, and Spark .

n

Key Responsibilities

n

- n
- Design, develop, and support AWS and Databricks data pipelines using SQL, Python, PySpark, and Spark.n
- Develop and maintain Bronze, Silver, and Gold data layers using Databricks and Delta Lake.n
- Work with source-system data, business rules, mappings, and integration requirements.n
- Review and validate data solutions delivered by implementation partners.n
- Perform data profiling, reconciliation, testing, and troubleshooting.n
- Support data quality, metadata, lineage, and Unity Catalog activities.n
- Optimize Databricks pipelines and queries for performance, scalability, and cost efficiency.n
- Participate in code reviews, CI/CD, deployment, and production support.n




- Maintain technical documentation and support knowledge transfer to internal teams.n
- Collaborate closely with BI and analytics teams on downstream reporting and data requirements.n
- Support integration with Tableau and Power BI for reporting and analytics use cases.n

n

Required Skills

n

- n
- 8+ years of experience in data engineering, ETL/ELT, or data integration.n
- Strong hands-on experience with Databricks .n
- Strong experience with AWS data services and cloud-based data platforms .n
- Strong SQL and Python skills.n
- Strong experience with PySpark / Apache Spark .n
- Experience designing and implementing data pipelines and data transformation frameworks.n
- Strong understanding of data integration, transformation, reconciliation, and data quality.n
- Robust analytical, troubleshooting, and communication skills.n

n

Preferred Skills

n

- n
- Databricks, Delta Lake, and Unity Catalog .n
- AWS Databricks / Databricks on AWS experience.n
- Medallion/Lakehouse architecture.n
- Experience with Tableau reporting and Power BI .n
- Git, CI/CD, and DevOps practices.n
- Experience with data governance, metadata, lineage, and data quality frameworks.n
- Databricks certification is a plus.n

n

n

📌 Data Engineer (Aurangabad)
🏢 SoftEdge
📍 Aurangabad