19 Sep
|
Tata Consultancy Services
|
India
19 Sep
Tata Consultancy Services
India
Hi Folks ,
We have Organised a Walkin drive for Below post at Indore office-
Super Corridor Rd, Tigaria Badshah, Indore, Madhya Pradesh 453112
Job Title: AWS Databricks Engineer Exp - 5+ yeas Location-Indore We are seeking a skilled
Data Engineer with deep expertise in AWS and Databricks
to design, build, and optimize our next-generation data platform. In this role, you will implement scalable data pipelines, manage cloud infrastructure, and enable advanced analytics and machine learning capabilities. You will bridge the gap between raw data and actionable business insights by leveraging the power of the Databricks Lakehouse architecture on AWS.
Data Pipeline & Architecture Engineering Design and deploy
robust, scalable data pipelines (ETL/ELT) using Databricks, Apache Spark (Python/Scala/SQL), and AWS services. Implement Lakehouse architectures
utilizing Delta Lake to ensure data reliability, ACID transactions, and high performance. Optimize Spark jobs
for performance, cost, and memory management within Databricks clusters. AWS Cloud Integration Manage and integrate
Databricks with AWS native services including Amazon S3, AWS Glue, IAM, Amazon EMR, RDS, and Redshift. Implement security and governance
controls, ensuring secure data access via Unity Catalog, IAM roles, and encryption (KMS). Operations & Dev Ops Automate deployments
and workflows using CI/CD pipelines (e.g., Git Hub Actions, Git Lab, Jenkins) and Infrastructure as Code (Terraform). Orchestrate workflows
using Databricks Workflows, Apache Airflow, or AWS Step Functions. Monitor platform health , cluster utilization, and cost optimization metrics to ensure efficient cloud spending. Technical Skills Databricks Expertise:
Strong experience with Databricks runtimes, Delta Lake, Unity Catalog, Photon engine, and Databricks SQL. AWS Cloud:
Proficient with core AWS data and infrastructure services (S3, IAM, VPC, EC2, Lambda, Glue). Programming:
Advanced programming skills in Python (PySpark) or Scala, along with expert-level SQL. Data Modeling:
Solid understanding of data warehousing concepts, star/snowflake schemas, and dimensional modeling. Dev Ops/IaC:
Experience with Terraform and version control (Git) for managing data infrastructure. Experience & Qualifications Education:
Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related field. Experience:
3+ years of hands-on experience building production-grade data pipelines on Databricks and AWS. Certifications (Preferred): Databricks Certified Data Engineer Qualified / Associate AWS Certified Data Engineer – Associate or AWS Solutions Architect
📌 Data Engineer (India)
🏢 Tata Consultancy Services
📍 India