CAREER CATEGORY

Engineering

JOB DESCRIPTION

Senior Data Engineer:

Roles & Responsibilities:
Design, develop, and maintain scalable databricks pipelines to support
structured, semi-structured, and unstructured data processing across the
Enterprise Data Fabric.
Implement real-time and batch data processing solutions, integrating data
from multiple sources into a unified, governed data fabric architecture.
Optimize big data processing frameworks using Apache Spark to ensure high
availability and cost efficiency.
Work with metadata management and data lineage tracking tools to enable
enterprise-wide data discovery and governance.
Ensure data security, compliance, and role-based access control (RBAC) across
data environments.
Optimize query performance, indexing strategies, partitioning, and caching
for large-scale data sets.
Develop CI/CD pipelines for automated data pipeline deployments, version
control, and monitoring.
Implement data virtualization techniques to provide seamless access to data
across multiple storage systems.




Collaborate with cross-functional teams, including data architects, business
analysts, and DevOps teams, to align data engineering strategies with
enterprise goals.
Stay up to date with emerging data technologies and best practices, ensuring
continuous improvement of Enterprise Data Fabric architectures.

Must-Have Skills:
Develop pipelines using Databricks (Delta Lake, Spark, notebooks)
Hands-on experience in data engineering technologies such as Databricks
PySpark, SQL, and Scaled Agile methodologies.
Proficiency in workflow orchestration, performance tuning on big data
processing.
Robust Programming skills and lead teams with technical acumen
Experience with Data Fabric, Data Mesh, or similar enterprise-wide data
architectures.
Ability to quickly learn, adapt and apply recent technologies
Solid problem-solving and analytical skills
Excellent communication and teamwork skills
Experience with Scaled Agil

📌 Senior Data Engineer Hyderabad
🏢 AMGEN
📍 Hyderabad