CAREER CATEGORY

Engineering

JOB DESCRIPTION

Senior Data Engineer:

Roles & Responsibilities:

* Design, develop, and maintain scalable databricks pipelines to support
structured, semi-structured, and unstructured data processing across the
Enterprise Data Fabric.
* Implement real-time and batch data processing solutions, integrating data
from multiple sources into a unified, governed data fabric architecture.
* Optimize big data processing frameworks using Apache Spark to ensure high
availability and cost efficiency.
* Work with metadata management and data lineage tracking tools to enable
enterprise-wide data discovery and governance.
* Ensure data security, compliance, and role-based access control (RBAC) across
data environments.
* Optimize query performance, indexing strategies, partitioning, and caching
for large-scale data sets.
* Develop CI/CD pipelines for automated data pipeline deployments, version
control, and monitoring.
* Implement data virtualization techniques to provide seamless access to data
across multiple storage systems.




* Collaborate with cross-functional teams, including data architects, business
analysts, and DevOps teams, to align data engineering strategies with
enterprise goals.
* Stay up to date with emerging data technologies and best practices, ensuring
continuous improvement of Enterprise Data Fabric architectures.

Must-Have Skills:

* Develop pipelines using Databricks (Delta Lake, Spark, notebooks)
* Hands-on experience in data engineering technologies such as Databricks
PySpark, SQL, and Scaled Agile methodologies.
* Proficiency in workflow orchestration, performance tuning on big data
processing.
* Robust Programming skills and lead teams with technical acumen
* Experience with Data Fabric, Data Mesh, or similar enterprise-wide data
architectures.
* Ability to quickly learn, adapt and apply new technologies
* Strong problem-solving and analytical skills
* Excellent communication and teamwork skills
* Experience with Scaled Agil

📌 Senior Data Engineer (Hyderabad)
🏢 AMGEN
📍 Hyderabad