Career Category
Engineering
Job Description
Senior Data Engineer:
Roles & Responsibilities:
- Design, develop, and maintain scalable databricks pipelines to support structured, semi-structured, and unstructured data processing across the Enterprise Data Fabric.
- Implement real-time and batch data processing solutions, integrating data from multiple sources into a unified, governed data fabric architecture.
- Optimize big data processing frameworks using Apache Spark to ensure high availability and cost efficiency.
- Work with metadata management and data lineage tracking tools to enable enterprise-wide data discovery and governance.
- Ensure data security, compliance, and role-based access control (RBAC) across data environments.
- Optimize query performance, indexing strategies, partitioning, and caching for large-scale data sets.
- Develop CI/CD pipelines for automated data pipeline deployments, version control, and monitoring.
- Implement data virtualization techniques to provide seamless access to data across multiple storage systems.




- Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals.
- Stay up to date with emerging data technologies and best practices, ensuring continuous improvement of Enterprise Data Fabric architectures.

Must-Have Skills:
- Develop pipelines using Databricks (Delta Lake, Spark, notebooks)
- Hands-on experience in data engineering technologies such as Databricks PySpark, SQL, and Scaled Agile methodologies.
- Proficiency in workflow orchestration, performance tuning on big data processing.
- Strong Programming skills and lead teams with technical acumen
- Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures.
- Ability to quickly learn, adapt and apply new technologies
- Solid problem-solving and analytical skills
- Excellent communication and teamwork skills
- Experience with Scaled Agile Fram

📌 Senior Data Engineer (Hyderabad)
🏢 AMGEN
📍 Hyderabad