**Senior Data Engineer:** **Roles & Responsibilities:** + Design, develop, and maintain scalable **databricks pipelines** to support structured, semi-structured, and unstructured data processing across the Enterprise Data Fabric. + Implement real-time and batch data processing solutions, integrating data from multiple sources into a unified, governed data fabric architecture. + Optimize big data processing frameworks using Apache Spark to ensure high availability and cost efficiency. + Work with metadata management and data lineage tracking tools to enable enterprise-wide data discovery and governance. + Ensure data security, compliance, and role-based access control (RBAC) across data environments. + Optimize query performance, indexing strategies, partitioning, and caching for large-scale data sets. + Develop CI/CD pipelines for automated data pipeline deployments, version control, and monitoring. + Implement data virtualization techniques to provide seamless access to data across multiple storage systems. + Collaborate with cross-functional teams, including data architects, business analysts, and DevOps teams, to align data engineering strategies with enterprise goals. + Stay up to date with emerging data technologies and best practices, ensuring continuous improvement of Enterprise Data Fabric architectures. **Must-Have Skills:** + Develop pipelines using **Databricks (Delta Lake, Spark, notebooks)** + Hands-on experience in data engineering technologies such as **Databricks PySpark** , SQL, and Scaled Agile methodologies. + Proficiency in workflow orchestration, performance tuning on big data processing. + Strong Programming skills and lead teams with technical acumen + Experience with Data Fabric, Data Mesh, or similar enterprise-wide data architectures. + Ability to quickly learn,



adapt and apply new technologies + Strong problem-solving and analytical skills + Excellent communication and teamwork skills + Experience with Scaled Agile Framework (Secure), Agile delivery practices, and DevOps practices. **Preferred / Strategic Skills (Aligned to Future Data Strategy)** + Certification: + Relevant certifications in Databricks, cloud platforms (AWS/Azure/GCP), or modern data engineering technologies are a plus + Experience with: + **Delta Lake / Lakehouse architectures** + **Data Fabric / Data Mesh concepts** + Familiarity with: + **Streaming data (Kafka, event-driven pipelines)** + **Data orchestration tools (Airflow, Databricks Workflows)** + Exposure to: + **AI/ML data pipelines and feature engineering** + **Unstructured data processing** + Understanding of: + **Data governance frameworks and cataloging tools** + **Security and privacy controls for sensitive data** **Good-to-Have Skills:** + Good to have deep expertise in Biotech & Pharma industries + Experience in writing APIs to make the data available to the consumers + Experienced with SQL/NOSQL database, vector database for large language models + Experienced with data modeling and performance tuning for both OLAP and OLTP databases + Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and Dev Ops **Education and Professional Certifications** + 8 to 12 years of Computer Science, IT or related field experience + Databricks Certificate preferred **Soft Skills:** + Excellent analytical and troubleshooting skills. + Strong verbal and written communication skills + Ability to work effectively with global, virtual teams + High degree of initiative and self-motivation. + Ability to manage multiple priorities successfully. + Team-oriented, with a focus on achieving team goals. + Ability to learn quickly, be organized and detail oriented. + Strong presentation and public speaking skills.

📌 Senior Data Engineer (Hyderabad)
🏢 AMGEN
📍 Hyderabad