We are seeking a talented and motivated DataBricks expert to join our team as a Senior Programmer Analyst.
Required Skills & Qualifications
• 3 to 5 years of hands-on experience in data engineering, software development, or cloud data platform delivery.
• Expert-level Apache Spark (PySpark) skills.
• Robust proficiency in SQL.
• Solid programming skills in Python for automation and data processing.
• Hands-on experience with a major cloud platform such as Microsoft Azure (Azure Data Factory, ADLS) or AWS.
• Experience with workflow orchestration tools like Databricks Workflows, Delta Live Tables (DLT), or Apache Airflow.
• Experience with Infrastructure-as-Code (Terraform).
• Experience with containerization tools (Docker / Kubernetes).
• Build and support Databricks data pipelines for enterprise analytics.
• Understanding of the SDLC and Agile methodologies.
• Communication with the customer and producing the daily status report.
• Bachelor's or Master's degree in Computer Science,
Engineering, or a related quantitative field.
• Databricks Certified Data Engineer Associate or Professional certification.
• Should have positive oral and written communication.
• Should be a valuable team player and able to work with business stakeholders.
• Should be proactive and adaptive.
Key Responsibilities
• Design, build, and maintain scalable ETL/ELT data pipelines using Databricks and Apache Spark to support enterprise analytics.
• Transform, validate, and optimize large datasets using PySpark and advanced SQL.
• Implement and maintain Delta Lake and the lakehouse/medallion architecture (bronze, silver, gold layers).
• Work with data analysts, data scientists, and business stakeholders to define data requirements and deliver trusted datasets.
• Implement DevOps practices, version control (Git), and deployment automation for data pipelines.
• Apply access controls, data quality frameworks, and data gove