BE | BTech | MCA | OR equivalent degree in Computer Science, IT or a related field candidates having 5 to 10 years of experience in Python, PySpark, SQL Unix + Big Data (Hadoop + Hive), Airflow, Datawarehousing, Data modelling.Required skills :- Strong hands-on experience in Python Development.- Extensive experience with PySpark / Apache Spark.- Expertise in Data Engineering, SQL Unix + Big Data (Hadoop + Hive), Airflow, Datawarehousing, Data modelling and data processing frameworks.- Strong knowledge of SQL and database concepts.- Experience working with large-scale datasets in distributed environments.- Understanding of Data Warehousing concepts and dimensional data modeling.- Exposure to cloud platforms such as Azure, AWS, or GCP.- Experience with version control tools such as Git.- Experience with Kubernetes, Java or python + Spark SQL, Delta Lake.- Knowledge of Airflow, ADF, or other orchestration tools.- Experience in CI/CD implementation.- Familiarity with SAP BW/4HANA, SAP S/4HANA, or SAP Banking solutions.SAP Skills :- Exposure to SAP FSDM/FPSL.- Exposure to SAP Financial Services Data Management (FSDM).- Understanding of SAP FSPL (Financial Services Package for Liquidity) + Big Data (Hadoop + Hive) processes and data structures.- Knowledge of financial services, banking, risk, liquidity,



or regulatory reporting domains is preferred.Role & Responsibilities :- Partner closely with Product Managers, Data Scientists, and AdOps teams in an Agile/Scrum environment to translate ambiguous business requirements into scalable, production-grade ETL/ELT data pipelines and dimensional data models.- Design and maintain end-to-end cloud Lakehouse architectures on Python using PySpark, Databricks, and Delta Lake, establishing standardized frameworks for automated data ingestion, schema enforcement, and data reconciliation.- Manage production workflow orchestration using Apache Airflow, building automated failure recovery, log monitoring, and SLA-driven alerting to guarantee 99.9% data pipeline uptime and reliable downstream analytics delivery.- Mentored junior data engineers through pair programming and code reviews; established team-wide engineering best practices around PySpark performance tuning, Git version control, and CI/CD pipelines.- Conducted routine cluster cost optimizations and query tuning across cloud infrastructure, adjusting Spark driver/executor configurations, energetic allocation, and storage layouts to maximize compute efficiency. (ref:hirist.tech)

📌 PySpark/Big Data Developer (Chennai)
🏢 L A Consultancy
📍 Chennai