Responsibilities

Direct Responsibilities

- Design & developrobust ingestion, transformation, and enrichment pipelines withPython,PySpark, andSQL.

- Write and optimizecomplex SQL queries, analytical UDFs, and window functions for data aggregation and reporting.

- Collaboratewith CEFS data architects, data scientists, and business analysts to translate functional requirements into technical specifications.

- Unittest, integratetest, and reviewcode.

- Maintain CI/CD pipelines(Git, Jenkins, Docker) for automated build, test, and deployment of jobs.

- Monitorproduction workloads and troubleshoot performance bottlenecks, memory issues, and job failures.

- Documentdata lineage, pipeline design, and operational runbooks in Confluence/SharePoint.

- Keep up to date with latest technologies and trends and provide input, expertise and recommendations.

Contributing Responsibilities

- Contribute towards innovation (e.g. AI/ML); suggest new technical practices for efficiency improvement.

- Participate inAgile ceremonies(sprint planning, daily standups, retrospectives) and help groom the backlogs.

- Mentor junior engineers and champion best practices in Python coding, Spark optimization, and dataengineering patterns.

- Evaluate emerging technologies and deliver proofofconcepts for CEFS.

Technical & Behavioral Competencies

- Resourceful to quickly understand complexities involved and provide the way forward.





- Valuable experience in technical analysis of n-tier applications with multiple integrations using object oriented, APIs & Microservices approaches.

- Strong knowledge about design patterns and development principles.

- Inclination and prior experience of working across SQL, Python and ETL.

- Strong Hands-on experience - Python (NumPy, pandas, Python Frameworks, Restful APIs, MS-SQL or Oracle.

- PySpark- DataFrames, Spark SQL, Structured Streaming, performance tuning (partitioning, caching, broadcast joins).

- Advanced SQL complex queries, stored procedures, query optimization.

- Positive Knowledge and experience to use Python packages such as Pandas, NumPy, etc. for cleaning up of Data, Data Wrangling, Analysis of Data and Visualization of Data.

- Good experience in development and maintenance of code/scripts in both functional and technical specifications of all applications component, bug fixing and production support.

- Good knowledge on Linux/Unix environment (basic commands, shell scripting, etc.), testing phases, documentation and new framework.

- Some experience of working with build tools like Maven & DevOps tools like Bitbucket, Jenkins.

- Knowledge of Agile, Scrum, DevOps.

- Development experience in Data Engineering environment.

- Ability & willingness to learn & work on diverse technologies (languages, frameworks, and tools).

- Self-motivated, good interpersonal skills and inclination to constantly upgrade new technologies and frameworks.

- Good communication and co-ordination skills.

📌 Python Software Developer (Chennai)
🏢 hackajob
📍 Chennai