02 Nov
|
Papigen
|
Chennai
We are seeking an experienced Python Data Engineer / Database Developer to join our enterprise data team supporting large-scale data transformation, reconciliation, migration, validation, and reporting initiatives. The ideal candidate will possess strong expertise in Python development, advanced SQL programming, data analysis, data quality management, source-to-target mapping, ETL validation, and Azure-based data solutions. This role will work closely with business users, data architects, application teams, database administrators, project managers, testing teams, and stakeholders to ensure accurate and reliable data movement across multiple enterprise platforms including Murex, FDR, Data Warehouses, Azure Data Platforms, APIs, Reporting Systems, and other internal applications. The successful candidate will play a critical role in analyzing complex data flows, designing reconciliation frameworks, validating business rules, building Python-based automation solutions, developing SQL objects, supporting migration programs, and ensuring data integrity throughout the software development lifecycle.
Key Responsibilities
Python Development & Automation Design, develop, maintain, and enhance enterprise-grade Python applications, reusable libraries, frameworks, and automation solutions. Develop Python-based utilities for large-scale data extraction, transformation, cleansing, validation, comparison, reconciliation, and migration activities. Build effective and scalable solutions utilizing Python libraries such as Pandas, NumPy, OpenPyXL, Requests, SQLAlchemy, PyODBC, XML, JSON, and CSV processing frameworks. Implement object-oriented programming concepts including inheritance, encapsulation, abstraction, and polymorphism. Develop reusable modules and components for enterprise-wide data validation and reconciliation processes. Create automated reporting solutions for business and technical stakeholders. Perform performance tuning and optimization of Python applications handling large data volumes. Implement exception handling, logging frameworks, monitoring capabilities, alerting mechanisms, and troubleshooting procedures. Participate in code reviews and enforce coding best practices and standards. Data Analysis, Profiling & Reconciliation Perform detailed analysis of enterprise data across multiple systems including Murex, FDR, Data Warehouses, APIs, files, and reporting platforms. Conduct source data profiling and target data validation activities. Analyze data trends, anomalies, variances, exceptions, inconsistencies, and data quality issues. Execute reconciliation activities between source and target systems. Identify missing records, duplicate records, data mismatches, aggregation issues,
and transformation defects. Investigate production data discrepancies and perform root cause analysis. Prepare reconciliation reports and exception management dashboards. Establish data quality rules and validation frameworks. Work closely with business teams to interpret data requirements and define acceptance criteria. Database Development & SQL Engineering Design, develop, and support relational databases and analytical data structures. Create And Maintain: Tables Views Materialized Views Stored Procedures Functions Indexes Constraints Sequences Triggers Develop Complex SQL And T-SQL Queries For: Data Validation Data Analysis ETL Validation Reconciliation Migration Data Aggregation Reporting Performance Optimization Analyze and optimize query execution plans. Perform indexing and database tuning activities. Develop reusable database utilities and validation scripts. Support data warehousing and dimensional modeling initiatives. Data Mapping & Migration Activities Develop comprehensive Source-to-Target Mapping (STM) documents. Create field-level mapping specifications. Define transformation logic and business rules. Maintain data dictionaries and metadata documentation. Support data lineage and traceability initiatives. Validate migration data requirements and transformation rules. Participate in migration dress rehearsals and mock migration activities. Monitor production migrations and post-migration verification activities. Document mapping assumptions, business logic, exceptions, and reconciliation checkpoints. ETL & Data Validation Validate end-to-end ETL processes and transformation logic. Perform extraction, transformation, loading, reconciliation, and balancing activities. Support batch processing, scheduled pipelines, and real-time integrations. Design data quality controls and audit mechanisms. Execute data comparison and balancing across environments. Validate inbound and outbound data feeds. Prepare ETL validation evidence and defect reports. Ensure data completeness, consistency, accuracy, and integrity. Testing & Quality Assurance Prepare And Execute: Functional Testing Data Testing ETL Testing Migration Testing System Integration Testing Regression Testing Reconciliation Testing User Acceptance Testing Support Design comprehensive test scenarios and test cases.
Develop expected results and validation criteria. Create SQL validation scripts and reconciliation queries. Track defects and coordinate resolution activities. Maintain test evidence and audit documentation. Participate in release validation and production support activities. Azure Data Engineering Support And Develop Solutions Utilizing: Azure Data Factory (ADF) Azure SQL Database Azure Functions Azure Blob Storage Azure Data Lake Storage Gen2 Azure Key Vault Microsoft Entra ID Azure DevOps Monitor and troubleshoot Azure-based data pipelines. Support CI/CD deployments through Azure DevOps. Configure secure authentication and authorization mechanisms. Automate operational processes using Azure cloud services. Implement monitoring and logging for cloud solutions. Production Support Responsibilities Investigate and resolve production incidents. Troubleshoot data pipeline failures and integration issues. Analyze data quality issues and recommend corrective actions. Review logs and audit records to identify root causes. Participate in on-call support and critical issue resolution. Coordinate with cross-functional teams to ensure system stability. Maintain technical documentation and support procedures. Mandatory Skills & Experience Python Development (Mandatory) Minimum 5+ years of hands-on Python development experience. Strong Knowledge Of: Python Programming Object-Oriented Programming Modular Development Design Patterns Exception Handling Logging Frameworks Debugging Techniques Unit Testing Performance Optimization Experience With: Pandas NumPy OpenPyXL Requests SQLAlchemy PyODBC SQL & Database Technologies (Mandatory) Minimum 5+ years of SQL development experience. Strong Expertise In: SQL T-SQL Stored Procedures Functions Views Indexing Query Optimization Data Modeling Performance Tuning Experience Working With: SQL Server Azure SQL Database Data Warehouses Data Engineering & Analysis (Mandatory) Source-to-Target Mapping Data Lineage Data Governance Data Migration Data Reconciliation Data Profiling Data Validation Data Quality Assessment Data Dictionary Preparation Business Rules Validation Root Cause Analysis Testing Experience (Mandatory) Experience In: ETL Testing Data Testing Reconciliation Testing Functional Testing Migration Testing Integration Testing System Testing Regression Testing UAT Support Preferred Skills Murex FDR Platforms Dremio (TViews/CViews) Azure Data Factory Azure Functions Azure DevOps Power BI Data Warehousing Banking Domain Experience Financial Services Experience Capital Markets Experience Agile Methodologies DevOps Practices
Skills:
ci/cd,fdr,datawarehouse,data structures,pandas,sql,numpy,python,azure,murex
📌 Python Data Engineer (Chennai)
🏢 Papigen
📍 Chennai