20 Nov
|
HCLTech
|
Chennai
Job Summary
We are seeking a highly skilled Data Operations Engineer with solid expertise in Databricks, Informatica, and SQL to support enterprise data platforms, data integration initiatives, and operational excellence. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data pipelines while ensuring data quality, reliability, and governance across the organization
Key Responsibilities
- Design, develop, and maintain end-to-end data pipelines using Databricks and Informatica.
- Build scalable ETL/ELT solutions to ingest, transform, and load large volumes of structured and unstructured data.
- Develop and optimize complex SQL queries, stored procedures, views, and database objects.
- Monitor production data workflows and resolve data failures, performance bottlenecks, and operational incidents.
- Implement data quality validations, reconciliation processes, and root cause analysis for data discrepancies.
- Collaborate with business stakeholders, data architects, and analytics teams to understand data requirements and deliver solutions.
- Optimize data processing performance within Databricks utilizing Spark SQL, PySpark, and Delta Lake.
- Support data migration, data integration, and data modernization initiatives.
- Maintain operational documentation, runbooks,
and standard operating procedures.
- Participate in on-call support and ensure adherence to SLA and data governance requirements.
Skill Requirements
- 8+ years of experience in Data Engineering / Data Operations.
- Strong hands-on experience with Databricks.
- Expertise in Informatica PowerCenter or Informatica Intelligent Cloud Services (IICS).
- Advanced SQL development and performance tuning.
- Experience with ETL/ELT design and implementation.
- Strong understanding of Data Warehousing concepts and dimensional modeling.
- Experience with PySpark and Spark SQL.
- Knowledge of Delta Lake architecture and optimization techniques.
- Experience working with relational databases such as SQL Server, Oracle, PostgreSQL, or Snowflake.
- Proficiency in troubleshooting production issues and performing RCA.
Good-to-Have Skills
- Azure Data Factory (ADF).
- Azure Data Lake Storage (ADLS).
- Python programming.
- CI/CD implementation using Azure DevOps or GitHub.
- Data governance and data quality frameworks.
- Experience with cloud platforms such as Azure, AWS, or GCP.
Must Have Skill
Databricks,MS SQL,Informatica
Good to Have Skill
Azure Data Factory (ADF)
Other Requirements
Not Applicable
📌 Technical Specialist (Chennai)
🏢 HCLTech
📍 Chennai