Job Summary- Data Engineer Job Title: Data Engineer

Job Family: Engineering / Data & Analytics

Experience: 4 years

Location: Bangalore

Role Overview Looking for a hands-on Data Engineer to design, develop and maintain scalable data pipelines, data-processing services and integration solutions supporting enterprise applications, analytics and AI-enabled solutions. The candidate should possess strong Python and SQL programming skills, along with practical experience in data ingestion, ETL/ELT, API integration, relational and NoSQL databases, data modeling and data-quality engineering. The role requires someone who can independently develop production-quality data components while collaborating with application developers, architects, AI/ML engineers, business analysts and other engineering teams.

Job ResponsibilitiesKey Responsibilities

- Design, develop and maintain scalable data ingestion, transformation and processing pipelines.
- Develop production-quality data engineering components using Python.
- Write complex and optimized SQL queries for data extraction, transformation, validation and analysis.
- Develop and maintain ETL/ELT pipelines for structured and semi-structured data.
- Integrate data from databases, enterprise applications, REST APIs, files and other data sources.
- Develop reusable Python modules for ingestion, transformation, validation, logging and exception handling.
- Design and implement relational database schemas and data models.
- Work with SQL and NoSQL databases based on application requirements.
- Implement data-quality controls including validation, reconciliation, deduplication and completeness checks.
- Develop and consume REST APIs for data integration.




- Work with enterprise search technologies such as Elasticsearch/OpenSearch or Apache Solr where required.
- Optimize database queries and pipelines for performance and scalability.
- Implement appropriate error handling, retry, logging and monitoring mechanisms.
- Develop automated unit, integration and data-quality tests.
- Troubleshoot data, database, API and pipeline-related issues across development and production environments.
- Follow Git-based development, code-review and CI/CD practices.
- Work collaboratively with architects and engineering teams to translate requirements into technical implementations.
- Create and maintain appropriate technical documentation.

Key Sourcing Information NA

Skill RequirementsRequired Technical Skills Skill Area | Expected Proficiency

--- | ---

Python | Strong - Mandatory

SQL | Robust/Advanced - Mandatory

Data Engineering | ETL/ELT, ingestion, transformation, validation

Python Libraries | Pandas, NumPy, PyArrow or equivalent

Relational Databases | SQL Server / PostgreSQL / MySQL

NoSQL | MongoDB / document databases or equivalent

API Integration | REST APIs, JSON, authentication

Data Modeling | Relational modeling, schema design, dimensional concepts

Data Formats | JSON, CSV, XML, Parquet

Search Technologies | Elasticsearch/OpenSearch, Apache Solr

Version Control | Git / GitHub / GitLab

Testing | PyTest, unit testing, integration and data-quality testing

Linux | Shell commands,



scripting and troubleshooting

Containerization | Docker

CI/CD | Basic understanding of automated build/test/deployment pipelines

SQL Skills The candidate should have practical experience with:

- Complex Joins
- CTEs
- Subqueries
- Window Functions
- Aggregations
- Views
- Stored Procedures
- Transactions
- Indexes
- Query Execution Plans
- Query Optimization

The engineer should understand how database design and indexing decisions affect application and pipeline performance.

Data Engineering Skills The candidate should understand an end-to-end data flow such as:

- Source Systems Ingestion Validation Transformation Data Quality Storage Search/Analytics/Application Consumption

Experience should include:
- Batch data processing
- Incremental data loading
- ETL/ELT
- Schema validation
- Data cleansing
- Deduplication
- Reconciliation
- Error handling
- Retry/recovery
- Data lineage fundamentals
- Metadata handling

AI/GenAI exposure - desirable

Educational Qualification Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science or a related technical discipline, or equivalent practical experience.

Other RequirementsProfessional Competencies The candidate should demonstrate:

- Strong analytical and problem-solving skills
- Good programming discipline
- Ability to independently troubleshoot technical issues
- Understanding of software engineering best practices
- Ability to work within Agile development teams
- Effective communication and collaboration
- Ability to understand technical requirements and translate them into working solutions
- Willingness to learn emerging data and AI technologies.

📌 SeniorAdministrator - Extract Transform Load (ETL), Data Lakes (India)
🏢 HCL Technologies
📍 India