Description

Job Title: On-Premise Data Engineer

Job Summary

We are looking for a skilled On-Premise Data Engineer to join our team. The ideal candidate will have experience in designing, developing, and maintaining scalable data pipelines, data lakes, and databases on on-premise infrastructure. You will work closely with data scientists, analysts, and other stakeholders to ensure efficient data processing and availability across the organization.

Key Responsibilities

Data Infrastructure Design and Management:

Design, build, and maintain on-premise data pipelines, ETL processes, and data integration frameworks.

Set up and manage on-premise databases, data warehouses, and data lakes.

Ensure optimal performance of data architecture by maintaining high data throughput, low-latency, and reliable data delivery.

Data Integration and ETL

Develop and maintain ETL (Extract, Transform, Load) processes to ingest data from multiple sources into on-premise systems.

Perform data integration tasks and synchronize data from various sources like relational databases, flat files, APIs, and other enterprise systems.

Automate and optimize ETL pipelines to process large volumes of structured and unstructured data.

Data Quality and Governance

Implement data quality checks to ensure accuracy, integrity, and consistency of the data.

Ensure adherence to data governance policies and standards, including data security and access control.

Database Management

Set up and manage relational database: SQL Server, Oracle on-premise.

Tune database performance and optimize queries for better efficiency and reduced downtime.

Data Modeling and Architecture

Design and maintain data models, schemas, and optimized data structures for high-performance querying and storage.

Collaborate with data architects and business stakeholders to create efficient data models that support various business use cases.

Monitoring and Troubleshooting





Monitor and manage on-premise data infrastructure for availability, performance, and capacity.

Troubleshoot any issues related to data ingestion, processing, and storage.

Collaboration

Work closely with data analysts, data scientists, and business users to understand data needs and deliver data products that enable decision-making.

Collaborate with IT teams for system updates, patching, and hardware configurations.

Required Skills and Qualifications

Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field.

3-5 years of experience in data engineering or a related role.

Experience with on-premise databases and data warehouses SQL Server, Oracle.

Strong knowledge of ETL.

Proficiency in scripting languages: Python, Shell for automating data pipelines. (MUST)

Experience with data processing frameworks (e.g., Apache Hadoop, Apache Spark) in an on-premise environment.

Familiarity with batch (real-time data processing technologies is plus).

Proficiency in SQL and ability to optimize complex queries.

Knowledge of data modeling and schema design principles.

Understanding of data governance, data security, and compliance requirements.

Experience in performance tuning of databases and ETL processes.

Strong problem-solving skills and ability to troubleshoot complex data issues.

Familiarity with version control systems (e.g., Git) and CI/CD tools for data pipeline deployment.

Knowledge of containerization (e.g., Docker) (MUST) and on-prem orchestration (Cron Jobs) is added advantage

Preferred Qualifications

Experience working in hybrid environments (on-premise and cloud solutions).

Experience with message queues or streaming platforms (e.g., Kafka, RabbitMQ). is added advantage

DataOps and CI/CD pipeline experience for data products is added advantage

Knowledge of Snowflake is added advantage

Soft Skills

Solid communication and collaboration skills.

Ability to work independently and within a team.

Analytical mindset and detail-oriented approach to problem-solving

ETL Testing Lead

📌 On-Premise Data Engineer (Bengaluru)
🏢 Digitide
📍 Bengaluru