Senior Lead Data Engineer Location: Remote/ Bangalore

Experience: 11+ years

About The Role Are you passionate about building high-performance data systems? Do you thrive at the intersection of Data Engineering, Cloud, and Big Data technologies?

We’re looking for a Senior Lead Data Engineer to join our growing team! If you have deep expertise in ETL/ELT pipelines , Data Warehousing , Cloud platforms (GCP, AWS), and Hadoop ecosystems , we want to hear from you!

ETL Tools: primarily use Pentaho Data Integration (PDI) for designing and managing ETL workflows, including data extraction, transformation, and loading from various sources into target systems.

Databases

- MySQL for transactional data handling and intermediate data staging.
- Amazon Redshift as the primary data warehouse for analytics and reporting workloads.

Workflow Monitoring & Orchestration

- Azkaban is used for monitoring and managing data pipeline executions and ensuring task dependencies are handled correctly.

Job Scheduling & Scripting

- Crontab and Unix shell scripting are utilized for scheduling recurring jobs and automating data processes on a daily or hourly basis.

Cloud Services (AWS)

- Hands-on experience with:
- RDS for managing relational database services.
- Amazon Redshift for handling analytical queries and data warehousing.
- EC2 for running compute workloads.
- Basic IAM (Identity and Access Management) for managing user roles and access permissions.

CI/CD & Deployment





- Jenkins is used for automating the deployment of ETL scripts and scheduling code pushes to production environments.

What You'll Do

- Architect, design, and build robust ETL/ELT pipelines using Informatica PowerCenter and cloud-native solutions.
- Work hands-on with Data Warehousing concepts and Data Modeling to power analytics and insights.
- Build large-scale Hadoop workflows with tools like Hive, Pig, Spark, and HDFS.
- Collaborate closely with business and technical teams to translate requirements into scalable solutions.
- Produce high-quality technical documents: design docs, mapping specs, and unit test plans.
- Optimize big data processing and cloud infrastructure for performance, scalability, and cost.
- Mentor junior engineers and contribute to a culture of technical excellence.

Tech Stack You'll Work With

- Informatica PowerCenter
- GCP (BigQuery, Dataflow, Pub/Sub)
- AWS (Redshift, S3, Glue, Lambda)
- Hadoop (Hive, Pig, Spark, HDFS)
- SQL, Python (bonus!)
- DevOps, CI/CD, Docker/Kubernetes (nice to have!)

What You Bring

- 11+ years of IT experience with a strong focus on Data Engineering.
- Solid foundation in Data Warehouse architecture and Data Modeling principles.
- Deep understanding of Cloud and Big Data technologies.
- Excellent communication, documentation, and troubleshooting skills.
- Ability to thrive in a rapid-paced, agile environment.

Your Background

- Bachelor's or Master’s in Computer Science, Information Technology, or related field.

📌 Senior Lead AWS Data Engineer (Bengaluru)
🏢 Inoptra Digital
📍 Bengaluru