Job Summary

Provides expert level data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure. Utilizes the deepest dive analytical skillset on a variety of internal and external data. Leads, instructs, and mentors newer Data Engineering crew.

Role Summary

The Lead Data Engineer, Specialist is responsible for designing and delivering scalable data platforms, cloud-native data pipelines, and analytics solutions. This is a hands-on technical leadership role requiring deep data engineering expertise, strong problem-solving skills, and the ability to lead a team of Data and AI Engineers.

The role requires a strong product mindset to understand business needs, translate them into sustainable technical solutions, and take end-to-end ownership of delivery, quality, reliability, and continuous improvement. The individual will collaborate with business and technology stakeholders to deliver secure, high-quality, and future-ready data solutions while contributing to modern AI-assisted engineering practices.

Responsibilities

- Lead and mentor a team of Data and AI Engineers, providing technical direction, coaching, and support.
- Lead solution architecture and technical design, ensuring alignment with business objectives and product priorities.
- Provide technical leadership through architecture reviews, design reviews, code reviews, and engineering best practices.
- Design, develop, and optimize scalable data pipelines that ingest, process, and transform large datasets from diverse sources.
- Build cloud-based data platforms and storage solutions to support analytical and operational workloads.




- Design and optimize data models, schemas, data quality controls, and governance frameworks.
- Implement batch and real-time data processing solutions using modern orchestration and streaming technologies.
- Partner with product owners, business stakeholders, analysts, architects, and engineering teams to deliver data-driven solutions.
- Take end-to-end ownership of solution design, development, deployment, production reliability, and continuous improvement.
- Evaluate technical options and make informed decisions considering performance, scalability, security, cost, and maintainability.
- Implement CI/CD, automation, testing, observability, and DevOps practices to improve engineering productivity and platform reliability.
- Apply AI-assisted development practices and up-to-date engineering approaches to improve quality, efficiency, and innovation.
- Support production operations, troubleshoot complex issues, perform root-cause analysis, and implement preventive actions.
- Participate in application-support rotations and work in shifts or outside regular business hours when required during critical incidents, major releases, or other business-critical situations.

Qualifications and Skills

- Bachelor's degree in Computer Science, Information Technology, Engineering, Computer Applications, or a related discipline from a recognized institution.




- Minimum of 8 years of relevant technology experience.
- At least 4 years of hands-on experience designing and developing scalable data pipelines, ETL or ELT frameworks, and database or data platform solutions.
- Demonstrated experience in leading technical solutions and guiding engineering teams through design, development, deployment, and production support.
- Experience working in Agile product environments with strong ownership and accountability.

Must Have Skills

- AWS
- Redshift
- Databricks
- dbt
- Spark
- Python
- PySpark
- Pandas
- Advanced SQL

Additional Requirements

- Strong product mindset with the ability to connect technical decisions to business and customer outcomes.
- Ability to lead technical solution design and drive delivery from concept through production.
- Strong end-to-end ownership of deliverables, technical decisions, quality, reliability, and support.
- Proven ability to lead, mentor, and develop engineers while remaining technically hands-on.
- Strong analytical, problem-solving, troubleshooting, and debugging skills.
- Effective communication and stakeholder management across technical and non-technical teams.
- Ability to manage ambiguity, balance competing priorities, and proactively address risks and dependencies.
- Willingness to support critical production incidents and application-support requirements outside regular working hours when needed.

Location

Hyderabad, Telangana

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

📌 Data Engineer, Specialist (Hyderabad)
🏢 Vanguard Technologies
📍 Hyderabad