Job Description

A Senior Data Engineer Extraodinaire is a visionary leader in crafting scalable, effective

solutions for data processing and analysis. They possess advanced expertise in database

management, ETL processes, and data modeling, designing robust pipelines with cutting-

edge technologies like Apache Spark and Hadoop. With deep proficiency in cloud

platforms such as AWS, Azure, or Google Cloud, they architect resilient, scalable data

ecosystems. Collaborating closely with cross-functional teams, they translate business

requirements into data solutions that align with organizational goals. Through their

leadership, technical acumen, and innovation, they drive data-driven insights that

empower organizations to excel in the digital age.

Key Responsibilities

• Lead the design and development of scalable data pipeline architectures and ETL

processes using PySpark, SQL, and cloud-based big data technologies to drive

optimal performance.

• Spearhead the implementation of advanced analytics tools and data products,

delivering actionable insights through cloud platforms.

• Innovate and solve complex data engineering challenges by leveraging the latest

advancements in big data and cloud technologies.

• Establish and enforce best practices for data engineering, including optimizing

PySpark jobs, SQL queries, cloud resource utilization, and data security within cloud

environments.

• Translate business requirements into scalable, cloud-based data solutions and

serve as a key technical liaison.

• Mentor and lead a team of data engineers, fostering their expertise in Python,

PySpark, SQL, and cloud technologies.

• Evaluate and recommend new technologies to enhance the organization's data





engineering capabilities, with a focus on cost and performance optimization.

• Represent the company at industry conferences, sharing insights on cloud-based

data engineering, PySpark, and big data technologies.

• Collaborate in a energetic, cross-functional environment to drive innovation in data

engineering.

Required Skills and Qualifications

• Proficient in SQL, Python, Spark, and data transformation techniques.

• Hands-on experience deploying and managing data services on cloud platforms like

AWS, Azure, or Google Cloud (GCP).

• Proficient in data orchestration tools like Apache Airflow and Azure Data Factory

(ADF) for managing complex workflows.

• Experience with data platforms like Databricks, data lakes, and warehouses

(Redshift, SQL Server, BigQuery) for big data processing.

• Proficient in version control (Git, GitHub) and comfortable working with CI/CD

pipelines.

• In-depth knowledge of data governance, security, encryption, and compliance

practices in cloud environments.

• Strong analytical and problem-solving skills, with a focus on optimizing data

workflows.

Preferred Skills and Qualifications

• 8+ years of experience in data engineering or related roles.

• Bachelor's degree or equivalent degrees in computer science, Engineering, or a

related field.





• Hands-on experience with distributed computing and parallel data processing.

Good to Have

• Experience with streaming tools such as Kafka, Event Hubs, and Amazon SQS.

• Advanced knowledge of containerization using Docker and Kubernetes.

• Engage in peer review processes and present research findings at esteemed ML/AI

conferences such as NIPS, ICML, AAAI and COLT

• Experiment with latest advancements in Data Engineering tools, platforms, and

methodologies. Mentor peers and junior members and handle multiple projects at

the same time

• Participate and speak at various external forums such as research conferences and

technical summits

• Promote and support company policies, procedures, mission, values, and

standards of ethics and integrity

• Certifications in AWS, Azure, or GCP are a plus

• Understanding of modern data architecture patterns, including the Lambda and

Kappa architectures

Benefits and Perks

• Competitive Salary: Receive a market-leading salary, commensurate with your

experience.

• Health Insurance: Comprehensive health coverage, including medical, dental, and

vision plans for you and your family.

• Generous Parental Leave: Enjoy exceptional maternity, paternity, and parental leave

policies to support your family needs.

• Paid Time Off (PTO): Generous vacation, holiday, and sick leave allowances to

ensure a healthy work-life balance.

• Professional Development: Access to learning resources, certifications, and

opportunities to grow in your career.

• Additional Perks: Benefit from a range of perks, including wellness programs,

flexible working hours, and more.

📌 Sr. Data Engineer | 8+ YoE | Contract 3-6 months | Remote | Full time | Immediate Joiners (India)
🏢 IndusGuru Network Partners
📍 India