Insight Global is seeking a Senior Data Engineer for a leading insur-tech client. This individual will join a highly technical Data Engineering team focused on modernizing a core subrogation platform and revamping customer data integration capabilities. The ideal candidate is a deeply technical engineer who thrives in large-scale data environments, understands the business impact of data solutions, and can design highly effective data pipelines within AWS.

This role will be responsible for building and optimizing ETL/ELT frameworks, maintaining data lakes and feature stores, supporting machine learning initiatives, and enabling seamless customer integrations through APIs and Guidewire-based solutions. Success in this position requires strong Spark expertise, deep AWS knowledge, excellent communication skills, and the ability to connect technical decisions to measurable business outcomes.

Day-to-Day:

Design, build, and optimize ETL/ELT pipelines that transform customer and enterprise data into actionable datasets

Develop and maintain scalable AWS-based data platforms and data lakes

Partner with Product and Market teams to create reporting tools and dashboards that support business strategy

Build and maintain feature stores used by ML Engineering teams

Develop customer data integration pipelines via APIs, Guidewire, and other ingestion methods

Implement data quality controls and monitoring frameworks

Troubleshoot data issues across distributed systems and streaming architectures

Optimize Spark workloads and cloud resources for performance and cost savings





Collaborate with a globally distributed Data Engineering team

Gather business requirements and translate them into scalable data solutions

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances.

If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to [email protected] learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.





Skills and Requirements

Must-Haves:

Strong proficiency in Python and SQL

Recent, hands-on experience with Apache Spark in production environments

Deep expertise building and optimizing ETL/ELT pipelines

Extensive experience with AWS data services:

Glue

S3

EMR

Athena

Step Functions

DynamoDB

SQS

Experience with distributed processing technologies beyond managed platforms

Experience with data streaming technologies:

Kafka

Spark Streaming

SQS-based event architectures

Strong data modeling experience, preferably within insurance or financial services

Experience designing and maintaining data lakes

Knowledge of medallion architecture (Bronze/Silver/Gold)

Schema evolution and incremental processing experience

Familiarity with MongoDB and document-based data stores

Infrastructure as Code experience (Terraform or CloudFormation)

Experience troubleshooting and resolving data quality issues across distributed systems

Experience implementing data quality frameworks such as Excellent Expectations

Strong business acumen with the ability to translate technical solutions into business outcomes

Ability to optimize pipelines for performance, scalability, and cost efficiency

Excellent communication skills and ability to explain the business problem being solved Plusses:

Experience in Insur-Tech or Technology

Experience handling insurance claims data and complex feature relationships

Experience supporting machine learning platforms or feature stores

📌 Senior Data Engineer (Hyderabad)
🏢 Insight Global
📍 Hyderabad