30 Oct
|
ListenFirst
|
Jaipur
30 Oct
ListenFirst
Jaipur
Senior Backend Engineer – Data Ingestion & ETL Company
- ListenFirst Media (social media analytics) Location
- Jaipur Experience
- 4–5 years in backend / data engineering Type
- Full-time
About the role
ListenFirst collects social and entertainment data from platforms like Facebook, Instagram, TikTok, YouTube, X and LinkedIn, then turns it into analytics for brands and media companies. You will own the backend that does this: the ingestion layer that collects the data, and the ETL pipeline that loads it into our Redshift data warehouse every day. This is a hands-on role on a mature, production system. You will keep it running, fix data issues fast, and modernise it step by step. What you will do
• Ingestion: Build and maintain collectors for social platform APIs (Facebook Graph, Instagram, TikTok, YouTube, LinkedIn and others). Handle API version changes, rate limits, auth and pagination.
• ETL: Own the extract-transform step that turns JSON ingest events into warehouse rows. Add new tables, fields and transformations through config, code and tests.
• Data warehouse: Write and tune Redshift SQL: migrations, views, incremental loads, COPY from S3, compression encodings, WLM and query performance.
• AWS operations: Run the pipeline on AWS: S3, SQS (priority queues), Kinesis Firehose, DynamoDB, CloudWatch, SNS and EC2 workers. Reprocess data, scale workers and recover failed runs.
• Data issues: Investigate data gaps and wrong counts raised by Support and Client teams. Find the root cause, fix it and backfill.
• Modernisation: Upgrade old libraries (e.g. AWS SDK v2 to v3), add tests where they are missing, move secrets out of code, and improve monitoring and alerts.
• Cost: Help reduce infra cost (Redshift, EC2) without losing data coverage.
• Team: Write clear docs and runbooks. Review code. Use AI coding tools (Claude Code or similar)
in daily work. Must have
• 4–5 years of backend or data engineering experience on production systems.
• Solid in Ruby, or strong in Python/Go and willing to work mainly in Ruby. Our ETL and ingestion code is mostly Ruby.
• Strong SQL. Hands-on with a columnar warehouse: Amazon Redshift preferred (Snowflake / BigQuery also fine).
• Built data pipelines that move large volumes: batch loads, incremental loads, deduplication, backfills.
• Hands-on AWS: S3, SQS, EC2, IAM, CloudWatch. Kinesis or DynamoDB is a plus.
• Integrated third-party REST APIs: OAuth, rate limits, retries, schema changes.
• Comfortable on Linux: SSH, bash, cron, logs, process management.
• Debugs production issues end to end, from a client complaint down to the raw record.
• Git, code reviews, writing tests (RSpec, pytest or similar). Good to have
• Experience with social media platform APIs (Meta Graph API, TikTok, YouTube Data API).
• Redshift tuning: dist/sort keys, encodings, WLM, RA3 / Serverless.
• Go or C++ for performance-critical tools.
• Kafka, Athena, Jenkins or GitHub Actions, Capistrano.
• Experience modernising a legacy codebase safely. How we will judge success
• First 30 days: Set up locally, understand the ingest → ET → Redshift flow, fix first data issue.
• By 90 days: Own daily ETL runs and recovery. Ship at least one new collector or ETL table end to end.
• By 6 months: Measurable gains in reliability, test coverage or cost. Become the go-to person for pipeline issues. Interview focus
• SQL round: write and optimise warehouse queries; explain an incremental load.
• Pipeline design: design ingestion for a new social platform API, including failures and backfills.
• Debugging round: find why a metric dropped, using logs, S3 files and SQL.
• Code review: review a short Ruby or Python ETL change.
📌 Senior Backend Engineer (Jaipur)
🏢 ListenFirst
📍 Jaipur