Data Engineering Specialist

About the Role

We are looking for a Data Engineering Specialist to build and support reliable data solutions for BI, analytics, and AI use cases. The ideal candidate has strong Python and SQL skills, hands-on experience with data pipelines and cloud platforms, and experience working with BI and analytics platforms.

Key Responsibilities

- Design, build, and maintain scalable batch and streaming data pipelines across cloud-based data platforms.

- Develop data solutions using Python, SQL, and distributed processing technologies such as PySpark.

- Integrate data from APIs, databases, files, and event-based sources.

- Build reliable, analytics-ready datasets and data models for BI and analytical workloads.

- Develop and maintain workflows using Airflow or similar orchestration platforms.

- Work closely with BI and analytics teams to translate business requirements into trusted datasets and reporting solutions.

- Support BI platforms such as Power BI, Tableau, or QuickSight, including data models, reporting datasets, metrics, and data validation.

- Develop efficient SQL transformations and optimize queries, pipelines, and data models for performance, scalability, and cost.

- Implement data quality, validation, monitoring, alerting, and error-handling processes.

- Investigate production issues and data discrepancies, perform root-cause analysis, and drive resolution.

- Follow engineering, security, governance, and privacy best practices while maintaining appropriate technical documentation and operational runbooks.

Required Skills





- Robust Python and SQL programming skills with experience developing production-grade data solutions.

- Proven experience building and supporting ETL/ELT pipelines in production environments.

- Hands-on experience with cloud data engineering on AWS, GCP, Azure, or similar platforms.

- Experience working with cloud storage, data processing technologies, and cloud data warehouses.

- Good understanding of data warehousing and dimensional modeling, including facts, dimensions, and slowly changing dimensions.

- Experience with a workflow orchestration platform such as Airflow, Azure Data Factory, AWS Step Functions, or similar.

- Experience working with BI and analytics platforms, with an understanding of how data engineering solutions support reporting and business analytics.

- Experience with data transformation and testing frameworks such as dbt or similar tools.

- Working knowledge of Git, CI/CD, and Docker.

- Understanding of data quality, monitoring, governance, security, and production support.

- Strong problem-solving, troubleshooting, communication, and collaboration skills.

Good to Have

- Hands-on experience with Apache Airflow and Power BI.

- Experience with Kafka, Kinesis, CDC, Debezium, or Spark Structured Streaming.

- Experience with Databricks or other modern data platforms.

- Knowledge of Terraform and cloud security.

- Experience developing lightweight APIs using FastAPI, Flask, Django, or similar frameworks.

- Experience supporting data platforms for AI/ML or LLM applications.

- Experience working with regulated or sensitive data.

- Relevant cloud or data engineering certifications.

- Experience using AI/LLM tools to improve engineering productivity.

📌 Data Engineer (Indore)
🏢 BESTPEERS INFOSYSTEM
📍 Indore