Data Engineering :

- Design, develop, and maintain scalable batch and real-time data pipelines using Python, SQL, PySpark, and dbt.

- Build, optimize, and maintain ETL/ELT workflows for structured, semi-structured, and unstructured data.

- Develop scalable and reusable data ingestion frameworks to ingest data from APIs, databases, cloud storage, streaming platforms, and third-party systems.

- Design, develop, and optimize enterprise-grade Snowflake Data Cloud solutions, including databases, schemas, tables, views, tasks, streams, agile tables, Snowpark, and stored procedures.

- Develop reusable dbt models, macros, snapshots, seeds, and tests following software engineering best practices.

- Implement data quality checks, data lineage, documentation, and automated testing using dbt.

- Design and implement dimensional data models, including Star and Snowflake schemas, for analytics and reporting.

- Optimize SQL queries and data-processing workloads to support high-performance analytical applications.

- Implement data governance, security, data masking, and role-based access control (RBAC).

- Develop reusable Python libraries, utilities, and frameworks for data processing and engineering.

AI / Machine Learning:

- Design, develop, and deploy machine learning models for predictive analytics, NLP, recommendation systems, computer vision, and Generative AI applications.





- Build scalable feature engineering and data preprocessing pipelines for machine learning workloads.

- Train, evaluate, tune, and optimize machine learning models to improve performance and business outcomes.

- Design and develop Retrieval-Augmented Generation (RAG) pipelines using vector databases and Large Language Models (LLMs).

- Integrate OpenAI, Azure OpenAI, Anthropic, and AWS Bedrock APIs into enterprise applications and AI solutions.

- Build AI agents and workflow automation solutions using frameworks such as LangChain, LangGraph, and LlamaIndex.

- Deploy machine learning and Generative AI solutions following MLOps best practices.

- Implement model monitoring, performance tracking, data/model drift detection, and automated retraining processes.

Cloud & DevOps

- Design and build cloud-native data and AI solutions on AWS and Azure.

- Develop scalable data platforms using Snowflake, Databricks, and Apache Spark.

- Design and implement data orchestration workflows using Apache Airflow, Azure Data Factory, AWS Glue, or similar orchestration tools.

- Build and maintain automated CI/CD and deployment pipelines using Azure DevOps and GitLab CI/CD.

- Monitor application, data-platform, and pipeline performance and optimize cloud resource utilization, scalability, reliability, and cost.

- Implement cloud security, monitoring, logging, and operational best practices across data and AI platforms.

📌 Data Engineer (Pune)
🏢 The Digital Group Infotech
📍 Pune