Job Description – Senior Data Engineer / Consultant
About GyanSys:
GyanSys is a global systems integrator and technology consulting company that helps enterprises accelerate digital transformation through SAP, Cloud, Data & Analytics, AI, and Intelligent Automation solutions. We partner with leading technology providers to deliver innovative, scalable, and business-driven solutions across diverse industries worldwide.
Senior Data Engineer / Consultant
Location: Bengaluru | Pune | Hyderabad
Employment Type: Full-Time
Experience: 4+ Years
Joining: Immediate Joiners Preferred
Job Summary
GyanSys is looking for experienced Senior Data Engineers / Consultants to join our growing Data & Analytics practice. The ideal candidate will have strong hands-on experience in designing, developing, and optimizing enterprise-scale data engineering solutions across up-to-date cloud and data platforms.
The role requires expertise across technologies such as Microsoft Fabric, Azure Data Engineering, Databricks, PySpark, Apache Spark, Python, SQL, and cloud data platforms . The candidate will work closely with architects, business stakeholders, data scientists, and cross-functional teams to build scalable data pipelines and analytics solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines for enterprise data platforms.
- Develop data engineering solutions using Microsoft Fabric, Azure Data Factory, Databricks, PySpark, Spark SQL, and Python .
- Build data pipelines for structured, semi-structured, and unstructured data .
- Work with Microsoft Fabric components such as Fabric Data Factory, Lakehouse, Warehouse, Dataflows, and Notebooks.
- Develop and optimize Databricks notebooks, Spark jobs, Delta Lake pipelines, and Databricks Workflows .
- Implement Medallion Architecture and modern Lakehouse/Data Platform architectures.
- Integrate data from multiple sources including SQL Server, Oracle, PostgreSQL, APIs, cloud storage,
ERP systems, and SaaS applications .
- Design and implement efficient data models, data warehouses, data lakes, and Lakehouse solutions .
- Optimize data pipelines, Spark jobs, SQL queries, clusters, and storage for performance and cost efficiency .
- Implement data quality, validation, monitoring, governance, security, and compliance practices.
- Develop and maintain CI/CD pipelines using Azure DevOps/GitHub and Git .
- Troubleshoot production issues and provide root-cause analysis and performance improvements.
- Collaborate with solution architects, business analysts, data scientists, and BI teams.
- Participate in technical design, code reviews, documentation, and knowledge-sharing initiatives.
- Mentor junior engineers and contribute to technical best practices within the team.
Required Technical Skills
Primary Skills
- 4+ years of experience in Data Engineering
- Strong hands-on experience with Microsoft Fabric and/or Azure Data Engineering
- Strong experience with Databricks, PySpark, Apache Spark, Spark SQL, and Python
- Strong SQL skills and experience with relational databases
- Hands-on experience developing ETL/ELT pipelines
- Experience with Azure Data Factory / Fabric Data Factory
- Strong understanding of Data Lake, Lakehouse, Data Warehouse, and Data Modeling
- Experience with Delta Lake
- Experience with Medallion Architecture
- Experience with cloud platforms, preferably Microsoft Azure
- Experience with Git, Azure DevOps/GitHub, and CI/CD
- Experience working with databases such as SQL Server, Oracle, PostgreSQL, Snowflake ,
or similar platforms
Secondary Skills
- Power BI – report/dashboard development and data integration
- Tableau or other BI/visualization platforms
- Databricks Unity Catalog
- Databricks Workflows
- Microsoft Fabric governance and security
- Kafka, Azure Event Hubs, or other streaming technologies
- Azure Synapse Analytics
- Microsoft Purview
- APIs and real-time data integration
- Exposure to Microsoft Fabric Dataflows Gen2 / Direct Lake
- Agile/Scrum methodologies
Preferred Experience
- Experience designing end-to-end enterprise data platforms
- Experience migrating traditional ETL/data warehouse workloads to Azure, Databricks, or Microsoft Fabric
- Experience with real-time/streaming data pipelines
- Experience implementing data governance, security, lineage, and access controls
- Experience working with enterprise ERP systems such as SAP
- Experience with performance tuning and cost optimization of cloud data platforms
- Experience working with global clients and distributed teams
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering , or a related field.
- 4+ years of professional experience in Data Engineering or related roles.
Preferred Certifications
- Microsoft Certified: Fabric Data Engineer Associate
- Microsoft Certified: Azure Data Engineer Associate
- Databricks Certified Data Engineer Associate/Professional
- AWS Certified Data Engineer / Data Analytics
- Google Professional Data Engineer
Why Join GyanSys?
At GyanSys, you'll work on enterprise-scale digital transformation programs leveraging modern cloud, data, analytics, and AI technologies. You will gain exposure to Microsoft Fabric, Databricks, Azure, modern data architectures, and global clients , while working in a collaborative environment focused on continuous learning and professional growth.
📌 Senior Data Engineer / Consultant (Immediate Joiners Only – 4+ Years) (India)
🏢 GyanSys
📍 India