Freelance Senior Data Engineer – Cloud, ETL/ELT & Data Engineering

Job Type: Freelance / Independent Contractor
Work Arrangement: Remote
Engagement Type: Project-Based / Flexible Hours
Experience Required: 7–10+ Years of Relevant Experience

Job Overview

We are seeking an experienced Freelance Senior Data Engineer with at least 7–10 years of relevant hands-on experience in designing, developing, implementing, and maintaining scalable data engineering solutions.

The ideal candidate will have strong expertise in data pipelines, ETL/ELT development, cloud-based data platforms, data warehousing, big data technologies, and workflow orchestration. Candidates should be capable of independently handling technical assignments, troubleshooting complex data issues, and supporting enterprise-level data engineering projects.

This opportunity is suitable for experienced professionals interested in versatile, remote, freelance, and project-based technical engagements.

Key Responsibilities

- Design, develop, and maintain scalable data pipelines for batch and real-time data processing.
- Build and optimize ETL/ELT workflows for ingesting, transforming, validating, and integrating data from multiple sources.
- Develop complex SQL queries and data transformation logic to support analytics, reporting, and business intelligence.
- Implement cloud-based data engineering solutions across AWS, Microsoft Azure, and Google Cloud Platform (GCP).
- Develop and maintain data lakes, data warehouses, lakehouses, and enterprise data platforms.
- Work with distributed data processing frameworks to handle large-scale structured and unstructured datasets.
- Implement workflow orchestration, scheduling, monitoring, and automated pipeline recovery.
- Optimize data pipeline performance, query execution, resource utilization, and cloud infrastructure costs.
- Implement data quality checks, validation rules, metadata management, lineage, and governance controls.
- Support CI/CD, version control, infrastructure automation, and deployment processes for data engineering solutions.
- Troubleshoot production data issues, pipeline failures, data inconsistencies, and performance bottlenecks.
- Collaborate with data architects, analytics engineers, data scientists,



and business stakeholders to deliver project requirements.
- Prepare technical documentation and provide regular updates on assigned project deliverables.

Key Data Engineering Tools & Technologies

1. Programming & Query Languages

- Python, PySpark, SQL, Scala
- Advanced SQL, stored procedures, query optimization
- Bash / Shell scripting

2. Cloud Data Platforms

- AWS: S3, Glue, EMR, Redshift, Athena, Lambda, Kinesis
- Microsoft Azure: Azure Data Factory, Azure Databricks, ADLS Gen2, Synapse Analytics, Microsoft Fabric
- GCP: BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage

3. Big Data & Distributed Processing

- Apache Spark, PySpark
- Hadoop, Hive
- Apache Kafka
- Apache Flink
- Delta Lake, Apache Iceberg, Apache Hudi

4. ETL/ELT & Data Integration

- Informatica, Talend, SSIS
- Fivetran, Airbyte
- dbt (data build tool)
- Azure Data Factory, AWS Glue
- Matillion and similar data integration platforms

5. Data Warehousing & Lakehouse Technologies

- Snowflake
- Databricks Lakehouse
- Amazon Redshift
- Google BigQuery
- Azure Synapse Analytics
- Microsoft Fabric
- Teradata and Oracle, where applicable

6. Workflow Orchestration & Scheduling

- Apache Airflow
- Dagster
- Prefect
- Azure Data Factory Pipelines
- AWS Step Functions
- Cloud Composer

7. Data Modeling, Quality & Governance

- Dimensional modeling, star and snowflake schemas
- Data vault and enterprise data modeling
- Excellent Expectations, Soda
- Data lineage, metadata management, cataloging, and governance
- Data quality monitoring and reconciliation

8. DevOps & Infrastructure Automation

- Git, GitHub, GitLab, Azure DevOps
- Docker and Kubernetes
- Terraform
- CI/CD pipelines
- Infrastructure as Code (IaC)
- Automated testing, deployment, and release management

Required Qualifications

- 7–10+ years of relevant professional experience in data engineering, data integration, or data platform development.




- Strong hands-on programming experience with Python, PySpark, and SQL.
- Proven experience building and supporting production-grade ETL/ELT pipelines.
- Experience with at least one major cloud platform: AWS, Azure, or GCP.
- Hands-on experience with modern data warehouses, lakehouses, or big data processing platforms.
- Strong understanding of data modeling, distributed processing, performance optimization, and data quality.
- Experience with orchestration tools, monitoring, troubleshooting, and production support.
- Familiarity with Git, CI/CD, automated testing, and deployment best practices.
- Ability to work independently, communicate effectively, and deliver assigned technical tasks within agreed timelines.

Preferred Qualifications

- Experience designing enterprise-scale data platforms and modernizing legacy data pipelines.
- Knowledge of streaming data architectures and real-time data processing.
- Experience with data governance, security, access controls, and regulatory compliance requirements.
- Familiarity with master data management (MDM), metadata platforms, and data observability.
- Relevant cloud or data engineering certifications are preferred but not mandatory.

Compensation & Engagement

Compensation will be negotiated based on experience, technical specialization, project complexity, and mutually agreed hourly or project-based rates.

The engagement may vary depending on project availability, duration, and client requirements.

How to Apply

Interested candidates should submit an updated resume along with the following information:

- Total years of experience and relevant data engineering experience
- Primary cloud platform and data engineering specialization
- Key tools and technologies used in recent projects
- Experience with production pipelines and enterprise data platforms
- Relevant certifications, if any
- Current availability
- Expected hourly rate or project-based compensation

We welcome experienced Senior Data Engineers who are interested in flexible, remote freelance opportunities and project-based technical assignments.

Pay: ₹5,000.00 - ₹10,000.00 per week

Benefits:

- Flexible schedule

Work Location: Remote

📌 Freelance Senior Data Engineer – Cloud, ETL/ELT & Data Engineering (India)
🏢 Egon Solutions
📍 India