Data Engineer | U S Staffing

Position: Data Engineer

Location: Bengaluru, Karnataka

Experience: 3–6 Years

Employment Type: Full-Time

Work Mode: On-site / Hybrid

Shift: Day Shift

Industry: IT / Software Development

Job Summary :

We are looking for an experienced Data Engineer with robust hands-on expertise in building and maintaining scalable data pipelines and data processing solutions.

The ideal candidate should have solid experience with Python, SQL, ETL/ELT, Apache Spark, Databricks, cloud platforms (AWS/Azure), and modern data warehousing technologies. The candidate will work closely with data analysts, data scientists, software engineers, and business teams to deliver reliable and high-quality data solutions.

Key Responsibilities :

- Design, develop, and maintain scalable ETL/ELT data pipelines.
- Collect, transform, integrate, and process large volumes of structured and unstructured data.
- Develop data-processing solutions using Python, SQL, and Apache Spark/PySpark.
- Build and optimize data pipelines using Databricks.
- Design and maintain data models, data warehouses, and data lake solutions.
- Develop complex SQL queries, stored procedures, and data transformations.
- Integrate data from databases, APIs, cloud storage, applications, and third-party sources.
- Perform data cleansing, validation, transformation, and quality checks.
- Monitor data pipelines and troubleshoot failures and performance issues.
- Optimize data-processing workloads for performance, scalability, and cost.
- Implement data quality, governance, security, and access-control standards.
- Work with cloud platforms such as AWS or Microsoft Azure.
- Develop and maintain workflow orchestration using Apache Airflow / Azure Data Factory / AWS Glue.
- Implement automated deployment and CI/CD processes for data pipelines.
- Collaborate with Data Analysts, Data Scientists, BI Developers, and application teams.
- Create and maintain technical documentation for pipelines, data models, and architecture.




- Participate in Agile/Scrum meetings, code reviews, and technical discussions.

Required Technical Skills :

Programming & Database

- Python
- SQL
- PySpark / Apache Spark
- Advanced SQL queries and performance optimization

Data Engineering

- ETL / ELT
- Data Pipelines
- Data Integration
- Data Modeling
- Data Warehousing
- Data Lakes / Lakehouse architecture
- Batch and streaming data processing

Cloud

- AWS or Azure
- AWS: S3, Glue, Lambda, Redshift, EMR
- Azure: Data Factory, ADLS, Synapse Analytics, Databricks

Big Data & Modern Data Platforms

- Apache Spark
- Databricks
- Delta Lake
- Kafka knowledge is preferred
- Snowflake experience is an advantage

Databases

- PostgreSQL / MySQL / SQL Server / Oracle
- Snowflake / Redshift / Azure Synapse
- NoSQL knowledge is an advantage

Tools

- Git
- Airflow
- Docker
- CI/CD
- Jenkins / Azure DevOps / GitHub Actions

Required Qualifications

- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
- 3–6 years of professional Data Engineering experience.
- Strong hands-on experience with Python and SQL.
- Experience developing production-level ETL/ELT pipelines.
- Strong knowledge of Spark/PySpark and Databricks.
- Hands-on experience with AWS or Azure cloud services.
- Good understanding of data warehousing and data modeling concepts.
- Experience handling large datasets and optimizing data-processing workloads.
- Solid analytical and troubleshooting skills.
- Good written and verbal communication skills.
- Ability to work effectively with cross-functional teams.

Preferred Skills

- Snowflake
- Kafka
- Delta Lake
- Apache Airflow
- dbt
- Kubernetes
- Terraform
- Power BI / Tableau
- Real-time/streaming data pipelines
- Data governance and security
- Cloud certification in AWS or Azure

Key Skills

Data Engineering | Python | SQL | PySpark | Apache Spark | Databricks | ETL | ELT | Data Pipelines | AWS | Azure | Snowflake | Airflow | Kafka | Data Warehousing | Data Lake | Delta Lake | Data Modeling | CI/CD | Git

📌 Data Engineer (Bengaluru)
🏢 Clicksoft Solutions
📍 Bengaluru