We are seeking a
skilled and detail-oriented Integration Consultant with 5 to 6 years of
experience to join our team. The ideal candidate will have expertise in
designing, building, and maintaining data pipelines and ETL workflows,
leveraging tools and technologies like AWS Glue, CloudWatch, PySpark, APIs,
SQL, and Python.
Key
Responsibilities:
1. Pipeline
Creation and Maintenance :
-
Design, develop, and deploy scalable data pipelines.
-
Optimize pipeline performance and ensure data accuracy and integrity.
2. ETL
Development :
- Create
ETL workflows using AWS Glue and PySpark to process and transform large
datasets.
- Ensure
compliance with data governance and security standards.
3. Data
Analysis and Processing :
- Write
efficient SQL queries for data extraction, transformation, and reporting.
-
Develop Python scripts to automate data tasks and improve workflows.
4. Monitoring
and Troubleshooting :
-
Utilize AWS CloudWatch to monitor pipeline health and performance.
-
Identify and resolve issues in a timely manner to minimize downtime.
5. API
Integration :
-
Integrate and manage APIs to connect external data sources and services.
6.
Collaboration :
- Work
closely with cross-functional teams to understand data requirements and provide
solutions.
-
Communicate effectively with stakeholders to ensure successful project
delivery.
Requirements
Required
Skills and Qualifications:
Experience – 5-6
Years
- o9 solutions
platform exp is Mandatory.
- Strong experience
with AWS Glue and CloudWatch .
- Proficiency in
PySpark , Python , and SQL .
- Hands-on experience
with API integration and management.
- Solid understanding
of ETL processes and pipeline creation.
- Robust analytical
and problem-solving skills.
- Familiarity with
data security and governance best practices.
Preferred
Skills:
- Knowledge of other
AWS services such as S3, EC2, Lambda, or Redshift.
- Experience with
Pyspark, API, SQL Optimization, Python
- Exposure to data
visualiz