Roles and Responsibilities
Associate Architect- Data Engineer is responsible for overseeing the design, development, and management of data infrastructure and pipelines within an organization. This role involves a mix of technical leadership, project management, and collaboration with other teams to ensure the efficient collection, storage, processing, and analysis of large datasets. The Lead Data Engineer typically manages a team of data engineers, architects, and analysts, ensuring that data workflows are scalable, reliable, and meet the business's requirements.
Responsibilities:
● Lead the design, development, and maintenance of data pipelines and ETL processes
architect and implement scalable data solutions using Databricks and AWS.
● Optimize data storage and retrieval systems using Rockset, Clickhouse, and CrateDB.
● Develop and maintain data APIs using FastAPI.
● Orchestrate and automate data workflows using Airflow.
● Collaborate with data scientists and analysts to support their data needs.
● Ensure data quality, security, and compliance across all data systems.




● Mentor junior data engineers and promote best practices in data engineering.
● Evaluate and implement new data technologies to improve the data infrastructure.
● Participate in cross-functional projects and provide technical leadership.
● Manage and optimize data storage solutions using AWS S3, implementing best practices for data lakes and data warehouses.
● Implement and manage Databricks Unity Catalog for centralized data governance and access control across the organization.
‍
Qualifications Required
● Bachelor's or Master's degree in Computer Science, Engineering, or related field
● 10+ years of experience in data engineering, with at least 6 years in a lead role
● Solid proficiency in Python, PySpark, and SQL
● Extensive experience with Databricks and AWS cloud services
● Hands-on experience with Airflow for workflow orchestration
● Familiarity with FastAPI for building high-performance API

📌 Associate Architect-Data Engineer (Secunderabad)
🏢 Fission Computer Labs Private
📍 Secunderabad