A Data Engineer is responsible for designing, developing, and managing the company's data infrastructure and systems. They collaborate with cross-functional teams to gather data requirements, design and implement data models, and ensure data integrity and availability. The ideal candidate should have a strong background in data engineering, ETL processes, and database management.
Responsibilities:- Design, develop, and maintain data pipelines and ETL processes to transform and load data into the company's data warehouse.
- Collaborate with data scientists and analysts to understand data requirements, and implement data models and schemas.
- Optimize data infrastructure and systems for scalability and performance.
- Ensure data quality and integrity by implementing data validation and cleansing processes.
- Manage and monitor databases, ensuring availability and security.
- Develop and maintain documentation for data infrastructure and systems.
- Stay up-to-date with the latest trends and technologies in data engineering.
- Train and mentor junior data engineers.
Preferred Candidate:
- Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
- Minimum 8 years of experience in data engineering or a related role.
- Strong knowledge of data engineering concepts and best practices.
- Experience with ETL tools and frameworks (e.g., Apache Spark, Apache Beam, Talend).
- Proficient in SQL and database management systems (e.g., PostgreSQL, MySQL, Oracle).
- Experience with cloud-based data technologies (e.g., AWS Redshift, Google BigQuery, Azure Data Lake).
- Strong programming skills in Python, Java, or Scala.
- Experience with data modeling and schema design.
- Knowledge of data warehousing concepts and architectures.
- Experience with version control systems (e.g., Git).
Application URL
Source : www.bayt.com