Follow us on the social media
All rights reserved ©
Caspian Innovation Center

Data engineer

By agreement
3186
Full-time
Baku, Azerbaijan
05.03.25 - 05.04.25

Working conditions

  • Design, develop, and optimize ETL/ELT workflows using ETL/ELT tools (DBT, Apache Nifi) from various sources to big data platforms;
  • Design, develop, and optimize ETL/ELT workflows using SAP SLT, SAP BW for data replication from SAP ERP systems to SAP;
  • HANA and big data platforms;
  • Implement data integration solutions using technologies like Hadoop, Spark, Kafka, or similar;
  • Develop data models for big data storage and processing;
  • Develop dimensional data models for data warehouse and OLAP storages;
  • Develop datamarts as a data products for data science and business intelligence workloads;
  • Write and optimize code in programming languages such as Python, Java, or Scala to process large datasets;
  • Automate data processing tasks and develop custom data solutions;
  • Develop realtime and batch data ingestion pipelines using Apache Nifi and other data ingestion/mediation rules;
  • Develop streaming ELT/ETL data pipelines using Kafka, Apache Spark, Apache Flink;
  • Develop and finetune existing CI/CD pipelines using Gitlab;
  • Develop and finetune process orchestration using Apache Airflow;
  • Develop and manage ETL/ELT processes to ensure efficient data flow;
  • Monitor and improve the performance of data extraction, loading, and transformation processes;
  • Implement data quality checks and validation procedures;
  • Ensure compliance with data governance policies, including data security and privacy standards;
  • Work with cross-functional teams, including data scientists, analysts, and business stakeholders, to meet data requirements;
  • Provide technical support and troubleshoot data-related issues;
  • Meal allowance;
  • Annual performance and project bonuses;
  • Corporate health program: VIP voluntary insurance and special discounts for gyms;
  • Access to Digital Learning Platforms;
  • 5 work days from 8-17 or 9-18.

Requirements

  • Bachelor's degree in Computer Science, Information Systems, or related field; Master's degree is a plus;
  • 3-5 years of experience in data engineering.
  • Experience with SAP reporting stack (SAP BW, SAP HANA, SAP BO Universe design);
  • Experience with big data technologies (e.g., Hadoop, Spark);
  • Basic experience with data streaming technologies (Kafka and Spark Streaming);
  • Strong programming skills in Python, Java, or Scala;
  • Experience with Apache Airflow;
  • Solid understanding of ETL/ELT processes and data warehousing concepts;
  • Solid understanding of SQL and NoSQL databases (MS SQL, PostgreSQL, SAP HANA, Cassandra);
  • Experience with containerization (Docker, Kubernetes as developer);
  • Basic understanding of oil and gas industry processes and data requirements is a plus;
  • Excellent problem-solving skills and attention to detail;
  • Strong communication and teamwork abilities.

Related vacancies