Data Engineer

Essential Duties And Responsibilities:

  • Build and maintain real-time and batch data pipelines across the advanced analytics platform.
  • Design, develop and orchestrate highly robust and scalable ETL pipelines.
  • Design and implement Dimensional and NoSQL data modelling as per the business requirements.
  • Develop highly optimal codebase and perform Spark optimizations for Big Data use cases.
  • Design, develop and deploy optimal monitoring and testing strategy for the data products.
  • Collaborate with stakeholders and advanced analytics business partners to understand business needs and translate requirements into scalable data engineering solutions.
  • Collaborate with data scientists to prepare data for model development and production.
  • Collaborate with data visualization and reporting application developers to ensure the sustainability of production applications and reports.
  • Maintain professional and technical knowledge by attending educational workshops, professional publications, establishing personal networks, and participating in professional societies.

Experience:

  • 4+ years of professional experience

Knowledge, Skills, and Abilities:

  • Natural sense of urgency, teamwork, and collaboration reflected in daily work ethic.
  • Expert in Spark or Databricks, Cloud Data Engineering Services preferably Azure, Streaming frameworks like Event Hubs or Kafka.
  • Proficient in Microsoft Office.
  • Familiarity with modern Machine Learning Operationalization techniques.
  • Agile methodologies.
  • Familiarity with Data visualization tools, such as Qlik or Power BI.