Data Engineer – Python & Airflow

Envision

Posted on: June 27, 2024

Closing: July 27, 2024

Position Type: Full Time

Job Description

As a Data Engineer working as part of our collaborative and talented team, you will work on our Python stack, which includes our data lake and processing pipelines.

This is a hybrid working opportunity and you will be required to work 3 days a week from our Cambridge office (CB1 2JH).

 

Role responsibilities 

How will you make an impact?

  • Design, develop, and maintain scalable and efficient data pipelines for ingesting, processing, and transforming large volumes of data.
  • Optimize and tune existing data infrastructure for performance, reliability, and scalability.
  • Find, extract and validate valuable data from publicly available and proprietary data sources to extend internal data sources.
  • Implement data governance and security measures to ensure the integrity and confidentiality of sensitive information.
  • Develop and maintain a long-term vision for key components of our data infrastructure, ensuring scalability, reliability, and performance.
  • Design and implement new features, demonstrating a deep understanding of data engineering principles and industry best practices.
  • Work with cloud platforms to deploy and manage data solutions.
  • Collaborate with cross-functional teams to understand business needs and provide data engineering support for various projects.
  • Monitor and troubleshoot data pipeline issues, ensuring timely resolution and minimal disruption to data workflows.
  • Stay current with industry trends and emerging technologies in data engineering and analytics to provide input during planning discussions to help sharp the direction of projects and initiatives.

 

Do you have what we are looking for?

  • Proven experience as a Data Engineer, with a strong portfolio of data-related projects.
  • Proficiency in modern Python programming, ideally Python 3.9 or later.
  • Familiarity with version control systems, such as Git.
  • Solid understanding of data modelling, database design, and ETL processes.
  • Experience with processing pipeline framework Airflow, or similar frameworks such as Kedro, Luigi or Argo.
  • Proficiency in SQL and experience with both relational and NoSQL databases.
  • Familiarity with data warehousing solutions and technologies.
  • Familiarity with cloud data processing and compute services e.g. EC2, S3 in AWS or equivalent in Azure or GCP
  • Experience with data processing libraries like Pandas, NumPy or Dask.
  • Experience with big data technologies such as Apache Spark, Hadoop, or similar frameworks is appreciated but optional.

 

Do you have experience in any of our bonus areas?

  • Familiarity with back-end technologies such as Alembic, FastAPI or Flask
  • Experience leading and/or coordinating implementation of (data-)-engineering features
  • Partnering with other technical teams to prepare technical requirements and designs
  • Mentoring junior colleagues to promote technical upskilling
  • A product-led engineering mindset

 

Magic happens when people care deeply about what they do!

We're on a mission to use Artificial Intelligence to transform healthcare as we know it.

We're pioneers. Everyone here is passionate about their impact on our products and teams and ultimately, the world. This is how we became leaders in what we do.

If you're looking for fascinating and complex projects and can thrive on change, you've found the right place.

Envision

Posted on: June 27, 2024

Closing: July 27, 2024

Position Type: Full Time

Similar Jobs


Poland

Project Director

The Project Director is responsible for oversight of the conduct of a pr...

Canada

Project Coordinator

The Project Coordinator works with Project Managers, Clinical Trial Mana...

Canada

Associate Director, Project Delivery

The Associate Director, Project Delivery is responsible for line managem...

Toronto, Canada

Downstream Process Subject Matter Expert...

Role Overview: We are looking for an experienced Downstream Process Spec...