Data Engineer - Python, Spark - London - £600 per day

  • Location

    London, England

  • Sector:


  • Job type:


  • Salary:

    £84 - £600.00 per day

  • Contact:

    Joe Rinaldi

  • Contact email:

  • Job ref:


  • Published:

    4 months ago

  • Duration:

    6 months

  • Expiry date:


  • Startdate:


  • Consultant:


The role
A data engineer is required to assist with the development and progression of a shared data environment for use across the organisation. You will be building data pipelines and tools that will enable data analysis and contributing to an integrated marketing platform.

Daily responsibilities will include:
*Setting up data processing pipelines utilising multiple sources of third-party data
*Implementation data processing and cleaning, working closely with the wider data team
*Improving the quality of code by reviewing the existing requests and mentoring and coaching
*Monitoring issues in operational systems

Experience required;
*A strong level of writing in Python
*Experience in either writing or maintaining apps in R / Shiny
*Spark or PySpark to process data at scale

It would be advantageous to have experience with the following;
*dealing with unreasonably large amounts of data
*experience in monitoring and making dashboards
*having used AWS, Google Cloud or Azure, in a manual or automated manner
*using Apache Airflow/Google Cloud Composer
*building a web interface with Flask or Django