Python Data Engineer

New Today

Python Engineer
Contract | £400£458 per day (Inside IR35)
Active SC Clearance is essential
Remote

We are seeking a highly capable Python-focused Data Engineer to join a delivery-driven team building and supporting complex data platforms in Azure. This role is heavily weighted towards Python software engineering rather than traditional ETL-only work. The successful candidate will be someone who writes clean, maintainable, and well-tested Python code, and is comfortable treating data pipelines as production-grade software.
A significant portion of the work involves designing and maintaining complex, test-driven Python data flows, with PySpark used as the execution engine rather than the primary focus. Strong Python fundamentals, testing discipline, and code quality are critical to success in this role.
What youll be doing
Designing and building scalable data pipelines with a Python-first approach
Developing complex data flows with a strong emphasis on clean architecture, reusable Python modules, and testability
Writing comprehensive unit tests and BDD tests (Behave), including mocking and patching
Using PySpark to process large-scale datasets while keeping business logic in Python
Creating, maintaining, and optimising Delta Lake tables for performance and reliability
Building and running applications in containerised (Docker) environments
Integrating Python applications with Azure services such as Azure Functions, Key Vault, and Blob Storage
Working closely with DevOps and engineering teams to support CI/CD pipelines
Debugging, tuning, and improving Python and Spark workloads in production
Following best practices in secure Python development, cloud security, and data governance
What were looking for
Strong, hands-on Python development experience (essential)
Proven experience writing test-driven Python code in production environments
Solid data engineering experience using PySpark
Experience working with Delta Lake
Hands-on experience with Docker and containerised workflows
Good knowledge of Azure and integrating Python applications with cloud services
A software-engineering mindset and comfort working in fast-paced, delivery-focused teams

TPBN1_UKTJ
Apply
Location:
Canning Town
Salary:
£400 - £458 per day, Benefits remote
Job Type:
FullTime
Category:
IT;IT

We found some similar jobs based on your search