Databricks Engineer (SC Cleared)
New Yesterday
Location: London (Hybrid)
Contract: 6 Months | £550-600/day (Inside IR35)
Start Date: ASAP (Dec 2025 / Jan 2026)
About the Role
We are seeking an experienced Databricks Engineer with active SC Clearance to join a high profile GovTech project. You will play a key role in supporting a large scale data migration and developing a cutting edge Azure Databricks platform for economic data modelling, analysis, and forecasting. This is an exciting opportunity to work on a mission critical program that influences national economic insights.
Key Responsibilities
Design, develop, and optimize data pipelines using Azure Databricks and PySpark.
Support data migration activities from legacy systems to modern cloud based architecture.
Implement Delta Lake for efficient data storage and retrieval.
Collaborate with data scientists and analysts to enable advanced economic modelling and forecasting.
Ensure data security and compliance within a government environment.
Troubleshoot performance issues and optimize Spark jobs for scalability.
Contribute to CI/CD processes and automation for data workflows.
Required Skills & Experience
Active SC Clearance (mandatory).
Strong hands on experience with Azure Databricks and Spark.
Proficiency in PySpark, SQL, and Python (Scala desirable).
Experience with Azure services: Data Lake, Synapse, Key Vault.
Proven track record in ETL pipeline development and data migration.
Familiarity with Delta Lake and data governance best practices.
Knowledge of CI/CD tools (e.g., Azure DevOps, Git).
If this looks like the role for you, apply now.
TPBN1_UKTJ
- Location:
- Sheffield
- Salary:
- £550 - £600 per day
- Job Type:
- FullTime
- Category:
- IT
