Summary: RED is seeking a strong PySpark Developer for a 6-month contract in London, with an ASAP start. The role requires hands-on development experience in PySpark, along with exposure to Databricks and the Azure data ecosystem, focusing on building and optimizing data pipelines. Candidates should possess strong communication skills and the ability to work autonomously in a fast-paced environment. The contract is inside IR35 and follows a hybrid work model, requiring 3 days on-site and 2 days remote. The position is full-time and involves supporting large-scale cloud data processing and developing data wrangling solutions.
Job Title: Pyspark Developer
Description:
PySpark Developer | London | Hybrid | 6 Month Contract | Inside IR35
RED is currently looking for a strong PySpark Developer to join a new project on an initial 6 month contract with an ASAP start. This role is ideal for someone with strong hands-on development experience in PySpark, alongside solid exposure to Databricks and the wider Azure data ecosystem. You will be focused on building and optimising data pipelines, developing data wrangling solutions, and supporting large-scale cloud data processing in a fast-paced delivery environment.
Key skills required:
- Strong hands-on development experience with PySpark
- Experience with PySpark DataFrames, partitioning, SparkSQL optimisation, and clustering
- Strong experience with Databricks
- Knowledge of Azure SQL, Azure Data Factory, and broader Azure Cloud services
- Experience working with Delta tables, Parquet, and CSV file formats
- Experience building and maintaining ingestion pipelines, data wrangling pipelines, and database integrations
- Good knowledge of Azure SQL DB, Storage Accounts, Key Vault, Application Gateways, VNETs, Azure Portal, and Power BI integration
- Strong communication skills and the ability to work effectively with stakeholders across multiple time zones
- Comfortable working autonomously, solving problems, and delivering with minimal direct guidance
Contract details:
- Contract Type: Inside IR35
- Start Date: ASAP
- Duration: 6 Months+
- Location: London
- Work Model: Hybrid – 3 days on-site, 2 days remote
- Workload: Full-time
If you are a strong PySpark Developer with Databricks and Azure experience, please apply with your latest CV
