Summary: The Lead Data Engineer role involves designing, building, and optimizing enterprise-scale data pipelines and cloud platforms in a fast-paced environment focused on innovative, smoke-free alternatives. The position requires collaboration across multi-disciplinary teams and external vendors to translate business and AI/ML requirements into scalable solutions. Key responsibilities include building data pipelines, supporting machine learning initiatives, and implementing MLOps. Candidates should be strategic and independent, excited to work at the forefront of data and AI innovation.
Key Responsibilities:
- Build and orchestrate data pipelines across Snowflake and AWS environments.
- Apply data modeling, warehousing, and architecture principles (Kimball/Inmon).
- Develop pipeline programming using Python, Spark, and SQL; integrate APIs for seamless workflows.
- Support Machine Learning and AI initiatives, including NLP, Computer Vision, Time Series, and LLMs.
- Implement MLOps, CI/CD pipelines, data testing, and quality frameworks.
- Act as an AI super-user, applying prompt engineering and creating AI artifacts.
- Work independently while providing clear justification for technical decisions.
Key Skills:
- Strong experience in data pipeline development and orchestration.
- Proficient with cloud platforms (Snowflake, AWS fundamentals).
- Solid understanding of data architecture, warehousing, and modeling.
- Programming expertise: Python, Spark, SQL, API integration.
- Knowledge of ML/AI frameworks, MLOps, and advanced analytics concepts.
- Experience with CI/CD, data testing frameworks, and versioning strategies.
- Ability to work effectively in multi-team, vendor-integrated environments.
Salary (Rate): undetermined
City: London
Country: United Kingdom
Working Arrangements: hybrid
IR35 Status: undetermined
Seniority Level: undetermined
Industry: IT
Job Title: Lead Data Engineer
Location: London (Hybrid)
Contract: 6 Months (Potential Extension)
Start Date: ASAP
About The Client
Our client is transforming their industry by replacing cigarettes with innovative, smoke-free alternatives. They are leveraging technology, data, and AI to drive a global shift toward a smoke-free world. This is a fast-paced, high-impact environment, perfect for candidates who are strategic, independent, and excited to work at the forefront of data and AI innovation
The Role
We are looking for a skilled Data Engineer to design, build, and optimize enterprise-scale data pipelines and cloud platforms. You will translate business and AI/ML requirements into robust, scalable solutions while collaborating across multi-disciplinary teams and external vendors. As a key member of the data architecture you will:
- Build and orchestrate data pipelines across Snowflake and AWS environments.
- Apply data modeling, warehousing, and architecture principles (Kimball/Inmon).
- Develop pipeline programming using Python, Spark, and SQL; integrate APIs for seamless workflows.
- Support Machine Learning and AI initiatives, including NLP, Computer Vision, Time Series, and LLMs.
- Implement MLOps, CI/CD pipelines, data testing, and quality frameworks.
- Act as an AI super-user, applying prompt engineering and creating AI artifacts.
- Work independently while providing clear justification for technical decisions.
Key Skills & Experience
Strong experience in data pipeline development and orchestration. Proficient with cloud platforms (Snowflake, AWS fundamentals). Solid understanding of data architecture, warehousing, and modeling. Programming expertise: Python, Spark, SQL, API integration. Knowledge of ML/AI frameworks, MLOps, and advanced analytics concepts. Experience with CI/CD, data testing frameworks, and versioning strategies. Ability to work effectively in multi-team, vendor-integrated environments.
Why This Role
Join a global, transformative initiative shaping a smoke-free future. Work with cutting-edge cloud, AI, and data technologies. Opportunity to influence technical and strategic decisions across enterprise data delivery. Dynamic, innovative environment where your work has real business impact.
