Data Engineer - DWP - SEO

Job summary

Please note this role requires you to pass Security Check clearance. For further information on vetting and UK residency requirements, please read the 'Security Clearance Requirement' section. 

Do you want to utilise best-in-class technologies to deliver modern cloud-based data products?   

Are you looking for ongoing professional development?  

DWP is the UK's largest government department. We help people into work and make payments worth over £195bn a year to support some of the most vulnerable people in our society.   

DWP Digital creates digital services that almost everyone in the UK relies on at some point in their lives. We’re carrying out a once-in-a-generation digital transformation - a task of size and scale unmatched elsewhere.  

You’ll join the DataWorks team that delivers innovative and insightful data products via Agile ways of working. The team makes Universal Credit data available to our analytical community - by the very nature of what we do, our data pool is extremely large (quite possibly the largest on the market), and you'll have access to all Universal Credit data. 

If you are looking to develop your skills, work on some of the most challenging and rewarding IT projects, shape solutions and contribute to new thinking, all with a great work life balance, we would love to speak with you. 

Job description

DataWorks is an AWS platform that provides Universal Credit data to our data scientists and performance analysts for reporting and statistical purposes. The DataWorks team provides the data products needed to evaluate the Universal Credit service’s performance, which in turn improves user experience and outcomes for millions of citizens across the country.  

The data products you develop will also be used to support ministerial decisions - your work will directly deliver benefits to UK citizens in receipt or need of Universal Credit. 

You’ll play a vital role in delivering data transformation products, ensuring that appropriate data quality, integrity and alerting is built into all solutions. There is a variety of source data in our vast data pools, and integration can be tricky - you’ll need sound judgement to choose the best approach. 

If you like collaborating with data scientists and analysts to understand how best to support them, and conducting proof of concept work, we think you'll enjoy working in this team. 

DWP Digital has a large data engineering community of practice, and opportunities to coach, mentor and share best practice are plentiful. We welcome fresh ideas on how to build data engineering knowledge and capability across our teams.  

DWP Digital is a great place to work, learn and grow your career. We provide comprehensive training and a collaborative environment that encourages continuous professional development.  

We care about our employees and can offer you an environment that is family-friendly, supportive and inclusive in an organisation with real social purpose. 

Person specification

When giving details in your employment history and personal statement you should highlight your experience in line with the essential criteria below:

• Experience of developing, testing, maintaining, deploying and supporting efficient, scalable, automated ETL pipelines in an AWS cloud environment using AWS S3 Cloud Object Storage 
• Strong coding skills using Hive SQL, Spark SQL, Python, PySpark and Bash 
• Experience of working with a wide variety of structured and unstructured data.

Knowledge and experience of the following tools:

- Hue

- GitHub/Lab 

- Azkaban (or similar scheduling tools)

- CI/CD (Concourse or similar)

- R-Studio

- Juypter-Lab

- Kafka 

If you would like to learn more about the role, please contact caicy.sandford1@dwp.gov.uk.

Technical skills

We'll assess you against these technical skills during the selection process:

• Technical Breadth

Benefits

Alongside your salary, the Department for Work and Pensions contributes 28.97% towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

We also have a broad benefits package built around your work-life balance which includes:

• Working patterns to support work/life balance such as job sharing, term-time working, flexi-time and compressed hours.
• Generous annual leave – at least 23 days on entry, increasing up to 30 days over time (pro–rata for part time employees), plus 9 days public and privilege leave.
• Support for financial wellbeing, including interest-free season ticket loans for travel, a cycle to work scheme and an employee discount scheme.
• Health and wellbeing support including our Employee Assistance Programme for specialist advice and counselling and the opportunity to join HASSRA a first-class programme of competitions, activities and benefits for its members (subscription payable monthly).
• Family friendly policies including enhanced maternity and shared parental leave pay after 1 year’s continuous service.
• Funded learning and development to support progress in your role and career. This includes industry recognised qualifications and accreditations, coaching, mentoring and talent development programmes.
• An inclusive and diverse environment with opportunities to join professional and interpersonal networks including Women’s Network, National Race Network, National Disability Network (THRIVE) and many more.

This job role may be suitable for hybrid working, which is where an employee works part of the week in their DWP office and part of the week from home. This is a voluntary, non-contractual arrangement and your office will be your contractual place of work. The number of days that anyone will be able to work at home will be determined primarily by business need but personal circumstances and other relevant circumstances will also be taken into account. If you are successful, any opportunities for hybrid working, including whether a hybrid working arrangement is suitable for you, will be discussed with you prior to you taking up your post.

Salary Information

Pay for this role is from £40,201 to £58,347.

The maximum salary for the grade is £43,347, however a Digital Allowance of up to £15,000 per annum is available for exceptional candidates, based on our assessment of your skills and experience.

Our offer to successful candidates will be based on an assessment of your skills and experience as demonstrated at interview.

Existing Civil Servants who secure a new role on lateral transfer should maintain their current salary.

Existing Civil Servants who gain promotion may move to the bottom of the next grade pay scale or 10% increase in salary whichever would be the greater.

Things you need to know

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Experience and Technical skills.

Stage 1: Application

Your application will consist of three parts:

1. A Personal Details application form.

2. Your employment history detailing your responsibilities, skills, accomplishments, plus your qualifications and relevant training. Please copy this information into the box field provided.

3. Technical statement up to 250 words answering the following question:

Technical Breadth - What is the most complex data engineering project you've worked on using AWS ecosystem, S3 (Cloud Object Storage), Python/PySpark or Hive/SQL? What made the project complex? How did you contribute?

Please use the STAR (Situation, Task, Action Result) approach to answer this question.

You may use bullet points if it helps clarify your answer and make it more concise.

An initial sift will be conducted using the technical statement (Technical Breadth). Candidates who pass the initial sift will be progressed to a full sift. The sift panel will use the information in your employment history to assess your experience, skills and knowledge against the essential criteria above.

For Hints and Tips on completing your application visit Applying for jobs at DWP Digital.

Applications will be sifted at regular intervals from the date the posts are advertised. Please apply as soon as you can, do not wait until the end of the campaign. 

Important Information

• You will be asked to complete your employment history any information that you would customarily share on a CV should therefore be entered onto the application form.
• Personal details that could be used to identify you including your name, contact details and address must be removed for your application to be considered.
• If your employment history or technical statement contains any personal details your application will be withdrawn.

Stage 2: Interview

If you’re successful at sift stage you will be invited to a video interview via Microsoft Teams. There, you will be assessed against the experiences listed in the essential criteria.

Interviews will take place from late October 2024. Interview dates to be confirmed.

Question in Advance

To help you prepare and settle into the interview you will be sent the experience questions in advance of the interview, seven days prior to your interview.

These questions should be treated as confidential and should not be shared. The interview panel may ask you other questions which will not be shared in advance, including follow-up questions, and those about your experience and technical abilities.

Candidates should be asked further probing questions if their response is deemed to require further information to determine suitability for the role.

Candidates are strongly encouraged to secure their interview slot promptly to ensure fairness in preparation time with the questions provided in advance.

Further information:

Find out more about Working for DWP 

A reserve list may be held for a period of 6 months from which further appointments can be made.

All successful candidates and those placed on reserve will be posted in merit list order by location.

Security Clearance Requirement

You must meet the security requirements before you can be appointed. The level of security needed is security check.

For meaningful checks to be carried out, you will need to have lived in the UK for a sufficient period of time, to enable appropriate checks to be carried out and produce a result which provides the required level of assurance. Whilst a lack of UK residency in itself is not necessarily a bar to a security clearance, and expectation of UK residency may range from 3 to 5 years. Failure to meet the residency requirements needed for the role may result in the withdrawal of provisional jobs offers.

For further information on National Security Vetting please visit the following page https://www.gov.uk/government/publications/demystifying-vetting

Reasonable Adjustment

At DWP we value diversity and inclusion and actively encourage and welcome applications from everyone, including those that are underrepresented in our workforce.

We consider visible and non-visible disabilities, neurodiversity or learning differences, chronic medical conditions, or mental ill health. Examples include dyslexia, epilepsy, autism, chronic fatigue, or schizophrenia.

If you need a change to be made so that you can make your application, you should: Contact Government Recruitment Service via DigitalRecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.

Complete the “Reasonable Adjustments” section in the “Additional requirements” page of your application form to tell us what changes or help you might need further on in the recruitment process. For instance, you may need wheelchair access at interview, or if you’re deaf, a Language Service Professional.

For further information on reasonable adjustments, terms and conditions and how we recruit visit the How we recruit page. 

The Civil Service values honesty and integrity and expects all candidates to abide by these principles. Please ensure that all examples provided in your application are taken directly from your own experience and that you describe the examples in your own words. Applications will be screened and if evidence of plagiarism or copying examples/answers from other sources is found, your application will be withdrawn. Internal DWP candidates may also face disciplinary action.

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a criminal record check.

Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).

See our vetting charter (opens in a new window).

People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).

The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).