Senior Data Operations Engineer - OFGEM - G7

Use your data expertise to help us change the future of how we use data.  
  
It is important that we become a fully digital regulator so that we can transform the energy sector. We are therefore embarking on a high-profile data and digitalisation programme, and it is within this scope of work that you could find a career-defining challenge.   
  
Our team is working with the most cutting-edge cloud and data technologies, so this is a great opportunity to further your knowledge with the support of experienced colleagues. As such, you will help deliver our new Strategic Common Data Platform, which will connect internal and external data sources. Taking responsibility for designing and developing Azure pipelines (which will include data transformation and cleansing), you will create accessible data sets that can be easily distributed and analysed. You will also engineer manual data flows that are both scalable and repeatable, and you will document source-to-target mappings. Across all that you do, you will align your projects and datasets with Purview.   
  
This is an exciting opportunity to network extensively with both internal colleagues and external partners. You will therefore be a natural fit in our team if you have good communication skills and you can bring a proactive mindset to identifying and resolving issues in databases, and data processes, products and services. With a background of working with Azure Data Lakes, data warehousing, and storage accounts, you will be capable of building all pipeline facets including data interfaces. Proficient in in T-SQL and Python, you will be able to transform data into appropriate storage formats, engineer data for analytics and devise error handling.   
  
Ofgem relies on having a workforce that reflects the society we serve, so we welcome candidates from all backgrounds, and especially those from underrepresented groups. While this is a full-time role, flexible working patterns and job shares are welcome, and we offer a fully hybrid environment. We’ll also support you with excellent training and development opportunities, plus a competitive pension and generous annual leave.

Job description

We play a key role in supporting Ofgem to be a dynamic, inclusive and high performing organisation, both striving to continuously improve our efficiency and effectiveness in meeting the needs of the organisation, and also setting clear corporate standards and controls in order to ensure the organisation operates within our statutory and legal requirements, and good practice standards.    

The Data Operations Engineer is an expert role to help accelerate data driven decision making, digital transformation and innovation across Ofgem through the provision of high quality, timely and valuable data and analytics in a resilient and consistent manner.   

The role will be key to achieving Ofgem’s vision of Digitisation and supporting the innovation and growth agenda of: 

• Making OFGEM a recognised digitalisation leader with innovative approaches in the exploitation of data using emerging technologies 
• Establishing the capabilities to focus on specialist areas including data analytics/insights, data innovation, artificial intelligence, data sourcing and automated data pipelines  
• Training our workforce to be comfortable and skilled in understanding, using and asking questions of data, visualisations and analytics output 
• Achieving cost efficiencies in targeted areas e.g. automate data preparation and processing, automate production of commonly used reporting, share data live online  

This role oversees the service provided by our modern cloud data platform.  This platform includes; data ingestion, data processing, data quality improvement*, master data management*, data modelling*, data science* and data visualisation*.   

Items with a * have peer team leads responsible for specifying and leading development of new content but this role could help deliver and will be responsible for ongoing running of our end to end data analytics pipelines. 

This role will work with the Head of Data and AI Enablement to monitor our end to end service, establishing trust in the data and analytics provided and will aim to optimise the usefulness and value it provides. 

Your initiative and ideas on how to measure, grow and improve the following will be required: 

1. Levels of usage of data/data products e.g. user session count, user session duration 
2. Levels of value provided by each data product e.g. dashboard 
3. Levels of value provided by each underlying data set e.g. source file 
4. User trust and satisfaction e.g. net promoter scores or review feedback 

This role works closely with the Head of Data Management and Operations to deliver priority improvements, oversee current work and improve data platform service levels.  This along with the proposed benefit of each proposed action will help this role holder demonstrate service value ongoing as well as the planned functional improvements each work package will deliver. 

Person specification

Key Responsibilities

Delivery of set of team deliverables, and managing and co-ordinating successful outcomes across team members and collaborating teams 

• Sets clear and transparent work objectives, milestones and success metrics at team and individual level to oversee and co-ordinate successful team outcomes.
• Collaborates closely with other teams to manage interdependencies, risks and resourcing to support portfolio delivery.
• Jointly responsible with the PDL on welfare and pastoral care of all colleagues.
• Demonstrates effective diversity and inclusive team management within their team and the wider organisation. 

Essential Criteria 

• Experience monitoring impact, service quality or value delivered from work, platform and service (LEAD)
• Experience using and building modern cloud data technologies (e.g. Microsoft Azure Data Factory, Microsoft Azure Synapse, Microsoft Azure Machine Learning, Microsoft Power BI service) to deliver resilient and scalable data pipelines (LEAD)
• Demonstrable experience in data analytics, data technologies or development processes
• Experience prioritising development time, effort or financial resource across a portfolio of work
• Clear written and verbal communication at a senior management level
• Experience of using master data management techniques to improve data quality and integrity 

Desirable Criteria 

Experience in:

• Service design or implementation
• DevOps processes and methodologies
• Agile/scrum process and methodologies 

Experience in:

• ETL development programming in Python and Pyspark

Behaviours

We'll assess you against these behaviours during the selection process:

• Changing and Improving
• Working Together
• Delivering at Pace

Technical skills

We'll assess you against these technical skills during the selection process:

• Please refer to the Candidate Pack and Role Profile attached for full details.

Benefits

Alongside your salary of £45,614, OFGEM contributes £13,214 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

Ofgem can offer you a comprehensive and competitive benefits package which includes; 30 days annual leave after 2 years; Excellent training and development opportunities; The opportunity to join the generous Civil Service pension which also includes a valuable range of benefits; Hybrid working, flexible working hours and family friendly policies. Plus lots of other benefits including clean and bright offices based centrally, engaged networks and teams and an opportunity to contribute to our ambitious and important targets of establishing a Net Zero energy system by 2050. This exciting blend of professional challenge and personal reward identifies career opportunities at Ofgem as something to get excited about.

Things you need to know

Selection process details

This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.

When you press the ‘Apply now’ button, you will be asked to complete personal details (not seen by the sift panel), and upload a copy of your CV anonymising all details where necessary.  

You will then be asked to answer 3 Technical Questions evidencing how you meet the essential and desirable skills and capabilities listed in the role profile. Please ensure you demonstrate clearly, within these answers how you meet each of the essential and desirable skills and capabilities. 

The Civil Service values honesty and integrity and expect all candidates to abide by these principles. Ofgem take any incidences of cheating very seriously. Please ensure all examples provided are of your own experience. Any instances of plagiarism or other forms of cheating will be investigated and, if proven, the relevant applications will be withdrawn from the process. 

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a criminal record check.

Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).

See our vetting charter (opens in a new window).

People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).

The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).