Senior Data Engineer - ADF & Data Lake
Royal Dutch Shell
Information Technology (IT)
Job Family Group:
Posting Start Date:
Projects and Technology
Where you fit in
This role is within Shell's Trading and Supply business, one of the world's largest energy trading operations in the world with key hubs in Singapore, Dubai, Houston, London, and Rotterdam. Based in Shell Energy Europe and Environment Products, the role supports trading across European Power, Gas, regulated Carbon and global weather markets, with a focus on the net zero agenda.
The role involves creating optimal data pipeline architecture, assemble large, complex data sets that meet functional / non-functional business requirements. You will work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. The role offers opportunities to work with the latest technologies, top-notch professionals, and hands-on exposure to complex projects which is bound to help one’s career growth.
What’s the role
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal ETL/ELT of data from a wide variety of data sources using SQL and Azure, AWS ‘big data’ technologies
Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other KPI metrics
Keep our data separated and secure across national boundaries through multiple data centres and Azure, AWS regions
Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
Work with data and analytics experts to strive for greater functionality in our data systems
What we need from you
9+ years of experience in a Data Engineer role and 4+ years of relevant experience in Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Databricks, PySpark, and Spark SQL
Hands-on experience with Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure Functions, Serverless Architecture, and ARM Template
Has technical expertise with relational SQL/NoSQL databases, file handlings and API integrations
Good background with object-oriented/object function scripting languages: Python, SQL, Scala etc.
Nice to have experience with any of these toolsets: Kafka, Stream sets, Alteryx, HANA, SLT and BODS
Advanced working SQL knowledge, experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases would be good to complement
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement would be an advantage
Shell is a global energy company where we work towards powering progress through more and cleaner energy solutions. We use advanced technologies and take an innovative approach to help build a sustainable energy future. In India Shell has its businesses footprint in Information Technology, Projects & Technology, Finance Operations, Integrated Gas, Downstream & Upstream spread across more than 7 main locations.
An innovative place to work
There’s never been a more exciting time to work at Shell. Everyone here is helping solve one of the biggest challenges facing the world today: bringing the benefits of energy to everyone on the planet, whilst managing the risks of climate change.
Join us and you’ll add your talent and imagination to a business with the power to shape the future – whether by investing in renewables, exploring new ways to store energy or developing technology that helps the world to use energy more efficiently.
An inclusive place to work
To power progress together, we need to attract and develop the brightest minds and make sure every voice is heard. Here are just some of the ways we’re nurturing an inclusive environment – one where you can express your ideas, extend your skills and reach your potential.
We’re creating a space, where people with disabilities can excel through transparent recruitment process, workplace adjustments and ongoing support in their roles. Feel free to let us know about your circumstances when you apply, and we’ll take it from there.
We’re closing the gender gap – whether that’s through action on equal pay or by enabling more women to reach senior roles in engineering and technology.
We’re striving to be a pioneer of an inclusive and diverse workplace, promoting equality for employees regardless of sexual orientation or gender identity.
We consider ourselves a flexible employer and want to support you finding the right balance. We encourage you to discuss this with us in your application.
A rewarding place to work
Combine our creative, collaborative environment and global operations with an impressive range of benefits and joining Shell becomes an inspired career choice.
We’re huge advocates for career development. We’ll encourage you to try new roles and experience new settings. By pushing people to reach their potential, we frequently help them find skills they never knew they had, or make career moves they never thought possible.
Please note: We occasionally amend or withdraw Shell jobs and reserve the right to do so at any time, including prior to the advertised closing date. Before applying, you are advised to read our data protection policy. This policy describes the processing that may be associated with your personal data and informs you that your personal data may be transferred to Shell/Shell Group companies around the world. The Shell Group and its approved recruitment consultants will never ask you for a fee to process or consider your application for a career with Shell. Anyone who demands such a fee is not an authorised Shell representative and you are strongly advised to refuse any such demand. Shell is an Equal Opportunity Employer.