Data Engineer - Scientific Engine (Airflow, Dvc) - Cdi

Detalles de la oferta

he Wise Seeker is the leading HR technology company in unbiased talent evaluation. With over 15 years in the industry analyzing the needs and demands of the job market, we are capable of identifying the best talent for each company thanks to our team of professionals and our SaaS platform integrated with Artificial Intelligence. We are efficient, evaluate talent objectively without bias, and close hiring times in record time, delivering optimal results. ABOUT YOUR ROLE Due to our consistent growth, we are expanding our Data, Software and DevOps team. We are seeking profiles dedicated to data engineering. At the core of the development of our scientific engine modeling climate phenomena, your main missions will be to create, improve and maintain the data pipelines used to train our model and infer the different scenario to make a climate risk assessment. You will have to take initiative and assess the viability of proof of concept projects. You will have to work with data scientists and software engineers to run and develop our models. You will be working along DevOps engineers to reliably put models in production and selected the compute/store instance needed to perform these tasks. Your secondary mission will be to automate the flow of information between the tech and business to monitor climate events. KEY MISSIONS Setup, automate, maintain and update: Connections to external and internal APIs; Data preparation process; Model training and inference process; Data storage process; Associated CI/CD pipelines; Associated package versioning and releasing pipeline; Modularization of code base; Notification tools to inform the team of the status of the operations. Setup data storage, data processing and data visualizing tools, by : Assessing the pains and needs of the teams; Benchmarking the open source and private solutions; Assessing the security, price and reliability of data architecture; Following the development the evolution of technologies on the topic; Forecasting the usage of the tools; Tracking the cost of the tools. Participate in: Tech stack selection; Discussions with tech partners; Training of software and underwriting teams; Support and debug of internal users. TECH STACK ? Cloud provider: GCP Code versioning tool: Git Gitlab OS: Windows Container: Docker Container orchestrator: Kubernetes Website architecture: LAMP Code base: Python Notification tool: Slack DATA STACK ? Types: images, timeseries, Storage: GCP bucket Version: DVC (roll out in progress) Pipeline: Airflow (PoC stage) Data base: to be setup depending on the use cases In our project, data is collected by sensors (satellite, weather station, IoT). We don't work with personal or sensitive data, in most cases the data is publicly available (earthquake magnitude, cyclone track, precipitation …). ABOUT YOU EXPERIENCE & QUALIFICATIONS ?? [Hard skills] Knowledge of the tech stack or equivalent tools; Experience converting python code to efficient data engineering tools (eg: spark); Experience with Docker; Experience with a cloud provider (GCP, AWS or azure); Experience automating a CI/CD pipeline; Good knowledge in English and fluency in French. [Soft skills] Desire to train junior developers and explain CI/CD and cloud tools; Desire to suggest improvements to the architecture. [Nice-to-have] Experience working data science project or scientific code; Experience with Kubernetes; Experience in HPC; Contribution to an open source project. MINDSET Strong interest with climate issue (it's not a hoax, many people suffer from it); Being comfortable to work alongside corporate insurers (some still wear suits ); You enjoy CI/CD automation (or at least appreciate the elegance of a well-crafted pipeline); Strong team spirit and ability to work (you'll have to review code and have your code reviewed); Rigorous, creative and meticulous mind (we handle large insurance, we take our time); Strong desire to learn (there's no limitation to the tech used, we're happy to test and learn new tools); Eagerness to work in a multi-cultural environment (policies and teams are from all around the world ?). RECRUITMENT PROCESS Step 1: Call and HR Interview with our Talent Recruiter Step 2: Technical project submitted via GitHub Step 3: Technical interview Step 4: Manager interview Step 5: Final round interview with the team (Candidates can opt to have the manager interview before the technical project and interview) LI-YM1


Salario Nominal: A convenir

Fuente: Adzuna_Ppc

Requisitos

Técnico/A De Prl - Ribarroja

¿Tienes titulación de Técnico Intermedio o Técnico Superior de Prevención de Riesgos Laborales? ¿Tienes la experiencia en inspecciones periódicas en campo? ¡...


Tradivel - Otra

Publicado 8 days ago

Multilingual Opportunities In Barcelona & Madrid

About the job Join Us in Sunny Spain Do you speak Dutch, German, Danish, Swedish, or French? Love sales, marketing, or customer service? We've got awesome ca...


Recruit4Work Sl - Otra

Publicado 8 days ago

Email Specialist

Detta är en tillfällig tjänst för att täcka föräldraledighet under perioden Januari 2025 till December 2025. Är du redo att ta nästa steg i din karriär inom ...


Dustin Ab - Otra

Publicado 8 days ago

Desarrollador/A De Software

Se trata de una vacante estable de contratación directa con la empresa, con proyección de carrera interna, por lo que podrás olvidarte de la incertidumbre de...


Your Talent Side - Otra

Publicado 8 days ago

Built at: 2024-11-09T01:35:50.569Z