OverviewAs a Data Engineer, you will provide solutions on the data processing, engineering, and enrichment phase for various projects. You will be responsible for some aspects of the data life cycle including design, maintenance, and building of cloud and data architectures. These activities will deliver on ingestion, transformation, and building of high-quality data structures.You will be responsible for:Creating, monitoring, and troubleshooting ELT jobs to ingest and transform data from various systems using batch and stream processing; Driving and advocating for state-of-the-art methods of handling data in the company, e.G., data lake, stream processing, IaC, DevOps, DataOps; Collaborating with analytics and business teams to improve data feed.As a Data Engineer, the majority of your work will involve performing new data ingestions from various sources such as databases, APIs, and files. This will require your skills in extracting, loading, and transforming data into a usable format for downstream analysis. The remaining of your work will involve supporting the maintenance of existing ETL jobs. This will require your attention to detail and strong problem-solving skills in monitoring, troubleshooting, and resolving any issues that arise. Your role will be critical in ensuring the successful ingestion and processing of new data while maintaining the integrity and efficiency of existing ETL jobs.You will be working with the following technologies:Python, Prefect (preferred), Dask, SQL; Kafka (preferred);AWS and/or Azure; Docker;Linux, bash,git; CI/CD.What will you needto be successful in the role?Demonstrated proficiency in the data engineering area; Intermediate Python;Strong SQL; Understanding of cloud solutions; Understanding of Docker;Familiarity with Linux; Experience with ETL.Whatdo we offer?A lot of mentoring and training. We want you to thrive as a member of our team and for that reason, we will provide you with a lot of opportunities to grow both personally and professionally; A relaxed working environment with lots of support, nurturing, and fun; A good work-life balance. We believe that only happy, well-rested employees can work well and do good.About dyveniadyvenia supports some of the biggest and most respected organizations in Europe in leveraging Cloud infrastructure, advanced analytics, and solid data foundations so that they can gain actionable insights and make impactful business actions based on their data. We work with state-of-the-art technologies, have some of the best people on our team, and, yes, we also get to have fun in the process.Salary: Up to 45,000.00€ per year#J-18808-Ljbffr
