Moffatt & Nichol is renowned for specializing in large, complex waterfront infrastructure projects and is acknowledged as a global leader in this sector.
We are currently seeking a Data Engineer team in Valencia or Algeciras.
The role involves supporting data infrastructure and pipelines within a large-scale cloud environment, enabling efficient data ingestion, transformation, and storage to support analytical and operational needs.
This position requires implementing robust ETL processes, optimizing data flows, and ensuring adherence to data engineering best practices and standards across microservices ecosystems.
Duties and Responsibilities: Design and implement big data architectures to support scalable ingestion, processing, and storage of large datasets efficiently by using Databricks and Azure. Design, develop, and maintain robust data pipelines to support real-time and batch processing of great volumes of data. Implement ETL processes to collect and transform data from various sources into usable formats. Optimize ETL/ELT processes to move and transform data efficiently between cloud services, data lakes, and databases. Optimize data workflows for performance, scalability, and cost-effectiveness. Monitor and troubleshoot pipeline performance, identifying and resolving issues to ensure continuous data flow. Work on database management, data storage, and data lake/warehouse solutions. Implement best practices for data governance, security, and compliance on cloud platforms. Implement MLOps best practices and tools to track and productionize predictive models. Qualifications: Bachelor's degree in Computer Science, Data Engineering, or a related field. 3-6 years of experience in data engineering, with a focus on cloud platforms, specifically Azure and Databricks. Strong hands-on experience with Apache Spark, Kafka, Flink, and Data Lakes. Experience with NoSQL databases such as Hadoop. Experience building real-time data streaming pipelines using Kafka. Solid understanding of ETL/ELT processes, data transformation, and models productization. Experience productionizing machine learning models with MlFlow. Strong knowledge of data security and governance practices in the cloud. All new hires will be required to successfully complete and pass a pre-employment (post offer) background check in compliance with NIST 800-171.
Working Conditions: Work is generally performed in an office or client office setting.
Ability to use standard office equipment is required.
Travel may be required occasionally.
Moffatt & Nichol's EEO Statement: As a global business, Moffatt & Nichol relies on diversity of culture and thought to deliver on our goal of Creative People, Practical Solutions serving our client needs, and ensures nondiscrimination in all activities.
#J-18808-Ljbffr