About us: We are currently building a company-wide Data Lake in the Cloud (AWS) that enables us to optimize our Sales priorities, Marketing spend & Pricing.
We are working with leading platform providers (Databricks, DataIKU, AWS, Snowflake, ThoughtSpot, and many more) to aggregate, speed, and enable information to flow.
The team has a worldwide impact with members located on all continents.
You will be leading a team of 8+ data engineers, who are responsible for bringing any data requested by the teams of data scientists into this data lake and ensuring that those assets are properly built (data architecture, efficient ETL processing), documented, and evangelized to a broad audience of users.
Key responsibilities:
Manage large amounts of production-level data pipelines across multiple platforms and types.Assign responsibilities, provide direction, leadership, and coaching, and remove barriers as needed to enable teams to execute their roles and achieve objectives and goals.People recruitment and development.Cost management and direction-setting within the area of responsibility.Give guidance, advise, and bring into our data lake any data required, constrained by our context and technology stack.Leverage our stack in the fastest & most efficient manner: S3, Kubernetes cluster, Dataiku DSS, Spark (pySpark, Scala, Spark R), Python, R, Julia languages, data partitioning.Guide and coach customers to access and use this data appropriately. Requirements:
Typically, 8+ years of relevant experience.Bachelor's degree in Computer Science or Engineering.Experience directly leading and developing a team.Familiar with Agile Project Management.In-depth understanding of technology in direct responsibility.Deep understanding of Spark (experience in Spark optimization would be ideal).Proficiency with pySpark.Experience working in a big data environment is required (HDFS, partitioning concept).SQL proficiency, experience in ETL activities.Experience with Cloud environment (AWS preferred).Experience with Kubernetes, Docker. Nice to have:
Go & Julia.Experience with Dataiku DSS.Python & R, Scala is a plus. What we offer:
Opportunity to work in an international organization with colleagues coming from all over the world.Diverse, continued internal growth and career opportunities, including HP's own learning platform and LinkedIn Learning.An attractive benefits package.Our Women Network organizes activities such as networking, the promotion of STEM vocations, talks on improving business acumen, work-life balance, and skills of the future. Entity sales and services
#J-18808-Ljbffr