The HP Ecommerce CRM enablement team is seeking for its CDP program a proven and hands-on Databricks process architect and engineering leader to drive our Databricks transformation, pipelines architecture, governance and engineering patterns. Our CDP delivers a unified Customer data model across our global CRM operations. In this role, you will be responsible for driving our Databricks migration, designing and executing a comprehensive pipeline architecture strategy that aligns with CDP team's skillsets, business objectives and drives compliant, efficient and effective data management.
ResponsibilitiesDrive our AWS Databricks transformation from current relational DB setup.Collaborate with our diverse teams, SQL developers/data engineers/CRM analysts, to develop, implement and maintain sustainable, high-performance, growth-ready data-processing and data integration systems.Provide best practice on CI/CD techniques with Git and Databricks.Provide best practice on Databricks cluster management.Reviews and evaluates designs and project activities for compliance with privacy, architecture, security and quality guidelines and standards; provides tangible feedback to improve product quality and mitigate failure risk.Design and maintain an appropriate metadata repository to enable the organization to understand its data assets.Build and manage relationships throughout the organization.Drive innovation and integration of new technologies into projects and activities in the big data space.Collaborate and communicate with project team regarding project progress and issue resolution.Provide guidance and mentoring to less experienced staff members.RequirementsBachelor's or Master's degree in Computer Science, Information Systems, Engineering, or equivalent.10+ years of experience in data architecture, database design, and data management.Expert in AWS Databricks.Expert in SQL & relational database systems (PostgreSQL preferred).Proficient knowledge in working with distributed data processing frameworks like Apache Spark.Hands-on experience in Python and Pandas.Experience in overall architecture of big data systems within AWS.Experience in Global Companies a great advantage.Knowledge & SkillsExtensive experience with designing data engineering solutions to manage data streams in complex, distributed and parallel systems.Excellent analytical and problem-solving skills. Ability to collaborate with and analyze source systems to drive data profiling/cataloging. Can present clear insights to colleagues to support the end use of the data.Extensive experience with data engineering pipelines, frameworks to mine, cleanse and explore data.Strong problem management. Initiating and monitoring actions to investigate patterns and trends to resolve problems. Determining preventative measures.Evaluating forms and processes for database architecture testing and methodology, including writing and execution of test plans, debugging, and testing scripts and tools.Excellent written and verbal communication skills; mastery in English and local language.ComplexityFrequently contributes to the development of new ideas and methods.Works on complex problems where analysis of situations or data requires an indepth evaluation of multiple factors.Acts as an expert providing direction and guidance to process improvements and establishing policies.Exercises significant independent judgment within broadly defined policies and practices to determine best method for accomplishing work and achieving objectives.Provide mentoring and guidance to lower level employees.
#J-18808-Ljbffr
