Data engineer will have the unique combination of business acumen needed to interface directly with key stakeholders to understand the problem along with the skills and vision to translate the need into a world-class technical solution using the latest technologies
This person will be a hands-on role who is responsible for building data engineering solutions for NMG Enterprise using cloud based data platform. They will provide day-to-day technical leadership and active oversight for technical design, development and support for data engineering workloads. In this role, you need to be equally skilled with the whiteboard and the keyboard.
Work with business partners and data science teams to understand business context and craft best-in-class solutions to their toughest problems
Develop deep understanding of business knowledge of data and Retail domain
Create robust and automated pipelines to ingest and process structured and unstructured data from source systems into analytical platforms using batch and streaming mechanisms leveraging cloud native toolset
Lead building a scalable next gen Cloud platform to so support data transformation, ML and AI Strategy including migration of current platform to AWS.
Work with data scientists to operationalize and scale machine learning training and scoring components by joining and aggregating data from multiple datasets to produce complex models and low-latency feature store
Lead and guide the evolution of various data management functions (Data Catalog, Data Prep, Data Lineage & Metadata Management functions)
Provide hands-on technical leadership in all aspects of data engineering design and implementations including data ingestion, data models, data structures, data storage, data processing, and data monitoring at scale
Develop data engineering best practices with considerations for high data availability, computational efficiency, cost, and quality
Lead, Design and develop automation to optimize data platform compute and storage resources
Evaluate and make recommendations on new technologies, tools and guide the team on up-to-date technologies, standards, and practices
Coach and mentor more junior resources
Contribute to the internal knowledge base to build expertise and awareness within the organization
Assist in our recruiting and interviewing process
Participate in developing projects plan, timelines and providing estimates
Participate in daily scrum calls and provide clear visibility to work products
BS in Computer Science or related field
6+ years of experience in the data and analytics space
Certification –preferably AWS Certified Big Data or any other cloud data platforms, big data platforms
4+ years experience developing and implementing enterprise-level data solutions utilizing Python (Scikit-lean, Scipy, Pandas, Numpy, Tensorflow) , Java, Spark, and Scala, Airflow , Hive and Python.
3+ years in key aspects of software engineering such as parallel data processing, data flows, REST APIs, JSON, XML, and micro service architectures.
2+ year of experience working on Big Data Processing Frameworks and Tools – Map Reduce, YARN, Hive, Pig, Oozie, Sqoop, and good knowledge of common big data file formats (e.g., Parquet, ORC, etc.)
6+ years of RDBMS concepts with Strong Data analysis and SQL experience
3+ years of Linux OS command line tools and bash scripting proficiency
Nice to have:
Kubernetes and Docker experience a plus
Prior working experience on data science work bench
Cloud data warehouse experience - Snowflake is a plus
Data Modeling experience a plus
Knowledge, Skills and Abilities:
A passion for technology and data analytics with a strong desire to constantly be learning and honing skills
Ability to deliver independently without oversight
Be productive even with ambiguity and highly fluid requirements during initial stages of projects
Flexibility to work in matrix reporting structure
Experienced in implementing large scale event based streaming architectures
Strong communication and documentation skills
Working knowledge of NoSQL, in-memory databases
Background in all aspects of software engineering with strong skills in parallel data processing, data flows, REST APIs, JSON, XML, and micro service architecture
Experienced in collaborating with cross-functional IT teams and global delivery teams
Solid Programing experience in Python - needs to be an expert in this 4/5 level
Working knowledge of data engineering aspects within machine learning pipelines (e.g., train/test splitting, scoring process, etc.)
Experience working in a scrum/agile environment and associated tools (Jira)
Apply now to have the opportunity to be considered for similar jobs at leading companies in the Seen network for FREE.
Zero stress and one profile that can connect you directly to 1000s of companies.
We’ll take it from there. After you tell us what you’re looking for, we’ll show you off to matches.
Boost your interview skills, map your tech career and seal the deal with 1:1 career coaching.
Join now and be seen.