The Data Engineer is responsible for the entire life cycle development, implementation, production support, and performance tuning of the Enterprise Data Warehouse, Data Mart. They support the design and implement table structures and the processes used to populate those structures with data from internal and external sources. The data they provide helps business teams drive improvement in key business metrics, customer experience, and business results.
Data Engineer works with the analytics teams who collect and analyze datasets for end-user requirements, evaluate the feasibility of these requirements from a technical perspective, Support the design and develop data models, identify root causes of data quality other related issues.
The Data Engineer will ensure that data pipelines are scalable, repeatable, and secure, and can serve multiple users within an organization; and help facilitate getting data from a variety of different sources, getting it in the appropriate formats, assuring that it adhere to data quality standards, and assuring that downstream users can get that data quickly.
Data Engineer is responsible for the infrastructure that provides insight from raw data and handles diverse data sources seamlessly; enable Data and batch/real-time analytical solutions that leverage emerging technologies. Additional responsibilities include developing prototypes and proof of concepts for the selected solutions and implementing complex Data projects focusing on collecting, parsing, managing, analyzing, and visualizing large data sets using multiple platforms; understand how to apply technologies to solve Data problems develop innovative Data solutions.
Supports Data and batch/real- time analytical solutions leveraging transformational technologies.
Works on one or more projects as a team member designing and developing software applications, testing, and building automation tools.
Works with the D&A team members to implement data solutions in alignment with the project schedule.
Codes, tests, and documents new or modified data systems to create robust and scalable applications for data analytics.
Creates data flow diagrams for all of business systems.
Expands and grows data platform capabilities to solve new data problems and challenges.
Implements security and recovery tools and techniques as required.
Builds automation tools.
Conducts logical and physical database design.
Designs key and indexing schemes and designs partitioning.
Develops, tests, implements, and maintains database management applications.
Constructs and implements operational data stores and data marts.
Resolves conflicts between models, ensuring that data models are consistent with the ecosystem model (e.g., entity names, relationships and definitions).
Conducts Level 2 support for ongoing engagements
Adheres to data management standards, policies and procedures best practices.
May interface with clients and provides ongoing status.
Participates in building and testing business continuity & disaster recovery procedures per requirements.
Evaluates and provides feedback on future technologies and new releases/upgrades.
Bachelor’s degree in computer science, computer engineering, other technical discipline, or equivalent work experience.
Typically requires 1-3 years of software development or database experience.
Familiarity with agile or other rapid application development methods.
Recommend having experience with object-oriented design, coding and testing patterns.
Recommend having experience with object-oriented coding with variety of languages.
Experience with ETL and data wrangling Tools.
Experience with relational database internals, including both query processing and query planning, or other data processing infrastructure.
Basic knowledge of key data structures and algorithms.
Knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases.
Familiarity with monitoring, backup, and disaster recovery of data systems.
Knowledge in different programming or scripting languages.
Strong verbal/written communication skills; must be able to communicate complex concepts effectively at all levels across the organization
Ability to work effectively in a collaborative and team-oriented manner with members of other work units
Solid Problem Solving, Multi-Tasking, dynamic, and creative thinking.
Strong implementation skills and knowledge: Delivery, documentation and circulation of an end to end Plan, including activity, dependencies, timing and ownership. Follow up with specific business owners to deliver components of the program/project on-time.
Interpersonal Skills: It?s likely that you will be dealing with clients, supervisors, staff, vendors and peers, and it?s important to be able to relate well to others
Strong analytical capability to visualize and capture reporting requirements
Ability to work independently and produce high quality deliverables
Knowledge of common data warehousing concepts such as dimensional modeling. Ability to design database schemas to best address business requirements and to best support BI applications.
Deep knowledge of relational databases. Understanding of different types of indexes and partitioning methods. Ability to identify and resolve query performance issues.
Ability to write complex SQL queries and stored procedures to answer complicated business questions. Extensive experience with Microsoft SQL Server and SQL Server Reporting Services
Deep knowledge of multi-dimensional databases (OLAP).
Some Experience In tapping data from NoSQL would be beneficial