Responsibilities
Data Engineering, Data Architecture, and ETL (60%)
- Designing, building, and maintaining scalable and efficient data pipelines to extract, transform, and load (ETL) data from various sources into Enterprise Data Platform (EDP).
- Designing and implementing data marts, including dimensional modeling, schema design, and optimization techniques.
- Integrating and consolidating data from diverse sources, such as databases, sftp’s, APIs, and streaming platforms, ensuring data quality, consistency, and integrity.
- Creating and maintaining data models, defining data structures, relationships, and data storage requirements, using techniques like entity-relationship diagrams and data flow diagrams.
- Developing data transformation processes, including data cleansing, normalization, aggregation, and enrichment, to prepare data for analytics and reporting.
- Identifying and resolving performance bottlenecks in data processing and storage systems, optimizing query performance, and improving overall data pipeline efficiency.
- Implementing data quality assurance processes, performing data validation, testing data pipelines, and resolving data quality issues.
- Monitoring data pipelines, diagnosing and troubleshooting issues, performing system upgrades and maintenance tasks to ensure data reliability and availability.
- Collaborating with cross-functional teams, including Business Liaisons, analysts, and software engineers, and documenting data engineering processes, workflows, and best practices.
- Supporting training of Harrison Street team members on how to properly organize findings and read data collected.
- Partnering with Technology Infrastructure and Support Operations team to identify, design, and implement internal process improvements.
- Partnering with Technology Infrastructure and Support Operations teams to build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Microsoft Fabric.
- Understanding how Harrison Street’s data platform integrates within the overall technical architecture.
Data Visualization (20%)
- Solid understanding of data visualization principles, best practices, and design aesthetics.
- Developing, maintaining, and managing advanced reporting, analytics, dashboards using Tableau and Power BI.
- Performing data analysis and validations.
- Ability to translate business requirements into technical solutions and effectively communicate with stakeholders.
- Experience in data analysis and interpretation to drive actionable insights.
Project Management (10%)
- Working closely with stakeholders to define project goals, scope, and deliverables.
- Creating project plans, tasks, and timelines using Azure Dev Ops.
- Monitoring progress to ensure timely completion of milestones and deliverables.
- Conducting meetings, providing progress reports, and managing expectations to ensure alignment and stakeholder satisfaction.
- Developing risk mitigation and contingency plans to address potential setbacks.
- Applying organizational and communication skills to ensure project objectives are met, and deliverables are of high quality within the defined constraints.
Cybersecurity Collaboration (5%)
- Collaborating with the Cybersecurity team to adhere to organizational security standards, secure coding practices, and compliance requirements.
- Working jointly with Cybersecurity teams on authentication, authorization, identity management, and secure Enterprise Data Platform development.
Miscellaneous (5%)
- Staying current on emerging technologies for data architecture, data engineering, cloud services, security practices, and AI platform capabilities.
- Documenting system configurations, platform architectures, processes, and operational procedures.
- Participating in training, workshops, and internal knowledge-sharing sessions.
- Performing other duties as assigned to support evolving business and technology needs.
Requirements
- Microsoft Certified – Fabric Data Engineer or Azure Data Engineer.
- Experience using Tableau and/or Power BI.
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
- Experience building and optimizing Azure data Factory pipelines, architectures, and data sets.
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with unstructured datasets.
- Build processes supporting data transformation, data structures, metadata, dependency, and workload management.
- A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
- Strong organizational skills.
- Experience supporting and working with cross-functional teams in a dynamic environment.
- 5+ years of experience in a Data Engineer role.
- Experience with Microsoft Fabric or Azure Data Factory.
- Experience with relational SQL databases.
- Experience with Azure cloud services.
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
- Able to compile and organize statistical information retrieved and present findings to management.
- Experience working with private and sensitive personal information.
- Confident in decision making and the ability to explain processes or choices as needed.
- Strong computer skills and ability to use necessary databases and software.
- Ability to complete milestones and work toward multiple deadlines simultaneously.
- Proficient in data warehousing solutions and ETL tools.
- Proficient in database design, data definition, data dictionary, and related concepts.
- Understands Machine Learning.
- Can leverage data APIs.
- Understands app/dev & SDLC.
- Knowledge of algorithms and data structures.
- Must be able to evaluate technical problems and determine solutions.
- Must have strong written and verbal communication skills.
- Must be able to follow and apply established security policies, procedures, and standards.
- Must be able to read and understand technical manuals and vendor documentation.
- Must be able to manage multiple technical workstreams independently.
- Must be able to maintain professional and effective working relations with supervisors and co-workers.
- Must be able to work flexible hours, including weekends and evenings.
- Must be able to learn new skills and technologies.
🇧🇷 Essa vaga exige inglês. Você está pronto?
A DevSpeak Academy prepara desenvolvedores brasileiros para conquistar vagas internacionais. Domine o inglês técnico com professores que entendem o mundo dev.
Conheça a DevSpeak Academy