IT Consultant (Cloud Data Engineer)
At Duke Health, we're driven by a commitment to compassionate care that changes the lives of patients, their loved ones, and the greater community. No matter where your talents lie, join us and discover how we can advance health together.
About Duke Health Technology Solutions
Pursue your passion for caring and innovation with Duke Heath Technology Solutions, which is dedicated to the transformation, development, and management of enterprise information technology solutions across Duke Health. By harnessing the power of innovative technologies like cloud computing and artificial intelligence — and pairing them with a forward-thinking approach — Duke Health Technology Solutions is revolutionizing the future of health care at Duke Health and beyond.
Primary Role
As a Cloud Data Engineer, you will play a crucial role in designing, implementing, and maintaining data solutions in Duke’s Azure Cloud computing environment as well as support API development, leveraging both on-premise and the cloud data solutions. This role will collaborate with cross-functional teams to ensure the successful delivery of data projects and contribute to the continuous improvement of our data infrastructure. As a member of the Duke Health Technology Solutions (DHTS)-Analytics Center of Excellence (ACE), this role will also provide administrative support for tools leveraged by the ACE Cloud Data Team.
Essential Tasks/Responsibilities
1. Data Architecture Design: Design and implement scalable, high-performance data architecture on Duke’s Azure Cloud. Define data models, schemas, and pipelines to support the organization's data needs
2. Data Integration: Collaborate with Cloud ELT Developers to ensure the quality and consistency of data from various sources migrated to the cloud data environment.
3. API Development: Design, develop, and maintain APIs for accessing and manipulating data stored in cloud environments. Implements APIs to enable seamless data access for internal and external stakeholders
4. Data Warehousing: Design and manage cloud-based data structures for storing structured and unstructured data.
5. Data Streaming: Implement and support real-time data solutions on cloud platforms. Design and deploy streaming pipelines for real-time analytics and monitoring..
6. Data Lake Development: Architect and build data lakes on the cloud platform to store and analyze large volumes of raw and unstructured data. Implement solutions for data ingestion, storage, and processing in data lake environments
7. Data Governance and Security: Implement the data governance processes and security measures to ensure compliance with regulations and protect sensitive data in the cloud environment. Monitor data access and usage to maintain data privacy and security standards.
8. Monitoring and Optimization: Monitor the performance and health of cloud data infrastructure. Identify opportunities for optimization and implement solutions to improve scalability, reliability, and cost-effectiveness
9. Collaboration and Documentation: Collaborate with cross-functional teams to understand data requirements and deliver solutions. Document data architecture, processes, and configuration for knowledge sharing and future reference.
10. Continuous Learning and Innovation: Stay abreast of the latest trends and technologies in cloud computing, data engineering, and API development. Provide input into the evaluation of new tools and technologies
Education/training
Bachelor's degree in a related field, or four years of equivalent technical experience required
Required Experience
• Proven experience as a Cloud Data Engineer leveraging data in cloud environments and API development
• Experience with data integration, ETL/ELT process, and data pipeline in the cloud environment
• Proficiency in programming languages such as Python, SQLk Java, Scala, or C# for data processing and scripting
• Experience with API development frameworks and technologies
• Knowledge of SQL and database technologies (ie: PostgreSQL, MySQL, etc.)
• Solid understanding of data governance, security, and compliance standards in the cloud environment
• Experience with cloud monitoring and management tools
Preferred Experience
• Excellent problem-solving and communication skills, with the ability to convey technical concepts to healthcare and research stakeholders.
• Experience working with healthcare professionals and researchers to understand their healthcare and research objectives
• Knowledge of containerization and orchestration tools such as Docker and Kubernetes in Azure environments
• Strong proficiency in Microsoft Azure services, particularly Azure Fabric, Azure Data Factory, Azure Databrick, and Azure Service Fabric
• Experience with data visualization tools and techniques for cloud-based environments
The information above describes the general nature and level of work assigned to this position. It is not intended to be an exhaustive list of all duties and responsibilities required of position incumbents.
Duke is an Affirmative Action/Equal Opportunity Employer committed to providing employment opportunity without regard to an individual's age, color, disability, gender, gender expression, gender identity, genetic information, national origin, race, religion, sex, sexual orientation, or veteran status.
Duke aspires to create a community built on collaboration, innovation, creativity, and belonging. Our collective success depends on the robust exchange of ideas—an exchange that is best when the rich diversity of our perspectives, backgrounds, and experiences flourishes. To achieve this exchange, it is essential that all members of the community feel secure and welcome, that the contributions of all individuals are respected, and that all voices are heard. All members of our community have a responsibility to uphold these values.
Essential Physical Job Functions: Certain jobs at Duke University and Duke University Health System may include essentialjob functions that require specific physical and/or mental abilities. Additional information and provision for requests for reasonable accommodation will be provided by each hiring department.
Explore Duke Health Technology Solutions
Want to see where you’d be working? Check out this location and everything it has to offer you.
As you explore and apply to career opportunities with Duke Health, we want to alert you of possible fraudulent scams and phishing schemes targeted at job seekers. Duke Health will never ask for any financial contribution or to provide sensitive personal information outside of our application and background check process.
Serving With Purpose
You’ve always been called to care – and we share your dedication to making a meaningful difference in the community. With the entire team united through this purpose, there are no limits to what we can achieve together. You’ll have all the resources you need here to leave a legacy of exceptional service because at Duke Health we support you in your calling to care.
Awards and Recognition
Duke Health is consistently recognized as a top employer for our dedication to the team members and the patients we serve.
On Forbes list of Best Large Employers, we are the #1 healthcare recipient in North Carolina for 2023.
Our hospitals are consistently recognized as LGBTQ Healthcare Equality Leaders by Healthcare Equality Index.
We are among about 9 percent of U.S. hospitals that hold the American Nurses Credentialing Center’s Magnet Recognition honor.
CHIME Digital Health Most Wired recipient since 2018, honoring our ability to elevate the health and care of communities around the world.
Jobs For You
As you search for opportunities that fit your skills, any jobs you view will appear here.
You haven’t saved any jobs yet, but when you find an opportunity that seems right for you, bookmark it so you can return to it easily.
Stephanie’s Story of Growth
Duke Health’s Nursing Career Ladder and supportive benefits helped Stephanie to move from Clinical Nurse I to Clinical Director for Advanced Heart Failure Services.
Building a Team Together
Team members at Duke University Health System and Duke University can refer qualified candidates for rewards.
Why Shondalyn Chooses Duke
Even after going to work at other hospitals, Shondalyn always comes back to Duke Health. She credits the community that makes her feel comfortable, fulfilled, and empowered here.