Skip to main content

Talent Network

Talent Network Sign Up

DHTS IT ANALYST - Deidentification Data Engineer

DHTS Full Time Durham, North Carolina
Requisition Number: 236559

Job Title:  IT Developer II (Deidentification Data Engineer)

Primary Role:  This position reports to the DHTS-Data Partnerships Director of Data and Analytics Platforms.  This position will be responsible for the management and execution of the deidentification processes applied to the Duke University Health System data assets to be included in the Federated Clinical Applications Platform (FCAP), and the management and administration of the FCAP deidentification cloud environment. 
The position will be a member of the Data Partnerships Data Engineering team and will additionally provide expertise in the development of data integration and delivery pipelines to deliver new data modalities into the FCAP and Duke’s Data Lake.  These solutions will capitalize on technologies to improve the value of analytical data, improve the effectiveness of information stewardship, and streamline the flow of data in the organization.  Solutions will focus on using state-of-the-art data and analytics tools including traditional and near real-time data warehousing, big data, relational and document-based databases using both extract, load, and transform (ELT) toolsets as well as REST APIs and FHIR.  The ideal candidate will be comfortable with data science platforms with proven experience leveraging DevOps and Automation/Orchestration tools. 

Essential Tasks/Responsibilities
•    Create and follow defined procedures in the deidentification of patient medical information
•    Maintain and tune the deidentification environment to perform optimally and comply with DUHS and DHTS policies and standards
•    Collaborate with the Duke partner on improving the deidentification programs and processes, and work with the partner and the Duke Cloud Team on troubleshooting issues
•    Assemble large, complex data sets that meet business requirements
•    Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
•    Recommend design of analytics solutions that improves data integration, data quality, and data delivery with an eye toward re-useable components
•    Create and maintain an optimal data pipeline architecture 
•    Articulate differences, advantages, and disadvantages between architectural solution methods
•    Work with Agile team members to document and execute test plans for data loading and data validation scripts. Support the code promotion process through development and production as required by using standard CI/CD processes
•    Develop, implement, and maintain schedule/dependency logic for automated ETL processing
•    Develop monitoring, logging, and error notification processes to ensure data is updated as expected and processing metrics reported
•    Participate in the creation and maintenance of standards for coding, documentation, error handling, error notification, logging, etc. 
•    Accountable for conforming to established architectural, developmental, and operational standards and practices including the creation of metadata
•    Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics
•    Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs

Education: Bachelor's degree in a related field, or four years of equivalent technical experience required

Required Experience:  We are looking for a candidate with 5+ years of experience in a Data Engineer role who should also have hands-on, professional experience with the following:
•    Relational SQL and NoSQL databases
•    Writing and executing Python programs and shell scripts on Linux
•    Intermediate Linux administration
•    Data Engineering on Microsoft Azure
•    Data pipeline and orchestration tools such as Azure Data Factory and SQL Server Integration Services
•    Developing on cloud-based analytic platforms such as Azure Synapse
•    Performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
•    Supporting and working with cross-functional teams in a dynamic environment
•    A successful history of manipulating, processing, and extracting value from large disconnected datasets

Required Skills:
•    Intermediate to advanced skills in Python programming
•    Intermediate to advanced skills in the Azure Cloud Data Engineering Stack
•    Intermediate Linux administration and shell scripting
•    Advanced working SQL knowledge and experience working with relational and non-relation database systems 
•    Strong analytic, documentation, and organizational skills

Desired Skills: 
•    Prior experience in health care IT
•    Working knowledge of Azure DevOps & Automation/Orchestration 
•    Knowledge of open-source software solutions and open-source as a business model 
•    Technical breadth across application development, enterprise architecture, or application integration
•    Understanding of Agile methodology
•    Knowledge of APIs, API Integration, and API Management 

The information above describes the general nature and level of work assigned to this position. It is not intended to be an exhaustive list of all duties and responsibilities required of position incumbents.

Duke is an Affirmative Action/Equal Opportunity Employer committed to providing employment opportunity without regard to an individual's age, color, disability, gender, gender expression, gender identity, genetic information, national origin, race, religion, sex, sexual orientation, or veteran status.

Duke aspires to create a community built on collaboration, innovation, creativity, and belonging. Our collective success depends on the robust exchange of ideas—an exchange that is best when the rich diversity of our perspectives, backgrounds, and experiences flourishes. To achieve this exchange, it is essential that all members of the community feel secure and welcome, that the contributions of all individuals are respected, and that all voices are heard. All members of our community have a responsibility to uphold these values.

Essential Physical Job Functions: Certain jobs at Duke University and Duke University Health System may include essentialjob functions that require specific physical and/or mental abilities. Additional information and provision for requests for reasonable accommodation will be provided by each hiring department.

large office building with trees in front

Explore Duke Health Technology Solutions

Want to see where you’d be working? Check out this location and everything it has to offer you.

As you explore and apply to career opportunities with Duke Health, we want to alert you of possible fraudulent scams and phishing schemes targeted at job seekers. Duke Health will never ask for any financial contribution or to provide sensitive personal information outside of our application and background check process.

Group of employees putting hands together in middle of circle to be a team and support each other in providing great care

Serving With Purpose

You’ve always been called to care – and we share your dedication to making a meaningful difference in the community. With the entire team united through this purpose, there are no limits to what we can achieve together. You’ll have all the resources you need here to leave a legacy of exceptional service because at Duke Health we support you in your calling to care.

Awards and Recognition

Duke Health is consistently recognized as a top employer for our dedication to the team members and the patients we serve.

Forbes list

On Forbes list of Best Large Employers, we are the #1 healthcare recipient in North Carolina for 2023.

LGBTQ Healthcare Equality Leaders

Our hospitals are consistently recognized as LGBTQ Healthcare Equality Leaders by Healthcare Equality Index.

Magnet Recognition

We are among about 9 percent of U.S. hospitals that hold the American Nurses Credentialing Center’s Magnet Recognition honor.

CHIME Digital Health

CHIME Digital Health Most Wired recipient since 2018, honoring our ability to elevate the health and care of communities around the world.

We are proud to be named as the #1 healthcare system in North Carolina on Fortune’s America’s Most Innovative Companies 2023 list.

Jobs For You

As you search for opportunities that fit your skills, any jobs you view will appear here.

You haven’t saved any jobs yet, but when you find an opportunity that seems right for you, bookmark it so you can return to it easily.

More About Duke Health

  • Upcoming Hiring Events

    Check out our calendar of events to stay up-to-date on information sessions, webinars, and other events featuring new opportunities at Duke Health.

    Learn More

  • Benefits You’ll Love

    We want to make your life better both in and out of work. That’s why we offer comprehensive benefits and unique perks that cater to you and your family.

    Learn More

  • What We Stand For

    Our values define the work we do. Team members here share in these values, creating a community that is connected in purpose and committed to excellence.

    Learn More

  • Other Locations

    With hospitals in Raleigh and Durham and over 140 specialty and primary care clinics around the Triangle region, there are many options to join Duke Health in the city and the suburbs.

    Learn More