Clinical AI Developer/Annotator

A highly skilled and detail-oriented Human Subject Matter Expert (SME) specializing in Clinical pre-annotation validation. The successful candidate will play a critical role in our incremental annotation process, focusing on human validation of pre-annotated data, refining annotation guidelines, and contributing to the development of high-quality golden datasets. While core clinical annotation skills are paramount, experience with the Johnson Labs (JSL) ecosystem and advanced NLP concepts are highly desirable.Key Responsibilities: • Perform rigorous human validation on pre-annotated data generated by commercial clinical NLP models (e.g., Amazon Comprehend Medical) or internal LLM/encoder-based tools. • Contribute to the iterative refinement of annotation guidelines and examples to improve inter-annotator agreement and overall data quality. • Identify and resolve disagreements between pre-annotations and human validations, ensuring the accuracy and consistency of the annotated datasets. • Collaborate closely with data science and NLP teams to provide feedback on model performance and contribute to the continuous improvement of pre-annotation models. • Assist in the creation and maintenance of golden datasets for training and evaluating NLP models. • Participate in discussions regarding data sensitivity and ensure adherence to all relevant data privacy and security protocols, especially for patient data. • Work on diverse clinical annotation projects (e.g., alcohol and smoking use cases, oncology (ECOG, Karnofsky scores), biomarkers, SDO, and mental health). Qualifications:To help recruiters identify the best fit, we've outlined the essential skills for this role and additional valuable assets that would further strengthen a candidate's profile.<>Must Have (Required Qualifications): • Bachelor's or Master's degree in a relevant field (e.g., Life Sciences, Linguistics, Computer Science, or a related healthcare discipline). • Proven experience in data annotation, specifically with clinical or biomedical text. • Strong foundational understanding of NLP concepts and terminology. • Exceptional attention to detail and ability to maintain high levels of accuracy in data validation. • Excellent communication and collaboration skills. • Ability to quickly adapt to various annotation tools (e.g., Inception, Label Studio, Prodigy). <>Good To Have (Preferred Qualifications/Skills): • Prior experience working within the John Snow Labs (JSL) ecosystem for modeling and annotation, including familiarity with their Health AI Lab and GenAI tool. • Experience with specific advanced annotation types such as named entity recognition (NER), relationship extraction, and assertion status annotation. • Direct clinical background or extensive practical experience working with sensitive patient data in a healthcare context. • Familiarity with incremental batch training processes in machine learning. Apply tot his job Apply To this Job