This project focuses on documenting the Assessing Risks and Impacts of AI (ARIA) dataset, which is used to evaluate risks as part of a NIST ARIA Program. The student will assist in the design and development of a comprehensive dataset card aligned with emerging standards for AI, including the principles of transparency, reliability, and reproducibility. This work supports the advancement of rigorous, standardized data documentation practices consistent with federal and international frameworks.
Research Title:
AI Dataset Documentation
Key responsibilities will include but are not limited to:
- Collaborate with AI researchers to develop a dataset card documenting the technical and operational parameters of datasets used in AI evaluations and emerging AI standards.
- Identify and describe the dataset’s technical parameters, including intended use, limitations, collection methods, and risk-relevant features.
- Conduct quality assurance checks to detect anomalies, inconsistencies, and completeness issues in the dataset.
- Assist in the preparation of the dataset for public release, including metadata curation, file organization, and access provisioning.
- Apply and assess data anonymization techniques to ensure privacy compliance and mitigate re-identification risks.
Deliverables
- A dataset card following relevant documentation standards.
- A finalized version of the dataset curated for public release, with accompanying metadata.
- A report summarizing identified data limitations, anonymization procedures applied, and mitigation strategies for potential risks.
- A checklist-based audit log of quality assurance checks performed.
Qualifications:
- Background in Data Science, Computer Science, Information Science, HCI, or a related field.
- Education level: graduate student
- Familiarity with dataset documentation standards and risk assessment frameworks (e.g., NIST AI Risk Management Framework)
- Strong written communication skills for technical documentation.
The university is an Equal Employment Opportunity/Affirmative Action employer that does not unlawfully discriminate in any of its programs or activities on the basis of race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity or expression, or on any other basis prohibited by applicable law.