Calibrated Peer Review Tool
Clients
My Role
Usability Team Lead
Duration
5 Months
Tools
Figma, Miro, Trello, Zeplin, JASP
Project Overview
In collaboration with IBM, the State University of New York at Oswego commissioned a software development team to create a "Calibrated Peer Review" (CPR) system. Calibrated peer review involves instructors creating assignments and sending them to students for completion. The instructors then review the solutions and distribute them to other students for assessment. My role in this project was to lead the Usability team, tasked with redesigning and developing the entire CPR platform for professors to utilize by Fall 2023. Given the predetermined timeline, it was crucial to avoid scope creep in order to meet the May deadline.
Problem Statement
How might we simplify the calibrated peer review process to reduce the time & effort required from instructors and encourage more student participation in the process?
Image by Freepik
Role & Responsiblities
In the initial phase of the project, I analyzed data and conducted user interviews to establish use cases and personas. Taking over from the previous design team, I led research efforts, collected feedback, and created user personas, empathy maps, and HTA. Later, I managed an 4-member usability team and oversaw user testing after eliciting stakeholder requirements.
Agile Development Process
Agile scrum methodology was utilized for the management of this project. This involved dividing the design and development processes into separate sprints of 7 days each. This approach helped us to prioritize the most important tasks for the minimum viable product (MVP) launch, facilitate better communication and adaptability, and ensure that the team met the project milestones.
Core Features
1. Personalized Navigation
A primary feature desired by the stakeholders - implementing a personalized navigation feature facilitates the efficient access and and course management for current and previous semesters or years.
2. Bulk Download
The bulk-download feature enables instructors to efficiently download all course materials, including course information, assignments, peer reviews, student and team details, and past years' data, based on their preferences.
3. Course Management & Team Communication
User have increased abilitiy to efficiently manage team information, allowing both instructors or students to choose avaiable teams and participate in a dedicated communication space.
The Design Process
-
Discovery
The "Discovery" stage of this CPR tool's design process involved thorough initial research, heaurtistic evaluations, stakeholder interviews, user interviews, empathy maps, and data analysis. By conducting in-depth interviews with stakeholders and users, the team gained valuable insights into their needs and goals. I carefully analyzed the collected data to identify patterns and uncover key user pain points. This stage laid the foundation for informed decision-making and guided the subsequent steps of the design process.
-
Define
In the "Define" stage of our design process, the team focused on gaining a deeper understanding of the project's target audience. By conducting user research, we uncovered valuable user themes and insights. I then developed user personas to humanize the design process. Utilizing hierarchical task analysis, the team identified key user tasks and interactions for the redesigned application. Finally, we consolidated these findings to define the core problem to be addressed, setting the stage for the next steps.
-
Ideate
The "Ideate" stage involved team brainstorming sessions to explore multiple design approaches. The team focused on identifying an optimal information architecture, organizing content, and addressing navigational issues to improve the user experience. By identifying key user tasks and creating user journeys, the team ensured a user-centered approach, mapping out interactions and pathways to enhance usability and engagement.
-
Design
In the "Design" stage, the team finally moved onto creating deignas for the CPR Tool. Beginning with initial sketches, we gradually progressed to crafting lo-fi and hi-fi mockups, ensuring a seamless user experience. Implementing a comprehensive design system further enhanced consistency and scalability. Finally, we brought it all together by creating a fully interactive Figma prototype, showcasing the project's functionality and aesthetic appeal
-
Testing
The "Testing" stage of the design process for the CPR Tool included rigorous IRB-approved user testing and A/B testing to gather valuable insights and measurements. By analyzing the results, we were able identify areas for improvement and validate the effectiveness of the redesigned solution. This stage concluded with a comprehensive handoff of the finalized project solution and relevant documentation to the client, ensuring their satisfaction and successful implementation of design.
-
Results
As a result of the platform redesign:
82%
of instructors are able to create a course & assignment in less than 1 minute - up from 28%88%
of instructor users are able to manage students and teams in less than 90 seconds92%
of instructor users made successful use of a facilitated download process
Future Goals
Accessibility
Ensure that the application is supportive of all assistive technologies and overcomes any accessibility barriers or issues that may be encountered by users with disabilities.
Enhanced Collaboration
Incorporate features to facilitate better collaboration among team members, file-sharing capabilities.
Increased Customization
Provide greater flexibility for users to customize the application to their preferences.
Improved Performance
Optimize the application's performance to enhance speed and reliability.
Want to work with me?
Feel free to contact me!
