Simonsen J. & Frieberg, K. (2014) Collective Analysis of Qualitative Data. In Situated Design Methods. Cambridge: MIT Press p. 99-118
-Conduct User Study 1 Contextual Interviews about portfolios with CMU students
UPCOMING DUE DATES
10/4 Review User Study 1: User Profile Insight Boards in class
10/6 User Study 1: Findings and Model Due
**Start planning your user study 3, meet with me for ideas about where and with who to conduct the research.
9/ 29 Assignments
A. Reading Reflection
Simonsen & Frieberg (2004) Collective Analysis of Qualitative Data. In Situated Design Methods. Cambridge: MIT Press p. 99-118
B. Pilot and conduct your interviews
10/5 User Profile Insight Boards Due
10/7 User Study 1 Findings & Work Model Due
Reading Reflection | Due 9/27
Hugh Beyer & Karen Holtzblatt. "Contextual Design." interactions 6, no. 1 (1999): 32-42.
Reflection Question Prompt:
How would you adapt the Contextual Design process and language in this article, which draws on workplace examples, for student-centered learning activities (e.g portfolio practices) in a classroom context? Focus your translation on the design directives in the Contextual Inquiry section and Work Model examples (flow, cultural and physical) for a learning context. Feel free to challenge or problematize assumptions built into the authors' Contextual Design approach that may unravel in a learning context.
NB: For User Study 1 - each team will need to create a “work model” for student portfolio practices based on your user study data, so pay attention to the different ways work models are represented in this reading.
Another good summary of Contextual Design online
Design Research Method: Directed Storytelling
Data Analysis Technique: Focused Listening
Design insights captured during the conversation with Nina Barbuto about with questions and storytelling framed around needs, opportunities, challenges and tools/methods for documentation and digital portfolio creation in informal learning contexts like ASSEMBLE.
A user study protocol is essential for organizing a research team and the study design. Use the template below as a starting point. Also read the Beyer & Holtzblatt Contextual Design reading and review the blog post notes on method tips.
On Tuesday in class, Irene and I will work with each of the groups on the user study planning and protocol draft.
User Study Planning Template
List the questions you want to be able to answer after this study.
(e.g. Motivations, goals, tools used, workflows, challenges, portfolio practices and processes)
How do you want to use the findings from your study? What part of the design problem will this study provide insights and perspectives on?
What design research technique or methods will best address your questions? Which will be the most time effective, and generate the needed level of detail?
Describe the participants you are intending to recruit, your rationale, and how you will sample. Who will lead the interactions with the participant? Who will take notes and record the data on your team (min. 2)? What questions will you ask and what prompts will you use? Will you have the user review their current portfolio, recount and show a specific past project and documentation process, or speak prospectively about a current project plans to document going forward?
Where will you conduct the study? What is the availability of team members and your participants? Who will do the recruiting and scheduling, and prepare an email introduction and request. Arrange for data recording equipment (audio, video, still camera, software apps (e.g. screen-capture), note –taking pads, observation sheet.
Leave time to pilot the protocol with a friend or each other, revise as needed.
Data Analysis & Reporting
List the data analysis techniques you plan to use for each type of data collected during your study. Give plans for presenting the data, be as creative and visual as possible.
Develop User Study Protocol
A user studies protocol should given enough detail that someone could repeat your study.
Section 1: Study Purpose
Include problem/issues statement, the objective of the study, and how it intends to shed light on the problem identified.
Section 2: Study Design:
How will you conduct this study? Summarize the study methods and data collection techniques used. If using a published method, cite it. In the appendix, add sample instruments (e.g. interview questions, survey form drafts, photos of tool kits, probes). A Materials Needed Checklist can be useful
Section 3: Analysis Plan
How will the raw data be prepared and processed for analysis? What techniques will you use to make sense of the data? A well-conducted contextual inquiry can yield immense amounts of data for analysis. Technique for analyzing and presenting this data can be broken down into variety methods and techniques of varying complexity and depth. Simple methods such posting photographs of user work environments, user profiles with brief narratives about users and their work, or simple lists of insights and issues can provide design teams with valuable direction.
More complex design issues require more in-depth analysis techniques such as affinity diagramming, user-task matrices, flowcharts, scenarios, or coding discourse, task and artifacts. Each of these methods addresses a different question or aspect of the design.
For example, a team may organize several rounds of affinity clustering: first to identify noteworthy statements and observations or single out outliers; conduct a second round of clustering to find related statements and observations, or identify by thematic correspondences, or organize by chronology or in hierarchical associations; and then finally complete a third round to distill the grouped particulars into general observations with implications for design. Determine best the portfolio practices model to represent your findings, or develop a new one. Look online
The Beyer & Hotzblatt reading outlines five work modeling approaches which can be adapted for a portfolio process:
Field Research Method
A situated ethnographic research method using observation and think-aloud interviewing techniques to document and reveal for analysis the underlying contextual goals, tasks, resources, and processes involved in work and learning.
For User Study 1 we will use a contextual inquiry user research method to explore current CMU student portfolios practices by observing, listening, and asking questions in the context of the participant's work space or learning setting. The aim is to gain insights into students' portfolio creation goals, tasks and workflow processes in order to help envision, integrate or improve a portfolio system design. Each team will need to do one contextual interview per person. Decide how to divvy up the user study data collection and start recruiting participants early. Try to recruit students from a range of disciplinary backgrounds with varying levels of "portfolio expertise." Small teams of two are less intimidating, so split observation and interviewing responsibilities per protocol suggestions below.
PART ONE – CONTEXTUAL INTERVIEWS
Understanding users goals, tasks, work process and practices
Observing user gestures, actions, and interactions
Documenting tools and resource use
[ ] Field Note Sheets
[ ] Audio/ video recorder or screen capture application
[ ] Camera + batteries
Understanding users underlying motivations needs and work flows
Asking the users to think aloud about their process and practices
Expanding your understanding of the design situation from others’ perspectives
[ ] Semi-structured question list – aimed at getting users to talk about issues of interest
[ ] IRB consent and media use form
Hackos, J.T. and Redish, J.C. (1998). Chapter 10 – Conducting Site Visits: Honing your Interviewing Skills, In User and task analysis for interface design, John Wiley and Sons: New York.
The Center for Creativity at the Allegheny Intermediate Unit's (AIU)
2016 STEAM Showcase, a regional celebration of STEAM education.
The Annual STEAM Showcase illustrates and celebrates the work of STEAM Grantees. Join AIU3 at the Circuit Center and Ballroom on Tuesday, September 27th, 2016 from 11am to 2pm.
Join the AIU as the recipients of last year’s STEAM Grants–28 school districts located throughout southwestern Pennsylvania– highlight how they have infused science, technology, engineering, the arts, and mathematics to redefine student learning.
This year’s Showcase will feature a record number of grant-funded projects and local exhibitors, with an award ceremony being held between 12-1 pm. We anticipate the event will reach near capacity, so please register all attendees to ensure admission. Free parking is available in the lot across the street from the venue, with overflow spots available for a fee in the parking garage next to the venue.
The STEAM Grants are awarded annually by the Center for Creativity at the Allegheny Intermediate Unit and are generously funded by Chevron and the Benedum and Grable Foundations. Now in its ninth consecutive year, nearly 175 grants have been distributed with over $3 million in funding provided to school districts throughout southwestern Pennsylvania.
Digital Portfolio Learning Ecosystem Map Assignment
Due: 9/20 for in-class review
Ecosystem mapping is a visual way to analyze the interactions and interdependencies betweens users and other actors and entities in a system. The primary goal of this activity is to give the newly formed teams time to create a shared, external representation of how you would collectively map the open portfolio design challenge space as you currently envision it.
Below is a starting list of primary and peripheral users, and stakeholder entities your group can draw on to think about mapping key interactions and relationships that would occur and influence an implemented open portfolio system. In thinking about your map you may want to consider how to represent changing contexts of portfolio use and learner roles as they move across settings, or focus on the learning interactions between users that a portfolio platform/device could support. Your team will need to decide how you best want to represent open portfolios as a design space.
Identify and situate Primary Users :
Add Peripheral Users and interactions :
Consider Stakeholders / Stakeholder Entities with vested interests and influence
How does learning flow through the system?
Think about locating opportunities for enhancing learning through portfolio interactions that support …
Adult/Peer critique, Inspiration, Design Precedent Seeking, Iteration, Articulation, Reflection, Collaboration, Modeling expert/professional practices, Opportunity Brokering, Identity building, Participation in Communities of Practice Vocation, Expertise seeking.
**Create a map in a digital form that graphically represents your group's abstracted view of the actors involve, and portfolio mediated interactions and linkages that would occur in an open portfolio learning ecosystem. Upload the digital file to Box folder > Assignment 5
As a group discuss what aspects of your open portfolio learning ecosystem map suggest an interesting design challenge space. Generate a running list of problems you could imagine trying to solve based on what you know about user needs and contexts. You may want to try the now classic "How might we...?" question formulation for some of your problem statements to brainstorm possible directions towards a solution.
**In class on Tuesday each team will be asked to informally present their maps, share some of problem statements generated and some early ideas for possible design solutions that came out of your group brainstorming session. So plan to show your map and give a brief (< 5min) conversational summary of your group's exploratory discussions.
Supplemental Reading (Optional)