Site Logo
Commentary on News

Measure Impact Faster: Qualitative Analysis of Job Shadows

Evidano4 min read

Harvey Mudd College ran a one-day, hometown job-shadow pilot (Aug 19, 2025 report) that placed 10 incoming students with alumni hosts across cities including Los Angeles, Redmond and Denver. The pilot (branded internally as “Muddship”) reimbursed students roughly $150 for travel and lunch and focused on Summer Institute participants who are first-generation or from underresourced high schools. For researchers and career-services teams who need to convert small‑n pilots into program decisions, a focused qualitative analysis of job shadows surfaces whether these low-stakes exposures change major choice, confidence, or network formation. This post shows a practical AI-enabled workflow (using tools to ingest host notes, student reflections, orientation docs and travel receipts) so you can produce thematic, frequency, and cross-segment analyses in days, not weeks. Read the original pilot writeup at www.insidehighered.com/news/student-success/life-after-college/2025/08/19/college-orchestrates-job-shadows-students and learn how to operationalize the findings with www.evidano.com.

Findings Snapshot

DatePilotParticipants (n)Avg reimbursementLocations (examples)Primary cohortSource
Aug 19, 2025Muddship (pilot)10$150Los Angeles, Altadena, Redmond, DenverSummer Institute; first-gen / underresourcedwww.insidehighered.com/news/student-success/life-after-college/2025/08/19/college-orchestrates-job-shadows-students

What happened, concise recap

Harvey Mudd matched Summer Institute students with alumni in students’ hometown metropolitan areas for one-day job shadows. Students and alumni completed pre-orientation for expectations and logistics; travel and lunch were reimbursed. Feedback suggested the experience helped some students refine major interests and expand local professional networks.

  • Pilot target: incoming students from underresourced high schools and first‑generation students.
  • Logistics reason: most students return home before sophomore year, hometown placement lowers cost and friction.
  • Scale in pilot: 10 students across multiple metros; staff consider expanding to winter/spring breaks but face funding limits.

Why this matters for qualitative researchers and career teams

For UX / Career‑services researchers

Small pilots like this are rich in qualitative signals (quotes, host notes, orientation feedback) but poor in statistical power; your analysis goal is to surface patterns that justify program investment or redesign.

Ask: Are students’ reported outcomes clustered (e.g., affirmation, major change, network formation)? Which contextual factors (travel cost, host preparation, host role) correlate with positive responses?

For program leaders

Use qualitative findings to prioritize low-cost scale levers: more host orientation, clearer branding (the pilot name confused participants), stipend policy changes, or seasonal timing changes.

Budget signal: a $150 average reimbursement enabled participation, quantify stipend elasticity in follow-up rounds.

How Evidano helps: map problems to AI-enabled solutions

Problem: Disparate inputs (emails, host notes, short surveys)

Solution: Ingest mixed documents and spreadsheets. Evidano consolidates interview notes, orientation materials, and student reflections into a single corpus for analysis.

Problem: Slow theme discovery in small‑n pilots

Solution: Thematic + frequency analysis. Evidano auto-suggests themes, counts occurrences, and highlights representative quotes so you can see which outcomes (e.g., 'major affirmation', 'new contact') dominate.

Problem: Comparing cohorts (first‑gen vs others; travel distance)

Solution: Cross-segment analysis. Tag participants by cohort and run side-by-side theme frequencies and co-occurrence networks to identify differential effects.

Problem: Follow-up needed but resource constrained

Solution: AI avatar interviewers. Deploy short, asynchronous avatar interviews to collect consistent follow-up reflections and transcripts without scheduling overhead.

Problem: Security and stakeholder concerns

Solution: Evidano encrypts data end-to-end and does not share your corpus to third-party model training, suitable for student data and sensitive program notes.

2‑week workflow: from pilot raw data to a decision memo

Follow this practical cadence to turn pilot artifacts into a fundable recommendation.

  • Day 0–2: Gather and ingest materials (student reflections, host notes, orientation slides, travel receipts) into Evidano; tag by participant and location.
  • Day 3–5: Run automated transcription (if audio) and a first-pass thematic extraction. Review and endorse suggested codebook.
  • Day 6–8: Apply codebook; produce frequency counts, cross-segment contrast (first‑gen vs others), and co-occurrence maps for theme clusters.
  • Day 9–10: Auto-generate representative quotes and a 1‑page insight summary. Validate with 2–3 stakeholder reviewers.
  • Day 11–14: Produce final decision memo with recommended next pilots (timing, stipend adjustments, branding changes) and one‑page slide deck for donors/leadership.

Quick FAQ: common questions when analyzing job‑shadow pilots

What should I code for?

Start with outcomes (major affirmation, skill interest shift), logistics (travel, stipend), and relational outcomes (new contact, follow-up meeting).

How do I compare small groups reliably?

Use thematic frequency as directional evidence plus rich quotes for plausibility. Evidano’s cross-segment reports show patterns without overstating significance.

Can I run follow-up interviews at scale?

Yes; AI avatar interviews can gather standardized follow-ups to increase N while keeping cost low; transcripts feed directly into thematic analysis.

Conclusion, next steps

Harvey Mudd’s Aug 2025 hometown job-shadow pilot shows how low-cost, localized experiences can surface program value quickly. For researchers and career teams, the priority is turning qualitative signals into clear, fundable recommendations.

Keep reading

Browse all articles