Research Updates

This fall marks the transition from data collection to analysis. I've returned to the University of Arizona as a visiting scholar to work with Dr. Reinhardt and the SLAT program on analyzing the comprehensive dataset we collected.

The Dataset

Our 300+ participants generated a rich, multi-layered dataset requiring both quantitative and qualitative analysis:

  • Quantitative Speech Data: 6,900+ audio recordings across five speaking tasks per participant (L1 baseline, four L2 English tasks including game recommendation, restaurant recommendation, gaming tutorial, genre discussion, and conversational agent dialogues)
  • Survey Responses: Gaming preferences and habits, speaking and listening patterns in gaming contexts, online communication styles, self-assessment of gaming fluency, motivation and interaction patterns, demographic and language background
  • User Experience Data: Conversational agent feedback, ratings, and qualitative comments about the libre software AI experience
  • Community Discourse: Reddit comments and forum discussions revealing attitudes toward gaming and language learning

Analysis Tools: PLSPP

For the quantitative speech analysis, we're using PLSPP (Pauses and Lexical Stress Processing Pipeline), an open-source tool that provides automated speech recognition, word and phoneme-level alignment, syntactic analysis, and detailed pause annotations. PLSPP categorizes pauses by syntactic position, which recent research shows reveals more about L2 proficiency than frequency alone.

We're running PLSPP locally and have forked the repository to add custom WhisperX arguments for our specific needs. Combined with our publicly available data collection code at https://codeberg.org/quanrong/videogames_survey, we now have an entirely open-source research pipeline from data collection through analysis, maintaining complete methodological transparency and institutional autonomy.

Conference Presentation

Research Integrity and Proprietary AI Services: Challenges and Open Alternatives

2025 Technology for Second Language Learning Conference, October 2025

This fall we also presented our open-source approach at TSLL 2025. The presentation generated significant interest from the research community, with many attendees expressing interest in learning about and potentially switching to free/libre software for their own language research projects. This indicates growing awareness of the importance of research integrity and institutional autonomy in an era of commercial AI dominance.

Looking Ahead

Spring 2026 will shift from methodological development and tool implementation to systematic analysis of fluency patterns, gaming discourse characteristics, and conversational agent effectiveness across our dataset.

After months of intensive recruitment, we successfully closed data collection in summer 2025 with over 300 participants.

Recruitment Success

Reaching 300+ participants required dedicated engagement with online communities. This involved becoming genuine members of gaming and language learning communities on Reddit, Discord, and specialized forums. Rather than simply posting recruitment messages, we participated authentically in discussions, followed community guidelines carefully, and built trust with moderators and members before sharing our study. This respectful, community-focused approach took considerable time but resulted in enthusiastic support and high-quality participants genuinely interested in contributing to language research.

We're deeply grateful to these communities for their generosity and support in making this research possible.

Study Completion

The €250 prize pool was distributed to 10 randomly selected winners in late summer. These 300+ participants provided rich, authentic data for investigating the relationship between gaming engagement and L2 English fluency.

Data Preparation

The remainder of summer focused on preparing the dataset for analysis. This involved organizing 6,900+ audio files, cleaning survey responses, anonymizing participant data, and establishing secure storage protocols following IRB requirements. This foundational work ensured everything would be ready for systematic analysis beginning in fall 2025.

The most significant milestone this spring is that our research is now fully operational and collecting data. Our investigation into the relationship between video gaming engagement and L2 English speaking proficiency has successfully transitioned from development to a live, functioning system at videogames.emmacaputo.com.

All major research components are functioning effectively:

  • Comprehensive survey platform with integrated voice recording
  • Speaking task framework (L1 baseline + 4 L2 English tasks)
  • Conversational agent using exclusively libre software
  • Complete data collection pipeline

About one-third of participants who start the study complete it entirely, indicating our methodology works well for engaged users. The preliminary data shows we're successfully collecting authentic L2 speech samples with natural repair strategies and complex meaning negotiation.

Conference Presentations

This spring we presented our libre software approach at two conferences:

Beyond Commercial AI Services: A Libre Software Conversational Agent for L2 English Research

CALICO 2025, March 2025

Beyond Commercial AI Services: A Self-Hosted Conversational Agent for L2 English Research

SOCALLT 2025 Conference (RE)Empowering Language & Culture Education at University of New Mexico, April 2025

These presentations established our work as a methodological example of ethical AI implementation in language research, demonstrating alternatives to commercial AI systems that prioritize research integrity and institutional autonomy.

Technical Performance

Our conversational agent maintains a 1.5-second average response time with a 10-message conversation context window. The system integrates voice activity detection (SileroVAD), speech recognition (Whisper.cpp), language generation (OLMo 2.0), and text-to-speech (Piper) - all using libre software for complete methodological transparency.

User feedback averages 2.4-2.6 on a 1-4 scale, with participants successfully engaging in extended gaming discussions that produce the authentic language data we need for analysis.

Open Source Release

We've released complete source code and documentation at https://codeberg.org/quanrong/videogames_survey, providing other researchers with a fully replicable libre software alternative to commercial AI systems and proprietary survey platforms.

Current Challenge: Recruitment

Our primary focus now is intensive participant recruitment. While our completion rates are strong, we need substantially more participants for meaningful statistical analysis.

The key achievement this spring: we've proven that rigorous language research can be conducted using entirely libre software while successfully collecting authentic L2 speech data that addresses our core research questions about gaming and language fluency.

The winter months have focused on refining and expanding our research tools while developing new applications for language education. Our conversational agent has undergone significant technical improvements to enhance its performance and reliability. We've also expanded our collaboration with the Arizona Games in Language Education group, leading to several exciting new projects that bridge research and practical classroom applications.

Survey Development

Working with Dr. Reinhardt, we've completed a comprehensive revision of our research survey, building on Sundqvist and Uztosun's 2023 validated survey in their article "Extramural English in Scandinavia and Asia: Scale Development, Learner Engagement, and Perceived Speaking Ability". Our latest refinements focus on capturing detailed insights into how learners interact with English in gaming environments. The survey now includes more nuanced analysis of gaming preferences and communication patterns, alongside detailed assessment of speaking and listening habits in gaming contexts. We've also integrated new tools for self-assessment of gaming fluency and expanded our metrics for understanding player motivation and interaction styles. These improvements will provide a more complete picture of how language learners engage with English through gaming.

Conversational Agent Improvements

Our AI-powered agent has undergone significant technical enhancements to create more natural and responsive interactions while maintaining our commitment to free/libre software principles. We've implemented Whisper.cpp for optimized speech-to-text processing, significantly improving accuracy and reducing latency in voice recognition. Response generation has been enhanced through the implementation of the Mistral 2 7B model, while text-to-speech capabilities have been improved using Piper for more natural voice output. The system now runs on a containerized deployment system for reliable scaling and performance, with robust security protocols ensuring research data integrity. These improvements have markedly enhanced the naturalness and timing of conversations while staying true to our commitment to using free/libre software.

Arizona Games in Language Education Projects

We've launched several new initiatives:

1. Educational AI Language Partners

Building on our research technology, we've created AI language partners that enable authentic, unpredictable communication practice in a low-anxiety environment.

  • Role-play scenarios with consistent AI personas
  • Real-world communication tasks
  • Comprehensive progress tracking
2. Hybrid Game Development

Our new game projects explore the combination of physical and digital elements to create more engaging language learning experiences that maintain valuable social interaction aspects.

  • Transhumancia Trail: Spanish language learning game combining physical map gameplay with digital resource management
  • Battle for the Brain: Team-based language production game with physical and digital components
3. Game Concept Generator Modernization

Updating Dr. Reinhardt's game deck (https://uagamedeck.weebly.com/game-deck) with enhanced features for language learning integration and game design components.

As a visiting scholar in the Second Language Acquisition & Teaching (SLAT) program, I'm developing my research on game-based language learning through coursework, collaboration, and study of game design.

Through weekly collaboration with the AGILE research group, I've been deepening my understanding of game design principles and mechanics for language acquisition. This work has involved studying how different game elements support language learning, and developing frameworks for analyzing player interactions in gaming environments. The AGILE group's expertise has been particularly valuable in helping refine my conversational agent design and methodology for audio tasks.

While at Arizona, I've been attending the SLAT Proseminar course, which has strengthened my understanding of language learning foundations. I've also presented my research at two conferences:

Developing a Generative AI Tool for Dialogue Data Collection in Online Environments

LatinCALL: CALL Challenges and New Horizons in the Age of AI, November 2024

A Generative AI tool for Dialogue Data Collection in Online Environments

2024 Technology for Second Language Learning Conference at Iowa State University, October 2024

Moving forward, I will begin pilot testing the conversational agent, followed by the primary data collection phase. These steps will build directly on the theoretical and methodological developments from my time at Arizona.

I'm excited to announce my upcoming three-month stay as a visiting scholar at the University of Arizona, Second Language Acquisition & Teaching (SLAT) program.

During my stay, I will be focusing on the following activities:

  1. Collaborating with Dr. Reinhardt and the Games in L2 Learning and Teaching (GL2TL) Research Group to refine my theoretical framework and research methods.
  2. Improving data instruments for analyzing fluency, intelligibility, and comprehensibility in online audio data collected from gamers.
  3. Developing and piloting qualitative data collection methods like game observation techniques.
  4. Exploring specific game design elements and their potential impact on language learning outcomes.

This opportunity will provide valuable feedback, unique collaboration, and access to new research contexts, significantly contributing to the development of my project.

Our current research focuses on developing two tools for language learning research: a conversational agent for L2 English dialogue data collection and a specialized dataset, or corpus, of L2 English usage in online gaming contexts including a conversational agent and a specialized dataset of L2 English.

Conversational Agent Development

Our research project involves developing a conversational agent to facilitate L2 English dialogue data collection. Utilizing a libre LLM and the Elixir programming language, we aim to create an agent capable of conducting brief, natural conversations with participants. A key focus is optimizing response times to closely simulate human interaction. Despite time constraints limiting proper validation, we anticipate that this tool will yield valuable data and increase participant engagement. The project is being developed in collaboration with a computer science student at the University of London. While not without limitations, this approach offers a practical method for gathering online dialogue data in support of our research objectives.

L2 English Dataset from Online Gamers

We are looking to create and analyze a corpus of L2 English from online gaming contexts. We plan to collect 30,000-50,000 words from recorded gaming sessions of non-native English speakers. Using the Natural Language Toolkit (NLTK), we will examine vocabulary usage, grammatical structures, and gaming-specific language across different gaming genres. This research will contribute to our understanding of L2 English usage in computer-mediated communication contexts, potentially informing both theory and pedagogy.

The first quarter of 2024 was dedicated to developing and testing the research instruments. We conducted several pilot studies to test our survey framework, refine methodologies and gain initial insights into participant preferences and technical requirements.

Custom Survey Framework

This project involved developing custom programs using JavaScript, PHP, HTML5, and CSS3 to enhance website functionality and implement a survey framework. This approach overcame the limitations of existing libre survey software, particularly in terms of design customization and functionality. The resulting program offers improved control over survey layout, navigation, and data storage, notably facilitating the collection and storage of audio data— a feature uncommon in standard survey software. Future program development contemplates rewriting the system using Go to prevent potential runtime errors, improve data accessibility and interpretability, and establish a foundation for future survey instruments.

Pilot results:

1. Topic preferences and task order (n=60)
Preferred task order
  1. Survey
  2. Native language task
  3. Three L2 English tasks
Preferred order for L2 tasks
  1. Monologue task
  2. Story re-telling task with video
  3. Dialogue
Topic preferences for monologue tasks
  • L2 English monologue: Hobbies
  • Native Language monologue: Hometown
2. Participation preferences (n=205)
  • ~75% prefer computer-human voice interaction in a second language
  • ~66% more likely to participate in online, self-paced studies without researcher interaction
3. Audio quality and participation challenges (n=15)
  • Tested survey framework and audio data collection
  • Revealed challenges in recruiting participants for dialogue tasks
4. Microphone and recording quality (n=15)
  • Tested microphone quality across various devices
  • Confirmed good overall audio quality for fluency studies
  • Identified potential issues with environmental noise

To support our research goals, we're continuously enhancing our technical skills. We're focusing on tools and technologies that will enable more sophisticated data analysis and improve our research methodologies. We are also dedicated to using libre software in order to guarantee participant privacy and security.

Planned software and tools:

  • Praat: For effective speech data analysis and interpretation
  • Python libraries:
    • NLTK (Natural Language Toolkit): For language processing, tokenization, and sentiment analysis
    • Pandas
    • NumPy
    • Parselmouth