UTEP Outreach: Soothing Resonance and the Fundamental Frequency for the Neurodiverse

Recently, I have made the effort to pursue community outreach in an effort to connect and broaden our insights into disability in the general public and the academic setting. Our research has continuously been reading several publications on the topic of disability and how art can be a therapeutic outlet or accommodation; however, we wanted to continue to explore the issues and conversations related to those who identify as having some degree of disability that is integrated into a campaign, organization, or project. My first initial interest lay in faculty members open about their cognitive disabilities or those who had a visible disability who were willing to share their stories or research projects. We want to avoid the implications of ableism but also learn more about research related to disability at The University of Texas at El Paso (UTEP).  Our team's Faculty Fellow, Dr. Warak, had thus created an advisory board of disabled faculty members at UTEP. I plunged deeper, hoping to find a project or organization that a UTEP faculty member may have some part in, taking note of the scarcity.

Research Project Outreach 

I had first heard of Dr. Eric Freudenthal’s project from his former student, Kevin Moreno, who had taken his course, Computer Architecture, as a sophomore at UTEP.   As Kevin Moreno states on his initial introduction and response to Dr. Freudenthal’s software demonstration of Soothing Resonance:  “I recognized a convergence between what he was saying and my own observation of internal behavioral characteristics. I began to ponder the merit behind the vibration of waves as a language model and their tendency to destruct each other from a computational perspective. Letting these paradigms stew in the noddle soup; all I knew was that I seemed to retain even more context of the material he was teaching because I could finally peer into his psyche.”

The project quickly captured my attention, as Dr. Freudenthal expressed his neurodiversity (ND) of medically diagnosed ADHD openly and positively, additionally relating the “intonation voice boosting” as a helpful tool in his classroom that allows him to codeswitch. I had reached out through Kevin, who was equally interested in the “intonation boosting project” and asked to meet for a live demonstration and for a chance to properly introduce myself and our own research project our humanities collaborative team–Jessica Gomez, Dr. Warak, and I– are working on in terms of the relationship between disability and art and potential collaboration with his research. Our first meeting during winter break proved to be incredibly insightful. We met in a lab located inside the UTEP Computer Science building, and after a brief introduction, Dr. Freudenthal flipped a switch on a stereo amplifier that sat on the main table, seemingly connected to his iPhone and a microphone that dangled freely above our heads. Initially, I didn’t notice anything different in the room until I reflected on the change in pitch level of my voice had risen to a higher amplitude that is deemed socially acceptable in conversation, with no effort on my part. It was fascinating to hear myself speak in a higher tone of voice, as I was dubbed by many as a quiet speaker and overall having difficulty in public speaking. Although I’m quiet, I believe that I am speaking in a “normal” tone. Dr. Freudenthal had then explained his own personal experience, an inversion to mine, as he speaks and lectures loudly unintentionally. The frequency recording that plays from the amplifier allows him to speak in a normal pitch. Additionally, it allowed him to interpret the emotions of others and properly express his own emotions efficiently with no room for miscommunication. Voice intonation is not arbitrary, as it produces a variety of pitches that conveys emotion and intent.

From left to right, respectively, Jess Gomez, Dr. Eric Freudenthal, Dr. Stephen Sands, Anthony Prudent, Kevin Moreno, and Katherine Villanueva in a “NE–ND” group meeting

Intonation Boosting through the Fundamental Frequency

In Nina Kaus’ Of Sound of Mind: How our Brain Constructs a Meaningful Sonic World, Kaus writes, “Other disorders that involve movement, such as aphasia, stuttering, difficulty with respiration, swallowing, and speaking, respond to music therapy. Therapy involving rhythm also has shown promise in addressing communication and social behavior in people on the autism spectrum. Children who cannot otherwise speak can form words and sentences when accompanied by a clear rhythm. There are children on the autism spectrum who will not engage in a verbal conversation but will gladly carry on a rhythmic conversation with another person on drums.”(68). There is a clear distinction between those who identify themselves as neurodiverse and a deficiency in speech prosody that rhythm and frequencies in music can help us understand better. The Natural Ear software, designed and created by Dr. Freudenthal, was his first attempt to explain the inability to discern categorical pitches— initially created for singers. Motivated by his conjecture that there must be some scientific explanation for what he describes as his wife's angelic harmonious voice as it differs from his own "dying cat" performances. The implications of this technology lead Dr. Freudenthal to transform his application into “Soothing Resonance," a more accessible web application with more than one use case. The overall premise of the tech implements the idea of verbal intonation boosting through an "acoustic laryngeal fundamental frequency(F0) accentuator." Essentially, the algorithm takes in the varying sine waves from the environment’s sound and reintroduces the fundamental frequency to a perceivable degree for everything in proximity. This is because in the brains of those who are described as neurodiverse, it appears that the cochlear translation from mechanical to digital signal of each harmonic series is a lossy one, to use a computational term. That is; there seems to be a dissonance between the hair cells responsible for capturing and propagating the fundamental frequency of an arbitrary sound and the electrical and digital signal it eventually gets processed into. Future and present studies on this phenomenon are currently pending conventional significance.

Image of the Soothing Resonance Website.

Within the whole team, there is a total number of ten of us that are broken up into two groups, the “NE-ND group” essentially discusses the humanitarian side, and the “Technical group” is responsible for the program itself: Dr. Freudenthal, an associate Computer Scientist (CS)  professor at UTEP,  Dr. Stephen Sands who is an external collaborator and developmental cognitive neuroscientist, Anthony Prudent, a student from the University of Rochester who is an undergrad majoring in cognitive neurology, Nick Sims who implements the software on the mobile platform and is an undergrad CS student at UTEP, Lianna Estrada, is additionally an undergrad CS student at UTEP who implements the fundamental frequency filter, Daniel Villanueva grad student in mechanical engineering at UTEP, and Kevin Moreno who is a biomedical computer scientist and lastly, the humanities team made up of Jessica Gomez, Dr. Warak, and I.

Discrimination and Accommodation 

The question that arises throughout Dr. Freudenthal’s research is whether you have pragmatic speech deficit or not, would Soothing Resonance help? Uncanny Valley is known as an emotion of discomfort brought upon by an “object’s” close resemblance to a human being. Similarly, “neurotypicals” (NT) may feel discomfort from atypical behavior expressed by someone who is neurodiverse as it causes cognitive delay. The offset of uncanniness in NTs often leads to a diagnostic gaze or assumption–if someone is identified as having “abnormal” speech, this may bring a negative depiction of ND or other negative identifications of character– as an ND's filter of emotion is not proper. Soothing Resonance offers accommodation to those who wish to enhance their awareness of their own and others’ intonation and volume.

Written by Katherine Villanueva, Undergraduate Research Fellow 

The University of Texas at El Paso, The Humanities Collaborative at EPCC-UTEP

References

Klaus, Nina. (2021). Of Sound Mind: How our Brain Constructs a Meaningful Sonic World. Cambridge, MA: MIT Press.

All images courtesy of Katherine Villanueva