MiracleProject

Last year, my friend and I participated in a 3-day hackathon where we would be making a functioning application for a non-profit organization. There were three non-profits that presented their mission to all the particpants, and the non-profit we felt most compelled to work with was the Miracle Project.

The description for the Miracle Project that we were given was:

The Miracle Project is a nonprofit organization that provides an inclusive theater, film, and expressive arts program that is focused on communicaton, self-esteem, and life skills for neurodivergent and disabled individuals. The nonprofit organization was seeking a website that allowed students to access course materials in the most accesible way since previously, they were using google drive to distribute material, which was extremely difficult for students and parents to navigate. 

The intuition behind our solution was to create a website where students would be able to access their courses with just the touch of the button. After extensive search, we came across and utilized a built-in Chrome speech-to-text API along with Google's Gemini to allow students to navigate to where they want on the site with their voice.

Code Sample:

import {GoogleGenerativeAI} from "@google/generative-ai";

// Access your API key as an environment variable (see "Set up your API key" above)
const genAI = new GoogleGenerativeAI("AIzaSyA14eZTGShAoOHx15smh8qpB6T6NWoGRsA");

export async function run(text_prompt) {
  // For text-only input, use the gemini-pro model
  const model = genAI.getGenerativeModel({ model: "gemini-pro"});

  const prompt = `write out the one class that is mentioned in this text without any dashes or extra spaces: ${text_prompt}`

  const result = await model.generateContent(prompt);
  const response = await result.response;
  const text = response.text();
  return text
}

When making this website, we heavily emphasized the possiblity that the user lacked reliable control or their hands. So, as seen in this example, we allowed the user to speak and tell us which class they wanted to access. We then passed the speech input into Google Gemini's API, and specifically opened the class that the user asked for.

Tech Stack

Vite (React), Node.js, Express.js, MySQL

If you would like to try this application by yourself, please feel free to follow the following terminal commands.

Installation

git clone https://github.com/PTruong9090/The-Miracle-Project
npm install

Starting the Program

npm run dev

cd server
node index.js

Credit

Parsa Hajipour, Phuc Truong