AudioCraft 24-hours Hackathon Summary

Every species on earth contributes in the balance of ecosystem.Birds are also an essential part of our ecosystem. They help to pollinate plants, control insect populations, and disperse seeds. But birds are in trouble. As over 1 in 5 bird species is now threatened with extinction. Monitoring changes in bird species numbers can reveal the effectiveness of restoration projects. Traditional observer-based surveys for this purpose are costly and logistically challenging. In contrast, passive acoustic monitoring (PAM) combined with machine learning tools enables cost-effective, large-scale, and high-temporal-resolution assessments of the impact of restoration efforts on biodiversity.

Team NatureAI

Problem: Film production teams, especially those with limited resources or tight schedules, struggle to create high-quality background sound effects that match the visual elements of their scenes. Traditional methods involve manually sourcing, editing, and integrating sounds, which is not only labor-intensive but can also result in a lack of synchronization with the on-screen action. This gap in sound quality can compromise the overall cinematic experience and viewer engagement. Solution: Our Movie Background Sound Effects Generator addresses this problem by harnessing the capabilities of the Audiogen API. This innovative tool automates the process of creating synchronized and immersive background soundscapes for movies. By leveraging cutting-edge AI and deep learning techniques, the generator analyzes scene visuals, identifies key elements, and intelligently selects and applies appropriate background sound effects. From bustling city streets to serene nature scenes, the generator ensures that every moment is accompanied by the perfect auditory atmosphere.

tech cats

SonicVision: The Pinnacle of Interactive Storytelling and Sensory Immersion In the ever-evolving landscape of gaming and interactive experiences, SonicVision stands as a groundbreaking innovation. Developed to be showcased at the AudioCraft Hack-a-Thon 2023, this transformative platform promises to redefine the way users engage with digital worlds. A Harmonious Blend of Art and Sound At the core of SonicVision is a revolutionary amalgamation of generative music and dynamic art, all woven into compelling stories that users can not only experience but also shape. Imagine entering a fantastical world where every decision you make not only progresses the story but also influences the art and music that envelops you. With SonicVision, this is not just a possibility; it's the standard experience. The Sonic Wonders of AudioCraft A crucial component that drives the platform is AudioCraft—an AI-driven music generation system that goes beyond mere background scores. Developed in-house, AudioCraft uses state-of-the-art AI models to generate music across all genres and styles. Whether you're venturing into an enchanted forest or a post-apocalyptic city, AudioCraft crafts the perfect auditory atmosphere, complete with sound effects that impeccably align with every situation. OpenAI: The Dungeon Master of Your Dreams SonicVision's immersive storytelling experience is powered by OpenAI's Chat-GPT, which serves as the Dungeon Master of your interactive journey. This is not just a chatbot; it's a narrative genius. It utilizes a tailored prompt layer that does more than merely guide the story. Chat-GPT dynamically commands the visual and musical elements of the game, adding layers of depth and interactivity previously unexplored in digital storytelling.

Sonic Meow

Creating a Symphony of Financial Data: Transforming Cryptocurrency Price Action into Music In the ever-evolving landscape of cryptocurrency, where markets surge and plummet within moments, enthusiasts and traders have long relied on charts and graphs to visualize these price dynamics. However, imagine a world where you not only witness these market fluctuations but also experience them as a unique musical composition. Welcome to "SoundCoin," an innovative project that merges cutting-edge technology, artificial intelligence, and creative expression to transform cryptocurrency price action into captivating music. The Vision Behind SoundCoin: SoundCoin was born out of a vision to bridge the gap between the analytical and artistic realms of cryptocurrency trading. Conceived by a team of tech enthusiasts and financial analysts, this project aims to provide a novel way for users to interact with and understand market data. Beyond traditional candlestick charts and complex technical analysis, SoundCoin introduces a sensory experience that transcends numbers and charts, making cryptocurrency trading not just informative but also enjoyable. The Impact of SoundCoin: SoundCoin transcends the conventional boundaries of financial analysis and creative expression. Here are some key aspects of its impact: - Education: Traders and enthusiasts gain a deeper understanding of market dynamics through auditory and visual means. The fusion of data and music provides a holistic perspective on price action. - Entertainment: SoundCoin introduces an element of fun and entertainment to cryptocurrency trading. Users can enjoy the creative and artistic aspects of market analysis. - Sharing Insights: The ability to export and share the created videos on platforms like YouTube extends the reach of financial insights. Users can use their unique compositions to convey their trading strategies and market observations.

SoundCoin

The challenge is to create a text-to-music generation AI application using Meta's Audiocraft that produces high-quality and coherent musical compositions from input text. This requires tackling issues related to algorithmic accuracy, diverse training data, music theory integration and real-time processing.We developed an efficient and high-quality text-to-music generation AI application using Meta's Audiocraft. The application can generate coherent musical compositions from textual input. It has ability to generate music from natural language prompts It has ability to download the music directly after generation.

PopeyeX

QuakeAI is an Audiobook Generator that enables Authors, Writers, and live Streamers/Broadcasters to generate Spoken stories with AI generated background music that brings life to it. QakeAI is leveraging the power of LLMs, Music Generations models and Voice Generation model to enable users to have to only provide and idea of a story or a story they've written themselves and make an Audiobook with amazing background music effects out of it. Authors and writers would never believe how easy it is to turn their stories written on papers to an audio spoken with their own voice or a premade one with high quality background music and publish it on Audible within a click of a button! Content creators of shorts & reels will generate music for their videos without worrying about demonetization or DMCA takedowns. Authors can brainstorm shorts stories with other author through a chat room and QuakeAI would make an Audiobook out of it. Try QuakeAI now to be amazed with it.

QuakeAI

Who is this for? This isn't a toy; it's a tool designed for dedicated musicians who see technology as an extension of their craft. If you're not afraid to embrace AI to enhance your creative output, then Sonic Meow is made for you. What Does It Do? Welcome to the future of remixing. Sonic Meow takes your original song, slices it, dices it, and reassembles it into something entirely new. And don't worry about jarring transitions—our sophisticated algorithm ensures your remix is a seamless auditory experience. How It Works Upload Your Track: Simply load up your audio file and let Sonic Meow take the reins. Set the BPM: Make sure you know your song's tempo. Input the Beats Per Minute (BPM) to keep everything in sync. Customize Your Preferences: Set the number of iterations, prompt duration, and min-max output duration to shape your remix the way you envision it. Seamless Splicing: Our intelligent algorithm keeps track of the song's bars, making sure each remix starts and stops at just the right moments. Hit Generate: Once you've set your parameters, click 'Generate' to craft your unique remix. Unique Every Time Worried about repetitive output? Fear not! Our semi-randomization feature ensures that no two remixes are ever the same—even when using identical settings. Why Wait? Start Remixing Now Experience a new level of creative freedom with Sonic Meow. Break barriers, push boundaries, and redefine what's possible in the realm of music production.

sonic meow remixers

🎶 Musicube: Where Creativity and Music Converge! 🎮🎵 Embark on a journey beyond traditional gaming with Musicube, an innovative 3D cube-based game that redefines the boundaries of creativity and music production. Designed to captivate both gaming enthusiasts and music aficionados, Musicube offers an unparalleled experience where players don't just play the game, but actively participate in crafting unique musical compositions. 🚀 Real-time Music Generation 🎶💡 What sets Musicube apart is its seamless integration of gaming and music generation. The instant you intersect cubes, your commands are sent to our cutting-edge MusicGen engine. This AI-powered technology transforms your actions into real-time musical output, providing an enchanting auditory experience that mirrors your gaming journey. Witness the magic unfold as your gameplay shapes the very music that accompanies it. 🌈 Limitless Exploration and Discovery 🔍🎮 Step into a universe where creativity knows no bounds. With a multitude of cube types, each representing distinct musical elements, Musicube encourages you to explore, experiment, and uncover hidden synergies. Delve into the world of harmonics, percussion, melodies, and more. Whether you're creating serene soundscapes or energetic compositions, every moment in Musicube is an opportunity to push the boundaries of your artistic expression. 🎉 Experience Musicube Today! 🌍🎮 Are you ready to embark on an unforgettable journey where your gaming skills fuel your musical prowess? Musicube invites you to explore, play, and compose your way to a symphonic adventure like no other. Elevate your gaming experience, unlock your inner composer, and witness the harmony of Musicube – where the cubes dance to your gaming, and the music sings to your soul.

Team Tonic

We have used Clarifai for image recognition, we give user the option to upload image and the AI can generate prompt according to the image it recognizes, based on that it generates music prompt to be passed to Musicgen for generating Music. Currently, with the MusicGen environment we have it takes approx 20 seconds - 22 seconds to generate music. The audio output we kept is upto 6 seconds for the user to hear it and download it. We also have an option to detect and recognize live webcam feed, the AI will generate prompt according to the image recognized and will generate prompt for MusicGen to generate music. In addition, the user can also simply write the prompts on their own to generate music.

Audiophiles

## Implementation - Built as a Chrome browser extension for ease of use - Uses JavaScript content scripts to analyze webpages and play lofi audio - Leverages AudioCraft's MusicGen AI model to generate the lofi tracks - Polished UI allows easy control over the music generation --- ## Our Custom Model We collected a dataset of original non-copyright lofi music. This gave us access to a large corpus of high-quality training data without any copyright issues. We split the lofi songs into 30 second audio clips and paired each clip with a text prompt describing the mood, instruments, tempo and other qualities of that segment. Examples include "slow chill hip hop beat with mellow piano and vinyl crackle" and "upbeat lofi with energetic drums and warm bassline". We formatted this dataset into the required .wav and .txt file pairs that musicgen_trainer expects. The text prompts would guide the model to learn the nuances of lofi hip hop. We then ran musicgen_trainer on this dataset, configuring it to use the small architecture for optimization purposes. We trained for 100 epochs with a learning rate of 1e-5 and batch size of 4. During training, musicgen_trainer used the audio/text pairs to fine-tune MusicGen on lofi music. The pre-trained weights were specialized to generate high quality lofi given descriptive prompts. After training finished, we saved the best performing model checkpoint. We now have a MusicGen variant skilled at generating original lofi tunes according to textual descriptions. --- ## Why Download Our Chrome Extension - Improve focus and concentration when reading - Make reading more enjoyable and relaxing - Boost productivity - Avoid listening fatigue - Portability - Ease of use - Less anxiety - Nostalgia

LoFi Focus

Hootmoo is an app designed to enhance early childhood education by putting the power of personalized learning in the hands of parents. Catering to toddlers as the primary beneficiaries, the app allows parents to specify the subjects and concepts they want their children to learn. With a few taps, Hootmoo generates vibrant and engaging flashcards that come alive with corresponding audio, transforming learning into an interactive and enjoyable experience. The app's user-friendly interface ensures that parents can easily customize flashcards to match their child's unique interests and developmental needs. Whether it's numbers, colors, animals, or even introductory language lessons, EduCard Connect covers a wide spectrum of subjects crucial for early learning. A standout feature of Hootmoo is its commitment to affordability and accessibility. The app operates mostly on a non-profit basis, with minimal profits generated to cover upkeep expenses. Funding is primarily sourced from unobtrusive elements like advertisements, sponsorships, and generous donors who share the vision of accessible early education for all. By harnessing the potential of modern technology, Hootmoo redefines educational toys and offers a cost-effective alternative for parents seeking interactive learning tools. It's a win-win solution that empowers parents to actively participate in their child's learning journey while fostering a love for learning from the earliest years.

Hootmoo

Raga Music Generation Pipeline: RagaCraft Our project, RagaCraft, bridges the gap between raw human emotion and the timeless art of raga music using cutting-edge AI. Here's a deeper dive into the underlying process: Customer Interaction: Users interact with our platform, sharing their current emotions and contextual information. For example, "I am feeling romantic today. It is Valentine's Day. I'd like a song to suit the mood." JavaScript Selection: Our system, powered by JavaScript, scans the user's input to select an appropriate raga that resonates with the given emotion. OpenAI Integration: To add depth and specificity, RagaCraft sends a refined request to OpenAI: "Generate a text-to-music prompt for a single romantic raga. Include parameters such as tempo, scale, pitch, and rhythm to optimize the romantic mood. Define ideal values for these features." OpenAI's Response: The API, enriched with musical knowledge, replies with precise musical direction. For instance, "For a romantic setting, employ the Hindustani raga Kamboji. Utilize a medium-slow tempo, major scale, and a high pitch with low undertones. The rhythm should be gentle with a 4/4 signature. Dynamics can vary, with crescendos and decrescendos, ensuring a light texture and smooth timbre." Audiogen Transformation: The detailed prompt from OpenAI is fed into Audiogen, which processes it and crafts a song that encapsulates the user's emotions. Delivering the Experience: Our user interface then presents the generated raga song to the user, completing a journey from raw emotion to personalized musical expression. Through RagaCraft, we're redefining the way users experience and interact with traditional music forms in the age of AI.

RMMM

Introduction Welcome to the World of Crafting Your Own Voice Wizard 🎙️ The concept is a personalized voice assistant that bridges the gap between humans and technology using voice-text transformation with Python and the Llama API. This is a highlight to unveil the secrets behind creating an interactive and enchanting Jarvis-like assistant. Voice Recognition (Listen for Command) The Art of Casting Spells with Your Voice 🎶 Explore the wonder of voice to text and back again using Llama API as it transforms spoken words into written commands and then back to speech again. Explore and share with friends how the "listen_for_command" method creates a magical bridge between user voice and digital interaction, bringing the assistant to life. Text-to-Speech (Generating Responses with Llama) Transforming Whispers into Majestic Speech 📣 Dive into the enchanting process of converting text into lifelike speech with the Llama API. Illustrate how the "text_to_speech" method weaves text into captivating auditory experiences, adding a personalized touch to interactions. Highlight the synthesis of natural-sounding voices, bringing forth an auditory dimension that connects users with their digital companion. Enhancements and Extensions Elevate and extend your assistant's capabilities beyond voice recognition and synthesis by teasing out the limitless possibilities: from controlling devices with voice commands to infusing emotional intelligence into speech. Conclusion The transformative power of Llama API and Python create a seamless human-computer interaction and makes a easy and fun to interact with all your devices just by talking to them! Our vision of the future where voice assistants understand context, emotions, and devices, leading to more immersive experiences. We are creating new spells that redefine how we communicate with machines. Thank You and Cheers!

Too much Base

You have outlined the process quite comprehensively: 1. Utilize the EnCodec model to encode audio files into vector representations, saved as text files. 2. Process these text embeddings using the "emojiintrospector" tool to generate emoji sequences that represent the audio. 3. Validate the emoji outputs across test audio samples to ensure that the harmonic relationships are maintained. Key points: - EnCodec encodes audio to discrete embeddings, output as text. - The "emojiintrospector" tool maps these text embeddings to emojis. - Generated audio samples with 3 harmonics are encoded. - Analyze the emoji outputs to identify common patterns representing harmonic frequencies. - This demonstrates that the pipeline retains the harmonic structure in the emoji mapping. - The resulting emoji sequences can be used for visualization or further analysis.

IntrospectiWaveVisioneers