Bubble with Speech to Text using AssemblyAI - Part 2

Introduction to Speech-to-Text APIs in Bubble

Welcome to part two of our mini-series looking at speech-to-text or speech recognition APIs and how you can easily add them into your Bubble app. In part one, we began by working through the AssemblyAI API documentation, and we have two calls in our API connector for AssemblyAI. One is to send off our audio file, and then the other one is to fetch the results once AssemblyAI has finished processing the file. Remember, this is different from Whisper. You don't get the response back as you wait; you have to check in once the transcript is complete, but you can automate this process using webhooks.

Using Webhooks with AssemblyAI

If we go back into the AssemblyAI documentation, we can look at the section called "using webhooks". By looking at this example, we can see that we can add a webhook into our JSON. This is basically saying when we send out our mp3 file that we want transcribed, we say, "When you have it ready, let this endpoint know that the file is ready." So I'm going to add it in here. In fact, I'm going to add in a space just so that the JSON is a little bit clearer. Okay, and then comment, and by webhook, I'm going to make this dynamic so that it is adaptive to whether I'm using my development or test version of my Bubble app or my live version, so I'm just going to call this webhook URL.

Setting Up the Inbound Webhook

So how do I set that up? The inbound webhook - well, I go in and I go on to API and I enable the API work for the API and backend workflows, and that then gives me an option up here. Now I create a new API workflow and I'm going to call this "inbound transcript". I have to write it without spaces if I'm going to make it a public API, which I need to have in order for AssemblyAI to be able to send the notification to me. Then I'm going to say "run without authentication" because the AssemblyAI webhook is not going to be able to authenticate, so I just want to do it. You might want to obscure the endpoint. I mean, there is the chance that someone could send data into your app if they were to guess this. It's up to you to judge how risky that is, but for this demonstration, we're going to have it public and have it run without authentication.

Demonstrating the Webhook Process

I now need to demo that, so I'm going to change this to the text request, copy this in. Actually, I'll open this in a new tab. I'm doing this because I need to teach Bubble what the inbound request looks like, and the easiest way to do that is to use detect, but that's only in detection mode by this box is up. I'm then going to go into my APIs and my AssemblyAI API, and right here we go. So my webhook URL, I paste into there, and I'm going to reinitialize. This is going to upload my tiny audio file again that I'm demoing this process with, and this time it's also supplying the webhook. We can see the status is queued.

Handling the AssemblyAI Response

Then I'm going to go back over here, and because it's a very small file, actually AssemblyAI has responded incredibly quickly, and so it's told me that the status for this transcript ID is now complete. What do I do next? Well, I go fetch the transcript ID. And how do I do that? Well, I've already set up the API call. I did this in the earlier video to get a processed transcript, and because it is an action, I can go into workflow and I give plugins get processed transcript. Instead of it being the test data I used, I can use data request transcript ID because that is the data that AssemblyAI are putting into this endpoint.

Saving the Transcript

I then send back another request that gets my transcript, and for the sake of making it simple, I'm going to save it as a message and content. This is set up from a previous video where I was demoing ChatGPT, but I'm just going to test it using this content and then text. Okay, so that should work. I should be able to upload my file, I should then be able to be notified by AssemblyAI through this webhook endpoint communicating into my Bubble app, I should then send an outbound request for the transcript using the transcript ID, and I'll create a new message with the content of my speech to text.

Testing the Workflow

Let's test it. So I'm going to here again say reinitialize. Cool, my file is uploaded. Ah okay, I've made a mistake. I need to get rid of initialize here. Right, reinitialize the call. You can see that it's been queued up, and then I think I can just go into data app data messages, and this is all from a previous demo where I was using the GPT-4, but there we go. Yeah, we saw that come in in front of our eyes.

Conclusion and Next Steps

So I can then confidently say that every step of my workflow is working. I think this is where we're going to wrap up part two. In part three, I'm going to demo how to create this on the front end to enable a user to upload a file or maybe even record a file and then send it off to the AssemblyAI and display the results back to the user.