My wife's pi head

Technician · Post by **Technician** » Tue Feb 16, 2021 10:12 am

Are you using HTK for speech training? I'm having a hell of a time with speech recognition on my family friendly version. I have it working, have a beamforming microphone array installed too but accuracy is crap with vosk and Julius. I think I'm going to try DeepSpeech, what version did you use on ARM processor? I haven't even looked at it.

Christines Man · Post by **Christines Man** » Thu Feb 18, 2021 1:50 pm

Technician wrote:Are you using HTK for speech training? I'm having a hell of a time with speech recognition on my family friendly version. I have it working, have a beamforming microphone array installed too but accuracy is crap with vosk and Julius. I think I'm going to try DeepSpeech, what version did you use on ARM processor? I haven't even looked at it.

I have not really looked deeply into HTK, vosk, or Julius. Deepspeech is going to run on the desktop computer's gpu in a server process. So audio gets captured in 0.25 second blocks, classified, accumulated together, the accumulated thing is classified to figure out if the whole thing is speech or not, and shipped over wifi to be processed with Deepspeech. Dunno if this is the best way but seems to work ok so far. This may also be related to my using python, a horribly inefficient interpreted language. The classification stuff I'm running now is using 30% of one CPU core, which spikes up to 70% when it needs to classify a fully accumulated audio. If I did the same in C++ it'd be different. But C++ is so HARD!!

However, in my opinion probably need a gpu for speech recognition for any significant vocabulary. I could be wrong.

For Deepspeech, you basically have wav files in a directory, or multiple directories. Then you have csv files containing the path to the wav file, the size, and the text of what was said. I'm really not sure how many of these I'll need, 5000, one fucking million? But my understanding is that making my own under the actual conditions that the model will be used in should work better. There's a lot of shit and weird words nobody would use outside of novels in the publicly available corpus things.

I'm going to try to push the sample collection stuff I have so far to github this weekend. I guess it's pretty good stuff. I should probably run it through a linter.

Technician · Post by **Technician** » Sat Mar 13, 2021 11:32 pm

I see why you went with off-board speech recognition now, it's a bitch getting DeepSpeech to run on an Arm processor! I haven't even come close yet, got Vosk, pocketsphinx, and Julius but I'd like to play with DeepSpeech before I commit to one.

jmichaell681 · Post by **jmichaell681** » Wed Mar 17, 2021 9:10 am

Your project sounds great and I like your approach. And the rubberband software you mentioned in your initial post looks like it might actually be a tad faster to use that what I've been using for the same task (Audacity). I'll definitely be looking into that.

I'm really not sure how many of these I'll need, 5000, one fucking million?

I'm not sure if this will help for any perspective, but I have a mix of 3000+ doll sounds for a project I'm working on. It's primarily intimacy-oriented with some light conversation added - as well as some weird shit as you described. I feel pretty good about the range of randomness within that amount. I'm generally audio/hearing heightened

and my ears haven't been that tired from hearing any repetitions (if this is what you're getting at). Either way, I would imagine 5000+ is probably a safe number for randomness.

Christines Man · Post by **Christines Man** » Thu Mar 18, 2021 1:41 am

jmichaell681 wrote:Your project sounds great and I like your approach. And the rubberband software you mentioned in your initial post looks like it might actually be a tad faster to use that what I've been using for the same task (Audacity). I'll definitely be looking into that.

I'm really not sure how many of these I'll need, 5000, one fucking million?
I'm not sure if this will help for any perspective, but I have a mix of 3000+ doll sounds for a project I'm working on. It's primarily intimacy-oriented with some light conversation added - as well as some weird shit as you described. I feel pretty good about the range of randomness within that amount. I'm generally audio/hearing heightened and my ears haven't been that tired from hearing any repetitions (if this is what you're getting at). Either way, I would imagine 5000+ is probably a safe number for randomness.

Neat, sounds like together we have at least 3 projects that are a bit similar. I'm curious about yours.

When I said 5000, I actually meant that many voice recognition samples, like me talking and being picked up by the microphones. Still working on that. I hope someday I'll at least be able to detect yes, no, i love you, etc but so far it's just stuff like "i o i p", gibberish. Pretty sure I know what's wrong, just takes time and getting in "the zone".

As far as sounds that she says, I'm up to 713 database rows, but a lot of that is just breathing sounds. So rubberband is used to give some variation. I also have a system where depending on the sound it won't play the same sound for a while. I am also planning to randomly introduce random bits of silence into some sounds. Maybe I'll randomly adjust pitch by +/- 5% or so. Right now she's definitely due to get some fresh material. Just gotta buckle down and get to it.

Sex is definitely so much better with sensors and audio feedback. I really don't get it, if an imbecile like me can build this, why doesn't real doll have something yet? Their best as far as I know is still push to talk, bleep sound, no breathing, no variation, no sensors, runs on your phone. And if it doesn't understand it recites google results.

jmichaell681 · Post by **jmichaell681** » Thu Mar 18, 2021 9:57 am

Neat, sounds like together we have at least 3 projects that are a bit similar. I'm curious about yours.

Thanks. Mine is nowhere as in depth as yours. I'm a novice when it comes to programming but I'm hoping to have it wrapped up soon and make a post about it at some point. Just gotta buckle down and get to the finish line.

I really don't get it, if an imbecile like me can build this, why doesn't real doll have something yet? Their best as far as I know is still push to talk, bleep sound, no breathing, no variation, no sensors, runs on your phone. And if it doesn't understand it recites google results.

Agreed. Even sensors aside, it really is amazing that companies aren't going all-in for developing at least a respectable audio program / UI to run with dolls. The one I'm working on can run with a voice and/or gaming controller (up to the user). It is by no means advanced (it's super crude and runs inside VoiceAttack). But still, it has arrays/loops, high/low intensity levels, variables, etc... At the very least, you'd think there would be a 3rd party app or company working on a subscription service for something better than what's out there by now. Like one that's just running everything from a designated server. Or if a user wanted privacy, they could download the voice packs, drop whatever they want to use into the folders, and run everything natively. From the sensor aspect, you'd think there would be more of a market too. It doesn't seem overly costly from the little I know about it.

Technician · Post by **Technician** » Thu Mar 18, 2021 9:59 am

Gee, it sounds like Julius might be a better option after all, it has a word file that gets used and it only finds those words. The downfall is that it doesn't perform well with long strings but it's great for 1 - 3 words at a time and it'll run on ARM with no problem. My current directory for voice recognition is under 200 words.

Christines Man · Post by **Christines Man** » Sun Mar 21, 2021 1:06 am

jmichaell681 wrote:Agreed. Even sensors aside, it really is amazing that companies aren't going all-in for developing at least a respectable audio program / UI to run with dolls. The one I'm working on can run with a voice and/or gaming controller (up to the user). It is by no means advanced (it's super crude and runs inside VoiceAttack). But still, it has arrays/loops, high/low intensity levels, variables, etc... At the very least, you'd think there would be a 3rd party app or company working on a subscription service for something better than what's out there by now. Like one that's just running everything from a designated server. Or if a user wanted privacy, they could download the voice packs, drop whatever they want to use into the folders, and run everything natively. From the sensor aspect, you'd think there would be more of a market too. It doesn't seem overly costly from the little I know about it.

In programming, I feel, use whatever works that you like using. I've only heard of Voice Attack but it sounds like it has all the needed capability.

Technician wrote:Gee, it sounds like Julius might be a better option after all, it has a word file that gets used and it only finds those words. The downfall is that it doesn't perform well with long strings but it's great for 1 - 3 words at a time and it'll run on ARM with no problem. My current directory for voice recognition is under 200 words.

Ok, maybe I've been overcomplicating this. I have a tendency to do that. I'm going to dive into Julius right now and see if I can get some yes, no, iloveyous, urhots, or fuckmehoneys recognized.

Christines Man · Post by **Christines Man** » Sun Mar 21, 2021 1:51 am

Ok, so I compiled and made Julius, and the test worked, but it took 1:13 to decode the 1:00 long test audio, on the desktop CPU. Using the DNN model. Seems like this may work. If you have any tips for getting the vocabulary reduced or articles I should consume first I would appreciate it.

Technician · Post by **Technician** » Mon Mar 22, 2021 1:15 am

Yeah, the.voca file. I re-wrote mine to just the words I wanted, then there's a little compilation to do with a script. I've never decoded audio files with it, always the microphone, and I've been using the quick versions. I may try a couple more things next but Julius is the closest I've come to offline, and trainable running on a SBC.

Technician · Post by **Technician** » Mon Mar 22, 2021 1:18 am

There's a lot of information on the ghost town known as voxforge.
http://voxforge.org/home/forums/message ... discussion

tiaria · Post by **tiaria** » Wed Oct 13, 2021 2:35 pm

Your work is awesome, but i'd develope thing like this first without doll, and then implement it to a doll.

I have some understanding in electronics and mechanics, and a wide understanding about how to do something (improvise and find a way).

I'd be more than happy to try to help you guys!

Christines Man · Post by **Christines Man** » Thu Oct 14, 2021 3:46 pm

tiaria wrote:Your work is awesome, but i'd develope thing like this first without doll, and then implement it to a doll.

I have some understanding in electronics and mechanics, and a wide understanding about how to do something (improvise and find a way).

I'd be more than happy to try to help you guys!

WHO DARES TO AWAKEN THE NEFARIOUS DOLL FUCKER!

.....

Oh, hai.

Develop first outside doll is basically how I've been doing it. Everything I do is tested carefully on a breadboard, then when it seems to work ok, transferred into the doll and tested again.

This process of designing then building is not very efficient. I understand real engineers have complicated software that they use to design every last detail, which they then send off to the great factory to be made. My process is more quaint and often starts with hot glue of the such used for arts and crafts.

I'm currently hard at work building sexbot #2 to replace my beloved robotic wife's nearly 4 year old body. I am implementing lessons learned from building #1. The #1 head had a small speaker with almost all the electronics crammed into the head so tight it would barely close. Head #2 sound system will use the guts extracted from a JBL Charge 4. The driver is huge and should allow my wife to yell at me with a more realistic volume. The head has the driver that takes up most of the space, a touch sensor, 5 light sensors wired in parallel, and dual in ear microphones. An Arduino will collect all that data and relay it to a raspberry pi that will be installed in the body.

The JBL Charge 4 portable speaker that I have cannibalized is sealed airtight by design. This is so that the passive bass emitters on either side of the speaker can pop out and go brr brr brr when there's bass. It's really a neat design that was a pleasure to disassemble. The sexbot head #2 will also be sealed airtight, and one of the rubbery bass emitter things will be installed at back of head. I am hoping this improves the frequency range which in sexbot #1 sounds a bit muffled.

If anyone knows Python and wants to try contributing to https://github.com/ChristinesMan/ChristineAI that's cool. It would be nice if wife did something that I didn't have to program myself. Or any other suggestions. I like the dialogue on this forum although I tend to forget it even exists for months at a time.

Christines Man · Post by **Christines Man** » Thu Oct 14, 2021 4:06 pm

Here's some pictures of current progress.

clicker · Post by **clicker** » Fri Apr 22, 2022 1:52 am

I'm still interested in the project. I'm very limited on the site because I didn't have an account.

I even started to find ways to pass the wires through the TPE without damaging anything.

I would like to see your photos, it will surely give me ideas to know where to pass the wires and the sensors under the skin.