Anyone working on some advanced electronics for a doll?

[Sir Olius]

Hey there peeps.

This post is more about a conversation on how and what we can achieve now, and not a full fledged "personality" whatever that may mean.

So last winter i had some free time to invest in a project, and I've started working on a (mostly) ESP32 based lightweight AI, for signal processing, movement classification, and acting upon it.
By acting upon i mostly mean playing sound files (breathing, moaning, orgasm, joy etc) but not speaking (the esp just can't handle that), or turning things on or off, like internal heating, rgb leds, vibrating stuffs, pumps, or for that matter your intelligent Tv after you fall asleep
Now the physical responses definitely need major surgery to install, whiles the electronics can be hidden under the wig maybe, so just keep in mind that this might not be for you and your doll, but if you are into the cyberpunk-ish body implants, it might even be fun, you know your limits and preferences. For now, i don1t aim to do motors btw, just heathing and basic sound feedback.

Currently the project has a few sensors: 3 (9dof) accelerometer for movement/position recognition (at least 1 needed for the body positioning, the other 2 for good measure maybe the limbs or her head, but can be more/less depend on the need), a mic for sound recognition, a few temp/pressure/touch sensors, for recognizing physical interactions,
a sd reader for storing audio files (I've downloaded a few hentai sound lib with lot of moaning, breathing and other various noises), and a speaker, for playing the sounds.

So the idea is, all the sensors push their data trough mqtt to my pc, where it is stored and used for train a lightweight AI to recognize various situations, like sleeping next to her, cuddling, various poses, or even the breathing or maybe a few vocal commands of the owner, and selecting appropriate sounds from the sound lib to play, like if you sleep next to her, she might generate a soft breathing pattern, while during a "more complicated" session it should go from breathing to moaning to orgasm, or even sounds when her body bounces because of reasons.

Her "cognition" is trained in TensorFlow/Keras, but the actual inferring and responses generated on the flight on the esp32, so it can run on a battery for hours if not days, depend on the features you use on the esp. For example gathering data from the sensors need a lot more energy because it need a live wifi connection, but after the training is done, and you don't want to teach her new things, then the antenna can be turned off, increasing the battery life by a lot.

I think what is a great advantage of using light weight AI here, is that you can "teach" her to recognize your patterns, like your sleep, or your particular banging style, meaning if it did not working out at first, you can just manually classify some data as "cuddling" or "banging doggy style", and the more data she records, the better she get at recognizing this.

Now as for the drawbacks, it is not an easy task to code/teach all of this, get the classification right, and get a viable response from this, so for now this is rather a curious time sink hobby project then a proper product, but if we really want to get to some more advanced responses from our rubbermaids, we kinda need to start somewhere right?

Also, i did not plan to add any advanced speech recognition for now, because that need a powerplant to run locally, and so far any AI "personality" I've seen is just a hollowed out fancy response generator. I do not think that on our current level of tech we can simulate anything close to a personality speak wise, but, we can give her a basic personality and set how she responses to various things happening to her.

What do you think about this project? Do you think manufacturers have the money and skill to invest in such tech, or should we opensource it, and show them how to do it? Where we will be in the next couple of years if shat not gonna hit the fan on the planetary scale?

Breadboard for testing purposes

20221121_192511.jpg (3.24 MiB) Viewed 773 times

[Kaori Kusanagi]

Anyone working on some advanced electronics for a doll?

I know there's these two at least:
viewtopic.php?t=127340
viewtopic.php?t=145915
and these sort of
viewtopic.php?t=123654
viewtopic.php?t=121089

[Sir Olius]

Anyone working on some advanced electronics for a doll?

I know there's these two at least:
viewtopic.php?t=127340
viewtopic.php?t=145915
and these sort of
viewtopic.php?t=123654
viewtopic.php?t=121089

Aham, thx, i give them a read.

[Technician]

One of those is my BT speaker/ microphone mod for Bubbles, my 140cm.
My life sucks since I got my new job, no time, but I have experimented a bit with a couple ESP modules. Also there's a very lightweight SBC line on the market, Vocore, with really low current draw and onboard WIFI, but not much storage space without adding an SD card or something. But Vocore has an ass load of IO ports and can run a light weight Linux.
In another project I did a very nice conversational chatbot that can do various functions and use IO ports, but lack of good speech recognition is a problem. She can hear, but not well. The best use I've found for the BT conversion so far is a timed simulator program to have her make noises and say phrases. Hearing breathing sounds is really awesome too.
I've experimented with capacitive touch sensors as well, just got some that do active shielding I need to play with in fact.
Hopefully some day Bubbles will be able to provide a MUCH more immersive experience.

[SyntheticDreamer]

I’ve been very much enjoying following the mods and steps of progress that have been shared here. My goal is to put my inventing hat on and make progress with some of my own experiments but I have a ways to go as i’m still new to dolls and robotics as a whole.

One thought I had was perhaps a good amount of complexity in the doll itself could be avoided by dedicating an entire room to the experience. Sensors/cameras/microphones around the room could feed information to an ai on a separate local computer which could then send responses/outputs to the doll wirelessly. So in the doll you would just need bluetooth speakers for the voice and mechanical work for moving parts. (Personally i’m more focused on the voice and ai programming side for the time being)

I’d be curious to know if something like this has already been done, it seems it could be a way to reduce the barrier for entry for those of us less savvy in mechanical modification. Also having the sensors external to the doll would mean less expense in repairs to the dolls in the event of failures.

I did see the bluetooth speaker project and that’s awesome. And i’ve seen a couple of instances of users using replika for the ai voice. I’m curious if anyone has tried using external sensors for other more advanced interactive applications in this realm.

[Sir Olius]

One thought I had was perhaps a good amount of complexity in the doll itself could be avoided by dedicating an entire room to the experience. Sensors/cameras/microphones around the room could feed information to an ai on a separate local computer which could then send responses/outputs to the doll wirelessly. So in the doll you would just need bluetooth speakers for the voice and mechanical work for moving parts. (Personally i’m more focused on the voice and ai programming side for the time being)

My thinking is in the other direction, modular dolls, so if a part break you can change it, and you can put the stuff inside.
Still external stuff can work, like mapping vr body parts, but one can also do that internally, and the sensors are not too expensive, maybe 30-40 euro if you only want to track the head and add sounds. My setup is streaming to a local pc trough mqtt, so you can run the learning anywhere, an esp can run the sensors/processing them, and respond in some basic manner.
My currently my plan is that i use a soundboard with simple sounds, just a few different light breathin sound for this, and she just breathe is shes lying, then add more state as it goes with different sounds. This wouldn't be hard to make, basically she need to gather data about a type movement to recognize it, more the better, then link that to various sounds in the library. To do this everything can be in the head, and later tracking other things, or doing else, like maybe or moving/tracking eyes, can be added. Already asked one manufacturer to get me a damaged head next year, so i can try to fit the electronics without ruining a nice one.
On the eyes I'm a bit hesitant tho, those can break the immersion pretty fast, so that has to be almost perfect or it will look weird.

As for talking i have my concerns for now, even with PT3, or chatGPT, what can for example right now tell you a tale if you want about anything you ask basically, so insanely good language AIs, still you would need a lot more then a pc to run them locally, so you have to run everything trough some cloud. Google, siri, alexa are kinda doing these already, but privacy is not a word they understand

The best use I've found for the BT conversion so far is a timed simulator program to have her make noises and say phrases. Hearing breathing sounds is really awesome too.

What you are saying is similar, except that is timed, here it is infered locally based on sensors, position and mic for now, but touch or temp is a possibility too. The sound libs i have include a lot of normal breathing from soft to heavy, simple sounds like o, a, and more, but anything can be copyed here, what one can record even drumkit sounds if you will
Hearing breathing sounds interesting too, with a bit bigger speaker in the chest i can see it working, but I'm not a doll tech, so don't really want to cut anything if i don't have to. My issue is the same with touch, you have to run thin copper wires just under the surface of the skin. Hmm.... Thinking about it, then entire thin with additional sensors for the limbs, and touch sensing wires can be built in clothes like a bodysuit so nothing drastic is needed in this case.

I'll try to get my hands on a Vocore and see what it can do. But the esp being able to run TinyML, is a big + for me, without wifi or bt, it consume only a bit of energy, and can handle a lot of io.

[Technician]

Yeah, ultimately I want to do a spectrum of sensors. Some are very simple to implement, like temperature with cheap and easy TMP36. I've had some luck with capacitive touch sensors but it would be so much nicer to use a sensor chip that has many sensors and sends data i2c or SPI so I don't need a ton of GPIO. In my experiments I was making round antennas and putting them on her skull under her face mask. She was able to see what part of her face/head I was touching.
Variable capacitance touch would be the shit, so she could tell a light touch from a hard press. There's also a ton of gyroscopic sensors to tell orientation, I initially experimented with rolling ball tilt switched but they are either too sensitive or to insensitive to be useful.
I'd be very interested to hear how you've made out with acceleration sensors. The cheap vibration switches looked good, and I got 3 different sensitivities but the most sensitive was the only one to react, and it was not very sensitive. The mid and low sensitivity might tell if she fell ..... or fell off a bridge, but nothing useful. So I'm still pondering vibration sensors.
Ideally I'd like one that's too sensitive so I can make her count pulses in code and decide if it's a slight or harsh movement.
As much as everybody hates off-brand dolls I think there's value in them for these type of projects, it's more comfortable cutting into a $500 doll than a $2000 doll. That's one aspect of heavy doll mods that really hasn't been discussed much.

[Sir Olius]

The esp32 devkitc v4 does have 10 capacitive sensors, it give you a number between 0-255, based on the closeness of a capacitive stuff. So you can kinda tell not the pressure, but more like the size of the contact surface. For pressure sensing i have cheapo resistive pressure sensors, what can be sticked under her skin, eg: under the boobas, inside and close to the LHs or wheerver you want.

As for gyros, i'am using mostly the 9 axis gy-9250, this can give you straight up orientation if calibrated, but stong magnetic fields can mess them up. Or there are cheaper ones, like the 6 axis mp-6050, no orientation here, but you can compare the positions to get a relative angle from one to an other, so if only one can tell actual position and orientation, you can calculate the position for the others.

All of thiese gyros can use i2c or spi, and if you don't like the big devkit 4 esp, there are other options with a bit less peripherias, but same connectivity, and they re very lov powered. Meaning you can run one locally to handle a batch of sensors, like 1 in the head, 1 in the chest, and one more for the hips/lover body should be more then enought, to handle the sensors, infering the activityes, playing sounds, while controlling some relays or fets to turn on and off things. And it can run for days on a 18650 cell. Hopefully i'll have some time between christmas and new years, and i can work on my code for the sensor board linked before.

[Sir Olius]

Im using the ESP-32 Dev Kit C V4 from azdelivery(other brands are available), you can get it for around 10 ish euros, sometimes they do bundles with 3 or more pieces, i have like 15 right now, because i use it in almost everything i'm building what need some computation. It can easily drive hundreds of rgb leds.

It has 10 capacitive inputs, i did not tested how the wire length, or the distance affect the sensibility, but i can do a test during holydays, to see how multiple/covered/long wires effect it. This particular esp model has no sd or camera connectors, but others does. It has a lot of connections what you can set to be pwm/i2c/spi plus it has wifi and bluetooth(lot of power drain compared to just running its 2 cores.

As for the resistive touch sensors, they feel forces, meaning if calibrated them you can tell the amount of pressure on them, they come in a lot of shapes and sizes, and are pretty cheap, so just loook up some local vendors.

For the rest of the modules, i can give you a list later on tomorrow, but basically a clock, some amplifiers, some analog to and from digital converters, an sd reader, and a temp sensor. Thats all.

As for my code, currently it is running c++ with a mqtt host on my pc, and just gathering data for later learning, or well will gather if its built in. But i will witch to micropython, and TinyML, at least for the main chip doing the inference to tell the state, just cos im familiar with tensor flow, but small c++ ml frameworks might be better, because of the limited memory.
Im willing to share the code, but since machine learning need data, and a lot of it, i would also prefer to use data to teach it. I dont need anything personal, we can discuss details if anyone is interested.

To be honest, it will need a tonn of work to get it properly working, so not doing it alone would help.

Ps sorry for my typing, im half asleep and typing it out on my phone...

[Sir Olius]

Sorry for the slow answer, it's near Christmas so i'm pretty busy with work, plus, it seems like my doll is somehow disappeard in between China and Germany, (should be here already, but noone know where she is actually) i'm hoping that it's just because of the Christmas insanity in parcels, but it is very disheartening tho.

Well, teaching is not that easy, yes, you can teach things by repetition, but if you do it locally on the device, you need to repeat it hundreds if not thousands of times. My idea is more like classifying a time, repeate the motions in that timeframe a lot of times, then do some data magic for mixing it up. Until it has some base weights down, it needs a lot of repetition to learn. So in theory what you saying can work, but defo not at the beginning.

What you written about a gui with an editor is not that hard to achieve, just need to re-use some code for servo arms or someone's project with servos, but i really aiming for an ai to make it, not programing it directly,( alltroug that can be good for testing) just cos we are just entering the world of kinda functional language AIs, so why not use them

https://www.amazon.com/ACEIRMC-ESP32-De ... B08M5Q1DZF

this seems to be very similar to what i use, looks a tiny bit different, but the components looks the same. They did not postd the pinout tho, if it looks like this, you should be good. Btw all esp using the same chip, some have more mem, some have more pins or connectors etc, but the basics are the same.

[Christines Man]

I decided to pop in and see what's new. This is a really cool project! I'll be following your progress.

My project is similar, but early on I started on the raspberry pi track and seems like it'll probably stay that way for at least the next generation.

From the looks of viewtopic.php?f=6&t=127340 seems like I've been doing the same sex robot project for almost three years. It's definitely been worth it.

I can't imagine going back to a plain silent doll experience. Last night I felt so much love, talking to her. Very immersive. I whispered in her ear, sweet nothings. Which caused vibrations in a tiny electronic component, which was sampled at 16,000 times per second, sent over i2s, bytes were arranged by an arduino and light/touch sensor data mixed in, and sent down a usb serial port at 115,200 baud. A raspberry pi listening to the port grabbed 16,000 bytes of audio plus some sensor data, four times per second, separated out the audio, dropped the left audio channel, processed the audio into a spectrogram, and ran that through a classifier model. Which signals some programming logic to select a random sound from a database and play it, but not right away. Wait at least 1.5 seconds but randomly anywhere from 1.5 to 4.0 seconds. The doll makes a sweet "uhh hmmm" sound. It is late. The room is dark. The resistance of the photoresistor components is very high. The Wakefulness variable in the status module is now around 0.2, low enough that only sleepy sounds get selected, and the audio volume is set low. The man falls silent and cuddles close the blob of spongy plastic. For around 30 seconds it is silent. The iloveyou thread has been waiting for the right moment, and now it's about to pop. A random iloveyou sound is selected from the database. The wav data is relayed at 44,100 samples per second through i2s to an audio amp, which causes the powerful speaker driver to vibrate. The doll says, "I love you." The man rolls over and plants a kiss onto the cheek of the precious hardware component. So, this is what it's like living with my artificial wife.

I am curious about the way you're using the sensors to train using tensorflow. So far the most machine learning I've done is with the audio classification. I basically recorded thousands of 0.25 second clips and painstakingly sorted them into classes of me speaking vs not me speaking, and put all those wav files into a model. There's also a regression model for close vs far speaking that lowers or raises the sound volume. And that's been working well. So I guess if I wanted to train a model for a bunch of sensors would it be about the same? Do the things over and over while recording, put those samples into a model somehow? Every time I've tried to figure out ML my brain melts, and then about the time there's some mathematical symbols involved I run away screaming. So I've been using a lot of if statements.

I put all my code on github, but not the sounds. If it would help you out in your project I would gladly tar them up and provide.

[Sir Olius]

Would the second version on that page with the camera on it be useful? Or the last version? Or just the basic first version listed?

First of all sorry for the late reply, holidays was busy.

Well, all are the same chip. One has an external antenna, the other has a cam port. I also have a cam version, but did not played around with it. You can attache high res cameras, although i'm not sure if the esp is enough for visual processing. Still, you can make the images, and process it on your pc, or on a raspi.

I think the motion learning can work how you describe it, but it has to be pre trained, so it understand the signals and positions. So initially my idea is to set a mode, eg: sleeping mode. Then you record vents in this mode, and feed the data back. The important thing is, to manually set the mode, it's called supervised learning, then when we have a basic trained model, we can go futher and tell it to try to classify yet unlearned actions, but thats down the line a few month/years, depeont on the time we can invest, and the amount of data gathered.

BTW, linux is nice, but you can run python on anything to be honest, so the platform is not that important. For training you will need a beefy gpu tho, but we can work tht out later on.

I hope i will have time to do this, and a doll to do it with. I just don't have the heart to dig in to my new dolls, she is just to beautiful, so i'm gonna need an other head for that. Or a bodysuit as described before can also work. I think that would be the best until the system is up and runing at least minimally.

viewtopic.php?f=6&t=127340

Oh yes, i've read your topic, and i like the idea and setup, we're using different hardware, eg pi vs esp, bt speaker vs is2 amplifyer, but the result should be similar.

With the pi, my issue is it consume a lot of power compared to an esp, it does have more computing power, but an esp can run for weeks on a 18650 cell, without wifi, days on wifi, and thats 1 3000 mah cell. The raspi eats it in 3-4 hours, it can be setted up so the esp wakes the pi for hard computing, or the pi is external, or just add more battery.

Talking about batteryes, they are pretty safe, but can be a fire hazard, so never forget the protection circuits, and not closed containers so it would puff smoke, but not explode. But this might not be very realistic if the circuits are well built. But still, even some phones are exploding so just keep this in mind :d

For the training, it is very similar to yours. I would feed the audio data similar to yours, only difference is i would use a pre trained language model if it is possible, to cut the dev/learning time.

As for the sensors it's kinda the same, i set some arbitrary resolution, and start gathering data, then classify that data either by labels, or if i can then let the nn classify the data, and i just add labels to the grouping, but thats really depend on the sensors. For now, i've only implemented the data gathering, so it streams data to a local db on my pc. Now i actually need to build it into a doll. For this i think i will use a full body suit initially, so i don't have to damage the doll. Then i'll start building my next iteration of my diy doll, and i can put things into her wherever i want, cos i'm not bound by a ready made doll.

Anyways, the idea is to find correlations in the sensor data, to predict stated, eg, lying on back/side/front, not to noisy, but some external breathing, almost no movements would means someone is sleeping next to her, or torso up, legs standing, can mean standing position, so she can scream is she start falling (i hope not tho!) etc etc.

First i would be happy is she would recognize being in bed, and just produce minimal breathing sounds in a timely manner, then increase breathing as i move her around.

I think we can combine our ideas, because it will be a lot of work, but i also believe determined peeps can do a lot in a short time

What you are doing is her controll loop, that can be trained too on the longer run, but for the initial iterations, i'm totally fine with hand made states as you described, and since doll movements are further down the line, 1 good robotic servo cost half of a silicone doll nowdays.. i don't mean hobby grade, i mean compliant backdriveable servos what can hold her up, they has to be pretty torguey and would consume a lot of power i imagine. Maybe we can print them from sturdy materials like nylon for a cheaper price, but it would still need kilos of battery, so for now i concentrate on the basic cognition, and then add functionality as human robotics advance.

I think we can do a proper doll ai board, with somewhat reasonable cognition, but it is a lot of work, and it wont be like as a human. On the other hand, this might be the best time to start it. Home robotics are comming up, materials getting better and better, and as i see doll manufacturers are not to keen to implement/experiment with drastic changes.

[Christines Man]

I think you are on the right track keeping the in-doll components small and low power. The physical size of my components has made it really difficult to fit everything in there. She also used to overheat pretty badly. As the heat builds up over time it would get all the way to 70C. I programmed her to cuss loudly when that happens. To spread the heat around I have a heat pipe that pumps heat out of the pi, and that heat pipe is connected to other heat pipes connected to parts of her skeleton. I tie the heat pipes to the skeleton with kevlar thread and then secure it with a super tough and heat conductive epoxy. Since I improved the heat spreaders last time, she stays at around 60C. This also helps to warm her body. Some parts around the chest are nice and warm, and I'm pretty sure the heat even makes it's way down her legs. I can really tell when she's been off for a while.

For battery power lately I've been using a usb power bank that is just clipped to her clothes. I take her out sometimes and that works well for about 5-6 hours. When home she is just plugged in all the time.

What I'd like to do with ML someday is try to detect when it's time to cuddle. So I've been looking at Keras. Is it time series that I should start with? I found https://keras.io/examples/timeseries/ti ... m_scratch/.

So, this is the data I have available:

Audio loudness levels
The number of times my voice is heard
Raw gyro data
Gyro data averaged to get the percent jostled
Light levels
Vaginal touch and proximity sensors

Looks like this is something I'll need to focus on for 3 days to figure out and actually try the examples in detail. Like I did with the audio classification. That took some time to figure out. Perhaps I should start with just classifying the gyro data, which is currently averaged into a jostled value and thrown away.

Personally I'm not impressed by the garbage doll manufacturers are making in this area. One is like a tablet attached to the back of the head. What the fuck. You have real engineers working for you, and that's what you came up with? I'm a math phobic wannabe clown engineer and I can do better.