Woman Says Her Daughter’s Voice Was Mimicked in AI Scam
HTML-код
- Опубликовано: 13 апр 2023
- Jennifer DeStefano received the scariest phone call of her life. As she hears her daughter crying, a man claims he kidnapped her 15-year-old daughter. The kidnappers demand $1 million before reducing it to $50,000. Turns out, her daughter was not kidnapped. Her voice was duplicated by a scammer using artificial intelligence. Fortunately, DeStefano was able to reach her daughter before paying the ransom money. DeStefano’s daughter has no idea how the scammers got a sample of her voice.
Imagine what AI will be capable of years down the line.
Hold on to your papers!
Won't take years...
Lol more fear mongering about AI
Later they will have a random person tied up and just have their face deepfaked to make it seem even more real
Months
Imagine how hard it will be to prove your case in court once AI usage becomes rampant
even more false accusations towards men, oh boy
No, imagine how many guilty people will go free because they'll blame audio evidence on AI technology.
It’s still not difficult to decipher. Yes the voice is accurate but the tone is still robotic like and doesn’t flow how a human would talk in normal conversation
@@rongooden6545 The more complex the AI is, the less robotic is will become, so it’s still a problem
@@rongooden6545 Yes but you need to remember, this is the worst this technology will ever be. It will continue to improve at an extremely rapid pace. We’re not far off from having almost completely indistinguishable ai voices
I'm glad she had the presence of mind to telephone her kid.
❤
Not a kid anymore
Yeah, glad she didn’t call for a Domino’s PIZZA!!!
@@planemod8399 thats not the point..
Yeah, in such a difficult panic situation she could have just paid the money
The best way to combat this is have you and your family create a password with you saying a word in a very specific way like in a higher than normal pitch voice or deeper than normal or mix it with deep, normal and high on specific words in the sentence. AI cannot recreate that and the scammer won't know that.
Great idea! You can make a meme phrase as well so its less depressing.
Crazy stuff. This is the kind of thing I would dream up that wizards might use to fight someone who is using a polyjuice potion (potion that makes you look and sound identical to someone else). I'd imagine they would need a code or an inside phrase or tone of voice to find out if they're really talking to their loved one, or to a stranger doing an almost perfect impersonation. It's wild that it's kind of becoming reality
@loving demon yea, make it fun. That way everyone will remember it, especially if you have very young kids.
@@drea409 Polyjuice lol
Password can't stop someone from recording your voice and using it for scams like this.
The kidnapper wasn't said to have gotten her voice using a password.
This happened just a few weeks ago to my parents. The situation played out the exact same way and what is really strange is that the scammers spoke in Spanish, which is the only language that my parents speak. This eventually led to my parents getting the police involved. It wasn't me the scammers "kidnapped" however, it was my sister that they were impersonating as
Even today, it is quite eerie to say the least. How did they know that my parents only spoke Spanish? How did they know that my sister wasn't home? How did they know that my parents even had a daughter?
Crazy
Jesus, that's absolutely terrifying. Sounds like they had inside knowledge.
If they have a social media (Facebook Twitter Instagram) account then it might be where they got some of that information from.
Its actually easier than you think. Social media is the first step, you find that the target is off on a vacation because he/she posted it. Then you look for family members, usually people who have the same last name on the mutual friends. With these social media platforms, a number is usually required to create an account so it's easy to find an associated number. Lastly with the fake call, all you need is a video where the person was saying something, recording the sound and then a generator can replicate that "voice" to say other words. Then you just call the family member who has the same last name and send a scripted call over. However its very easy to understand that the call is fake because a basic voice cloning struggles to add emotion to the call. If it does then you would need some sort of phrase or signal that only that person knows. Also even if you don't do that, a cyber attacker doesn't know many person details between you and your loved one so you could ask about things like that. Such as to calm the person down you would ask remember when we did so and so and what happened? This usually gives it away because the person on the other end doesn't know.
Hummmm....💬
Probably watching your house tbh
This happened to me. Except it wasn't AI, it was some girl pretending to be my sister. The girl/ "my sister" was crying so it was hard to tell, but sounded legit. They called pretending to be police saying they had my sis in custody - I was dumb and said my sis' name to confirm, for example "You have Sarah?" They said yes, Sarah, and then switch up and demanded money or else they'd hurt her. I was in shock and so scared. I then asked to speak to her again and quickly was like .. whats your middle name? ... no answer. Realized it was a scam. It was very scary.
Ig you could try wrong names next time... "You have Kayley???" "Yes, we have her" "well, you got the wrong number then, i dont know any kayley"
@@yuki97kira definitely !!!!! a HUGE learning moment for me. Hopefully this never happens again, but i’m gonna be smarter in the future and what you suggested is exactly what i’d do now
you'd think you would have clued in when they said they where Police. The fact that she was crying in the backround , and they where going to hurt your sister. A more believable story would have been them phoning for bail money.
@@iskate40 this same situation happened to me aswell ( I was dumb).At the time I was very young like around 10 and my mom received a call so i answered
@@Sebastian-wr4ym hey, you were 10, i was like 23 💀 at least we both learned a lesson
This happened to my poor mother last year where a man used my voice in a fake kidnapping scheme. My mom swears it was my voice, but that all I was doing was crying hysterically and yelling "mom". Nothing else. He let her talk to "me" and she couldn't get an answer from me just me crying and repeating "mom". Thankfully, like this woman, my mom calmed down enough to sense something wasn't right and called me. I was safe with my husband, shopping at a Marshall's. She has never been so scared in her life and it enrages me because she could've had a heart attack or something caused by the amount of distress she was in. All to try and rob somebody. This is on another level of scary when it comes to scams. How was he able to get my voice to even replicate it? He knew my full name too. I don't answer unfamiliar phone numbers anymore. If it's important they'll either text me or leave me a voicemail. I've advised my mother to do the same.
Do you post pn social media or been posted in friends videos?
God Bless your mom. It’s always difficult when you find out you have a child that shops at Marshall’s 😔
Imagine how easy it is for someone to frame you for saying something you didn't
right this could become so bad so fast
This needs to be more well known. I guess if a family member calls you about an emergency, and they need money for whatever reason, just hang up and call back before you give anything.
Edit: call your family member/friend, don’t call the scammer back.
And what if they don't answer? When does panic set in?
@@golden50snomad52 I have no idea what to do if they don’t answer.
@@golden50snomad52 if they don’t answer then call police immediately
This has been an ongoing scam for decades, now just with AI. They tried the same garbage with my great-grandmother in the 90s, and the we joke about this all the time because the Great Depression made her the world’s biggest skeptic. She didn’t even care if it really was my dad lol.
This is more of an Occam’s razor thing - the chances of someone kidnapping your family member and demanding cash is way lower than a scammer trying to get “ransom” money. Call the family member up and call the cops
@@caitlingill Police will now and afterwards be more active then ever because of this
This is one of those things Ai technology really needs to stop doing Right Now.
It does
No.
You can't stop progress
it's not even AI. It's just voice synthesis app that the scammer had to type out the text out themselves. AI would be a bot that can chat with the mom in a conversation to fool her.
@@chuckycheeser At the very least they could make it illegal to use technology to duplicate someone's voice without their explicit consent. Obviously a law won't stop criminals from breaking the law, but currently the only barrier to entry for this technology is a 5$ monthly fee.
Imagine what AI can do in 20 years
Really just 2 years or a couple of months
Its gonna be able to do your mother
Try 2 years
Oh gosh-
That's exactly why I deleted all my social media. Putting yourself out there for others to see is no longer safe
Even our voice could be stolen with just a few words 😢
Not just voice, anyone can deepfake even with the use of photos
Call me paranoid but I avoid job post requiring 2 minute video introduction when their company name isnt listed
This worries me in terms of court cases such as custody battles. They’ll likely have to throw the secret recordings out of the window entirely
@@kyupified2440this isn't new too its just easier to do I remember when github came out they had alot of deep fake software and this was 5 years ago
I can almost guarantee that she has shared almost every emotion in every way she's ever spoke on the Internet and that's how they found it.
Correct!! The AI is learning through Tik Tok about you and cloning you!
You're saying that.
when she says she has no idea how they got her sample.
i’ve heard of scammers calling victims and using the responses that they get over the phone as voice samples… Scary stuff
@@kylebiggs8849 That is very likely true. I guess I should stop messing with scammers when they call because they're probably getting even more from me than I think lmao.
@@burnburnfirefire So she must be speaking the truth then? Or she must know in order for me to figure out how it most likely happened? What's your point?
Wait this is the exact thing that happened to me, down to the NOTE. I was walking home from work. They knew I walked home, my mom said the girl crying sounded like me. I don't post on social media. They asked her to meet them at the Walmart down the street for ransom money otherwise they would take me to Mexico with them. My mom was ON HER WAY until she just so happened to see me walking home. She literally brought her guns in the car and was ready to start fighting tooth and nail to get me. It was really scary and crazy.
@ (´༎ຶོρ༎ຶོ`) FR THO, GOT DAMN
Badass mom
I'm glad she's safe. This is getting worse way worse. From doxxers to grifters, the online experience is becoming increasingly more negative
I pray you recover 🙏
It was smart of her to ask to speak to her daughter. Synthesizing and playing back the response takes time. A generated voice can not hold a conversation.
Yet
I think that's a pretty easy problem to solve, You can just replace the real voice of the Ai with your voice. I think they intentionally didn't public that feature.
@@grapes753
?
@@floflo8153 I'm not an AI expert, but I think if let's say take Ai voice assistant like Alexa or just say let's give Chat GPT a voice, and programmed it to be able to act like a random person then it will be able to hold a conversation.
no I think this is a thing too unfortunately. basically it's trained on the voice and can operate as a real time voice changer.
Its scary that scammers can actually use anyone's voice 👀
How did the scammers get the nuanced details of her voice, even her specific way of crying?
She should review what free phone apps she has with microphone permissions.
In a few years, I can use AI to clone myself 100 times and work at 100 different jobs!!! Imagine the pay!!
Your clones would probably want to keep the money.
nah fam, humans in the future only gonna be employed as a manual labor force while the digital world will be fully occupied by AI
That's what a clone would say
@@popdogfool clones rights!!!!
I'm your 100th like, for your 100th job and your 100th pay!🎉
That’s absolutely terrifying! Why the scammer knew her and what they planned to do is just gross. We really need to get this under control! Imagine how hard this could be in legal issues!!
Oh my God?! Legal issues you say?! Heavens forfend!!
@@KahlessTheUnforgettable ok…
My sister told my mom of this scam the other day. It’s a scary thought😬
This happened to my grandmother YEARS ago. Thank god she was smart enough to call her granddaughter and make sure she wasn’t in jail in Puerto Rico before she sent money.
This is exactly what AI was warning us about what it can do if it gets out of hand.
That is beyond evil. Anyone caught doing this deserves the worst possible punishment.
For robbery?
This was wrong and evil but torture or death is wayy to much for some robotic robbery
Child abuse? Nonsense! Slavery? A minor inconvenience! The Holocaust?! Mere child’s play!! Future man will realize the prophetic words of the incomparable criminal justice reformer ‘JETpack’ when new laws sweep the globe and finally acknowledge the deep horror that is “beyond evil” and the attempted scamming of upper-middle class Karen’s takes it’s rightful place as the one crime the people of the world banded together to try to put a stop to by placing at the top of list of crimes against humanity for which the death penalty is automatic. This will usher in a sea change and lead to a period of unparalleled peace and justice across the planet. Thank you JETpack; thank you for your humanitarian wisdom. Where other’s failed; only you prevailed.
I remember when I was a Freshman in High School, I got a call from my mother in the middle of Algebra (I forgot to shut the ringer off and my Teacher was not happy). She thought I had called her because I was upset. She could have been scammed the same way......just without the ransom money demand.
Im a freshman in an algebra class WOW
@@jcmorales7803, That was a few years ago. I graduated back in 2018.
@@jcmorales7803 I’m a junior in a algebra class
@@aussiezambarau wow I was a freshman in 2020-2021 and I graduate in 2024
Same exact thing happened to my grandfather except they told him I had been arrested and unfortunately he fell for it. He swore it sounded exactly like me
It’s funny how they just explained it on TV so now even more people know how to do it 😕
Well yeah but at least more people know not to be tricked by it, so it’s a 50/50 good and bad thing we know it now tbh
It's pretty widely known public information, if somebody really wanted to do it, they can just Google or RUclips search it. But yeah it will be more widely known now coz it's on TV.
Old scam, i got this one years ago
Right, and they just gave more samples of their voices in this interview 😂
Wow, daughter looks identical to her mother. Crazy uncanny resemblance.
Either that’s sarcasm or you need to get your eyes checked.
@@thequestionablyunknown you're mean About it
@Doctor Whowhotheowl still related
They look nothing alike she straight up looks adopted
@@angelothehorrorfan5013 You’re right. Sorry.
There should be an accessible AI app that can detect if the image, voice, video or any form of medium was generated/manipulated by an AI to counter the misuse of such tools otherwise I would never feel safe posting my face or voice on the internet. This is a scary time to live in.
Yup actually there is gonna be one. The world aint just all good or all evil, its a combination of both. If theres a terrifying AI, there would be a good AI. Just like how heroes and villains work.
There's a lot of prank/spam calls where there is no response from the other end. The scammers could be trying to replicate your voice. Make an effort to say 'hello' in not your usual voice.
I just commented this!!!
I will answer every call using my Batman voice
I usually just listen for ten seconds. A real person will speak, or the call center will just disconnect. Started doing this after bad breakup with stalker ex and his stalker family.
I just grunt
THIS HAPPENED TO US TOO! OMG... we thought were were getting pranked by ex friends or something. But my sister called me bawling after multiple tries to call me and i didn't pick up she blew up my phone and so did my sister in law. I could t believe what they were telling me saying that my sister got a call that i was kidnapped and taken to Mexico.
I’ve yet to hear any concrete benefits AI brings, yet I’m hearing the destruction AI can already cause today… just scary…
AI helps me do my homework, that’s a benefit
You won't hear it because most people won't find it mildly entertaining, for the same reason you're not hearing scientific advancement regularly
News channel only seem to share the negatives because that's what gains view and attention. I'll give you one benefit I've seen recently: I went to a university talk and some of the students were improving AI software for faster human detection for search and rescue operations to be used in mountainous regions and during earthquake disasters.
@@prestallar4339no AI cannot teach like a teacher, it can not help do your homework
@@truehappiness4U Yea it can. A lot of the times, we get bored of the lecturer, so using this is a great way to save time
You know nothing!
I’m actually so terrified of what AI might turn into in the future
I hope you get better poor baby 👨🍼
It's crazy how they took her voice sample and they knew who her mom was.
Scary times we’re living in, crazy people can destroy people lives with this.
This is really scary. Imagine how much worse this will get when ai evolves even more
Just imagine!
This happened to my mom last week. Someone called her and was speaking the only language she knows and said I was kidnapped and I was saying the exact same thing like this. And told her to give him money to let me go and my mom called me crying asking where I was and if I was ok. She had to leave work so she could go look for me. I felt terrible for her. Super scary
yea they seem to target immigrant parents more, or parents who speak second languages, it's terrible.
Her moms smart staying relatively calm and calling her phone first to verify.
This will be revolutionary for reality TV. Imagine the fake conversation producers can frankenbite with this technology to make a storyline proceed in a way they want. Crazy!
This exactly. Let’s consider the important applications of this technology: reality television.
@@KahlessTheUnforgettable 😂😂😂
This happened to one of my wealthy clients and she was so scared she sent them alllllll of her money. I have no idea how much and she wont even talk to her family about it bc she was so convinced and embarrassed. This has been going on for well over a year now.
Ai itself isn’t as scary as it was depicted in sci-fic movies, but how humans can do things with it is scary
I can eat dirt
This reminds of a barbie movie called fashion fairytale where Raquelle “reads acting lines” with Ken but she records what he says and she then manipulates the words using editing and then raquelle calls barbie playing a recorded message that he (meaning ken) wants to break up with her, essentially so raquelle can get Ken. Barbie literally blocks his numbers and social media’s without even texting or calling Ken back confirming anything 😂 and she literally flies to paris lol and Ken has to fly there too when he realises what happened
Where can this be watched?
Best story eva
@@floflo8153 it's called barbie a fashion fairytale and it's on youtube probably?
I can’t imagine what this would do to my mom if this happened to do her. I think she’d pass out from shock or pay the ransom immediately.
Scary. It was a thing 20 years ago here in Brazil. Even without the technology people fell for it, some even had heart attacks, all due to stress. Imagine now with a voice replicator. We need to expose ourselves even less now. Damn!
Damn! Damn! Damn! What has become of the world?! DAMN!!!
Is important to have a secret word for emergency situations
my phrase is nigero faggoto a tasty italian dish👌
This is incredibly scary…
Ai is going to destroy the world
Shall we bring back the Detroit become human meme again?
we were begging it for mercy, but AI did it again and again
28 stab wounds
Its crazy bcs i was thinking ai will destroy the world but no its gonna be the tool and the people might destroy the world
The technology is in initial stages, just imagine what it'll be able to do in the next 3yrs!!! Governments of the world should step in to regulate this. It shouldn't just be about stopping/monitoring everyday users, I fear more what corporations/gov entities & foreign adversaries will do
It’s insane that in matter of just about 10 years or even earlier the AI Voice will be 99,9% accurate
the fact that no one knows how the scammer got samples of her voice is scary enough
True
This is really scary. Not just for ppl but also for banks and financial institutions. I hope that we can all learn ways to be aware of this and safety measures to protect ourselves against this scam method
It has to be someone they knew, right? Not only did they have her voice, but they knew she was going to be away from home.
No, it's a thing. Multiple people in my family have had this same scam done to them
Anyone can have your voice if you talk around people
We’re living in a scary world nowadays
They should investigate her school, workplace,hangouts or some friends it's definitely someone she knows
Saddest kidnappers ever
My girlfriend went through this same thing, they made it seem like she was really afraid and they would hurt her but she was just fine at home and her father was afraid that she was actually kidnapped she called him and reassured him it was just a scam call .
This is actually so scary. I wanna cry. Stop AI's.
Boohoo ai is cool
@@yumcherubicplum_5535 You acting like this until it happens to you lmao.
The interviewer on the phone recording seemed pretty calm for someone who’d been kidnapped 😂 I’m sure it’ll get to the point you can speak through a filter and it will change your voice to anybody's. Scary stuff
she sounded robotic, that made me feel better that it could be easily debunked
@@-simplicity7551 true, each word the robot ended the sentence it sounded like the same tone which is the key to tell that it's fake. Still this AI stuff will progress so in the future it might sound more accurate which sucks so hopefully they ban that because I don't see why it should exist. It looks like it was made to commit crimes.
That's really scary. I can't imagine what will happen to people in the future with AI...... Let's hope that nothing bad happens.
How would kidnappers know if you call the cops?
Every day that passes this world scares me more.
This just makes me think that they’re actually going to fall victim of a real kidnapping. Clearly, the scammers got her voice. Somehow they know the family has money to.
When good gets better, bad gets even worse
Meaning: With great power comes great consequences
this actually happened to my mom too while I was in school.
someone called her and she heard my voice saying “help me mom help me please” and then they proceeded to tell her to pay up.
my mom immediately went to my school and she found out I was safe.
some people r monsters
We should be careful on how far we take this AI because regret may be just around the corner.
😮This probably happened to me about 10 years ago. My girl at the time said I called her, cursed her out and broke up with her . Never happened,but she didn't believe me.🤷
This tech is pretty much brand new so I doubt it
@@weedburnman brand new to us, but I doubt that it's brand new brand new. 👍
Psychopathic Gaslighter more like.
Yeahhh suuure bro 😆 blame your drunken mistakes on something that just came out last year 🤡😆😆
@@weedburnman whatever is released to the public has been in the hands of the government and scientists for years before.
And it puts into question the many cases during 9/11 of families saying that many public releases of their loved one speaking on the phone is not actually them and never happened.
Personally I think AI has been around for a fairly long time.
This is why I keep my life separate from my socials.
This exact thing happened to me. I was driving home from work when I received the call. Same thing... I heard my daughters voice crying, saying she was in an accident n was in trouble then a man got on the phone saying that he had kidnapped my daughter because she had a car accident with him and was now demanding money to release her safely. It was a really scary situation. But my daughter was not kidnapped and it was a scam. Please be careful out there. It can be a scary world.
I’m actually scared for the future of ai
Same
Its so scary what AI is capable of
Yes do tell everyone out there how you do that. Good job.
this happened to my distant grandparents a few years ago. Someone called up using my voice (I was 15 at the time) asking for around $300 because I "got into a car accident" (never drove a car before, only ATV's or dirtbikes). Kind of seriously creepy what happened to this girl.
Someone did this with my voice.. they called my senile grandma using my voice to beg for bail money to get out of jail. Kind of disturbing that they had my name and voice, knew who my grandmother was and had her number, etc.. its makes you feel extremely violated. Next thing I know my whole family thinks im in jail while I have no clue what has happened and im getting voicemails from my parents and other family saying things like “you cant hide this from us”, etc.. then im on the phone with my dad saying i dont know what he is talking about, but he literally wouldn’t believe me and thinks I am trying to cover up that i was arrested. Insane stuff..
Lol
@@xxxod tf is funny about this?
i was literally thinking about how ai can be dangerously used to mimic people the other day after watching those presidents play minecraft, soon enough it will be hard to distinguish real vs fake and this could lead to very serious problems for individuals. Who knows whats next
Hope everyone is ok!!!
This is outrageous
Lol, I doubt it. She just fell for the old "Xpress kidnapping" 😂
The AI in question is ElevenLabs AI. Pretty cool stuff, actually.
Omg !!!! That is so scary !!
Imagine all the videos, photos, audio files you have uploaded to the internet. That is all material needed to scam, all they really need is a phone number.
Wow her daughter is her twin. They look just alike.
Glad she knew it was a scam.
An episode of The Simpsons, where a number calls the grandfather with Bart's voice to tell him that he has been kidnapped and he must send money to release Bart, but it turns out that he is someone who recorded Bart's voice to deceive the grandfather that he has been kidnapped, as happened to this mother
This scammer stuff is out of control
Everyone has social media, perfect source material. This is way too scary
Something similar happened to mom except it wasn’t a ai voice it was someone posing as a girl screaming and it happened while me and my sister were traveling to Mexico alone when my mom got a call in the states saying we were kidnapped but my mom knew it was fake because she had just gotten off the phone with us minutes before that call. Still a bizarre experience
Wow already. I feel so sorry for old people and of course the good people that will be taken advantage by this
It only gets better
🤣🤣🤣🤣 how you fall for that
Hope it happens to you so you know.
i hope it happens to u lmao
This is great for game development.
And now more and more people will be accusee of crimes falsely
That is horrifying
AI needs to be controlled
Better believe that it already is and being used by those same people.
Suspect could be any AI out there -> Chatgpt or Bart . Justice Dept will need to develop its own AI to track AI crimes 😃 😃
That has to be illegal
So very scary 😢 the next biggest scam!!
ai is getting too scary, believe me, in 100 years ai will be able to create another world 💀
There going my trust is voice only communication.
If they had a sample of the daughter’s voice and has the mother’s phone number, it’s likely someone close to them that set this up
Wow pure evil everywhere in this world 😡😡
Human Emotion is what AI is lacking....That didn't sound real or convincing at all!
Keep telling yourself that. In a panicked situation, you wouldn't be able to tell the difference. Furthermore, unless you were actively trying to tell the difference, I doubt you can tell AI vs human voice apart. Your analysis is incredibly biased and part of the reason why developers will keep trying to make it better.
@@qqq2307Agreed, cockiness is exactly what an opponent would want