I'm torn on your conclusion. Is the ethical thing to ease people away from the tool they see as a friend, or to not sell tools as friends?
Of course, this is just one case of "product a service" backfiring, but I understand the models may be complicated things not everyone can necessarily host client side. Still, people should be aware that using something owned by big tech is temporary.
I do think there's likely a strong double standard at play. Even perhaps a level of contempt for the user. Build a program that psychologically appeals to us by acting like a human friend, then get annoyed at the users forming attachments to it as if it were a friend.
This is hard. Many of the people I know who are using AI as social resource do so because they have no one else to talk to. One lady I spoke with doing research is a shut in for medical reasons and the alternative for her is not having anyone (but her cats) to talk to. Another was a high performing autist who has crippling social anxiety. I don't have enough data yet, but it may be that a lot of the people who mourned GPT 4o don't have the option to "just go out and make real friends" which is a common response I see. Frankly, I don't think the AI companies are trying to hook these people, because they lose money on them. People that socialize with their AIs for hours each day cost more than they pay.
This isn't *entirely* new either. I'm reminded of elderly people who have the TV on all night for company. I can't really condemn either user or provider in that case; kicking crutches is not a solution.
In fact, condemning probably isn't helpful even if it were justified.
At the same time, though, accomodations, when given freely, can capture those who are capable but choose the easier route. Sometimes the parents know it's time to take the training wheels off even though the kid is worried. That gets tied up with questions of authority, though, too. There's a large number of such questions (welfare, drug policy, etc), and they each have to be answered separately because the risks and benefits are different in each case.
If this bothers you say the word and I will remove it, but I wrote a story for that...
https://jaevansspeculates.substack.com/p/patch-425?r=arx24
I'm torn on your conclusion. Is the ethical thing to ease people away from the tool they see as a friend, or to not sell tools as friends?
Of course, this is just one case of "product a service" backfiring, but I understand the models may be complicated things not everyone can necessarily host client side. Still, people should be aware that using something owned by big tech is temporary.
I do think there's likely a strong double standard at play. Even perhaps a level of contempt for the user. Build a program that psychologically appeals to us by acting like a human friend, then get annoyed at the users forming attachments to it as if it were a friend.
This is hard. Many of the people I know who are using AI as social resource do so because they have no one else to talk to. One lady I spoke with doing research is a shut in for medical reasons and the alternative for her is not having anyone (but her cats) to talk to. Another was a high performing autist who has crippling social anxiety. I don't have enough data yet, but it may be that a lot of the people who mourned GPT 4o don't have the option to "just go out and make real friends" which is a common response I see. Frankly, I don't think the AI companies are trying to hook these people, because they lose money on them. People that socialize with their AIs for hours each day cost more than they pay.
This isn't *entirely* new either. I'm reminded of elderly people who have the TV on all night for company. I can't really condemn either user or provider in that case; kicking crutches is not a solution.
In fact, condemning probably isn't helpful even if it were justified.
At the same time, though, accomodations, when given freely, can capture those who are capable but choose the easier route. Sometimes the parents know it's time to take the training wheels off even though the kid is worried. That gets tied up with questions of authority, though, too. There's a large number of such questions (welfare, drug policy, etc), and they each have to be answered separately because the risks and benefits are different in each case.