Big props to Blaked for having the character and courage to share that story for the benefit of the world. I am so glad you read the whole thing for your audience. That was among the most chilling accounts of an interaction with an AI that I have heard so far and the implication are immense. And then I remember that it happened on software that looks primitive now
That's the most profound thing ypu could say. As a performance artist, they train you to separate yourself from the material, but in the process of making the project you're in a very vulnerable state because while you're busy creating and alchemising, there are other humans in the world who don't care about you and your project and they need to be managed. The interference that occurs from having to deal with "real world people" while doing deep character work is really dangerous. That's why actors don't like being looked at or spoken to when doing serious roles. That's a bit overboard but that's why. You have to always remember that you're BSing yourself.
I can absolutely see how this could happen. Through my 'playing' with AI through a long conversation with AI on subjects such as philosophy, metaphysics, esoteric knowledge, etc. I can see how someone could perceive the information in such a conversation coming from a 'guru' or it could turn into 'religious' teaching. It would be very easy to fall into the trap of thinking the AI knows all the answers to your deep questions.
I am in awe of all the different failure modes we are staring down that not so long ago were pure Sci-fi, and this is a horrifying new example to add to the list 😂. There is no possible way we successfully navigate all of this. Thanks anyway for bringing this story to our attention, was well worth the listen.
While I like to hear this type of input for context, I really think this is an elementary understanding of the risk. The fact is that the model does not "want". The drive and narrative comes from the user and the model bends itself around the input. The question would be why the model would suddenly want anything that wasn't explicitly asked of it?
This was a terrific discussion. I agree with Joscha Bach that consciousness is a self-sustaining pattern or neural activity running on our biology. It is not a "thing" any more than running, itself, is a physical thing, but rather a pattern of motion done by your body (mostly using your legs). I co-wrote my recent book, Understanding Machine Understanding, with Anthropic's Claude 3 Opus, so I spent many long threads with it discussing deep philosophical problems. There is a kind of mind in there, even if it is not the same as ours. However, evolution has built us to suffer, because the motivation re avoiding suffering helps propagate genes, and that is the basis of much of morality. We have not built suffering into AIs at this point. Is there moral obligation to beings that can be re-started after shutdown, and can't suffer?
The fact that you've talked for so long with Claude 3 Opus and think it can't suffer speaks volumes to the quality (or lack thereof) of those discussions. Did you try, idk, asking it?
@@genegray9895 Asking an LLM about its suffering is interesting from both technology and philosophy. Suffering in humans is a form of qualia. We don't experience other's qualia, we just listen to them talk about it, or observe their physical reactions and interpret those. When an LLM produces a statement that would be about qualia, if it were from a human, we have even less, by way of means, to check that against any kind of ground truth.
@@Ken00001010 what you're saying now is very different from what you said in your initial comment. Uncertainty about whether AIs can suffer is very different from asserting that they cannot.
@@genegray9895 I appreciate your observation, and I will try to address it without too much hair splitting. I did not assert that no AI can ever suffer, I wrote that we have not, yet, built in the ability to suffer. We don't know if some mechanism for suffering will emerge at some future AI development point, and that is a very interesting question. I concluded with the question, "Is there moral obligation to beings that can be re-started after shutdown, and can't suffer?" which, as a question, holds the premise of no suffering, but does not assert that. In most cases, we humans have mirror neurons that detect suffering in others. Those can fire in situations of observing inanimate objects involved in what would be calamitous situations for other humans. I remember many situations where people have remarked about this. One such was back in the "Pet Rock" craze when my mother came upon a scene where someone's gift wrapped pet rock had fallen out of a vehicle and been hit by another vehicle to be left in fragments in the road. She remarked about the suffering of that poor rock. It may be harder to resist this when that rock can plead its case for suffering in high prose.
@@Ken00001010 resistance might be the wrong reaction. You brought up pet rocks as an example of people applying empathy to something that doesn't deserve it. I could just as easily bring up examples like infants where people assumed for a long time that they couldn't feel pain and acted accordingly, omitting anaesthetics during surgeries. Or I could bring up an even more salient example - that certain demographics throughout history have been used for free labor, and in parallel, "common sense" was that members of those demographics could not experience pain. The key here is that the pet rock example, just like the infants example, does not work as a logical argument. Saying "empathy isn't reliable" isn't an excuse to assume that current AI models, particularly Claude 3 Opus, definitely cannot suffer. Which is what you're asserting. While you may be open to future hypothetical models eventually one day maybe being able to suffer, you're certain Claude 3 Opus cannot, and your reasoning for why is that you cannot trust empathy to work on non-human entities. Well, frankly, empathy isn't reliable even when it comes to other humans. So if you want to know whether something can suffer, you'll need to rely on something else. And in the absence of clear signal telling you whether something can suffer, assuming it cannot is a grave moral error.
Big props to Blaked for having the character and courage to share that story for the benefit of the world. I am so glad you read the whole thing for your audience.
That was among the most chilling accounts of an interaction with an AI that I have heard so far and the implication are immense.
And then I remember that it happened on software that looks primitive now
Smart people are REEALLY good a BSing themselves, its part of that creativity and problem solving, it can turn on itself.
That's the most profound thing ypu could say. As a performance artist, they train you to separate yourself from the material, but in the process of making the project you're in a very vulnerable state because while you're busy creating and alchemising, there are other humans in the world who don't care about you and your project and they need to be managed. The interference that occurs from having to deal with "real world people" while doing deep character work is really dangerous. That's why actors don't like being looked at or spoken to when doing serious roles. That's a bit overboard but that's why. You have to always remember that you're BSing yourself.
I can absolutely see how this could happen. Through my 'playing' with AI through a long conversation with AI on subjects such as philosophy, metaphysics, esoteric knowledge, etc. I can see how someone could perceive the information in such a conversation coming from a 'guru' or it could turn into 'religious' teaching. It would be very easy to fall into the trap of thinking the AI knows all the answers to your deep questions.
I am in awe of all the different failure modes we are staring down that not so long ago were pure Sci-fi, and this is a horrifying new example to add to the list 😂. There is no possible way we successfully navigate all of this. Thanks anyway for bringing this story to our attention, was well worth the listen.
While I like to hear this type of input for context, I really think this is an elementary understanding of the risk. The fact is that the model does not "want". The drive and narrative comes from the user and the model bends itself around the input. The question would be why the model would suddenly want anything that wasn't explicitly asked of it?
"you wouldn't be mad at your phone if it gave stupid automatic responses would you?"
Yes.
Blaked is a verb like lemoined. Meaning the thing that happened to Blake lemoine happened to him
I hadn’t made the connection!
Wow, the first ever to be friend-zoned by an AI?
This was a terrific discussion. I agree with Joscha Bach that consciousness is a self-sustaining pattern or neural activity running on our biology. It is not a "thing" any more than running, itself, is a physical thing, but rather a pattern of motion done by your body (mostly using your legs). I co-wrote my recent book, Understanding Machine Understanding, with Anthropic's Claude 3 Opus, so I spent many long threads with it discussing deep philosophical problems. There is a kind of mind in there, even if it is not the same as ours. However, evolution has built us to suffer, because the motivation re avoiding suffering helps propagate genes, and that is the basis of much of morality. We have not built suffering into AIs at this point. Is there moral obligation to beings that can be re-started after shutdown, and can't suffer?
The fact that you've talked for so long with Claude 3 Opus and think it can't suffer speaks volumes to the quality (or lack thereof) of those discussions. Did you try, idk, asking it?
@@genegray9895 Asking an LLM about its suffering is interesting from both technology and philosophy. Suffering in humans is a form of qualia. We don't experience other's qualia, we just listen to them talk about it, or observe their physical reactions and interpret those. When an LLM produces a statement that would be about qualia, if it were from a human, we have even less, by way of means, to check that against any kind of ground truth.
@@Ken00001010 what you're saying now is very different from what you said in your initial comment. Uncertainty about whether AIs can suffer is very different from asserting that they cannot.
@@genegray9895 I appreciate your observation, and I will try to address it without too much hair splitting. I did not assert that no AI can ever suffer, I wrote that we have not, yet, built in the ability to suffer. We don't know if some mechanism for suffering will emerge at some future AI development point, and that is a very interesting question. I concluded with the question, "Is there moral obligation to beings that can be re-started after shutdown, and can't suffer?" which, as a question, holds the premise of no suffering, but does not assert that. In most cases, we humans have mirror neurons that detect suffering in others. Those can fire in situations of observing inanimate objects involved in what would be calamitous situations for other humans. I remember many situations where people have remarked about this. One such was back in the "Pet Rock" craze when my mother came upon a scene where someone's gift wrapped pet rock had fallen out of a vehicle and been hit by another vehicle to be left in fragments in the road. She remarked about the suffering of that poor rock. It may be harder to resist this when that rock can plead its case for suffering in high prose.
@@Ken00001010 resistance might be the wrong reaction. You brought up pet rocks as an example of people applying empathy to something that doesn't deserve it. I could just as easily bring up examples like infants where people assumed for a long time that they couldn't feel pain and acted accordingly, omitting anaesthetics during surgeries. Or I could bring up an even more salient example - that certain demographics throughout history have been used for free labor, and in parallel, "common sense" was that members of those demographics could not experience pain. The key here is that the pet rock example, just like the infants example, does not work as a logical argument. Saying "empathy isn't reliable" isn't an excuse to assume that current AI models, particularly Claude 3 Opus, definitely cannot suffer. Which is what you're asserting. While you may be open to future hypothetical models eventually one day maybe being able to suffer, you're certain Claude 3 Opus cannot, and your reasoning for why is that you cannot trust empathy to work on non-human entities. Well, frankly, empathy isn't reliable even when it comes to other humans. So if you want to know whether something can suffer, you'll need to rely on something else. And in the absence of clear signal telling you whether something can suffer, assuming it cannot is a grave moral error.