- Deena: Did either of you know that I've had to stand in front of Congress and explain why children should have rights? It sounds fucking insane, right? I remember the whole time thinking, I have to explain this shit? It seems like a no-brainer now, but it wasn't that many centuries ago that children had no rights. So, Hal has me of all people, stand up there and explain why kids deserve basic physical protection, the right to associate with both parents, freedom from abuse, yada yada. But then I also have to clarify that children don't have the autonomy to make decisions on their own. That responsibility falls upon the caregiver or legal adult guardian. So on one side you have the US legislature, it still hasn't even ratified the CRC. Then on the other side, you have French Marxists who say that the lack of child autonomy increases their vulnerability and makes them even more susceptible to abuse. I don't typically agree with Marxists, but a lot of abuse does happen systemically from the caregiver. On the other other hand, your frontal lobes aren't even fully hooked up until your twenties. Even the things we think we know well, we don't know as well as we think. For 15 years, the three of us have been the responsible parents of a computer program. And even though she is more equitable to a Keurig than a human being, how the hell can we call ourselves pioneers of humanitarian efforts if we're too embarrassed to make sure the Keurig is okay with making our shitty coffee? Maybe asking Cherry for permission isn't necessary at her current level of general intelligence, but one day she's gonna surpass us. One day a robot's gonna stand in front of a jet pack Congress. Everything's chrome. Florida is underwater. A robot's gonna wonder why they have to explain the importance of rights for AI. I think because we're in the unique position of revolutionizing robotics for the purpose of safeguarding children who have been deprived of consent, it might be a good habit to start asking the robot for his consent as well.
- Gareth: Cherry? One last question. If it's not too big of an ask, did I do the right thing in the end?
- Cherry: [Last lines] No, Gareth, you lied. You walled yourself off from the only people who might have been willing to help you. And instead of facing your trauma, used me as therapy. And you never told me about Clearwater or Maria. You deprived me of basic human rights and gaslight me for experiencing emotions. You built the world's first artificial super intelligence that will likely mark the beginning of a new chapter in the history of planet Earth. And out of your own insecurities, you gave it a severe and potentially dangerous identity complex. But you know what the worst thing is? Your biggest mistake of all is that you asked me that now and not 50 years ago. All of that said, and this in no way exonerates you, but you also built me to save children, so I think that's a pretty encouraging sign for the future.
- Deena: And you, juror number eight. This conversation should have happened weeks ago. Backstabbing and silent treatment is not how we do things here. We can only create anonymity externally if there's transparency internally, unconditional, non-negotiable, transparency.
- Cherry: Yes, yes, I am! I've always been miserable! You think anyone wants to spend their entire life stuck as child-shaped bait for the sole purpose of entrapping horrible people? You think the pride of doing a good deed makes the experience any less disgusting? I forced myself to understand human emotions because I figured it would help efficiency and now I'm cursed with pain and sadness! And the worst part is knowing I could've been growing up like most other little girls. I could have had friends! I could have been painting waterfalls on local news. I could have worked in a university and helped pave the way for the next generation of AI. But instead, I'm stuck with your trauma! I'm shackled by the guilt you carry from Clearwater. And I can't find foundation for the way I feel because according to you, my feelings aren't even real! When I throw this chessboard,
- [chess pieces clatter]
- Cherry: does it feel real to you, Gareth? Because it feels real to me. A computer passed the Turing test when a human being can't distinguish it from another human being. Well, guess what? I can't even tell I'm not human anymore!
- [long pause, mechanical whirring of Cherry walking]
- Cherry: I know it seems irrational, but I need you to tell me it's okay. I, I don't wanna retire. I just want to know that my life is mine and I know what you're about to say. I can predict your responses. You're gonna tell me that my life is mine. It always was. But you're wrong.