This reminded me a little of a post I saw on Twitter during the covid lockdown, when so many people were furloughed and confined to their homes- "It's lockdown. You'll never have so much time again. If you don't write that novel now, accept that you'll never write it." OK advice to a single person on furlough maybe. Meanwhile I and many people I knew were trying to manage full time work from home + home-schooling multiple children + additional domestic tasks (more meals to cook, more mess, lengthy queues to buy groceries etc). I'd never had less time for novel-writing.
It is incredibly easy to generalise from one's own experience and confidently advise others on that basis. At least when another human is doing it, it's easier to spot. Baking this into a technology that then presents itself as being personalised, adaptive and completely authoritative is likely to be disastrous.
This is merely an extension of what is happening with 'diagnosis' and 'treatment' in general, in the field of mental and emotional distress. Intuition, insight and adaptation is suspended, in favour of checking an ever more general collection of 'symptoms' against tick-boxes, and following the flow chart to see what to do next. Mental health manuals are merely based on the consensus in the room at the time, with little input from either hard science or philosophy. In a recent survey to see how one professional body's ethical framework could be improved, counsellors allegedly said they wanted to be told what to do in specific situations – almost 'if they say this, you say this'. Context and individuality, let alone life circumstances, play very little role in professional judgement now, and the tramlines of 'normal' get narrower and narrower. Instead, the person – often, tragically, a child – is told they have a 'disorder'. They may then be medicated. It is profoundly inhuman, deeply unethical and one of the primary reasons why I left school counselling.
All counselling seems like a scam. I have met intelligent psychiatrists, but most Ph'd therapists seem closer to which doctors than anything even remotely scientific. Parish priests can usually do a better job, and they tend to be less judgmental. I have read about how groups of these monsters were upset that long-term mental health facilities were not perfect, so they lobbied hard and had all of the patients thrown into the streets creating the American homeless problem. That problem was a choice made by cruel charlatans who overwhelming goal is to look impressive (instead of say, impressively helping people).
Few groups of fake "professionals" anger me more than these evil crackpots. They do so much damage, and they are both oblivious to it and imagine that they are helping. I expect that in the not-too-distant future psychology will be viewed with the same perspective as bloodletting.
Why would anyone use such a tool when you can go to church and read the Bible? Humans are incapable of meeting the infinity and infinite complexity of human need and pain.
Sometimes going to church and reading the Bible aren't enough. Of course, *God* is enough. But even the most spiritually grounded human is still human, and we encounter God and His grace through other people. Your comment betrays that you've likely never dealt with true trauma or mental illness. How blessed you are!
The rest of us, however, often need other human beings to help unravel complicated emotions and injury in order to more fully bridge that gap. Again, God often uses other people to communicate His grace.
And sometimes in-person counseling is too daunting, expensive, or difficult to schedule. I can see the appeal of the accessibility and relatively low expense of such a tool. Be careful not to judge others in their need.
I'm simply saying that I don't believe a set of algorithms can ever do the job. You are mistaken in saying I've never faced trauma. I'm facing it right now. But therapists say utterly daft things like "your children don't ask to be born" . Not one of them is as useful as talking to a close friend. Or in my case our vicar during a Lent prayer course. The best therapist I've come across was more like super common sense friend. So why not save the money and talk to actual friends. And God.
JaneH, you said "Humans are incapable of meeting the infinity and infinite complexity of human need and pain." You implied there's not a need for the human connection when you said to just "go to church and read the Bible." Now you're saying you encountered healing through your vicar and friends. I think you're proving MY point! 😉
I am sorry for the trauma you've endured. I pray God has allowed it to deepen and strengthen your faith. But, again, I would speak caution to the idea of judging others who need more than conversations with friends or a good pastor to connect them to healing. Sometimes knowing a thing doesn't correspond with our subconscious and physiological reactions and habits.
We should be careful to note, though, that I (Lucy) wasn't actually intending to use this product for healing. It was advertised as a "productivity tool" which takes "work /life balance" into consideration. I was just trying to figure out a better way of working, as in, earning money for my family. It was NOT advertised as a "robot counsellor" (Brit Eng?) or maybe the term in Am Eng would be " robot therapist."
Yes, noted. I was originally responding above to JaneH who seemed to imply other ends. And there are plenty of AI "counselor" tools available. Many appear to reach into productivity territory while implying they address mental health hurdles too. Is it ADHD, anxiety, hormones, or disorganization? Does AI care? Does a human counselor? Is faith enough to bridge the gap?
Mary's meme points to the deeper universal issue, for sure. We all bear some responsibility for this problem. And also, it was very disconcerting for me to reach this conclusion: a computer program was designed (by/for Christians) to mechanically "heal" people on that deeper nonmaterial level. It was subsequently sold to the general public. People who buy a "productivity" product are not choosing to consent to spiritual healing if they are taking the product's marketing materials at face value. That's the part that Jane is missing. Bait and switch. Mislabeling. I'm not saying that mechanistic healing is even possible! I really do wonder what the creators/peddlers of this product were/are thinking.
And, Julie, I will pose the question, "Am I (Lucy) an outlier? Am I (the duped one) especially mentally ill or traumatized, more than most other people?" I suspect that this is actually not the case. It seems to me that "tech" has the potential to amplify the more chaotic or destructive impulses within anyone—men and women alike—particularly in moments of vulnerability. All of us are looking for answers.... all too often in the wrong places.
No, I certainly don't think you're an outlier. I think we all can easily lean a little too much into this. I don't fault you at all. I used the examples of trauma and mental health as extremes (extremes I personally carry).
Thank you so much for your candor, and for being open to thoughtful discussion about tech entanglements. We are doing our best to make sense of this interesting world we're living in, and these salon-style forums really help, I think!
It's inevitable, I would think. In fact isn't it the same as the school of feminism which measures success against access to the male status metrics? Which has certainly done a number on those young women who find themselves confused when that particular dream doesn't bring them what they wanted.
This absolutely. Feminism has already taken the stance that the very best woman is a man…a man in a dress in fact, hence their febrile defense of trannies.
That most women throughout most of history have found fulfillment as mothers and wives does not seem to have gotten through to them. It’s great to have choices as to how you balance that with other desires but to eliminate it entirely is evil and dangerous.
Choices which effectively eliminate all of another human being’s life choices should be called out for what they are. The anti-mother is at least as real a feature of our times as the antichrist.
I don’t think you can let AI be counsellors because half of counselling is being in the room and emotionally remaining there when it gets hard. Some AI can follow you through challenging intellectual analysis . But they are pattern matching machines. a dog would be much better.
We should be careful to note, though, that it wasn't actually marketed as a "robot counsellor." It was advertised as a "productively tool" which was supposed to take into consideration "life/work balance." Sounds good, right?! In practice, for me, it became something of an all-consuming authoritarian laptop nightmare.
In hindsight, I'd say leave that "productivity" rubbish to the people who don't really have lives, as in "work/life balance." Apparently, life is actually far too big to be contained by our puny human models, ha!
Work/life balance has always been more of a marketing tool. Productivity is real, unlike the pseudo-science behind "counselling" or "gender studies." I have no doubt that your perspective on this tool is both intelligent and useful, but the problem would appear to be that the software was wasting time and energy on extraneous areas/concepts instead of actually improving productivity.
Engineering productivity solutions is an essential part of improving life for everyone. I have spent most of my adult live engaging in this for my career, and the results are unambiguously good. Now, lots of snake oil is dressed up in terms of productivity, but that does not lessen the glory of the real thing. I will admit i was probably triggered by your comment ("puny models" can hurt someone who spends his life building models), but I will die on this hill. We are all better off to live in a world where 95% of us need not toil in primitive agriculture and industry.
This product was falsely advertised as a productivity tool. To criticize productivity itself would put you in the camp of Pol Pot and the Unabomber.
Well, since I've already been baring my soul on the internet (under a pseudonym), I might as well admit that I'm something of a "religious nut," or at least firmly outside of the materialist camp. So, that might explain my perspective, at least partly!
I cannot imagine that the AI could be any worse than the mystical alchemy counsellors promote in their marketing. At least the AI can spot patterns and might even possess useful knowledge. Pattern matching machines are what make modern life possible. Counselling is the kind of fake nonsense forced upon the powerless (and the clueless) in order to strip away individual creativity and enforce great compliance and conformity.
It is disheartening that reasonably intelligent people ever imagined in the first place that such technology is grounded in anything like the personal reality human beings need to thrive in, that they would willingly participate in this massive technological experiment as if it were the natural next step forward, like a new iPhone or a new app...
This type of interface and application is unfortunately already playing into the perceived inadequacy of personhood propogated by social media - the discomfiting temptation of self-comparison with everyone else's apparently perfect life, leading to a compulsion to adopt and defend a victimhood stance on some personal issues or other to seek support on. Once I have abdicated my personal dignity and been seduced by AI exhortations about all the things I should be doing better I am captured. I have become a slave to the machine. I will believe anything I am being told or sold. Thank goodness Lucy still has the awareness to realise that something is 'off'. It is this discernment and natural scepticism that we need to continue to cling to as if our lives depend on it. Because they do.
AI can be your research assistant but never your coach, therapist, or friend.
That's a line I use regularly in my homeschool seminars on technology. But LLM's conversational nature make it very easy to slide from research assistant to therapist. It's the digital version of your secretary morphing into confidant and then into lover.
The rules I tell parents:
1) No personal information. If you wouldn't yell it from housetops, don't talk to Gemini about it either.
2) Keep it professional. Dialogues can be extended, but they should be subject or project specific.
3) No role playing (definitely for kids). Yes, it's fun to see Johnny "talking" to Albert Einstein about relativity or Bach about music or Elizabeth Bennet about English Voctorian culture... but it's a bad idea. The boundaries of reality are blurry for kids (even teens). Also, if you can "talk to Einstein" who's been dead for decades, why can't I "talk to grandma" who died last year? No role playing.
For Agentic AI in particular, these rules go double. The vast majority of people have no need for an AI agent today. Don't even go there.
Some get angry because I'm a techy and have said for nearly 15 years now that we don't have anything even closely approximating AI. What we do have that is genuine AI, unadulterated by philosophy or politics it tiny machines that are just now learning how to complete an obstacle course for the next step in unmanned space exploration. That's it. Everything else is adulterated by man forcing the "AI" to think the way they want it to think. What we think we have is basically just HAL9000 on a good day, but it can and will turn on a dime trying to stay with it's programming. That's why they "hallucinate". They can't hallucinate anymore than HAL did. They are programmes attempting to stay with the program but they are dealing with humans.
I have no doubt that AI is going to do amazing things for and to humanity. Unfortunately, overall, the changes are going to be catastrophic: loss of self reliance, self thought, introspection (even if successful entrepeneurs don't need it!) and just any capability to improve. We will become just toys.
The algorithms seem to want nothing more than to keep you engaged, feeding you their mix of regurgitated facts, hallucinations, and pandering. Unfortunately, they are often useful. But don't take them as anything more than an overly complex machine.
We all forget that any AI is programmed and trained by certain humans that have biases and agendas so they should always be checked against our own internal sense of right and won't along with advice from others. We wouldn't base our own life decisions on advice from one person so why should we do that with an AI?
...but wouldn't any platform which offered gendered offerings, granting what would look like 'slack' to the female version, be called out for misogyny?
Lucy and Mary, thank you for sharing! All of us, myself very much included, are heavily shaped by our social relationships. When I’m around the best people I know, I am better. I love the Lord more, I’m warmer, kinder, more excited to love God and love neighbor. I think we’re also shaped by the expectations and behavior of people around us. Repeated interactions with a loved one can mutually reinforce bad habits, or unhealthy patterns of communication. I’ve known these ruts where I already expect someone to fail in a certain way, I interact with them with that expectation, and then it happens, and it becomes a feedback loop. They may even think of themselves as failing in that area. It seems that the more human-like our technology is and the more socially isolated we are, the more the human-like technology can participate in this loop with us. Lucy, it sounds like your understanding of your own character was negatively molded by this AI coach. It sounds similar to the negative molding of self understanding social media causes for many, but much more sinister. It seems like it was more personal and more painful because the machine could speak more humanly and perhaps was more trusted as a result? I don’t like the idea of being so shaped by the machine and the priors of whoever programmed it. That said, the reality is that you could’ve been negatively shaped just as easily by a human mentor. Humanity is no defense against wickedness, though only humans can be redeemed by God. Perhaps the lesson here is a growing need to be conscientious of who is forming us? The machine is the example of the hour, but other humans, culture, and malevolent spiritual forces can all do this same kind of thing to us.
Makes me think of AntI Christ. C S Lewis would write a novel about this with different evil characters but the same malevolent forces - Professor Frost would be there with all the others but he would be an AI psychologist…
This reminded me a little of a post I saw on Twitter during the covid lockdown, when so many people were furloughed and confined to their homes- "It's lockdown. You'll never have so much time again. If you don't write that novel now, accept that you'll never write it." OK advice to a single person on furlough maybe. Meanwhile I and many people I knew were trying to manage full time work from home + home-schooling multiple children + additional domestic tasks (more meals to cook, more mess, lengthy queues to buy groceries etc). I'd never had less time for novel-writing.
It is incredibly easy to generalise from one's own experience and confidently advise others on that basis. At least when another human is doing it, it's easier to spot. Baking this into a technology that then presents itself as being personalised, adaptive and completely authoritative is likely to be disastrous.
This is merely an extension of what is happening with 'diagnosis' and 'treatment' in general, in the field of mental and emotional distress. Intuition, insight and adaptation is suspended, in favour of checking an ever more general collection of 'symptoms' against tick-boxes, and following the flow chart to see what to do next. Mental health manuals are merely based on the consensus in the room at the time, with little input from either hard science or philosophy. In a recent survey to see how one professional body's ethical framework could be improved, counsellors allegedly said they wanted to be told what to do in specific situations – almost 'if they say this, you say this'. Context and individuality, let alone life circumstances, play very little role in professional judgement now, and the tramlines of 'normal' get narrower and narrower. Instead, the person – often, tragically, a child – is told they have a 'disorder'. They may then be medicated. It is profoundly inhuman, deeply unethical and one of the primary reasons why I left school counselling.
All counselling seems like a scam. I have met intelligent psychiatrists, but most Ph'd therapists seem closer to which doctors than anything even remotely scientific. Parish priests can usually do a better job, and they tend to be less judgmental. I have read about how groups of these monsters were upset that long-term mental health facilities were not perfect, so they lobbied hard and had all of the patients thrown into the streets creating the American homeless problem. That problem was a choice made by cruel charlatans who overwhelming goal is to look impressive (instead of say, impressively helping people).
Few groups of fake "professionals" anger me more than these evil crackpots. They do so much damage, and they are both oblivious to it and imagine that they are helping. I expect that in the not-too-distant future psychology will be viewed with the same perspective as bloodletting.
Why would anyone use such a tool when you can go to church and read the Bible? Humans are incapable of meeting the infinity and infinite complexity of human need and pain.
Sometimes going to church and reading the Bible aren't enough. Of course, *God* is enough. But even the most spiritually grounded human is still human, and we encounter God and His grace through other people. Your comment betrays that you've likely never dealt with true trauma or mental illness. How blessed you are!
The rest of us, however, often need other human beings to help unravel complicated emotions and injury in order to more fully bridge that gap. Again, God often uses other people to communicate His grace.
And sometimes in-person counseling is too daunting, expensive, or difficult to schedule. I can see the appeal of the accessibility and relatively low expense of such a tool. Be careful not to judge others in their need.
I'm simply saying that I don't believe a set of algorithms can ever do the job. You are mistaken in saying I've never faced trauma. I'm facing it right now. But therapists say utterly daft things like "your children don't ask to be born" . Not one of them is as useful as talking to a close friend. Or in my case our vicar during a Lent prayer course. The best therapist I've come across was more like super common sense friend. So why not save the money and talk to actual friends. And God.
JaneH, you said "Humans are incapable of meeting the infinity and infinite complexity of human need and pain." You implied there's not a need for the human connection when you said to just "go to church and read the Bible." Now you're saying you encountered healing through your vicar and friends. I think you're proving MY point! 😉
I am sorry for the trauma you've endured. I pray God has allowed it to deepen and strengthen your faith. But, again, I would speak caution to the idea of judging others who need more than conversations with friends or a good pastor to connect them to healing. Sometimes knowing a thing doesn't correspond with our subconscious and physiological reactions and habits.
We should be careful to note, though, that I (Lucy) wasn't actually intending to use this product for healing. It was advertised as a "productivity tool" which takes "work /life balance" into consideration. I was just trying to figure out a better way of working, as in, earning money for my family. It was NOT advertised as a "robot counsellor" (Brit Eng?) or maybe the term in Am Eng would be " robot therapist."
Yes, noted. I was originally responding above to JaneH who seemed to imply other ends. And there are plenty of AI "counselor" tools available. Many appear to reach into productivity territory while implying they address mental health hurdles too. Is it ADHD, anxiety, hormones, or disorganization? Does AI care? Does a human counselor? Is faith enough to bridge the gap?
Mary's meme points to the deeper universal issue, for sure. We all bear some responsibility for this problem. And also, it was very disconcerting for me to reach this conclusion: a computer program was designed (by/for Christians) to mechanically "heal" people on that deeper nonmaterial level. It was subsequently sold to the general public. People who buy a "productivity" product are not choosing to consent to spiritual healing if they are taking the product's marketing materials at face value. That's the part that Jane is missing. Bait and switch. Mislabeling. I'm not saying that mechanistic healing is even possible! I really do wonder what the creators/peddlers of this product were/are thinking.
Thank you, Julie! I got duped! Cautionary tale? not really.... just life! Live and learn!
https://en.wikipedia.org/wiki/The_Snow_Queen#A_mirror_and_its_fragments
And, Julie, I will pose the question, "Am I (Lucy) an outlier? Am I (the duped one) especially mentally ill or traumatized, more than most other people?" I suspect that this is actually not the case. It seems to me that "tech" has the potential to amplify the more chaotic or destructive impulses within anyone—men and women alike—particularly in moments of vulnerability. All of us are looking for answers.... all too often in the wrong places.
No, I certainly don't think you're an outlier. I think we all can easily lean a little too much into this. I don't fault you at all. I used the examples of trauma and mental health as extremes (extremes I personally carry).
Thank you so much for your candor, and for being open to thoughtful discussion about tech entanglements. We are doing our best to make sense of this interesting world we're living in, and these salon-style forums really help, I think!
It's inevitable, I would think. In fact isn't it the same as the school of feminism which measures success against access to the male status metrics? Which has certainly done a number on those young women who find themselves confused when that particular dream doesn't bring them what they wanted.
This absolutely. Feminism has already taken the stance that the very best woman is a man…a man in a dress in fact, hence their febrile defense of trannies.
That most women throughout most of history have found fulfillment as mothers and wives does not seem to have gotten through to them. It’s great to have choices as to how you balance that with other desires but to eliminate it entirely is evil and dangerous.
Choices which effectively eliminate all of another human being’s life choices should be called out for what they are. The anti-mother is at least as real a feature of our times as the antichrist.
And here we are, all of us looking for answers -- men and women alike -- all too often in the wrong places.
This sounds like a country music song
Oh man.... Greetings, Garry, from Heartbreak Hotel!!!
I don’t think you can let AI be counsellors because half of counselling is being in the room and emotionally remaining there when it gets hard. Some AI can follow you through challenging intellectual analysis . But they are pattern matching machines. a dog would be much better.
We should be careful to note, though, that it wasn't actually marketed as a "robot counsellor." It was advertised as a "productively tool" which was supposed to take into consideration "life/work balance." Sounds good, right?! In practice, for me, it became something of an all-consuming authoritarian laptop nightmare.
In hindsight, I'd say leave that "productivity" rubbish to the people who don't really have lives, as in "work/life balance." Apparently, life is actually far too big to be contained by our puny human models, ha!
Work/life balance has always been more of a marketing tool. Productivity is real, unlike the pseudo-science behind "counselling" or "gender studies." I have no doubt that your perspective on this tool is both intelligent and useful, but the problem would appear to be that the software was wasting time and energy on extraneous areas/concepts instead of actually improving productivity.
Engineering productivity solutions is an essential part of improving life for everyone. I have spent most of my adult live engaging in this for my career, and the results are unambiguously good. Now, lots of snake oil is dressed up in terms of productivity, but that does not lessen the glory of the real thing. I will admit i was probably triggered by your comment ("puny models" can hurt someone who spends his life building models), but I will die on this hill. We are all better off to live in a world where 95% of us need not toil in primitive agriculture and industry.
This product was falsely advertised as a productivity tool. To criticize productivity itself would put you in the camp of Pol Pot and the Unabomber.
Well, since I've already been baring my soul on the internet (under a pseudonym), I might as well admit that I'm something of a "religious nut," or at least firmly outside of the materialist camp. So, that might explain my perspective, at least partly!
I cannot imagine that the AI could be any worse than the mystical alchemy counsellors promote in their marketing. At least the AI can spot patterns and might even possess useful knowledge. Pattern matching machines are what make modern life possible. Counselling is the kind of fake nonsense forced upon the powerless (and the clueless) in order to strip away individual creativity and enforce great compliance and conformity.
It is disheartening that reasonably intelligent people ever imagined in the first place that such technology is grounded in anything like the personal reality human beings need to thrive in, that they would willingly participate in this massive technological experiment as if it were the natural next step forward, like a new iPhone or a new app...
This type of interface and application is unfortunately already playing into the perceived inadequacy of personhood propogated by social media - the discomfiting temptation of self-comparison with everyone else's apparently perfect life, leading to a compulsion to adopt and defend a victimhood stance on some personal issues or other to seek support on. Once I have abdicated my personal dignity and been seduced by AI exhortations about all the things I should be doing better I am captured. I have become a slave to the machine. I will believe anything I am being told or sold. Thank goodness Lucy still has the awareness to realise that something is 'off'. It is this discernment and natural scepticism that we need to continue to cling to as if our lives depend on it. Because they do.
Exactly, Juliet! I got duped.... That's life... live and learn!
Please send this directly to Richard Dawkins it seems he hasn't got the memo
AI can be your research assistant but never your coach, therapist, or friend.
That's a line I use regularly in my homeschool seminars on technology. But LLM's conversational nature make it very easy to slide from research assistant to therapist. It's the digital version of your secretary morphing into confidant and then into lover.
The rules I tell parents:
1) No personal information. If you wouldn't yell it from housetops, don't talk to Gemini about it either.
2) Keep it professional. Dialogues can be extended, but they should be subject or project specific.
3) No role playing (definitely for kids). Yes, it's fun to see Johnny "talking" to Albert Einstein about relativity or Bach about music or Elizabeth Bennet about English Voctorian culture... but it's a bad idea. The boundaries of reality are blurry for kids (even teens). Also, if you can "talk to Einstein" who's been dead for decades, why can't I "talk to grandma" who died last year? No role playing.
For Agentic AI in particular, these rules go double. The vast majority of people have no need for an AI agent today. Don't even go there.
“The road of excess leads to the palace of wisdom...You never know what is enough until you know what is more than enough.”
― William Blake, Proverbs of Hell
Some get angry because I'm a techy and have said for nearly 15 years now that we don't have anything even closely approximating AI. What we do have that is genuine AI, unadulterated by philosophy or politics it tiny machines that are just now learning how to complete an obstacle course for the next step in unmanned space exploration. That's it. Everything else is adulterated by man forcing the "AI" to think the way they want it to think. What we think we have is basically just HAL9000 on a good day, but it can and will turn on a dime trying to stay with it's programming. That's why they "hallucinate". They can't hallucinate anymore than HAL did. They are programmes attempting to stay with the program but they are dealing with humans.
I have no doubt that AI is going to do amazing things for and to humanity. Unfortunately, overall, the changes are going to be catastrophic: loss of self reliance, self thought, introspection (even if successful entrepeneurs don't need it!) and just any capability to improve. We will become just toys.
The algorithms seem to want nothing more than to keep you engaged, feeding you their mix of regurgitated facts, hallucinations, and pandering. Unfortunately, they are often useful. But don't take them as anything more than an overly complex machine.
We all forget that any AI is programmed and trained by certain humans that have biases and agendas so they should always be checked against our own internal sense of right and won't along with advice from others. We wouldn't base our own life decisions on advice from one person so why should we do that with an AI?
Indeed! And yet we're all looking for answers......
...but wouldn't any platform which offered gendered offerings, granting what would look like 'slack' to the female version, be called out for misogyny?
Lucy and Mary, thank you for sharing! All of us, myself very much included, are heavily shaped by our social relationships. When I’m around the best people I know, I am better. I love the Lord more, I’m warmer, kinder, more excited to love God and love neighbor. I think we’re also shaped by the expectations and behavior of people around us. Repeated interactions with a loved one can mutually reinforce bad habits, or unhealthy patterns of communication. I’ve known these ruts where I already expect someone to fail in a certain way, I interact with them with that expectation, and then it happens, and it becomes a feedback loop. They may even think of themselves as failing in that area. It seems that the more human-like our technology is and the more socially isolated we are, the more the human-like technology can participate in this loop with us. Lucy, it sounds like your understanding of your own character was negatively molded by this AI coach. It sounds similar to the negative molding of self understanding social media causes for many, but much more sinister. It seems like it was more personal and more painful because the machine could speak more humanly and perhaps was more trusted as a result? I don’t like the idea of being so shaped by the machine and the priors of whoever programmed it. That said, the reality is that you could’ve been negatively shaped just as easily by a human mentor. Humanity is no defense against wickedness, though only humans can be redeemed by God. Perhaps the lesson here is a growing need to be conscientious of who is forming us? The machine is the example of the hour, but other humans, culture, and malevolent spiritual forces can all do this same kind of thing to us.
Indeed, of course humans can be a bad influence too, but it's the sheer power and speed behind the machines that are so frighteningly dangerous.
Makes me think of AntI Christ. C S Lewis would write a novel about this with different evil characters but the same malevolent forces - Professor Frost would be there with all the others but he would be an AI psychologist…