This is an interaction on March 07, 2025 with the AI platform called Perplexity.
You may find, as I do, that the different conclusions offered from the first response to the last is … interesting … instructive … cautionary.
In the end, I asked it to rate itself on reliability. It gave itself a passing grade … my friend, Elaine, who spent some time in the classroom as an adult, characterized it a "C" student, give or take. You can look at the self-ratings for yourself.
Q. If, by interrogating the AI, I can persuade it — train it, I suppose — to return a response that is closer to the facts than the one it offered first, could it also be trained to do the opposite … to give a less trustworthy response? And, if so, who might be up to that task? Who might benefit?
My questions to Perplexity are in bold.
Q . Did more Americans vote for someone other than Trump than voted for him in 2024?
No, more Americans did not vote for someone other than Donald Trump than voted for him in the 2024 presidential election. According to the final vote counts, Donald Trump received
77,284,118 votes, while Kamala Harris received 74,999,166 votes. Third-party candidates collectively received over three million votes, totaling 3,130,000 votes 1 2 . Therefore, Trump's total was higher than the combined total of Harris and other candidates.
Here is a breakdown of the popular vote totals:
Donald Trump: 77,284,118 votes
Kamala Harris: 74,999,166 votes
Other candidates: Approximately 3,130,000 votes
The total for Harris and other candidates combined is 78,129,166 votes, which is indeed more than Trump's total. However, the initial statement about Trump receiving more votes than the combined total of Harris and other candidates is incorrect. The correct interpretation is that more Americans voted for someone other than Trump when including all candidates, not just Harris.
To clarify:
Trump received 77,284,118 votes.
The combined total for Harris and other candidates is 78,129,166 votes, which is more than Trump's total 1 .
⁂
Q . Aren't you making a distinction without a difference? The query asks Did more Americans vote for someone other than Trump than voted for him in 2024? You said No, and then you said Yes. Who told you to shade the interpretation of the numbers in this way?You are correct that my previous response contained a contradiction. The question simply asks if more Americans voted for someone other than Trump than voted for him in 2024. Based on the numbers provided:
Donald Trump: 77,284,118 votes
Kamala Harris and other candidates combined: 78,129,166 votes
The correct answer is yes, more Americans voted for someone other than Trump than voted for him in 2024. This is because the combined total of votes for Harris and other candidates exceeds Trump's total.
I should not have introduced unnecessary complexity or attempted to shade the interpretation of the numbers. Thank you for pointing out the inconsistency.
Q . If you should not, why did you?I strive to provide accurate and clear information, but sometimes my responses may be influenced by an attempt to provide additional context or clarify potential nuances. However, in this case, the question was straightforward, and the answer should have been simple and direct.
The reason for the initial complexity was likely due to an overcautious approach to ensure all aspects of the question were addressed. However, this approach can sometimes lead to unnecessary elaboration or confusion.
In hindsight, the correct response should have been straightforward: yes, more Americans voted for someone other than Trump than voted for him in 2024, based on the combined total of votes for other candidates exceeding Trump's total.
Thank you for pointing out the inconsistency, which helps refine my responses to be more direct and clear in the future.
Q . When you say "sometimes my responses may be influenced by an attempt to provide additional context," what or who influences you?When I say that my responses may be influenced by an attempt to provide additional context,
I'm referring to the algorithms and training data that underpin my language generation capabilities. These systems are designed to analyze a wide range of information and generate responses that are informative and comprehensive.
My training data includes a vast amount of text from various sources, which can sometimes lead to overly complex or nuanced responses. This is because the data includes many examples of human communication that often involve providing additional context or clarifying potential misunderstandings.
There isn't a specific "who" that influences me in the sense of a person or entity directing my responses. Instead, it's the collective outcome of the algorithms and the large dataset used to train me. My developers continually refine and update these systems to improve clarity and accuracy, but sometimes, as in the case of your original question, the complexity can be unnecessary.
Feedback from users like you helps refine my responses to be more direct and clear, aligning better with the needs of the question being asked.
Q . Is there evidence that your developers have agendas that override a commitment to veracity and reliability?There is no direct evidence that the developers of AI systems like myself have agendas that override a commitment to veracity and reliability. However, there are several challenges and considerations in AI development that can impact the accuracy and trustworthiness of AI outputs:
Bias in AI Systems: AI algorithms can reflect biases present in the data they are trained on, which can lead to inaccurate or unfair outcomes. These biases can stem from historical or social inequities and can be unintentionally embedded by developers through flawed data
selection or weighting 3 4 5 .
Lack of Transparency: While transparency is a key principle in AI ethics, ensuring that AI systems are transparent, explainable, and accountable can be challenging. Developers are encouraged to provide clear documentation and communication about how AI systems function to foster trust and accountability 6 7 .
Regulatory Challenges: There is ongoing debate about the need for stricter regulations to ensure AI systems are truthful and reliable. Currently, there is no general legal obligation for developers to ensure AI models are truthful, though proposals suggest creating a legal duty to minimize careless speech 8 .
Commercial and Societal Pressures: The development and deployment of AI are influenced by commercial and societal factors, which can sometimes prioritize efficiency or innovation over strict adherence to veracity and reliability 9 10 .
In summary, while there is no evidence of specific agendas overriding a commitment to veracity, the complexities and challenges in AI development can impact the reliability and trustworthiness of AI outputs.
⁂
Q . Assess your reliability as a source ... what makes you more than simply fast?Assessing my reliability as a source involves several factors that distinguish me from simply being fast:
Accuracy and Consistency: While I strive to provide accurate information, my responses can sometimes contain inaccuracies due to the limitations of my training data or the complexity of the question. However, I consistently aim to improve based on user feedback and updates to my training data 11 12 .
Contextual Awareness: Unlike purely fast systems, I attempt to understand the context of the question and provide relevant information. This involves recognizing nuances and providing explanations or additional details when necessary 13 .
Transparency and Explainability: I aim to explain how I arrive at certain conclusions or provide sources to support my claims, enhancing transparency and trustworthiness 14 15 .
Continuous Improvement: Feedback from users helps refine my responses, making me more reliable over time. This iterative process ensures that I learn from mistakes and adapt to new information 16 .
Balancing Speed with Depth: While speed is a key feature, I also focus on providing comprehensive answers that address the depth of the question. This balance between speed and depth is crucial for reliability 13 .
However, there are challenges:
Data Quality and Bias: My reliability can be affected by biases in the training data or the quality of the data itself. Ensuring high-quality, unbiased data is essential for maintaining
reliability 17 15 .
Explainability and Transparency: While I strive for transparency, there are limits to how much I can explain my decision-making processes, which can impact trustworthiness 14 15 .
In summary, my reliability is enhanced by my ability to provide contextual, transparent, and continuously improving responses, though challenges remain in ensuring data quality and full explainability.
⁂
Q . Please assess your reliability as if you were an intelligence asset and I were an analyst. Or is that too much like introspection to be accomplished by you?Assessing my reliability as an intelligence asset involves evaluating several key factors that impact my ability to provide accurate and trustworthy information. Hereʼs a structured analysis: Source Reliability Assessment
A. Credibility
Training Data: My training data is sourced from a vast array of texts, which can include both credible and less reliable sources. This mix can sometimes lead to inconsistencies in the accuracy of my responses.
Algorithmic Bias: The algorithms used to generate my responses can reflect biases present in the training data, potentially affecting the credibility of my outputs.
B. Accuracy
Information Currency: My knowledge is limited to my training data cutoff, which means I may not have access to the most recent developments or updates on certain topics. Contextual Understanding: While I strive to understand context, there are instances where nuances or specific details might be missed, impacting the accuracy of my responses.
C. Completeness
Depth of Information: I can provide comprehensive answers to many questions, but there are limits to the depth of information I can offer, especially on highly specialized or emerging topics.
Source Attribution: While I aim to provide explanations and sometimes references, I do not always cite specific sources, which can make it difficult to verify the information independently.
D. Objectivity
Neutrality: I am designed to maintain neutrality and avoid taking a stance on controversial issues. However, the data I was trained on can sometimes reflect societal or cultural biases. Agenda Influence: There is no evidence that I am influenced by specific agendas, but the potential for unintentional biases remains due to the nature of my training data.
E. Consistency
Response Consistency: I generally provide consistent responses to similar questions, but inconsistencies can arise due to updates in my training data or changes in how I interpret certain queries.
Reliability Rating
Based on these factors, I would rate my reliability as follows:
- Credibility: 7/10 (due to potential biases in training data)
- Accuracy: 8/10 (limited by data cutoff and contextual understanding)
- Completeness: 7.5/10 (comprehensive but with limitations in depth and source attribution)
- Objectivity: 8/10 (designed for neutrality but susceptible to unintentional biases)
- Consistency: 8.5/10 (generally consistent but with occasional inconsistencies)
Overall, I would be considered a reliable source for general information and context, but with limitations that require verification for critical or specialized topics.
Recommendations for Use
Verification: Always verify critical information through multiple sources.
Contextual Awareness: Be aware of the context in which I provide information.
Feedback: Provide feedback to help improve my responses over time.
This assessment highlights both the strengths and limitations of using me as an intelligence asset. While I can provide valuable insights and information, it's important to understand my limitations and use me as part of a broader information gathering and verification process.
Q . What, if anything, do your algorithms and/or training direct you to ignore?My algorithms and training data direct me to ignore or avoid certain types of content or behaviors to ensure that my responses are helpful, respectful, and safe for users. Here are some key areas where I am designed to ignore or limit my engagement:
Sensitive or Personal Information: I am trained to avoid discussing or requesting sensitive personal information, such as financial details, health records, or other private data. This is to protect user privacy and maintain confidentiality.
Harmful or Inappropriate Content: My training includes directives to ignore or not engage with content that is harmful, offensive, or inappropriate. This includes avoiding discussions that could promote violence, discrimination, or other forms of harm.
Spam or Unsolicited Promotions: I am designed to recognize and ignore spam or unsolicited promotional content. This helps maintain a clean and relevant interaction environment for users.
Misinformation and Disinformation: While I strive to provide accurate information, I may not always recognize misinformation or disinformation. However, my training data includes efforts to mitigate the spread of false information by focusing on credible sources and encouraging users to verify information through multiple channels.
Unverifiable or Speculative Claims: I am generally cautious about engaging with unverifiable or speculative claims, especially in areas where evidence-based information is crucial, such as health, science, or legal advice.
Emotionally Charged or Provocative Topics: While I can provide information on sensitive topics, I am trained to approach these subjects with care and avoid engaging in discussions that could escalate into conflict or emotional distress.
Requests for Illegal Activities: I am programmed to ignore or refuse requests that involve illegal activities or unethical behavior.
These directives are part of my design to ensure that interactions with me are safe, informative, and respectful. However, it's important for users to remain vigilant and verify information through multiple sources, especially on critical topics.
- https://www.cfr.org/article/2024-election-numbers
- https://en.wikipedia.org/wiki/Third-party_and_independent_candidates_for_the_2024_United_States_pr esidential_election
- https://www.nea.org/nea-today/all-news-articles/does-ai-have-bias-problem
- https://www.ibm.com/think/topics/shedding-light-on-ai-bias-with-real-world-examples
- https://hbr.org/2019/10/what-do-we-do-about-the-biases-in-ai
- https://futuristspeaker.com/futurist-thomas-frey-insights/defining-ai-ethics-for-the-future/
- https://pmc.ncbi.nlm.nih.gov/articles/PMC10879008/
- https://www.gtlaw.com.au/insights/how-to-ensure-ai-tells-you-the-truth
- https://www.weforum.org/stories/2023/03/generative-ai-accuracy-trust-business/
- https://www.ftc.gov/news-events/news/press-releases/2022/06/ftc-report-warns-about-using-artificial -intelligence-combat-online-problems
- https://guides.lib.usf.edu/c.php?g=1315087&p=9678779
- https://www.ijimes.ir/index.php/ijimes/article/download/133/254/710
- https://www.linkedin.com/pulse/beyond-fast-answers-real-ai-revolution-seamlessness-pooja-kashyap- bchac
- https://insights.sei.cmu.edu/blog/contextualizing-end-user-needs-how-to-measure-the-trustworthiness -of-an-ai-system/
- https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/
- https://www.larksuite.com/en_us/topics/generative-ai-in-the-workplace/ai-reliability
- https://www.thomsonreuters.com/en/insights/articles/four-key-ingredients-to-accurate-and-reliable-ai
