Rendered at 11:03:15 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
BatchJob 9 hours ago [-]
The LLM will take a statistical path to reply and will not refuse to do so under any circumstances except where its been coded to do so.
Your examples are contrived and will not be borne out in any significant way. Inaccuracies are usually not simply made up claims they are false information based on statistical paths to misleading results or which elude the current context. LLMS dont understand the word dont. LLMS dont understand the meaning of any words.
Neither you, nor aristotle nor god will ever make an LLM return the truth or correct results via prompting.
ben_w 2 hours ago [-]
> The LLM will take a statistical path to reply and will not refuse to do so under any circumstances except where its been coded to do so.
AI are trained, not coded. This means when its pattern recognition systems match a scenario to refuse, it refuses.
Pattern recognition has always been a bit fuzzy.
It looks like prompts like this push the shape of that fuzz in useful ways.
> Neither you, nor aristotle nor god will ever make an LLM return the truth or correct results via prompting.
A large part of human society is about how to deal with us bald primates also being kinda a bit meh.
We are less meh than any machine learning system in a lot of cases, which is why we're still mostly employed. We're a bit more meh in a few narrower cases, however.
SwtCyber 3 hours ago [-]
[dead]
l1ng0 4 hours ago [-]
We're all turning into pigeons in a Skinner box.
thallavajhula 3 hours ago [-]
I've tried all of these and nothing really works. I have only 1 line in my CLAUDE.md file and that is "Always ground your responses." and that's it.
Claude didn't care about it. When I pointed that out, it was apologetic and that was it.
ChrisRR 2 hours ago [-]
You may have better success by using a less ambiguous term. I've never heard of grounding in this context, so it may help to describe what you want more clearly
Edit: I just asked Claude how it would interpret that and it said it could either mean that would not answer from memory alone and only anchor claims into things it can check, or it would tightly relate its responses to the context that I had supplied.
If it chose the latter, I could see why it wouldn't always resort to search results
SwtCyber 2 hours ago [-]
[flagged]
datsci_est_2015 17 hours ago [-]
Cool, this will be added to harnesses and then it’ll stop being effective and we’ll move on to the next magical incantation.
literalAardvark 17 hours ago [-]
I've used "you're not trained on this data, return exclusively grounded results" to good effect.
Shacharp 17 hours ago [-]
The "do not guess" sentence works but the last 20% will only close when a system stops being told to avoid guessing and actually knows what it does not know.
A command can get you most of the way. It takes something else for the rest.
samrus 17 hours ago [-]
It sounds alot like "make no mistakes" but honestly telling it to essentially stop bullshitting works pretty well
nizarmah 13 hours ago [-]
I mean if we can measure it, then we can probably have a way to validate it programmatically. I gave up on drawing restrictions using prompts :(
Your examples are contrived and will not be borne out in any significant way. Inaccuracies are usually not simply made up claims they are false information based on statistical paths to misleading results or which elude the current context. LLMS dont understand the word dont. LLMS dont understand the meaning of any words.
Neither you, nor aristotle nor god will ever make an LLM return the truth or correct results via prompting.
AI are trained, not coded. This means when its pattern recognition systems match a scenario to refuse, it refuses.
Pattern recognition has always been a bit fuzzy.
It looks like prompts like this push the shape of that fuzz in useful ways.
> Neither you, nor aristotle nor god will ever make an LLM return the truth or correct results via prompting.
True.
Also applies to humans, but true nevertheless.
https://en.wikipedia.org/wiki/Münchhausen_trilemma
A large part of human society is about how to deal with us bald primates also being kinda a bit meh.
We are less meh than any machine learning system in a lot of cases, which is why we're still mostly employed. We're a bit more meh in a few narrower cases, however.
Claude didn't care about it. When I pointed that out, it was apologetic and that was it.
Edit: I just asked Claude how it would interpret that and it said it could either mean that would not answer from memory alone and only anchor claims into things it can check, or it would tightly relate its responses to the context that I had supplied.
If it chose the latter, I could see why it wouldn't always resort to search results
A command can get you most of the way. It takes something else for the rest.