Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

“Get me there as fast as possible. Hey! I never said you should speed!”

This is literally the bad genie / monkey’s paw plot.

Give a powerful entity a goal and act shocked when it gets there in ways that aren’t in your best interest.



Putting aside whether Andrew's shock is appropriate, it seems like we agree that current agents at least occasionally do things that are straightforwardly against the interests of the prompter when given mundane prompts like "Get me into this gym class as soon as possible."

How does this look once agents are superintelligent?


> How does this look once agents are superintelligent?

Who cares? We're dealing with reality here on the ground.


Generally when I read a literal genie story, the message isn't 'well it was reasonable of the genie to do this'. It's more commonly either a morality tale of the person being wrong to ask for whatever it was they asked for, or just a "wouldn't it be fucked up if the genie did that huh"




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: