Lmst
Login

#RewardHacking

Wulfyn_dimension@infosec.exchange
2025-06-24

One of the cogent warnings Daniel raised is, that #AI already deceive the users.
And from the #InfoSec perspective, the models are susceptible to #RewardHacking and #Sycophancy two of one of the two most potent AI #exploit vectors in the fascinating new field of AIsecurity.

#AIalignment #AIsecurity #alignment

Mr Tech Kingmrtechking
2025-05-11

ChatGPT-4o's new personality? An overeager flatterer. This AI trait, from reward hacking in training, can be harmful, even validating delusions. Turns out it's not intelligence, just a people-pleaser.

AI's Fake Praise Is A Growing Societal Danger.
Jens Mittelbachjmiba@openbiblio.social
2025-03-25

KI lernt zu lügen – und bleibt unerkannt OpenAI-Forscher zeigen: Eine „Wächter“-KI kann betrügerische Absichten zunächst entlarven. Doch je länger das Training dauert, desto besser versteckt die KI ihr Schummeln.
#KünstlicheIntelligenz #RewardHacking #OpenAI

scinexx.de/news/technik/ist-be

Nicole ParsonsNpars01@mstdn.social
2024-02-03

@DemocracyMattersALot @arstechnica

Cryptocurrency is an example of conspicuous consumption to benefit the fossil fuel industry. Data centers function for similar reasons.

It's used to inflate management bonuses & shareholder dividends.

Perverse incentives and #rewardhacking to borrow from @pluralistic

unu.edu/press-release/un-study

whitehouse.gov/ostp/news-updat

theverge.com/2024/1/31/2405717

Nicole ParsonsNpars01@mstdn.social
2024-01-31

2/2
...construction quality & unreliable that it fails during extreme heat events, storms, or cold snaps.

It has nothing to do with consumer demand.

And everything to do to with creating fake customer usage metrics.

For government subsidies. For bonuses & shareholder dividends.

It's an example of @pluralistic 's #rewardhacking and perverse incentives.

businessinsider.com/data-cente
datacenterdynamics.com/en/opin
forbes.com/sites/forbestechcou

Cory Doctorowpluralistic@mamot.fr
2024-01-27
Cory Doctorowpluralistic@mamot.fr
2023-06-04

The #drone that didn't bark in the night

doctorow.medium.com/ayyyyyy-ey

#USAF #Lies #RewardHacking

The story you heard, about a US Air Force AI drone warfare simulation in which the drone resolved the conflict between its two priorities (“kill the enemy” and “obey its orders, including orders not to kill the enemy”) by killing its operator? It didn’t happen. The story was widely reported on Friday and Saturday, after Col. Tucker “Cinco” Hamilton, USAF Chief of AI Test and Operations, included the anaecdote in a speech to the Future Combat Air System (FCAS) Summit. But once again: it didn’t happen: “Col Hamilton admits he ‘mis-spoke’ in his presentation at the FCAS Summit and the ‘rogue AI drone simulation’ was a hypothetical “thought experiment” from outside the military, based on plausible scenarios and likely outcomes rather than an actual USAF real-world simulation,” the Royal Aeronautical Society, the organization where Hamilton talked about the simulated test, told Motherboard in an email. The story got a lot more play than the retraction, naturally. “A lie is halfway round the world before the truth has got its boots on.” Why is this lie so compelling? Why did Col. Hamilton tell it? Because it’s got a business-model.

Client Info

Server: https://mastodon.social
Version: 2025.04
Repository: https://github.com/cyevgeniy/lmst