Yeah, METR is an industry group funded by Google, Microsoft, Anthropic etc. I happened to summarize their work just yesterday actually: https://lemmy.blahaj.zone/comment/21358850 but one thing I think it’s important to point out is that there is no second study, not yet. The blog, and the open source dataset, are all we have, because the researchers decided to adjust their method. So the intimation arstechnica gives that they completed a second study is false.
And the selection effects they are talking about are pro-ai software developers being unwilling or unable to complete complex tasks without their precious ai anymore, whereas a year ago they were more willing to do both ways in order to prove how powerful ai is. In my opinion, as I stated, this reflects poorly on their levels of self confidence from using ai, more than saying anything about a speed boost/decline.
Yeah, METR is an industry group funded by Google, Microsoft, Anthropic etc. I happened to summarize their work just yesterday actually: https://lemmy.blahaj.zone/comment/21358850 but one thing I think it’s important to point out is that there is no second study, not yet. The blog, and the open source dataset, are all we have, because the researchers decided to adjust their method. So the intimation arstechnica gives that they completed a second study is false.
And the selection effects they are talking about are pro-ai software developers being unwilling or unable to complete complex tasks without their precious ai anymore, whereas a year ago they were more willing to do both ways in order to prove how powerful ai is. In my opinion, as I stated, this reflects poorly on their levels of self confidence from using ai, more than saying anything about a speed boost/decline.