Discussion about this post

User's avatar
Li-Lian Ang's avatar

> As the pacing exercises gain momentum, I expect more Western AI companies would announce their own pacing exercises

Out of curiosity, what makes you expect this?

Ivan French's avatar

One thought from healthcare. Clinical trials have independent safety boards that can stop a trial early. The key detail is that the stopping rules are written before the trial starts, and the sponsor isn't the one who pulls the trigger. That's what makes it credible... There's also a lesson from factory inspections: announced ones produce clean factories oninspection day, which is why the FDA moved to showig up unannounced.

So maybe the first few exercises do exactly what you describe, on the company's own schedule, to work out definitions and figure out what evaluators actually need to see. But the goal after that is handing someone else the calendar. Stopping when it's convenient is the warm-up. Stopping when a run is going well and the timing is terrible is the thing that would actually build trust.

1 more comment...

No posts

Ready for more?