More the later than the former… they are better than purely marketing focused stuff pushed out by the LLM companies, and if you dig through them and read between the lines you can occasionally sift out useful details. Like here is a pretty solid sneer digging through Mythos’s ‘system card’ and pointing out all the ways it contradicts the hype and press headlines Anthropic was pushing.

But even so they have some big problems…

• the benchmarks the system cards reference are kind of useless and heavily gamed
• LLM companies want to keep lots of details secret from competitors, so fundamentals like number of model weights or parameters or size/quality of the training data set or other training specs are deliberately left out
• lots of the stuff they reference is booster garbage and/or doomer crit-hype
• they tend to be long, wastefully so, imitating the length of academic papers without having the corresponding amount of depth or information
• despite their length and wordiness they also neglect basic practical usage advice that isn’t even proprietary (or at least would be bound to leak if you poke around with the model at all and thus not worth keeping secret in the first place). Like not even big picture stuff I mentioned in my second bullet, but really simple stuff…