![]() | |
Video game tips. Constant mistakes and hallucinations, in my experience. Seen this across a lot of different games. Even in really well documented games, such as OSRS (multiple fantastic wikis). |
|
![]() | |
They don't generate keyword search queries very well. They can overcome this by brute force but if you watch what they search you will cringe. nhl toronto scores
nhl hockey toronto scores
"nhl hockey" toronto score today
nhl "hockey score toronto"
"hockey" who won toronto etc. Somehow being good at semantic search makes them bad at keyword search, for whatever reason. |
|
![]() | |
I’ve noticed this too but it hasn’t been obvious to me that this style of search is not a learned behavior. Tool calling is very much part of the post training phase, I would expect that these style searches just naturally emerge during training. This is just my prior though. |
|
![]() | |
These models seem to be bad at writing prose or text. Many of the sentence structures seem to be unvaried. |
|
![]() | |
Short answers to simple questions. |
|
![]() | |
Playing Chess without letting it write a chess engine. |
|
![]() | |
It's dishonest. On several occasions team members have asked Claude to do things like analyze Gitlab CI timings and a lot of the numbers are outright fabricated. Said team members assume the numbers are good and continue with their work. Some hours are spent. Then finally someone realizes that the numbers don't look quite right and confronts Claude. Claude melts down and admits that it made it all up. You wouldn't tolerate this kind of duplicity from a human coworker, but AI is so fast and efficient at lying, so it's OK. |
|
![]() | |
science?!! but I'm working to fix that... |
|
![]() | |
Suggesting business names for businesses, I mean they are great, but they already exist, multiple times even. |
|
![]() | |
whenever I ask it for anything load bearing |
|
![]() | |
Claude is still not perfect at reading and interpreting noisy graphical data (imagine something like an EKG or chromosomal microarray plot). Still better than an average person but makes mistakes, not sure if this fits your description. |
|
![]() | |
being consistent when being asked the same question multiple times |
|
|
![]() | |
Spatial reasoning and 3d rigging and animation. Oh, you said simple. Speaking like a human |
|
|