It is starting to become clear to more and more people that Sam is a dyed in the wool True Believer in AGI. While it's obvious in hindsight that OpenAI would never have gotten anywhere if he wasn't, seeing it so starkly is really rubbing a lot of people the wrong way.
Well, in the world where AGI is created and it goes suboptimally, everybody gets turned into computronium and goes extinct, which is a prospect some are miffed about. And, in the world where it goes well, no decision of any consequence is made by a human being ever again, since the computer has planned every significant life event since before their birth. Free will in a very literal sense will have been erased. Sam being a true believer means he is not going to stop working until one of these worlds comes true. People who understand the stakes are understandably irked by him.