NorcalAthlete put together a very accurate post regarding some of the factors that separate golf. It isn't a sprint sport, its the epitome of a marathon. No other sport offers a lifetime pursuit with failure as the constant. When a serious golfer has a shot within the distance of their tools, the goal is to put the ball in the hole. Mentally, balancing the drive to achieve while accepting the results over and over will mentally beat down most humans.
Along a different path: the massive amount of competitors that are all individually golfing and comparing scores at the end is a format in and of itself.
There are extremely few tournaments where the leader walks away from the field. Commonly, there is a group that forms where the difference is a tiny. The elite group is talented.
As to the life of a golfer. The coordination, physical conditioning, and mental approach - it's special in that the responsibility is solely on the golfer. Some choose to make it their life, others pursue it at whatever level they feel is proper. But, the golfers working on their game put millions of balls down range.
Swimming has a similar requirement. Where you truly hone your skills by swimming. Accessory workouts help fitness and condition the body, but swimming is how you tie the muscles and coordination together to swim fast and far. Golf, you have to hit balls. The elite golfers can tell you withing 1 to 2 millimeters where the face impacted the ball. That's after accelerating a club to over 100mph in a short period of time. Think about the driver. The elite golfer is targeting a 10-15 foot circle over 1000 feet away. Some driving holes are easier than others, but the elite golfer is focused on that specific target. No other sport drives the object ball without chemistry being involved.
The simplest sectors of the game: the tee box, the approach, and putting are each individually handled and combined by the player. If you haven't pursued it. The frustration of "feeling" 2 and not having any grasp of the 3rd is enough to cause grown men to throw their club. I've seen it, and it's hilarious.
Now, that's just a glimpse of the individual and their clubs. That isn't even starting to address that it's 18 holes to a course, and there is zero consistency between holes throughout the world. Sure, similar feel or aspects. But, the greens keeper moves the hole position on greens. If they have a fight with their wife that morning, you may not be able to putt the ball close enough to feel confident you can drop the ball in the hole. That's not common, but it happens. Will provide a link to a recent story where the college women's tournament put a hole where the average score was 3 over par.
The complexities of golf surpass most offerings in sports. It is slower. Takes a lot of interest to enjoy. But, if you get bit by the bug, it becomes a part of you. It's not for everybody. It's too hard