Yep meant welding brazing sorry. Was just a thought thinking out loud--what i'd do. Probably isn't as much clearance as it feels out at the end of the shaft.
& the hole is likely not round so maybe not the best [my best] plan. All good--carry on.
ps, i've forgotten exactly how it looks on that model but be good if a splined lever could be clamped direct to the change shaft without the linkage. But i guess its linkage'd for a reason like chain gets in the way. Not yamahas best design.

The play between a new shift cover, and shift shaft (which is not itself badly worn), is excessive, and this in combination with the hole in the case, where the shift shaft fits, being worn oval, means that the gear change will never be great.

A bushed shift shaft housing will remove all play from the outer cover. The case ideally needs to be bored and a bush fitted. However, with great care the worn crankcase hole can be smeared with something like Devcon F, then the gear change shaft fitted, after coating with some sort of release agent.

Leave it a couple of days to fully harden, and remove the shaft and clean everything up, before reassembling. If the cover has a grease nipple fitted, and it's lubed with heavy oil, that repair will last for quite a while.