It doesn't say "slower traffic unless you're going the speed limit."
No one is forcing you to break the law, the sign is forcing you to move the fuck out of the lane when safe to do so. It doesn't say speed up and break the law, it says SLOWER traffic KEEP right.
The intent of the law is to keep traffic flowing, regardless of its speed. The reason being, we spend hundreds of millions of dollars widening our freeways, just so someone can sit in the far left lane, side by side with another vehicle, which is doing the same to the next two lanes, effectively turning a 4 lane highway into a 1 lane highway. That is why the slower traffic keep right sign exists.
It's not your concern if the vehicle behind you is speeding, your concern should be obeying the regulatory (by your I mean people violating the sign, I know you specifically aren't doing it) which is directly applicable to your driving. Which in this case, is the one telling you to keep right because there's a faster moving vehicle behind you.
I don't work the highway, but as I said, if I did... I would write that citation every-single-minute of every-single-day. Just about nothing else does more to fuck up traffic than left-lane bandits.
The catch-all that you would be violating is CVC 21461, failure to obey a regulatory sign.