I have a little background here I won't go into, however, I type at 2 words per minute so just trust me on my earned expertise.

So, first question, how old are you? If over 19..
Have you considered Master's swimming?
USMS is HUGE in this nation, especially in CA and your area. Masters come in all shapes, sizes and speeds......workouts can be modified any which way to your particular speed, needs etc. So, you can only do freestyle right now due to injury? no biggie, you just alter the sets accordingly.
Walnut Creek Masters produces swimmers qualifying for Olympic Trials...at 50-55 age group.
Davis Aquatic Masters was founded by Dave and Vern Scott; Every Triathlete in my area belongs to DAM.
You get organized workouts keyed to your goals. And yes, lots of camaraderie to keep that weight coming off and yup, hot chicks in swim suits.
Not only do I have "a bit" of a background but my kid's Dad is a level 5 Olympic Swim coach; coached 2 Olympians to top 5 finshes...Scott Weltz being the latest in London. So just do it. The coaches will take it from there. You're welcome.